They bounce against air molecules, causing them to have an effective diameter larger than their real diameter. However, the effective diameter is at all times LESS than that of a particle with a (slightly) larger diameter. The net result is that smaller particles get captured LESS often. The Brownian motion might cause them to be captured  more often than what would be the case in a vacuum, it is still much much less than 95%. Lets say their effective diameter becomes 10x larger (which I doubt) then the holes are still 200 times larger and they'll fly through with the air unimpeded.