It made sense, then I was thrown. By technically I assume that is in a perfect world, where luck wouldn't matter to begin with? Or maybe on a time line that extends so far that it isn't really that relevant... I must say I am a bit thrown still... and just as I thought I was understanding.
Over an infinite period of time, you would expect luck to be infinitely close to neutral regardless of mining speed/pool size. Since that's not reasonable, you end up with the basic variance. The smaller the pool, the more wildly it can move for a given time frame.
Think of it like a wave on a graph, steadily moving up and down, centered on the axis (the axis being neutral luck). For a smaller pool, its stretched out so it spends a lot of time above or below the line, but it steadily moves between the two. So if you take a given slice (2 weeks), you may end up quite a bit above or below the average line depending on where in the wave that 2 week period landed in.
For a larger pool, that wave is squished, so it fluctuates up/down very rapidly. But since it's moving between the two so frequently, taking a 2 week slice of that graph will likely average closer to 0 since it never spends too much time away from the axis.