N is average of last few block's share sum.
354 block height's block sum 100
355 block height's block sum 110
356 block height's block sum 200
357 block height's block sum will be ((100 + 110 + 200) / 3) = 136
If 357 block height's block sum was 200, then first 64 shares will be skipped and only 136 shares from backward will be calculated.
It's just rough calculation to describe what's happening at PPLNS. It's much more complex calculation.
I think you are concerned about fair distribution. It's quite fair because pool distributes to miners according to their time and difficutly.
You are collaborating and competing the block reward with other miners in same pool.
