I found a pool that offers a fixed diff of 32, 64 and 128. What's the point of that? Why do I choose 128 if the 32 is easier? What's the difference between 32 and 128?
Re-read
your previous question. I'll try to make it easier.
If you work at diff 32 and send a result to the pool, it will have value '32'. If you work at diff 128, each result has value '128'.
At the end of the round, those values make up your share of the reward proportionally.
Alright, but in the end both diffs mines the same, right?
Only if you have enough hash. At the extreme, you can mine at network diff and at that point you'll be subject to normal variance.
If your diff is so high you miss to send result across a whole round you'll most likely be screwed somehow. Pools don't reward share value exactly and the time distribution is important. Producing less than 2-3 results per minute might incur in quite some loss, especially if your pool has high rej% such as the case for p2pool.