Assuming a VPN is only going to add a tiny delay to their current connection, sounds like you've never looked at how a typical public VPN performs
Yes, it's just a tiny delay if
1- Your internet connection is fast in the first place
2- The VPN server is pretty close
3- The pool server is close to the VPN server you connect to
4- It's a paid VPN with a decent server
Of course, if you use a free service that is fully loaded with other users and is located on the other side of the globe then you are asking for trouble, but most VPN services now have servers all over the world, it's pretty easy to find a decent VPN server nearby.
This isn't a matter of what I or you think, anyone can try it and see the results to themselves.
You seem to have some idea that adding %'s here and there and elsewhere don't matter.
It does matter, every share counts, and I rather get 0% stale shares, but if mine is 0.5% or 1% does that mean I should shut down my gears and move on with my life? it does seem like that is what you are suggesting, as a pool operator you have the code in place that takes care of this, if the work is submitted too late, you reject it, it's as simple as that.
Your concern is high latency, not the use of VPN, miner "A" could be mining to your pool with no VPN but with very high latency, on the other hand, miner "B" on a VPN may have very low latency to your pool compared to miner "A", to you, it does not matter, a share is either accepted or not, not sure what part of that you don't understand.
I can certainly say I am glad you don't mine on my pool
That makes two of us, I would be very pissed if I mined for almost a year and got zero rewards while wasting power bills and burning a few hash boards along the way.