Well, back to basics:
If you can do a ping test (like 1000 packets) from the router to your destination pool successfully (not more than 1 packet lost) then your Internet connection is OK and you can try the following:
1. Change the Ethernet cable from the miner to the switch and the cable from switch to router (one-by-one)
2. Make sure it is not getting intense interference from any parallel running power cord very close to it
3. Try a different switch in case the one you have has problems
4. Connect only the miners to the switch in case another PC on the net is broadcasting packets and causing congestion, and then connect only that switch to the router.
5. Connect each miner one-by-one to the switch and connect the switch to the router.
6. Connect each miner one-by-one to the router directly
In each of these cases try a "ping -n 1000" (windows) or "ping -c 1000" (linux) from the miner to the router and you should get 1000 replies.
Work out what causes the problem from the results.
Plain old Ethernet debugging