In case anyone finds themselves in a similar position, here is what Bitmain advises:
Dear L Lanaio,
Thank you for contacting us. The error message "Socket connection failed" has several possible causes. Please test as follows to determine the cause of the error on your miner:
Please upgrade the firmware:
https://shop.bitmain.com/support.htm?pid=00720160617153633091CybeR3fo06DA1. Confirm that your internet connection is stable. Network configurations or security settings in the router can cause this error.
2. If the network is good then please disconnect all the hash boards from both the power supply unit and the controller. Reboot the miner with only the controller connected. If the error still appears then we need to replace the controller. Please follow these instructions to create the ticket:
https://bitmainhelp.zendesk.com/hc/en-us/articles/115000024827. Please reply with the ticket number and we will send a replacement.
3. If the error message does not appear with only the controller connected then please run the miner with one hash board at a time. If one of the hash boards is defective please create a repair ticket and return the hash board (for S9 return the whole miner) to us for repair:
https://shop.bitmain.com/workOrderGuide.htmPlease let us know if the problem is still not resolved or you have further questions.
Best regards,
Barbara
Bitmain
It looks like I got several bad controller boards.