I've got no memory issues on host, and I created a new VM and it still had the problem. The issue is with the files somehow. How can I keep most of the blocks but have Bitcoin Core just download the last few again without reindexing or redownloading the entire blockchain? What files should I delete/change?
There is no "issue with the files somehow". There is an issue with your hardware or software. In my limited experience the most common problem with corrupted files on Windows, even with "fresh installs", is caused by the buggy antivirus or security packages. Edit: The second most common is that people who claim "fresh installs" actually don't do a fresh install but some sort of "OEM restore" that riddles the Windows with crapware, which although not malicious is frequently ultra buggy. The true "fresh install" is from non-writable original Microsoft disk or from a writable disk/image with SHA1 verified with MSDN/Technet/other official Microsoft channel source, not through an OEM. End of edit.
Bitcoin lets you be your own bank. Then just be your own bank and put your wallet on a backup computer (you do have a backup computer, right?). You'll quickly learn that you have problem with your primary computer.