No because usually when you need 32gb your OS/application also needs several CPUs. You rarely need 32 GB to use something with only 1-2 CPUs, otherwise your server will crash as you guess. I don't remember seeing a site where you can customize your CPU
You may try to visit lowendbox or webhostingtalk and ask for an offer you will receive a lot of them
Now and thinking about it, are these CPUs or Cores? I managed to find something much cheaper with OVH and it has one CPU (4 core, 8 threads) and 32GB RAM:
https://www.ovh.com/world/dedicated-servers/hosting/1901host01.xmlBut again, the problem is that they don't accept bitcoin.
Hostforlife.eu. You can consider this hosting provider if you're looking for windows platform. OVH server is cheap but their support is very pathetic. I have tested their services, I submitted inquiry, no one reply in 3 days. Maybe this is the reason why the server is quite cheap.
I've checked their website and just like the website suggested above, it looks expensive.
I don't think I care that much about the support, I'm not planning to run something that could cause problems or I might need help with, as long as the server is running 24/7 that's all I need.