You did not mention whether you are looking for a managed VPS or an unmanaged one. Unmanaged VPS is way cheaper than managed VPS. For unmanaged VPS, you may try Amazon AWS, Google App Engine or Heroku and set up as per your need. For managed VPS, you may try the one quoted below. I have tried their shared hosting service and I dont have any complaint so far...
Looking for something that is reliable and takes bitcoin. If they are good I do not mind signing up via a referral link. The VPS just runs a small IRC bot...nothing shady.
You might
compare VPS packages here and order your chosen one. Monthly price starts @ 10 USD.
Bitcoin accepted here.
You might check feedback of other forum users on our thread -
https://bitcointalk.org/index.php?topic=617213.0. Wishing to see your feedback as well.