Bitcoin Forum
December 08, 2016, 06:29:39 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: Linode compromised and it's effect on TradeHill  (Read 6645 times)
Jered Kenna (TradeHill)
Sr. Member
****
Offline Offline

Activity: 420



View Profile WWW
March 02, 2012, 10:41:24 AM
 #1

From tradehillblog.com


Dear Clients,

Yesterday, March 1st, one of our hosting providers was compromised. Linode.com contacted us several hours ago via email stating:



" As a valued customer, the security of your account is our top priority.  Today we suffered from an unauthorized access of our system that resulted in eight customer accounts being compromised. Unfortunately, your account was one of the accounts targeted.

You should begin a compromised system recovery procedure immediately -- we recommend a complete redeployment -- on your  Linodes. "




It appears that hackers were targeting accounts owned by Bitcoin businesses.

We built TradeHill with security as a priority and make use of multiple data centers to protect core infrastructure. Due to our system architecture design it appears that no user data was compromised, and all wallet files are safe, however we are in the process of a more thourough audit and will provide updates when more information is available.

We are taking TradeHill offline at this point as a precaution and appreciate your patience.

Updates can be found  here and on twitter via @tradehill and @jeredkenna

Regards,

Jered Kenna
Chief Executive Officer
TradeHill

moneyandtech.com
@moneyandtech @jeredkenna
1481178579
Hero Member
*
Offline Offline

Posts: 1481178579

View Profile Personal Message (Offline)

Ignore
1481178579
Reply with quote  #2

1481178579
Report to moderator
1481178579
Hero Member
*
Offline Offline

Posts: 1481178579

View Profile Personal Message (Offline)

Ignore
1481178579
Reply with quote  #2

1481178579
Report to moderator
1481178579
Hero Member
*
Offline Offline

Posts: 1481178579

View Profile Personal Message (Offline)

Ignore
1481178579
Reply with quote  #2

1481178579
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481178579
Hero Member
*
Offline Offline

Posts: 1481178579

View Profile Personal Message (Offline)

Ignore
1481178579
Reply with quote  #2

1481178579
Report to moderator
1481178579
Hero Member
*
Offline Offline

Posts: 1481178579

View Profile Personal Message (Offline)

Ignore
1481178579
Reply with quote  #2

1481178579
Report to moderator
deepceleron
Legendary
*
Offline Offline

Activity: 1470



View Profile WWW
March 02, 2012, 10:51:38 AM
 #2

It would seem the attacker used backdoor administration access that was not logged (and not publicized as being present) to reset shadow passwords and gain access. They were quite quick in withdrawing funds from slush mining pool, bitcoin faucet, and bitcoinica, but I wouldn't rule out any kind of compromise or future wallet emptying, as it seems that many think this came from inside Linode themselves using tools only their personnel would have access to on any reasonably administered system. It would be wise to go as far as considering your entire VPS file system cloned and logged into with root access, then think of what the intruder might do with the data.

Jered Kenna (TradeHill)
Sr. Member
****
Offline Offline

Activity: 420



View Profile WWW
March 02, 2012, 11:24:45 AM
 #3

It would seem the attacker used backdoor administration access that was not logged (and not publicized as being present) to reset shadow passwords and gain access. They were quite quick in withdrawing funds from slush mining pool, bitcoin faucet, and bitcoinica, but I wouldn't rule out any kind of compromise or future wallet emptying, as it seems that many think this came from inside Linode themselves using tools only their personnel would have access to on any reasonably administered system. It would be wise to go as far as considering your entire VPS file system cloned and logged into with root access, then think of what the intruder might do with the data.

I hesitate to speak too soon and we're going through it right now but it looks fine. The idea was that Linode could be compromised without compromising TradeHill and without compromising the servers where everything happens. We built the exchange to protect against things like this and the engineer that laid it out did fine job. I wasn't enjoying paying the extra data centers every month but I certainly don't regret the decision now.

Linode is done though, they were great to work with and we'll see what happens.

Jered

moneyandtech.com
@moneyandtech @jeredkenna
zby
Legendary
*
Offline Offline

Activity: 1431


View Profile
March 02, 2012, 11:29:52 AM
 #4

It is reassuring to see you posting here - is TradeHill customer service still working?
Jered Kenna (TradeHill)
Sr. Member
****
Offline Offline

Activity: 420



View Profile WWW
March 02, 2012, 11:36:57 AM
 #5

It is reassuring to see you posting here - is TradeHill customer service still working?

Thanks. Yes customer service is still working we're just swamped trying to get all the funds out and deal with a lot of individual requests.
When the banks simultaneously closed our accounts it created a huge backlog. We'll get to the emails as fast as we can. Almost everything has been processed at this point and is under control. We'll have the Linode situation sorted soon as well.

-Jered

moneyandtech.com
@moneyandtech @jeredkenna
Etlase2
Hero Member
*****
Offline Offline

Activity: 798


View Profile
March 02, 2012, 01:14:31 PM
 #6

Good work, TradeHill. In the interest of protecting from future events like this, would you disclose what you did that made your accounts immune to this attack?

Technomage
Legendary
*
Offline Offline

Activity: 1624


Affordable Physical Bitcoins - Denarium.com


View Profile WWW
March 02, 2012, 01:28:39 PM
 #7

This is 10 points for Tradehill. I hope you launch Bitcoin.com soon, I'm interested in that site. You've earned the trust of the community and we're excited for what's coming.

Denarium - Leading Physical Bitcoin Manufacturer - Special Xmas deals now live!
zby
Legendary
*
Offline Offline

Activity: 1431


View Profile
March 06, 2012, 01:28:32 PM
 #8

It is reassuring to see you posting here - is TradeHill customer service still working?

Thanks. Yes customer service is still working we're just swamped trying to get all the funds out and deal with a lot of individual requests.
When the banks simultaneously closed our accounts it created a huge backlog. We'll get to the emails as fast as we can. Almost everything has been processed at this point and is under control. We'll have the Linode situation sorted soon as well.

-Jered
What is the ETA for dealing with the requests?
Jered Kenna (TradeHill)
Sr. Member
****
Offline Offline

Activity: 420



View Profile WWW
March 06, 2012, 07:24:18 PM
 #9

It is reassuring to see you posting here - is TradeHill customer service still working?

Thanks. Yes customer service is still working we're just swamped trying to get all the funds out and deal with a lot of individual requests.
When the banks simultaneously closed our accounts it created a huge backlog. We'll get to the emails as fast as we can. Almost everything has been processed at this point and is under control. We'll have the Linode situation sorted soon as well.

-Jered
What is the ETA for dealing with the requests?

We're currently dealing with them but we are being very careful and doing them all manually.
I'd estimate a few days at most we'll be caught up and they shouldn't take more than a day after that point.
We don't want to screw anything up considering everything that has happened.
We've also been dealing a lot with our lawsuit against Dwolla.
See tradehillblog.com for more info on that or the other post in this forum.

-Jered

moneyandtech.com
@moneyandtech @jeredkenna
Michael_S
Sr. Member
****
Offline Offline

Activity: 277


Bitcoin-Note-and-Voucher-Printing-Empowerer


View Profile
March 07, 2012, 03:59:48 PM
 #10

I haven't seen my funds yet, and no reply from TH since 16 Feb 2012.

Generally I fear that my TH funds (EUR and USD) are gone forever (hope I am wrong), because I do not trust TH any more.

Why? --> See here.

casascius
Mike Caldwell
VIP
Legendary
*
Offline Offline

Activity: 1344


The Casascius 1oz 10BTC Silver Round (w/ Gold B)


View Profile WWW
March 07, 2012, 04:22:58 PM
 #11

SOmething I thought hilarious, I read the TradeHill complaint (lawsuit) on Scribd.com, which plastered it with banner ads for Linode "hosting as little as 65 cents a day" between every page.  Not just once, but the majority of the ad spots between the 19 pages went to Linode.

Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable.  I never believe them.  If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins.  I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion.  Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice.  Don't keep coins online. Use paper wallets instead.
Jered Kenna (TradeHill)
Sr. Member
****
Offline Offline

Activity: 420



View Profile WWW
March 07, 2012, 07:56:51 PM
 #12

I haven't seen my funds yet, and no reply from TH since 16 Feb 2012.

Generally I fear that my TH funds (EUR and USD) are gone forever (hope I am wrong), because I do not trust TH any more.

Why? --> See here.

I'm pasting this from the other thread. We take this very seriously.

Regarding the withdrawals the site will be back up very soon and either have an interface or provide withdrawals via email.
Send an email to info@tradehill.com now if you like and I'll process it as soon as we finish recovering from the Linode hack.

About the email with the information CC'd. We take privacy very seriously. This was an isolated incident where one employee made a mistake and sent out an email to several customers using CC instead of BCC. It's not excusable and the employee has been reprimanded. Despite this only happening once with over 100,000 emails sent I want to make it very clear that it was not our policy and I will take responsibility for this. It was not our intention and I would like to seriously apologize to the people effected.


SOmething I thought hilarious, I read the TradeHill complaint (lawsuit) on Scribd.com, which plastered it with banner ads for Linode "hosting as little as 65 cents a day" between every page.  Not just once, but the majority of the ad spots between the 19 pages went to Linode.

That is funny as hell.

-Jered

moneyandtech.com
@moneyandtech @jeredkenna
ctoon6
Sr. Member
****
Offline Offline

Activity: 350



View Profile
March 07, 2012, 10:34:03 PM
 #13

It scares me that websites dealing with the amount of money you guys deal with even have the remote consideration of using these types of services (clouds and vps). simply a joke and a very good reveal of the security taken by all these companies.

Jered Kenna (TradeHill)
Sr. Member
****
Offline Offline

Activity: 420



View Profile WWW
March 08, 2012, 12:39:34 AM
 #14

It scares me that websites dealing with the amount of money you guys deal with even have the remote consideration of using these types of services (clouds and vps). simply a joke and a very good reveal of the security taken by all these companies.

We've never kept any wallet files or records on Linode. I'm not as technical as the guys that set it up so I don't want to comment on specifics and make an incorrect statement.

That said it was essentially a decoy and it's compromise didn't cost TradeHill anything other than time and if we were still up with our engineers it would have been insignificant. It worked exactly as intended. We were very concerned about these types of attacks and others which is what we looked at when designing it. I'll see someone more qualified on the subject wants to speak but they have moved on to other projects and are no longer working with us.

-Jered

moneyandtech.com
@moneyandtech @jeredkenna
coiningz
Newbie
*
Offline Offline

Activity: 9


View Profile
March 10, 2012, 09:32:25 AM
 #15

It looks to me like you did in fact lose bitcoins because of Linode.  Why else are you stalling to process refunds from Tradehill?  Did you really lose all of your coins?

The timing of Linode getting hacked, Tradehill's refusal to manually process withdrawals, and the lawsuit against Dwolla all leads me to believe you have nothing left to pay withdrawals with, and you sued Dwolla both as misdirection and as your last hope of recovering the lost funds. 

Am I wrong?  Prove it by either putting Tradehill.com back online so people can withdraw through automated means (why hasn't this been done yet?) or manually processing every withdrawal request in your queue.
Melbustus
Legendary
*
Offline Offline

Activity: 1554



View Profile
March 11, 2012, 09:11:20 AM
 #16

I wouldn't mind an update... I still have some BTC as well as USD in my Tradehill account.

Bitcoin is the first monetary system to credibly offer perfect information to all economic participants.
But Bitcointalk & /r/bitcoin are heavily censored. bitco.in/forum, forum.bitcoin.com, and /r/btc are open.
Best info on Casascius coins: http://spotcoins.com/casascius
Jered Kenna (TradeHill)
Sr. Member
****
Offline Offline

Activity: 420



View Profile WWW
March 11, 2012, 11:50:37 PM
 #17

I wouldn't mind an update... I still have some BTC as well as USD in my Tradehill account.

I cab move all the coins to an address I announce ahead of time to show that they're still under my control.
Instead of bringing the site up and paying for all the servers we're going to do withdrawals via email.
There aren't very many accounts left with funds in them and it should be straightforward.

The goal was to have this done this morning but it is taking longer than anticipated.
I'll have an update soon. My apologies on this taking far longer than expected.
It was somewhat of a perfect storm. We were almost completely wrapped up when this hit along with Paxum ceasing to do Bitcoin business.
I'm also busy moving to a new house so I've just given up on sleep.

Jered

moneyandtech.com
@moneyandtech @jeredkenna
Melbustus
Legendary
*
Offline Offline

Activity: 1554



View Profile
March 12, 2012, 07:07:30 AM
 #18

I wouldn't mind an update... I still have some BTC as well as USD in my Tradehill account.

I cab move all the coins to an address I announce ahead of time to show that they're still under my control.
Instead of bringing the site up and paying for all the servers we're going to do withdrawals via email.
There aren't very many accounts left with funds in them and it should be straightforward.

The goal was to have this done this morning but it is taking longer than anticipated.
I'll have an update soon. My apologies on this taking far longer than expected.
It was somewhat of a perfect storm. We were almost completely wrapped up when this hit along with Paxum ceasing to do Bitcoin business.
I'm also busy moving to a new house so I've just given up on sleep.

Jered


Thanks for the update - much appreciated. I can wait... Tradehill's reputation to date has been rock solid, so as long as you guys are responsive on this board, I'm happy. I understand things are crazy over there right now - hope you guys get all this behind you soon. 

Bitcoin is the first monetary system to credibly offer perfect information to all economic participants.
But Bitcointalk & /r/bitcoin are heavily censored. bitco.in/forum, forum.bitcoin.com, and /r/btc are open.
Best info on Casascius coins: http://spotcoins.com/casascius
coiningz
Newbie
*
Offline Offline

Activity: 9


View Profile
March 14, 2012, 06:16:51 PM
 #19

I wouldn't mind an update... I still have some BTC as well as USD in my Tradehill account.

I cab move all the coins to an address I announce ahead of time to show that they're still under my control.
Instead of bringing the site up and paying for all the servers we're going to do withdrawals via email.
There aren't very many accounts left with funds in them and it should be straightforward.

Then do so with haste.  Every day you hold those funds is a day that your former customers can't have use of them.  The best way you can prove to the world that you didn't lose all your coins is to send them back to everyone who's been waiting a week to withdraw.
Yankee (BitInstant)
Legendary
*
Offline Offline

Activity: 1078


Charlie 'Van Bitcoin' Shrem


View Profile WWW
March 14, 2012, 06:47:36 PM
 #20

Attn BTC and USD Holders on TradeHill:

I can confirm that the funds are safe. Gareth and myself are working hard, in addition to our own work, in helping Jered and his team get everything back up to speed and get your funds out.

For all those who think Jered is delaying, he is not. We are all on different timezones, ranging from CA, to NY to the UK and Jered has been staying up late to make sure everything is worked out.

If my word means anything, as far as I know (and I've seen) your funds are safe and should be ready for withdrawal ASAP.

We are all start-ups here and not mega corporations. We've never had to deal with situations like this, so contingency plans are difficult to create. We ask that you cut us some slack.

Thank You

Charlie Shrem, CEO
Bitinstant LLC

Bitcoin pioneer. An apostle of Satoshi Nakamoto. A crusader for a new, better, tech-driven society. A dreamer.

More about me: http://CharlieShrem.com
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!