Suspect #1: Linode admins/insiders

[Vandroiy]

I wonder why everybody assumes the hacker is outside Linode.

Isn't the most likely person to know of such security issues someone within the company? I didn't even know Bitcoinica was hosted there. Also, it reeks of sloppy admin password policy:

compromised credentials used by this intruder (quote directly from Linode!)

IMO, Linode is responsible, either by using the typical ridiculous internal security, or directly (admin, higher-up person, etc.). Anyone serious about their reputation would pay back what they likely took.

Also, their press release is a joke. "Only eight accounts were compromised," no mention that it happened to be exactly the accounts the thief needed.

[1714157853]

1714157853

[1714157853]

1714157853

[DarkEmi]

I think it is more fair to assume that could be an action from a single employee than the whole compagny.

That's usually what happens.

[Jered Kenna (TradeHill)]

I think it is more fair to assume that could be an action from a single employee than the whole compagny.

That's usually what happens.

I really doubt a company like Linode would even consider risking anything for this amount of Bitcoin. If it was internal then it would most likely be a single employee. That is a possibility and if it was internal the chance of the individual being caught goes up considerably.

Jered

[Vandroiy]

I think it is more fair to assume that could be an action from a single employee than the whole compagny.

That's usually what happens.

Sure, the company couldn't spend the BTC anyway. However, the company can spend the savings from less effort on security... and there's no way to tell what position the thief might have inside the company. The problem is: a customer should not have to care about the internal structure of a company he trades with. Especially on a free market, there needs to be someone responsible.

Linode is clearly responsible here. I would demand that they either prove that their security handling was appropriate, e.g. it was a hard-to-find flaw or a flaw in a system usually expected safe, or else pay the full amount. We can't have a market in which people can just write a post containing "lol I got hacked" and have other peoples' money disappear. That press release is insufficient, more of an insult if you ask me.

Just as a disclaimer, I'm not involved in this, just an external observer who finds the behavior absurd.

If I entrust A with X, and X ends up with some random person somewhere else, I demand a very good explanation from A. I advise that everyone demand a very good explanation from Linode.

[DarkEmi]

Yeah I agree with you here.

[Matthew N. Wright]

I wonder why everybody assumes the hacker is outside Linode.

Hi. For your information, not everyone does. We're looking into the matter to get to the bottom of that little detail as well.

Isn't the most likely person to know of such security issues someone within the company?

They were a victim of their own lack of security. Unfortunately, it's the same bullshit that MtGox, the polish exchange and MyBitcoin put out when "someone used credentials" to hack. It's the perfect excuse afterall. "Oops! I was hacked! I also found a sports car in the garbage yesterday!"

I didn't even know Bitcoinica was hosted there.

Nothing but the payment functions need to have been hosted there. Zhou made a grave mistake by not collocating as he was advised to do by DCAO representatives when he first joined. He held the belief that there was bigger chance of outside security threats or single collocation operator trust issues than with major companies. He still hasn't necessarily been proven wrong for that, because we don't know if Linode was behind it, but we definitely know now that that logic is flawed because it assumes that only employees and not hackers with employee credentials could be the culprits. At $200,000USD, that might be the most expensive lesson of 2012 for Bitcoin related services.

Also, it reeks of sloppy admin password policy:

compromised credentials used by this intruder (quote directly from Linode!)

It was never said outright who the culprit was, they were careful on that point. They never said it wasn't someone who worked there, and we haven't ruled it out. It's being looked into seriously and I am pushing for a multiple party suit against them for damages.

For all I can tell, the most likely scenario is that Linode stole this money, either by using the typical ridiculous internal security, or directly. Anyone serious about their reputation would pay back what they took.

Linode themselves doing this is not likely, you're wrong. What is likely is that a single underpaid employee did this. But they are claiming it was just someone with knowledge of their management panels for employees only. Possibly an ex-employee. Linode themselves should not be bashed. You don't want to put them on the defense when you need their cooperation legally and might get a settlement from them through insurance.

Also, their press release is a joke. "Only eight accounts were compromised," no mention that it happened to be exactly the accounts the thief needed.

What a dumb statement. How would they know what the "thief needed". And if they were to publicly state that "a large amount of bitcoins were taken worth hundreds of thousands of dollars", do you know how easily that would seal the case against them in court for damages? Let the grownups do their job.

[Vandroiy]

For all I can tell, the most likely scenario is that Linode stole this money, either by using the typical ridiculous internal security, or directly. Anyone serious about their reputation would pay back what they took.

Linode themselves doing this is not likely, you're wrong. What is likely is that a single underpaid employee did this. But they are claiming it was just someone with knowledge of their management panels for employees only. Possibly an ex-employee. Linode themselves should not be bashed. You don't want to put them on the defense when you need their cooperation legally and might get a settlement from them through insurance.

I edited this; of course Linode doesn't steal *as a company*, I didn't mean it that way.

Also, their press release is a joke. "Only eight accounts were compromised," no mention that it happened to be exactly the accounts the thief needed.

What a dumb statement. How would they know what the "thief needed". And if they were to publicly state that "a large amount of bitcoins were taken worth hundreds of thousands of dollars", do you know how easily that would seal the case against them in court for damages? Let the grownups do their job.

The hell? Yes, I expect them to say exactly that. So the "grownups" are to not speak the truth these days? Hide it between dodgy wording? I prefer not to be classified with them in this case, thank you.

Yes, I expect people to tell the outright truth, or else I expect they have facts to hide or twist. And even though profit seems to be the measure of everything these days, I don't see how going with that flow is grown up at all. Keh. Court games, involving simply not saying the most important fact. Makes me sick.

[N12]

Yes, I expect people to tell the outright truth, or else I expect they have facts to hide or twist. And even though profit seems to be the measure of everything these days, I don't see how going with that flow is grown up at all. Keh. Court games, involving not saying the most important fact, such that they don't have to pay what they fucked up. Makes me sick.

Free market, man.

Free market.

[Matthew N. Wright]

I edited this; of course Linode doesn't steal *as a company*, I didn't mean it that way.

Oh alright then. ^^

The hell? Yes, I expect them to say exactly that. So the "grownups" are to not speak the truth these days? I prefer not to be classified with them in this case, thank you.

I was more speaking as devil's advocate for their side. Obviously it's better for us if they said that, I am just asking you-- did you really expect them to? I mean, as a business with lawyers, a stake in the financial future of their employers and assets, etc? You've never held a management position have you? Did you know merely trying to help someone in a car crash and having them die can get you sued for causing their death for holding them wrong or making a mistake? Did you know that if someone breaks into your house and gets wounded by your careless arrangement of knives or something they can sue you for that in some cases? We're not talking about sanity here, we're talking about law. They did the right thing legally to protect themselves by saying that. You want to argue morals, start a thread about morals and dishonest business. We're talking about getting Zhou's money back here. Morals will not be a factor.

Yes, I expect people to tell the outright truth, or else I expect they have facts to hide or twist.

How about they have investors to protect, so that even if they have details that they know for a fact will not help anyone find the crooks, they're rather not go bankrupt for their errors? What if you're in a country where sex is punishable by death and you have sex with your girlfriend in private. Are you going to publicly announce it the next day? What if someone asks? Morality. Ho hum.

And even though profit seems to be the measure of everything these days, I don't see how going with that flow is grown up at all.

No offense, but I question if you're even over the age of 18 yet. You don't sound like someone who's ever held a job either. I certainly am not impressed by Linode's security or their actions, nor am I particularly impressed by their response. I was the first one to respond to slush and Zhou tong (who are in our DCAO group) and recommend a multiple party lawsuit against them for damages. The thing anyone who has ever run a company would know though, is that it's not losing in itself, it's how you lose. If they gave Zhou back $500k just to keep quiet and not push issues publicly so that they could fix their problems and keep their business going, do you think that's bad when the alternative is bankrupting them in court fees, and Zhou gets nothing? What about insurance? What if Linode can only claim the insurance to pay Zhou back if they never publicly admit it was their fault or how much was stolen? Can you wrap your head around the idea that maybe, just possibly, less is more?

Keh. Court games, involving not saying the most important fact, such that they don't have to pay what they fucked up. Makes me sick.

No one has said they won't pay. And yes, court games can be very trying and bothersome, but part of the game is knowing how to play. People who say "Herp Derp just tell the truth and everything will be okay!" don't know how to play. Are you American? Linode is in America. In US law, "What you say can and will be used against you in a court of law". That includes the honest things you say. The less the judges know, the better for everyone.

If the court knows that Zhou was running Bitcoinica, what if they found a law that says because it was not registered in the US as a trading site by US rules, they will not process the case, wouldn't it have been better for them not to know that little bit of information about bitcoinica, and just mention it as a 'Linode customer'?

Welcome to the Dark Side.

[Jered Kenna (TradeHill)]

Keh. Court games, involving not saying the most important fact, such that they don't have to pay what they fucked up. Makes me sick.

No one has said they won't pay. And yes, court games can be very trying and bothersome, but part of the game is knowing how to play. People who say "Herp Derp just tell the truth and everything will be okay!" don't know how to play. Are you American? Linode is in America. In US law, "What you say can and will be used against you in a court of law". That includes the honest things you say. The less the judges know, the better for everyone.

If the court knows that Zhou was running Bitcoinica, what if they found a law that says because it was not registered in the US as a trading site by US rules, they will not process the case, wouldn't it have been better for them not to know that little bit of information about bitcoinica, and just mention it as a 'Linode customer'?

Welcome to the Dark Side.

Keeping your mouth shut is rarely a bad idea.

 I can think of far more times I wish I had remained silent than wish I had spoken up.
You can be completely in the right and lose for speaking the truth. That being said if everyone spoke the truth all the time then we wouldn't run in to these problems 

If Linode isn't 100% sure then I completely understand why they didn't give more details. This just happened and I'm sure they'll be forth coming as they investigate. They're a solid company and have plenty to lose by being shady or dishonest. I've really enjoyed working with them in the past and they've been more than helpful.

My only real hope is the media does't some how try to spin this as a Bitcoin failure.

-Jered

[marked]

...(who are in our DCAO group)...

DCAO?

marked

[Matthew N. Wright]

Keeping your mouth shut is rarely a bad idea.

Ironic that I'm an idea guy who always has his mouth open.  

I can think of far more times I wish I had remained silent than wish I had spoken up. You can be completely in the right and lose for speaking the truth.

This has happened to me quite a few times in the US court system where knowledge of how to twist laws is what decides your right more than a moral judge. Spirit of the law is seldom used outside of TV courtrooms, especially when $200k is at stake.

That being said if everyone spoke the truth all the time then we wouldn't run in to these problems  

lol

If Linode isn't 100% sure then I completely understand why they didn't give more details. This just happened and I'm sure they'll be forth coming as they investigate. They're a solid company and have plenty to lose by being shady or dishonest. I've really enjoyed working with them in the past and they've been more than helpful.

I think all the sane people here agree with you. That said, they most certainly do have insurance and if data stolen from their centers was due to a hacker who was able to gain entry through their own private employee-only gateways and not even remotely related to the security of their customer's applications, that is a pretty clear cut case for responsibility, even if the insurance doesn't cover it.

My only real hope is the media does't some how try to spin this as a Bitcoin failure.

Some members of the DCAO have already advised the Bitcoin Magazine to take the defensive on this one, but I think it's a non-issue. What'd be a better use of our expensive pages is outlining how Linode dropped the ball and finding out if it was an internal job (employee, not evil anti-bitcoin CEO).

P.S. Did you change your name? I always thought your name was "Jared". I also always thought Mark's last name was "Karples".

DCAO?

http://dcao.org

[wumpus]

I do not think it was Linode itself. As they control the systems that run the VMs they could have trivially copied the wallets without leaving any trace, leaving everyone to wonder how the private keys leaked.

However, the hackers had to change the root password to get in... If it was an insider it was a very dumb one, or someone with limited permissions working alone.

[Jered Kenna (TradeHill)]

My only real hope is the media does't some how try to spin this as a Bitcoin failure.

Some members of the DCAO have already advised the Bitcoin Magazine to take the defensive on this one, but I think it's a non-issue. What'd be a better use of our expensive pages is outlining how Linode dropped the ball and finding out if it was an internal job (employee, not evil anti-bitcoin CEO).

P.S. Did you change your name? I always thought your name was "Jared". I also always thought Mark's last name was "Karples".

I'm not going to jump to any conclusions on who did it yet but I'm up to discuss possible scenarios.

I'm not a security expert so maybe someone who is could speak up. A lone employee (or 2 working together etc) might not be "Linode dropping the ball". There is always a human element and it's always the hardest to protect against. If it turns out some employee went off and did this and Linode comes clean / takes responsibility / makes it right and it wasn't easily preventable then I'll maintain my respect for them. On the other hand if someone can point out how they "really screwed up" when the facts come out please do it.

Regarding my name: it's always been "Jered" and I've probably lost a lot of emails over the years because of it. I'll be sure to remind my parents when I go visit.

-Jered

[lonelyminer (Peter Šurda)]

It certainly sounds fishy. I have a linode box, it doesn't use bitcoin in any way, and it wasn't restarted, for example. Based on this information, what Linode published and what others reported, it looks like only those that were running bitcoind were restarted. So the attacker must have figured out how to request a password reset only for particular machines. How do they figure out which? They can do that, for example, based on the IP. To find out the IP is not difficult, since it shows up on the Bitcoin network. But knowing the IP does not automatically allow one to find out which server it corresponds to in a control panel. So either the mysterious "customer service portal" was compromised thoroughly (= security fiasco) or the attacker had inside knowledge (= also fiasco, but a different sort).

[Gabi]

I wonder why everybody assumes the hacker is outside Linode.

Isn't the most likely person to know of such security issues someone within the company? I didn't even know Bitcoinica was hosted there. Also, it reeks of sloppy admin password policy:

compromised credentials used by this intruder (quote directly from Linode!)

IMO, Linode is responsible, either by using the typical ridiculous internal security, or directly (admin, higher-up person, etc.). Anyone serious about their reputation would pay back what they likely took.

Also, their press release is a joke. "Only eight accounts were compromised," no mention that it happened to be exactly the accounts the thief needed.

It's the first thing i tought that morning when i read about all these hackings.

Someone noticed UNENCRYPTED wallet.dat, happily copy-pasted the privaye key and ta-dah, moved the BTC

And yup, it's probably one employer and not the whole company, of course the company is getting hurt by this, once more customers lose trust over cloud/VPS/thing-you-have-to-trust things

Funny thing, i tought about that months ago when i backuped my wallet.dat on varios email/skydrive etc services, and that's why i encrypted the file before uploading it.

And since BTC leave no traces, since there is no way to know who moved the btc... well, good luck for everything.

Of course Linode should repay the losses, after all they confirmed that something weird happened

[Vandroiy]

@Matthew:
So now, it's about... hiding facts to stay within an insurance policy? That's fraud! Yes, I understand, it's not about morals, just about getting the most money for one's own position/company/whatever. But if this is the only purpose, where is the difference to the thief? (At the girl example, I don't see why one can have an obligation to tell anyone about a private relationship. Unless you like slavery, and someone "entrusted" someone else with the girl... otherwise, I don't see the analogy.)

I'm not actually a kid. But maybe my ideology is stuck with that of an 18 year old, I dunno. I have never been so desperate for money that I'd trade it in.

@lonelyminer:
That's interesting... so, if I got this right, the password reset requires information an outside attacker should not have? I don't know how Linode handles administration, but that sounds quite important.

[Matthew N. Wright]

So now, it's about... hiding facts to stay within an insurance policy? That's fraud!

Yea well, I don't work for them so I don't really care what they do-- our immediate issue is Zhou getting his money back. Second issue is catching the thief. If Linode lies to their insurance company in order to survive the losses, what does anyone here care? I wouldn't recommend doing it, and I've never even filed insurance on so much as a car wreck before, always just eat the losses myself.

Yes, I understand, it's not about morals, just about getting the most money for one's own position/company/whatever. But if this is the only purpose, where is the difference to the thief?

You're talking too much about how Linode should be honest, and not enough about punishing the messenger. Do you know anything about laws in the US? Do you know that their bad employee reflects them 100%? If you like Linode and want them to succeed, would you want this to shut them down because of their choice to hire a thief?

You're not thinking this the whole way through because you're stuck in a moral haze. Be sensible.

Also, no I am not advocating that they get special treatment. I have no dog in the fight personally, I just telling you what I believe they are doing. I'm not saying it's right. If you look at my posts on this forum, you'll know that I play devil's advocate most of the time.

I'm not actually a kid. But maybe my ideology is stuck with that of an 18 year old, I dunno. I have never been so desperate for money that I'd trade it in.

It's not always about money. Sometimes it's just about staying in business. Your discounting an awful lot of things with your statements.

[farfiman]

From their terms of service:

"Therefore, subscriber agrees that Linode.com shall not be liable for any damages arising from such causes beyond the direct and exclusive control of Linode.com. Subscriber further acknowledges that Linode.com's liability for its own negligence may not in any event exceed an amount equivalent to charges payable by subscriber for services during the period damages occurred. In no event shall Linode.com be liable for any special or consequential damages, loss or injury. Linode.com is not responsible for any damages your business may suffer. Linode.com does not make implied or written warranties for any of our services. Linode.com denies any warranty or merchantability for a specific purpose. This includes loss of data resulting from delays, non-deliveries, wrong delivery, and any and all service interruptions caused by Linode.com."

I'm not a lawyer but this more or less says they aren't responsible for almost anything ?

[Matthew N. Wright]

From their terms of service:

"Therefore, subscriber agrees that Linode.com shall not be liable for any damages arising from such causes beyond the direct and exclusive control of Linode.com. Subscriber further acknowledges that Linode.com's liability for its own negligence may not in any event exceed an amount equivalent to charges payable by subscriber for services during the period damages occurred. In no event shall Linode.com be liable for any special or consequential damages, loss or injury. Linode.com is not responsible for any damages your business may suffer. Linode.com does not make implied or written warranties for any of our services. Linode.com denies any warranty or merchantability for a specific purpose. This includes loss of data resulting from delays, non-deliveries, wrong delivery, and any and all service interruptions caused by Linode.com."

I'm not a lawyer but this more or less says they aren't responsible for almost anything ?

Rule #1 of law: States and courtrooms decide damages, not silly internet contracts. Numerous times, big players like eBay and Paypal have had judges call their user contracts "ridiculous and verbose" and had cases lost because of it.

Rule #2 of law: states differ on what is actually allowed in a contract and what is not.

Rule #3 of law: If this contract was supposedly "air tight", then what do you think would happen if their employees openly admitted to having robbed the customer while working there? You think the law would not be able to prosecute them because of the contract? It doesn't mean anything.