Bitcoin Forum

I wonder why everybody assumes the hacker is outside Linode.

Isn't the most likely person to know of such security issues someone within the company? I didn't even know Bitcoinica was hosted there. Also, it reeks of sloppy admin password policy:


IMO, Linode is responsible, either by using the typical ridiculous internal security, or directly (admin, higher-up person, etc.). Anyone serious about their reputation would pay back what they likely took.

Also, their press release is a joke. "Only eight accounts were compromised," no mention that it happened to be exactly the accounts the thief needed.

I think it is more fair to assume that could be an action from a single employee than the whole compagny.

That's usually what happens.

I really doubt a company like Linode would even consider risking anything for this amount of Bitcoin. If it was internal then it would most likely be a single employee. That is a possibility and if it was internal the chance of the individual being caught goes up considerably.

Jered

Sure, the company couldn't spend the BTC anyway. However, the company can spend the savings from less effort on security... and there's no way to tell what position the thief might have inside the company. The problem is: a customer should not have to care about the internal structure of a company he trades with. Especially on a free market, there needs to be someone responsible.

Linode is clearly responsible here. I would demand that they either prove that their security handling was appropriate, e.g. it was a hard-to-find flaw or a flaw in a system usually expected safe, or else pay the full amount. We can't have a market in which people can just write a post containing "lol I got hacked" and have other peoples' money disappear. That press release is insufficient, more of an insult if you ask me.

Just as a disclaimer, I'm not involved in this, just an external observer who finds the behavior absurd.

If I entrust A with X, and X ends up with some random person somewhere else, I demand a very good explanation from A. I advise that everyone demand a very good explanation from Linode.

Yeah I agree with you here.

Hi. For your information, not everyone does. We're looking into the matter to get to the bottom of that little detail as well.


They were a victim of their own lack of security. Unfortunately, it's the same bullshit that MtGox, the polish exchange and MyBitcoin put out when "someone used credentials" to hack. It's the perfect excuse afterall. "Oops! I was hacked! I also found a sports car in the garbage yesterday!"

Nothing but the payment functions need to have been hosted there. Zhou made a grave mistake by not collocating as he was advised to do by DCAO representatives when he first joined. He held the belief that there was bigger chance of outside security threats or single collocation operator trust issues than with major companies. He still hasn't necessarily been proven wrong for that, because we don't know if Linode was behind it, but we definitely know now that that logic is flawed because it assumes that only employees and not hackers with employee credentials could be the culprits. At $200,000USD, that might be the most expensive lesson of 2012 for Bitcoin related services.

It was never said outright who the culprit was, they were careful on that point. They never said it wasn't someone who worked there, and we haven't ruled it out. It's being looked into seriously and I am pushing for a multiple party suit against them for damages.

Linode themselves doing this is not likely, you're wrong. What is likely is that a single underpaid employee did this. But they are claiming it was just someone with knowledge of their management panels for employees only. Possibly an ex-employee. Linode themselves should not be bashed. You don't want to put them on the defense when you need their cooperation legally and might get a settlement from them through insurance.


What a dumb statement. How would they know what the "thief needed". And if they were to publicly state that "a large amount of bitcoins were taken worth hundreds of thousands of dollars", do you know how easily that would seal the case against them in court for damages? Let the grownups do their job.

I edited this; of course Linode doesn't steal *as a company*, I didn't mean it that way.



The hell? Yes, I expect them to say exactly that. So the "grownups" are to not speak the truth these days? Hide it between dodgy wording? I prefer not to be classified with them in this case, thank you.

Yes, I expect people to tell the outright truth, or else I expect they have facts to hide or twist. And even though profit seems to be the measure of everything these days, I don't see how going with that flow is grown up at all. Keh. Court games, involving simply not saying the most important fact. Makes me sick.

Free market, man.

Free market. 8) 8) 8) 8)

Oh alright then. ^^

I was more speaking as devil's advocate for their side. Obviously it's better for us if they said that, I am just asking you-- did you really expect them to? I mean, as a business with lawyers, a stake in the financial future of their employers and assets, etc? You've never held a management position have you? Did you know merely trying to help someone in a car crash and having them die can get you sued for causing their death for holding them wrong or making a mistake? Did you know that if someone breaks into your house and gets wounded by your careless arrangement of knives or something they can sue you for that in some cases? We're not talking about sanity here, we're talking about law. They did the right thing legally to protect themselves by saying that. You want to argue morals, start a thread about morals and dishonest business. We're talking about getting Zhou's money back here. Morals will not be a factor.

How about they have investors to protect, so that even if they have details that they know for a fact will not help anyone find the crooks, they're rather not go bankrupt for their errors? What if you're in a country where sex is punishable by death and you have sex with your girlfriend in private. Are you going to publicly announce it the next day? What if someone asks? Morality. Ho hum.

No offense, but I question if you're even over the age of 18 yet. You don't sound like someone who's ever held a job either. I certainly am not impressed by Linode's security or their actions, nor am I particularly impressed by their response. I was the first one to respond to slush and Zhou tong (who are in our DCAO group) and recommend a multiple party lawsuit against them for damages. The thing anyone who has ever run a company would know though, is that it's not losing in itself, it's how you lose. If they gave Zhou back $500k just to keep quiet and not push issues publicly so that they could fix their problems and keep their business going, do you think that's bad when the alternative is bankrupting them in court fees, and Zhou gets nothing? What about insurance? What if Linode can only claim the insurance to pay Zhou back if they never publicly admit it was their fault or how much was stolen? Can you wrap your head around the idea that maybe, just possibly, less is more?

No one has said they won't pay. And yes, court games can be very trying and bothersome, but part of the game is knowing how to play. People who say "Herp Derp just tell the truth and everything will be okay!" don't know how to play. Are you American? Linode is in America. In US law, "What you say can and will be used against you in a court of law". That includes the honest things you say. The less the judges know, the better for everyone.

If the court knows that Zhou was running Bitcoinica, what if they found a law that says because it was not registered in the US as a trading site by US rules, they will not process the case, wouldn't it have been better for them not to know that little bit of information about bitcoinica, and just mention it as a 'Linode customer'?

Welcome to the Dark Side.

Keeping your mouth shut is rarely a bad idea.

 I can think of far more times I wish I had remained silent than wish I had spoken up.
You can be completely in the right and lose for speaking the truth. That being said if everyone spoke the truth all the time then we wouldn't run in to these problems  :D

If Linode isn't 100% sure then I completely understand why they didn't give more details. This just happened and I'm sure they'll be forth coming as they investigate. They're a solid company and have plenty to lose by being shady or dishonest. I've really enjoyed working with them in the past and they've been more than helpful.

My only real hope is the media does't some how try to spin this as a Bitcoin failure.

-Jered

DCAO?

marked

Ironic that I'm an idea guy who always has his mouth open.  :'(

This has happened to me quite a few times in the US court system where knowledge of how to twist laws is what decides your right more than a moral judge. Spirit of the law is seldom used outside of TV courtrooms, especially when $200k is at stake.

lol

I think all the sane people here agree with you. That said, they most certainly do have insurance and if data stolen from their centers was due to a hacker who was able to gain entry through their own private employee-only gateways and not even remotely related to the security of their customer's applications, that is a pretty clear cut case for responsibility, even if the insurance doesn't cover it.

Some members of the DCAO have already advised the Bitcoin Magazine to take the defensive on this one, but I think it's a non-issue. What'd be a better use of our expensive pages is outlining how Linode dropped the ball and finding out if it was an internal job (employee, not evil anti-bitcoin CEO).

P.S. Did you change your name? I always thought your name was "Jared". I also always thought Mark's last name was "Karples".


http://dcao.org

I do not think it was Linode itself. As they control the systems that run the VMs they could have trivially copied the wallets without leaving any trace, leaving everyone to wonder how the private keys leaked.

However, the hackers had to change the root password to get in... If it was an insider it was a very dumb one, or someone with limited permissions working alone.

I'm not going to jump to any conclusions on who did it yet but I'm up to discuss possible scenarios.

I'm not a security expert so maybe someone who is could speak up. A lone employee (or 2 working together etc) might not be "Linode dropping the ball". There is always a human element and it's always the hardest to protect against. If it turns out some employee went off and did this and Linode comes clean / takes responsibility / makes it right and it wasn't easily preventable then I'll maintain my respect for them. On the other hand if someone can point out how they "really screwed up" when the facts come out please do it.

Regarding my name: it's always been "Jered" and I've probably lost a lot of emails over the years because of it. I'll be sure to remind my parents when I go visit.

-Jered

It certainly sounds fishy. I have a linode box, it doesn't use bitcoin in any way, and it wasn't restarted, for example. Based on this information, what Linode published and what others reported, it looks like only those that were running bitcoind were restarted. So the attacker must have figured out how to request a password reset only for particular machines. How do they figure out which? They can do that, for example, based on the IP. To find out the IP is not difficult, since it shows up on the Bitcoin network. But knowing the IP does not automatically allow one to find out which server it corresponds to in a control panel. So either the mysterious "customer service portal" was compromised thoroughly (= security fiasco) or the attacker had inside knowledge (= also fiasco, but a different sort).

It's the first thing i tought that morning when i read about all these hackings.

Someone noticed UNENCRYPTED wallet.dat, happily copy-pasted the privaye key and ta-dah, moved the BTC

And yup, it's probably one employer and not the whole company, of course the company is getting hurt by this, once more customers lose trust over cloud/VPS/thing-you-have-to-trust things

Funny thing, i tought about that months ago when i backuped my wallet.dat on varios email/skydrive etc services, and that's why i encrypted the file before uploading it.

And since BTC leave no traces, since there is no way to know who moved the btc... well, good luck for everything.

Of course Linode should repay the losses, after all they confirmed that something weird happened

@Matthew:
So now, it's about... hiding facts to stay within an insurance policy? That's fraud! Yes, I understand, it's not about morals, just about getting the most money for one's own position/company/whatever. But if this is the only purpose, where is the difference to the thief? (At the girl example, I don't see why one can have an obligation to tell anyone about a private relationship. Unless you like slavery, and someone "entrusted" someone else with the girl... otherwise, I don't see the analogy.)

I'm not actually a kid. But maybe my ideology is stuck with that of an 18 year old, I dunno. I have never been so desperate for money that I'd trade it in.

@lonelyminer:
That's interesting... so, if I got this right, the password reset requires information an outside attacker should not have? I don't know how Linode handles administration, but that sounds quite important.

Yea well, I don't work for them so I don't really care what they do-- our immediate issue is Zhou getting his money back. Second issue is catching the thief. If Linode lies to their insurance company in order to survive the losses, what does anyone here care? I wouldn't recommend doing it, and I've never even filed insurance on so much as a car wreck before, always just eat the losses myself.

You're talking too much about how Linode should be honest, and not enough about punishing the messenger. Do you know anything about laws in the US? Do you know that their bad employee reflects them 100%? If you like Linode and want them to succeed, would you want this to shut them down because of their choice to hire a thief?

You're not thinking this the whole way through because you're stuck in a moral haze. Be sensible.

Also, no I am not advocating that they get special treatment. I have no dog in the fight personally, I just telling you what I believe they are doing. I'm not saying it's right. If you look at my posts on this forum, you'll know that I play devil's advocate most of the time.

It's not always about money. Sometimes it's just about staying in business. Your discounting an awful lot of things with your statements.

From their terms of service:

"Therefore, subscriber agrees that Linode.com shall not be liable for any damages arising from such causes beyond the direct and exclusive control of Linode.com. Subscriber further acknowledges that Linode.com's liability for its own negligence may not in any event exceed an amount equivalent to charges payable by subscriber for services during the period damages occurred. In no event shall Linode.com be liable for any special or consequential damages, loss or injury. Linode.com is not responsible for any damages your business may suffer. Linode.com does not make implied or written warranties for any of our services. Linode.com denies any warranty or merchantability for a specific purpose. This includes loss of data resulting from delays, non-deliveries, wrong delivery, and any and all service interruptions caused by Linode.com."

I'm not a lawyer but this more or less says they aren't responsible for almost anything ?

Rule #1 of law: States and courtrooms decide damages, not silly internet contracts. Numerous times, big players like eBay and Paypal have had judges call their user contracts "ridiculous and verbose" and had cases lost because of it.

Rule #2 of law: states differ on what is actually allowed in a contract and what is not.

Rule #3 of law: If this contract was supposedly "air tight", then what do you think would happen if their employees openly admitted to having robbed the customer while working there? You think the law would not be able to prosecute them because of the contract? It doesn't mean anything.

Well, it does not necessarily mean that they shouldn't have had the information they had. If control panel was crap, or the privileges of the compromised account were too high, this could have been sufficient. My point is that either way, incompetence or fraud, it's a major screwup.
EDIT
Let me try to explain again. The attackers had a lot of information. This wasn't a script kiddie, it was carefully designed and swiftly and accurately executed. Of course, this does not imply the assistance of Linode employees or contractors. But this only shifts the nature of Linodes failure, it does not really lessen the magnitude.

Thats why I put a question mark at the end .   You are probably right if this ever gets to court.
Bitcoin itself will be on trial. A court will have to decide what it is first before it can deliberate about the rest no? ( once more a question mark....)

First, people need to decide if it's worth suing the company for 200K combined total. Linode might have a very good lawyer and it will tie up the case for many month, if not years.

Second, I don't think linode is in the business of storing and protecting valuables. You can't get much from a 50 dollars a month web host.

That is in fact a concern. Some of us think of Bitcoin already as a digital commodity, but to have ANY court decisions related to values of property loss related to Bitcoin will be a dangerous territory to get into because it can set precedence for things we can't easily take back later imo.

No, I don't believe so. It will be treated as a digital commody, just like if someone hacked your account then stole facebook credits. I don't think they need to define it anything further than just "damaged incurred due to the illegal entry" etc. It might be pushed further than that but I doubt it. Disclaimer, I'm not a lawyer.


hehe. Don't worry about me. I am a dog. I chew bones.

This is the key thing we should take away from this. Real currency stored by banks is also digital currency but is heavily protected physically, digitally and legally. Given that Bitcoin doesn't have legal protection (they can't be seized), digital protection is very hard (private keys need to be available to sign a transaction) then the bare-bones level of protection you should have as a holder of many bitcoins is physical security at the server access level. Letting third-party admins have access to your server and having admin panels exposed over the Internet is incredibly foolish.

The only individuals responsible for this are the criminals themselves, be they Linode employees or not. Linode has no moral responsibility to refund the victims. However, if they do not they risk gaining a seriously bad reputation forcing others to look for a VPS that assumes responsibility for loss or theft of a client's assets. Like all things, competition will allow for a diversity in quality of services. Some VPS's will be insured against theft, others will not. Some will refund victims, others will not. If Linode chooses not to refund the victims, so be it. If you don't like that about Linode, find a VPS provider that will.

I'm sorry, are you a circuit court judge? Why don't you let people who understand the laws better make such statements?

I'm not talking about a government law. I'm talking about principle. You're talking to an anarchist, somebody who has zero respect for the government and its definitions of right and wrong.

Oh, okay. Please carry on then. ^_^

I would be very surprised if any hosting company (VPS or otherwise) will assume responsibility in the form of economical compensation for incidents like this. Every Terms and Conditions I have read from virtually every hosting company I have worked with is very clear about no compensation for damages of any kind, for any reason.

That has its limitations. You can't have employees working at your company willfully stealing things from customers and saying "woops! You agreed! haha".

There needs to be an investigation and it's something Zhou, Slush, and the proposed attorney will be discussing over the next few days.

If there is enough of a demand it will happen. Sure, the VPS premiums are likely going to be a lot higher, but it can happen. What we're really talking about here is insurance. If the VPS doesn't supply the optional insurance, then individuals will have to get it themselves. Take Tradehill. They could have gotten some insurance before this situation ever happened. They could have been insured up to, what was it, 45,000 BTC? In the case they got ripped off the insurance would have kicked in, and the insurance provider would have the most interest in catching the criminal and recovering the stolen bitcoins. Now, I'm sure no CORPORATE, "legal" insurance company exists like this right now. But there is definitely a need for one. Wasn't there a poll the other day asking people what is most necessary for Bitcoin? One of the poll answers was "insurance." I didn't participate in the poll, but my answer was "insurance" when I read it. Nobody else picked that one though. Perhaps it's time for a Bitcoin insurance company to pop up, preferably one that is totally underground, i.e. not sanctioned by the government at all.

IMHO, it would be tough to get the losses directly from the courts.  Settlement is much more likely.  Linode had revenue of $10 million in 2010 and looked to be growing rapidly.  Bad press (servers hacked, assets lost) can cost them dearly.  

Finally, someone who understand US legal precedings.

No one will insure a speculative asset like Bitcoin.  Can you imagine the liability if the price spiked to 100 just before a heist?

That would not be a problem if they charge in bitcoins.

Matthew, don't get mad for what I'm about to say. It's not an attack on you or your organization, it's just something that makes sense.

So, Zhoutong told you guys where he was hosting his hot wallet... Sorry to say, but that makes you(DCAO) suspects also. It's a lot easier to steal something if you know where it is exactly.
I would step out of that investigation if I was in your place.
It would be the almost perfect crime: You steal and then you "help" to try and catch the "thieves"...

I'm having trouble finding where I said he told anyone where he held his wallet....

We all knew where he was hosted (everyone in the community) though.


Also, I love being suspect.  :D

When you guys are ready for an interview, I'll start with the first time I ran away from home at 7.

Are you going to tell me that when he described the part about less trust issues with major companies he didn't mentioned the company he was using? c'mon... ::)

Uhh. Yes. That's exactly what I'm telling you.

We were having a discussion related to collocation vs cloud in regards to general security for bitcoin applications.

I believe you.

http://t1.gstatic.com/images?q=tbn:ANd9GcR5TEW6j6pvuOKhO8G7BKaoj_QnuQ130KzQOfFM3iGQqW7jUNb9rtLOsWx04g

none of my intruders have sued me as yet  ;)

but then none of my knives are ever carelessly arranged

though a lawyer did come round once to talk about that, he's top right

http://help.bitcoinica.com/kb/faq/how-secure-bitcoinica-is

2nd para, 2nd sentence: "... And more importantly, we don't even operate a Bitcoin wallet, which means that hackers have nothing to steal. ..."


marked

Priceless!

Wish I also had "nothing" to steal...   ::)

I'm not a security expert either but I've watched many an episode of Mission Impossible & seen the Ocean's 11, then for $2k Benjies sans armes, ni haine, ni violence (http://en.wikipedia.org/wiki/Albert_Spaggiari) & the prospect of much more (or less) if they hadn't already had an ongoing peak somehow & timed it for max balances which it doesn't sound like, the MI scenario starts to sound quite plausible - get a temp job on Linode's cleaning services Co team & install key-loggers or spy-cams, a telephone sanitizer dood with smarts, most likely though borrow an admin colleague's log in details or say that yours must have been pinched & for plausible deny-ability say that you found a USB stick on the ground in the company car park one morning & foolishly plugged it in to your admin company computer out of curiosity (it had gay porn on it so you dumped it) - I wonder if he/they got more or less than they'd hoped for, anyway here's hoping that they get a nice long term inside to write a memoir on their heist & it can all be good for the inevitable The Bitcoin Story film that already has plenty of juicy Ocean's 11 type plot material