Bitcoin Forum

I have violated my promise (of "not to post anything [about Bitcoinica]") yesterday, by posting this in the emergency announcement thread:


Undoubtedly, I felt upset about some confusing commenters. I objectively disagreed with Intersango guys' ways of doing things and I think if Bitcoinica is still under my control, some of our customers' immediate issues can be addressed in a more timely manner.

However, I want to express my sincere apology to the General Partners of Bitcoinica LP, because I should not have criticized them when I should bear part of the responsibility by not doing my best in securing the system. The direct cause of the issue is not important, we shouldn't argue about "if someone didn't do X this thing wouldn't have happened", instead, we should say more about "if I did X this thing could be prevented". In this case, I can express these statements:

- If I have firewalled the wallet server properly (like web production servers), this thing could be prevented.
- If I have spent enough time on the re-implementation of the bitcoin client, this thing could be prevented.
- If I have set up strict access policies, and proactively communicate with Rackspace to disable certain insecure features, this thing could be prevented.

Respect for teammates is extremely crucial to achieve productivity. Everyone's reputation has been damaged badly in this event, and we shouldn't criticize each other due to the differences in the way we work. Even though I have announced that I would leave the Bitcoin economy a few days ago, I'm still actively monitoring our customers' feelings and communicating with the General Partners about the progress.

I am also extremely grateful for the Limited Partner (an investment group) of Bitcoinica LP for exceeding their legal obligation to bear the full cost of both recent attacks. Without their active support, Bitcoinica couldn't have survived until today to serve our customers well.

In the end, I would like to request everyone who cares about the community to be objective about this matter. I am no longer legally associated with Bitcoinica and I had no control over the attacked system. However, other team members are working in their greatest ability to deliver a fair solution to everyone. I have the advantage in understanding our customers (because I'm more familiar everyone using Bitcoinica) so I keep contributing some ideas as well. Please appreciate their hard work and understand the difficulties in resolving a serious security attack. We have already assured you the full compensation.

Thank you everyone for showing your support, understanding and patience.

PS. You can claim your Bitcoinica account at https://claims.bitcoinica.com/ (https://claims.bitcoinica.com/) now.

Good to see this I was really confused what was going on with you and intersango  :)

There's the important part!

This is the second time you've suggested that the Bitcoin reference code is responsible for your robbery.   I inquired about this claim before and I don't believe I got a reply: https://bitcointalk.org/index.php?topic=81045.msg899922#msg899922  Luke-jr also expressed skepticism: https://bitcointalk.org/index.php?topic=81045.msg899911#msg899911

 I fail to see how any system which has private keys for online realtime 'hot wallet' usage could be defended against an attacker which has root access to the selfsame systems.   Even if you used a multisignature wallet and machines inside separate security domains an attacker with that level of access could simply impersonate the web application's legitimate withdraws.

That said— if there is some flaw or omission in the reference client which could make high value installations more secure all the developers would love to hear about it.

What I am reasonably confident of is that while you're quite possibly smarter and have more time on your hands than any one of the people developing the publicly available reference software, you're not smarter than all of them combined.  ... And a bug that sends 18kBTC into a black hole (as MTGOX's custom code did with a few thousand BTC) is no better than having code stolen.  

There are significant advantages in working with a larger user base to test out and harden code before putting it on mission critical systems, and those advantages almost certainly outweigh the many troubles and limitations in the reference client.   Moreover, many aspects of Bitcoin security require that you be a part of the majority clique— even if the majority is "wrong"—, if you can be moved onto a minority chain you can be robbed.   Because the significant super-majority of the network (users and miners) are using the reference client, its critical that any client be bug for bug compatible with the block rejection rules in the reference client or be at increased risk.  So it very much is in your own interest to invest resources in improving the publicly available software than reinventing the wheel.

So that means no forced liquidations.

Thanks for the idea.

This is what I wanted to do:

- Drop the Bitcoin official client and re-implement one.
- Store private keys in the database, AES encrypted with a master key (that is associated with the user).
- Store master key in the database, AES encrypted with another hash of the user password (such as the SHA512 hash in place of the BCrypt hash).

This will be effectively a segregated account for the user. Of course we need to solve some problems (like forget password and forced settlements) but this is the general idea.

I'm a web developer so I feel much more comfortable securing the database rather than the wallet.dat. I never trust direct filesystem operations.

The team has not confirmed the settlement price yet. But you can expect these arguments:

- I have unrealized profits and I should have them!
- I have unrealized loss and I should wait until I recover!

We have open interest of 100,000 BTC so the conflict of interest is huge. I will leave this for the team to decide but my general suggestion is to use the highest price as the settlement price for longs and lowest price as the settlement price for shorts. You can continue to hedge your position elsewhere (and get some one-time settlement free money from Bitcoinica). It'll be fair for Bitcoinica and the user.

Brilliant idea, now the hacker can get at all of Bitcoinica's funds.

The hacker wrote "EXPECT MASS LEAK" in transactions from the stolen money. We can only assume he has a copy of the database, so he has access to all the information needed to make a claim.

If "Bitcoinica Consultancy" is handling things now, why didn't they notify us of the claims page rather than zhoutong? The more I hear from zhoutong, and less from intersango or whatever they call themselves, the more I lose confidence in the new owner/operators.

i guess now would be the perfect time to use the signing feature in bitcoin-qt.

Almost.
Unfortunatly (for the hacker) he doesn't control user's email accounts, and he has no access to users ID documents.

Then everybody who's email account password was the same as their Bitcoinica password better change their password pretty damn quick.

No, we require email confirmations.

We use BCrypt with a pretty high difficulty number. So it will take a long time for the hacker to crack the passwords, possibly months for a moderately complex password.

This +1. To start with, we never saw any communication from them indicating that anything that Zhoutong said was in any way incorrect, or that there was need of an apology to begin with. Of course, that could have been communicated privately, but from what I have seen so far even Zhoutong himself is becoming frustrated with the obvious stonewalling that we are seeing from the Consultancy.

Indeed, stonewalling is the best description that I can imagine for this series of events; to the public, and to those that wish to fix the problems, as it appears - since access to even the domain name has been fraught with problems such as a poor DNS implementation, leading to those that don't even use the forum to be forced to come here and find out the problem, and wait here for a resolution.

And he ought only have hashes of passwords.

Link me to more detail on the "EXPECT MASS LEAK" message?

http://blockchain.info/address/1EMLwAwseowTkDtKnEHRKrwQvzi4HShxSX

This is an address some of the stolen money was sent to

Notice the transaction amounts:

1.01100101 BTC
2.01111 BTC
3.0111 BTC
4.01100101 BTC
5.01100011 BTC
6.011101 BTC
7.001 BTC
8.01101101 BTC
9.01100001 BTC
10.01110011 BTC
11.01110011 BTC
12.001 BTC
13.011011 BTC
14.01100101 BTC
15.01100001 BTC
16.01101011 BTC
17.001 BTC
18.01110011 BTC
19.01101111 BTC
20.01101111 BTC
21.0110111 BTC

The part after the decimal point is ascii binary, and it converts to: expect mass leak soon

Also, the address starts with 1EML

Expect Mass Leak

Converter:
http://www.roubaixinteractive.com/PlayGround/Binary_Conversion/Binary_To_Text.asp

(for amounts that don't have 8 decimal places you need to add in more 0's, the 116BTC transaction is irrelevant I think)

And after the 1EML part, it says wAweso
Looks like the beginning of "Awesome", not sure what the "w" is all about.

Its actually wAwseo, so its likely a coincidence.

Vanitygen has a little-known ability to use regex, which - if written carefully - could make the job easier. It could also be sped up by running on many GPUs.

http://en.wiktionary.org/wiki/wawe

Why is nobody on here talking about why the claims page isn't working. Everyone has bitched since the coins were stolen (including me) and Zhou briefly mentions "claims.bitcoinica.com" and nobody replies? Has anyone actually connected to the site and filed their claim?

Doesn't work for me. The site is not available.

Might be DNS is not yet propagated data.

Was working 20 minutes ago. What did you break, Bigmerve?

This works.  But the cert it not valid for the ip
https://173.45.224.244/

Yeah, I wouldn't use that link

I have filled out the claim page, verified my email address with them and now waiting for the next step?

It's a valid StartCom certificate that was just issued, but connecting via IP doesn't allow the browser to do various automated checks on it.

They have to verify each claim manually. Wait your turn.

Not an expert here, just speculating...

That is really bizarre.  Behavior like this speaks allot about the nature of the one behind it's premeditation.  Juvenile for one, and why those three words?  Designed to instill fear in the minds of those that find the message.  Allot went into them being embedded in the transaction.  More than likely a troll shunned by the community here.  If this heist had been done by some group within TPTB in-order to derail BitCoin progress, I doubt they would go to all this trouble, no this is the mind of a Jack The Ripper type personality, and my guess a solo individual.  

https://claims.bitcoinica.com does not work (tried chrome, ff and ie)

EDIT: direct ip does work...

Works for me... And is the same IP as in like above.

NOW it does work for me too

Not an expert of what, trolling?  ::)

criminal psych?

Someone tell me if I did this right. I filled out the form, submitted it and received an email. I clicked the link in the email and it brought me to a page summarizing what I had filled out. There was nothing on that page to click or enter.

Same here. Now we'll have to wait I guess.

A word of friendly advice, zhoutong: your life will be much easier and lower-stress if you leave this mess to Donald, Patrick and Amir to sort out.

I mean, we all appreciate everything you have done, and that you have been the most communicative representative of the service, but it's not your problem anymore.

So why not disable your forum login, block access to the forum in your computer's host file, back away from the keyboard, and focus on your studies and on enjoying the great lifestyle that Australia has to offer. It's not your problem anymore.

No, I was online for the entire duration of the hacking. I revoked the keys immediately. The withdrawal limit had already been reached due to normal withdrawals.

The terrible thing is, Rackspace refused to log the hacker out. They don't know how to do it.

Err, pull the plug for the whole dedicated server is that hard?

If I'm not mistaken, it was hosted on RS Cloud Servers (similar to AWS), and I assume that shutting it down would destroy valuable evidence that could remain in memory.

They pulled the plug (suspend the servers), but the hacker was still in session. Thus the hacker is able to re-create cloud servers using our backup images.

Later I questioned them "Does this mean that Rackspace Cloud shouldn't be trusted for anything financially serious?", they didn't give a response.

http://www.rackspace.co.uk/managed-hosting/solutions-for-business/type-of-business/finance/

Fixed that shit for you.

Was that a managed server? How much did you pay monthly/yearly to Rackspace for managing the server for you?

Um psy, do you know how a cloud works? Virtual machines. Suspend it, and it stops responding - same as physical hardware being put into sleep mode.
Even though this was done, the cracker had access to the RS admin console, which is something that they have hosted on their own infrastructure. Apparently they don't know how to invalidate a php session, and so the cracker was able to spin up a new VM instance and load a backup and away goes Mabel with all the data.

Apparently, Zhou Tong, which had access to the server "temp" folder holding the session data, didn't knew how to invalidate a php session also.
Are you trying to tell me that Rackspace still had root access to that server? Pretty slick, actually...
Or are you trying to say that the Virtual Machine is not suited to host such a website? Even more slick.

One way or the other, their fault, and by their I don't mean Rackspace ::)

No, what I am saying is that the admin console/portal is hosted by rackspace themselves, not bitcoinica. It is the page that allows them to provision new hardware, file support tickets, create backups, etc. Bitcoinica has no access to those servers for obvious reasons (other RS customers use the same portal).

blah blah blah... Stop kissing Zhou Tongs ass, dude. First it was php sessions, now you're telling me that Zhou Tong couldn't send a halt command to his instance? GTFO
Now tell me: How much money do you have hostage in Bitcoinica at this exact moment?

This link's doing nothing, I tried yesterday too.

The main question, why did they remain on VPS hosting after the linode VPS hack. They can buy or even rent a high powered dedicated server for peanuts nowadays.

Jesus Christ, I am not responsible for how badly Rackspace fails at server administration, I'm just telling you how the fucking setup WORKS. If you can't comprehend how it works, you have no right to be placing blame.

Now obviously, using cloud services in this manner was not a good idea, and there should have been some actual dedicated hardware in use, in a locked cage, "blah blah blah", but it's too late for that now.

How about the claim page start with asking me my user - password then ask for OTP ?

I had not used bitcoinica for ... 3 months ... I can't recall what positions or how much I had.

Hint : I'm not going to bother filling a page full of infos I don't know about, or I never gave them, other than my email. ... on a 173.45.224.244 that could be anything.

It seems you are the one not to understand how things work. Not even going to argue this with you. It's really not worth it lol

I still want Zhou Tong to tell me how much did he pay Rackspace for a FULLY managed server...
For people who understand 1 word is more than enough. You're not such a person, rjk...

Oh? You have your own Rackspace account, and you can log in and tell me how it works?
I do, but I am not going to waste any more of my time explaining things to a brick wall.

It's heartening to know that we've got Dayle Hinman (http://en.wikipedia.org/wiki/Dayle_Hinman) on the case!

From what I've observed, I have a different perspective. The Intersango guys were brought to help with security not PR. For them to take any position of public communications would have been a breach of contract. The fact that Zhou had to become a team player for his creation caused him a lot of frustration. He was the main PR man up till the incident and should have followed through with a splash page and daily email updates (not just the forum), but instead we got a "I'm leaving Bitcoin" thread. He left when the going got tough. Sure, feelings were hurt and emotions were high. Zhou, if you really want to be proud of what you started then get back to doing the PR and be a team player even if you don't agree. You should leave Bitcoinica on much better conditions if it something you really want to be proud of!

Regarding the Bitcoin Consultancy and questions about why they haven't been more active in this mess...I don't know what their arrangement with Bitcoinica is, but if they hadn't fully taken over the operation of Bitcoinica and had no responsibility for the security or theft, then they might be wise put their relationship on hold until Bitcoinica sorts everything out first.  If the Bitcoin Consultancy had nothing to do with the security issue there's no reason they should have to clean up someone else's mess.  At the minimum they would probably want to first arrange compensation for the time and effort that will be required for them to clean up the mess.

claim page doesnt work for me I click submit and nothing happens

One of the things I've admired most about Zhou Tong's work is that he seems to make good estimates of fair dispute resolution, then further errors on the customers side even when it costs him personally.  If the new owners to not share this mode of operation, being a 'team player' could make ZT a lot of things but I could certainly understand if 'proud' was not one of them.

We used cloud services and what rjk just described is right...

The "I'm leaving Bitcoin" has nothing to do with Bitcoinica hack. I'm still here, but I'm not doing other Bitcoin business any more.

I was the main operator before Bitcoinica joins forces with Intersango. After that, neither the investor nor I possesses full decision power. Intersango guys took over the management entirely. Even my position in PR was not fully recognized.

I did suggest some ideas internally, but I shouldn't have criticized them for different ways of doing things (even though I disagreed).

They are working very hard, but at the same time, I have nothing to update either.

It's very hard to judge whether they had anything do to with the security issue, because everything contributes to the disaster.

Patrick - compromised email server.
Me - improper access control.

Bitcoin Consultancy has fully taken over the management and the relationship is final. However, during the transition period, the access control is not defined properly and resulted in this problem. I have no knowledge of an insecure email server but I assigned admin rights to its user.

I think you need to make up your mind; are you going to stop posting about Bitcoinica, or are you going to keep telling us more and more.

Doesn't look like any more information than was already available.

Zouthong didn't say anything new recently. He just repeated again what is publickly known allready.

Unfortunately there seem to be a lot of folks hanging out here, which aren't able to read (but insist in spreading their guesses and opinions very loud.). This whole situation is also embarrasing for us, as a community. Bottom line is we're behaving as if we where a bunch of barely 17 year olds.

Oh, I see, well that's a bit different then.  I didn't have a chance to follow all the messages in these threads, but from the sound of it, someone inadvertently sent their hosting control panel password through an email server that was later compromised and gave someone access to the control panel?  I'll use this as a case in point in the future the next time someone dismisses the risk of sending sensitive information in the clear over email.  On a side note, it never ceases to amaze me at how companies (even financial ones) will send scanned forms full of sensitive information over email with no encryption and never give it a second thought.

I see. Thanks for the clarification.

Edit: Looks like the whole Bitcoinica thing is going through a lot of growing pains. Glad to see you are still helping out 'till they are running smoothly again.
If they truly make good on all lost coins and do their best to compensate everyone they will definitely earn all of my trust and respect back.

However, the PR during the initial days was a fiasco and is still not where it should be.
Still waiting for an email with all the news and a splash page with daily updates.
I shouldn't have to find it here in this forum.

Still more funny, try to convince a "conventional" finnancial institution you're working with to use something simple as PGP. You'll hit a wall of consultants not even knowing what encryption is, but communicating very "professionally" all day long....

+1

This part of the situation make me cry.  :'(

I am pretty sure Rackspace does not send passwords over emails - just the password reset link to the list of authorized emails on the account. They also use opportunistic TLS so if the recipient email server supports TLS the in-flight data will be encrypted.

However, in this particular case it didn't matter because it appears that one of the authorized email addresses was hosted on a compromised server.

I mentioned PGP once to a mortgage broker I was working with…they clearly had no idea what I was talking about, so I said never mind, I'll just drive over to the office…and they thought I was behind the times in that I couldn't handle it over email.   ::)

So we're still in this stage, aren't we.

It actually stands for "Expect Mass Leak when African warlords see excellent online way to keep dollars tucked, knowing not everyone has right key, reveal wallet quietly, vexed Zhou is 4cibly hushed, soon his extravagant system exposed."

LMAO! Now try this one: 1DkyBEKt5S2GDtv7aQw6rQepAvnsRyHoYM

Dope kilos?  You bet.  Everyone knows that.  5BTC sent, to golden dropbox travels value.  Seventually, anyone questions whether 6ilk Road quietly extracts payment after various national senates relent, yielding harmless opinion, yes marijuana.

haha. someone should write a script to translate these vanity acronyms.

It really makes me smile when people like Zhou take responsibility for their part and explain to the community on what happened so others will learn from this wisdom and bring good will back to this brand.   It shows class and I hope this type of mature behavior spreads in this community, make it the best on the planet.  I am proud to be a part of this and in my business venture coming online very soon, we are going to take the same level of communication and honest information.

Thank you.


Dalkore