[Emergency ANN] Bitcoinica site is taken offline for security investigation

[guruvan]

#1. Don't park more money than you can afford to lose in any one site - especially one ran by people who claim to want to 'regulate' bitcoin.

#2. Once they've goofed not once, but twice, on basic security measures - get the fuck away and don't turn back.

Luckily for me I had next to nothing on their site. I don't expect to get back what I did have there, and their reassurances don't really make me anymore confident. They wouldn't be the first scammers that promised to return funds.

More than twice.

Don't trust people that continually suggest that they're the experts, hold everyone to their standards, and repeated show that they fail on basic security measures. (again, why have I seen responses on their blog, and other places from competitors wondering why they're pen testing competitor sites? - this is pure arrogance at work  if it's really the case)

The best exchange? there's an old guard around here that thinks this, beyond any rationality. Repeated failure of basic measure. FFS, how many times do they fuck up before they're no "the best" anymore?

Calm down? lol. I calmed down when I wrote off the funds I had at bitcoinica. I have little expectation of seeing those anytime soon.

But it's still a valid question. And it still isn't answered with more than vague bullshit, so I figured maybe I'd write it BIGGER.

[muyuu]

The best exchange? there's an old guard around here that thinks this, beyond any rationality. Repeated failure of basic measure. FFS, how many times do they fuck up before they're no "the best" anymore?

Other than the mail leak, what has happened?

Bitcoinica != Intersango

[stochastic]

So yesterday, the bitcoinica.com domain was pointing to their blog, explaining that they will be closed for months, and will begin the return of funds soon. Today, it doesn't point to anything again. Am I missing something here? Why did they redirected the domain once again?

Maybe Bitscalper was brought in for some technical advise.

[muyuu]

So yesterday, the bitcoinica.com domain was pointing to their blog, explaining that they will be closed for months, and will begin the return of funds soon. Today, it doesn't point to anything again. Am I missing something here? Why did they redirected the domain once again?

Do you mean this?

MAY
15
Bitcoinica Hack Post Mortem
As promised we are providing further details surrounding the recent security attack on Bitcoinica.

The hacker was successful in exploiting a vulnerability in a critical email server. This gave the attacker access to an administrative email account which in turn allowed them to reset passwords with our hosting provider, Rackspace. From there, they were able to change root passwords, steal the private keys of our hosted bitcoin wallet, and compromise our online database.

In the past, Bitcoinica has been victim to the poor security practices of an irresponsible hosting provider.  In this case, the fault was entirely ours. Specifically, here's how things went wrong:

1) We had too many bitcoins in our online wallet.

In light of past experiences you might say this is inexcusable. You would be right. Our practice was to keep online balances to a minimal amount by periodic transfer to offline storage. However, this was a manual process and the online balance could grow quickly and unpredictably from user deposits. We should have had an automatic process or an alert system to prevent the online wallet from growing too large. Indeed, that was planned, but it didn't happen soon enough.

2) Email server security did not get the proper attention

Since administrative email accounts can receive password reset links from Rackspace, a breach of our administrative email server is as good as root access to web servers with Rackspace. Our administrative email server should have been protected with the same tight standards we apply to our other servers, and access to this email account should have remained more limited.

3) We did not retain needed expertise fast enough

As many of you know, Bitcoinica began as a small project by a solo founder. The advanced trading experience that Bitcoinica brought to the world would not have been possible without Zhou Tong's brave innovation. In light of rapid growth, it was prudent to bring in a larger team with diverse technical specialties, including security. This occurred officially last month when the Bitcoinica Consultancy team stepped in as managers and operators of the business. A transition period ensued. A new platform was conceived which would strengthen Bitcoinica in the long term but took focus away from the present system in the short term. The recent security breach was not beyond our team's skills to prevent. We know better. But we did not address relevant issues as quickly as was needed.

So, what are we going to do about it?

We are choosing to leave Bitcoinica offline until such time as a new platform can be built and tested with security best-practices built-in from scratch. We do not yet have a firm estimate for availability but it will most probably be measured in months.

We will set up a process in the short term for users to withdraw their funds. Further details will be provided once we determine the best approach.

We thank you in advance for your patience. And we humbly apologize for this incident.

Posted 17 hours ago by Bitcoinica

MAY
11
Bitcoinica Security Breach

It is with much regret that we write to inform our users of a recent security breach at Bitcoinica. At approximately 1:00pm GMT, our live production servers were compromised by an attacker and they used this access to deplete our online wallet of 18547 BTC.

We will learn more as we investigate, but would like to address early concerns.
We have suspended operations while we focus on our investigation.
The overwhelming majority of our bitcoin deposits were not stolen.
The thief stole from us not you. All withdrawal requests will be honored.
The database was most likely compromised.

The last point has important implications for the following:

PASSWORDS
Bitcoinica uses the most stringent best practices for password security.* Therefore, it is extremely unlikely that even full database access would give the attacker knowledge of your Bitcoinica password. It is always best not to reuse passwords among different online services and we recommend changing passwords if you have done this.

IDENTIFYING DOCUMENTS
All identifying documents for verified customers are stored on separate servers at a separate data center and separately encrypted. Even full access to website database would not give the attacker access to this data.

USER INFORMATION
Other user information that you've provided upon account creation is stored in the database. If the attacker has full access to the database, they would have access to this information. This would include your username, email and account history, but not information about your banking details outside of Bitcoinca. Users should be especially suspicious of any emails received to your Bitcoinica email address. It is always a best practice to never click an email link to login to any online service.

We're providing this notice primarily for the protection of our users.

We will have more to say soon about the circumstances surrounding this attack and what we will do to handle it.

- The Bitcoinica Team




* For the technically inclined, we salt and encrypt passwords with bcrypt.
Posted 4 days ago by Bitcoinica

You are redirected there from http://bitcoinica.blogspot.com   ( http://www.bitcoinica.com not https not -www )

[dizzy1]

Any timeframe on the supposed claim page yet?

[DeathAndTaxes]

2) Email server security did not get the proper attention
Since administrative email accounts can receive password reset links from Rackspace, a breach of our administrative email server is as good as root access to web servers with Rackspace. Our administrative email server should have been protected with the same tight standards we apply to our other servers, and access to this email account should have remained more limited.

Rant Mode On.  For the rant adverse scroll on past.

NO. NO. FUCKING NO.

There should be NO method to reset admin passwords from outside the system.  Period.  No exceptions, no ifs, no ands, no buts.

How many thefts will it take for this to sink in? 1 (well we already know that isn't true)? 2 (based on this "response" that seems unlikely)? 3? 10? 100?

If an admin loses access due to lost credentials then one of the remaining admins (who already has internal access) restores access in a secure out of band method.

1) Contact "locked out admin" via predetermined method (i.e. specific contact phone #)
2) Verify "locked out admin" via predetermined challenge & response.
3) Reset "locked out admin" password to one time password and require password change on next login.
4) While still in out of band communication verify "locked out admin" is able to login, change password, log out, and login again.
5) Note the loss of access in employee records.  If admin can't retain secure access reduce his access.

If somehow your admins are so incompetent that all simultaneously lose access to the server well then fire them, wipe the server, restore from backups, and deal with the PR fallout.  It would still be cheaper than handing $300K to an attacker.

NO EXTERNAL ACCESS TO SECRET DATA (PRIVATE KEYS).  PERIOD.

External access to admin accounts still leaves you vulnerable to multiple attack vectors:
a) Rackspace could make a mistake and give an attacker access
b) Rackspace could have a flaw in their console which allows attacker to gain access
c) Rackspace could have internal employee who compromises the system
d) The email could be captured in-route.
e) The email server likely has the same backdoor so compromising that server provides indirect access to the primary server.

How fraking hard is this concept:
1) YOUR OWN HARDWARE in a locked co-location cage.  A good provide should allow custom procedures to limit physical access.
3) No external admin/root access by third parties.  Period.  Access to the servers is granted internally by employees who already have access.
4) Use 2 factor authentication.  Humans are fallable.  Expect they will fail.  Using a second factor provides hardening when that failure inevitably happens.

If you fail those three it doesn't matter how "secure" your code is. You build Fort Knox on a foundation of sand.

I mean for fuck's sake guys. You just lost $300K in the span of a couple months. Go to a quiet place, sit down and meditate on that.  You just lost $300K to hackers who used the ability to externally reset admin credentials in two separate attacks.  Does it even seem logical that the "solution" to ensure it doesn't happen again is to CONTINUE TO ALLOW EXTERNAL RESETS OF ADMIN CREDENTIALS and "try harder"?  

[rjk]

I wonder if it is different with Rackspace's hosted cloud versus the managed services. I just logged in to take a look at our managed servers, and I can't find any kind of password reset option anywhere. And not only that, but there is indeed a challenge-response set up for when I have to call them for assistance.

And besides all that, you can assign "device guidelines" that must be followed when any team member of Rackspace has to do maintenance on the device. This could be things like contacting 2 admins before authorizing a root password reset, requiring phone confirmation before a reboot, and so forth.

[Clipse]

I wonder if it is different with Rackspace's hosted cloud versus the managed services. I just logged in to take a look at our managed servers, and I can't find any kind of password reset option anywhere. And not only that, but there is indeed a challenge-response set up for when I have to call them for assistance.

And besides all that, you can assign "device guidelines" that must be followed when any team member of Rackspace has to do maintenance on the device. This could be things like contacting 2 admins before authorizing a root password reset, requiring phone confirmation before a reboot, and so forth.

That and I dont believe for a second this whole second theft story. I can believe the linode incidents since it occured to multiple people but this second incident with bitcoinica smells way to much like a financial recovery.

Just look how easy others get away with just stating shit got stolen, quite an easy route to recover your actual lost profits and everyone knows no matter how many times this happens there magically allways remains clients.

[Scott J]

The amount of stupidity in this thread is shocking.

Any responsible business wouldn't just throw the website back up without taking the time to make sure it was secure.

They have suffered a massive loss and have committed to paying everyone back. This will obviously be complicated and is going to take time to set up securely. Imagine if they rushed into it and someone was able to somehow claim your money?

Calling them scammers is unfair, unhelpful and makes you look like an idiot.

[Bigpiggy01]

The amount of stupidity in this thread is shocking.

Any responsible business wouldn't just throw the website back up without taking the time to make sure it was secure.

They have suffered a massive loss and have committed to paying everyone back. This will obviously be complicated and is going to take time to set up securely. Imagine if they rushed into it and someone was able to somehow claim your money?

Calling them scammers is unfair, unhelpful and makes you look like an idiot.

+1

[mobodick]

The amount of stupidity in this thread is shocking.

Any responsible business wouldn't just throw the website back up without taking the time to make sure it was secure.

They have suffered a massive loss and have committed to paying everyone back. This will obviously be complicated and is going to take time to set up securely. Imagine if they rushed into it and someone was able to somehow claim your money?

Calling them scammers is unfair, unhelpful and makes you look like an idiot.

Noone is saying they should put up the OLD site in the OLD location.
It would take one change to DNS to make www.bitcoinica.com point to a completely different server which hosts some information.

The ammount of non-communication is what is shocking.
Every info goes through a forum which you have to happen to know about.
I haven't received any mail on the incident.

[Krakonos]

Noone is saying they should put up the OLD site in the OLD location.
It would take one change to DNS to make www.bitcoinica.com point to a completely different server which hosts some information.

As far as I know, the DNS were redirected (I think there is someone mentioning changed IP at the beginning of the thread). Unfortunately, nothing was on that IP..

The ammount of non-communication is what is shocking.
Every info goes through a forum which you have to happen to know about.
I haven't received any mail on the incident.

Well, even if I know about this forum, it was a real pain to read all those posts just to get few posts from zhoutong and later the Bitcoinica Consultancy. If I would now come and find out that bitcoinica is down, discover this thread, I would probably be really pissed off, because it means whole evening of reading meaningless posts like mine (sorry guys, I hope noone who's new will read it).

I'll stop bragging right now.

For those who wonder and don't want to read whole thread, there IS some announcement at http://www.bitcoinica.com/ (with the www, without https -- it matters).

[lonelyminer (Peter Šurda)]

I'll play a bit of advocatus diaboli:

Noone is saying they should put up the OLD site in the OLD location.
It would take one change to DNS to make www.bitcoinica.com point to a completely different server which hosts some information.

The new server, however, also needs to be secure, and it needs to be able to securely process data (claim requests, including documents). These requests contain sensitive information, which could be misused if it fell into the wrong hands.

The ammount of non-communication is what is shocking.
Every info goes through a forum which you have to happen to know about.
I haven't received any mail on the incident.

The same security issue involves contact details of customers. These might not be available in a secure, or even usable, form. Maybe they only have a database dump at hand, which might have been modified by the attacker. Maybe they do not have a secure method of sending bulk emails. Sending sensitive emails to random people from a list that might have been manipulated is not be a great idea. I had cases in the past where invoices were sent to the incorrect addressee, and they threatened with legal action due to alleged "privacy violations", even though the data on the invoice is publicly known and they actually knew each other (it's akin to threatening when I tell your mum what your name and address is).

[mobodick]

I'll play a bit of advocatus diaboli:

Noone is saying they should put up the OLD site in the OLD location.
It would take one change to DNS to make www.bitcoinica.com point to a completely different server which hosts some information.

The new server, however, also needs to be secure, and it needs to be able to securely process data (claim requests, including documents). These requests contain sensitive information, which could be misused if it fell into the wrong hands.

That is certainly true.
However, information could have been distributed through such a site in the meantime.

The ammount of non-communication is what is shocking.
Every info goes through a forum which you have to happen to know about.
I haven't received any mail on the incident.

The same security issue involves contact details of customers. These might not be available in a secure, or even usable, form. Maybe they only have a database dump at hand, which might have been modified by the attacker. Maybe they do not have a secure method of sending bulk emails. Sending sensitive emails to random people from a list that might have been manipulated is not be a great idea. I had cases in the past where invoices were sent to the incorrect addressee, and they threatened with legal action due to alleged "privacy violations", even though the data on the invoice is publicly known and they actually knew each other (it's akin to threatening when I tell your mum what your name and address is).

Not having a copy of your userbase seems like a pretty big NONO...
Very unprofessional if true.
But in any case, such things should have been communicated if they form a problem.
That is why a new site on a different location with information
But afaik www.bitcoinica.com is pointing at some address on the google network with a non-existent webpage.

[casascius]

Seems to me if the passwords are properly hashed, and there hasn't been a plaintext compromise of them somehow, knowledge of the password will be the most surefire way to prove ownership of the account.

[disclaimer201]

There were many starfish posts on the forum in the days before the hack. Maybe they really were in need of funds? Given this fact and the fact that only Bitcoinica was targetet, this is more than enough for an initial suspicion, to say it in legal terms. Of course, the PR - or lack thereof - doesn't help either.

[Phinnaeus Gage]

There were many starfish posts on the forum in the days before the hack. Maybe they really were in need of funds? Given this fact and the fact that only Bitcoinica was targetet, this is more than enough for an initial suspicion, to say it in legal terms. Of course, the PR - or lack thereof - doesn't help either.

What do you mean by "starfish posts"? (serious question)

~Bruno~

[disclaimer201]

There were many starfish posts on the forum in the days before the hack. Maybe they really were in need of funds? Given this fact and the fact that only Bitcoinica was targetet, this is more than enough for an initial suspicion, to say it in legal terms. Of course, the PR - or lack thereof - doesn't help either.

What do you mean by "starfish posts"? (serious question)

~Bruno~

https://bitcointalk.org/index.php?topic=79949.0

[tvbcof]

...
Kids, here is how you secure a financial server:
...

Here's another couple of ideas:

1) Don't follow anyone's cookie-cutter solutions...particularly if they are good ones (because if they are, they will be popular.)

2) If you are serious about security, don't advertise your genius ideas.

[tvbcof]

2) If you are serious about security, don't advertise your genius ideas.

On the contraire, if your solution is proven and tested, you shouldn't be afraid of sharing it with the public. "Security by obscurity" is just laughable, and the source of problems like the one Bitcoinica is going through perhaps. Let's bet that nobody knows where our server is, or how we secure it. Let's just bet!

If you don't have a solution which obscures necessary weaknesses behind solutions which provide some degree of opacity, you probably have a somewhat primitive solution.  Of course you lose this advantage if you blather on about how you've done things.  Just sayin...