[Emergency ANN] Bitcoinica site is taken offline for security investigation

[zhoutong]

Site note:

You can safely delete Bitcoinica from your Google Authenticator app.

The GA keys were stored in plain text, so we can't use it as an authentication method. In OTP authentication, we have to use the same key that was added to your smartphone to verify your OTP.

[Coinoisseur]

Even if they re-acquired the database, it's worthless for purposes of payouts, the trust chain is broken. Unless they kept up to date hash information on the database information they could verify records with, extremely unlikely.

IMO, they should have had an EULA that said "this site is for entertainment purposes only, no losses will be covered". This is also why non-financial institutions like to sell you a point system such as MS Points, Riot Points, whatever Sony's is called instead of keep a currency account open. Because buried in the terms for their points is the legal equivalent of "no guarantees".

@zhoutong
So the accounting stuff says I had some past balance or deposit and the amount of funds I claimed was in the ballpark of this. What now? My fake claim still goes through.

Let me make myself very clear here.

There is but ONE solution where Bitcoinica does not necessarily go bust.

You have to retrieve the database!

Do it either by posting a bounty or by catching the hacker.

Every passing day without database increases the likelihood for it to be entirely forged once it is released!

Blitz out.

[rjk]

So how about starting a wiki on the framework for recoverable and resilient systems? For instance, binlogging to an encrypted disk on a server in a different datacenter, because that would fix issues with deleted databases. And HSM devices or some equivalent for storing private keys. And daily and hourly backups to systems outside of the core network. Encrypted of course.
What other ideas do people have?

[Coinoisseur]

Reason there isn't a consolidated resource is because people and companies make a lot of money in the computer security sector. You can find a lot of good information piecemeal but the "how" of putting it all together has extra value from the hoarding of that knowledge.

So how about starting a wiki on the framework for recoverable and resilient systems? For instance, binlogging to an encrypted disk on a server in a different datacenter, because that would fix issues with deleted databases. And HSM devices or some equivalent for storing private keys. And daily and hourly backups to systems outside of the core network. Encrypted of course.
What other ideas do people have?

[LightRider]

If there is a lesson to be learned here, it is that you can't make the world a better place in the context of a monetary system. The only wealth we have is our relationships with our fellow human beings. Any technological advancement that is outside the context of making such relationships healthier works to degrade that wealth. Zhou is making the right call in recognizing that he cannot pursue his desires to make society better while working in a sick game of token exchange.

[rjk]

Reason there isn't a consolidated resource is because people and companies make a lot of money in the computer security sector. You can find a lot of good information piecemeal but the "how" of putting it all together has extra value from the hoarding of that knowledge.

So how about starting a wiki on the framework for recoverable and resilient systems? For instance, binlogging to an encrypted disk on a server in a different datacenter, because that would fix issues with deleted databases. And HSM devices or some equivalent for storing private keys. And daily and hourly backups to systems outside of the core network. Encrypted of course.
What other ideas do people have?

Very true. And, the "how" often changes so rapidly that maintaining a comprehensive resource on it is not workable. What I want to see is a list of minimum standards that should be expected from businesses dealing with our money. Simply a framework of technologies that you have to stir together in the right proportions, and a list of potential consequences of omitting one or more of them.

[RandyMarsh]

We have all kinds of accounting reports. They are outdated and incomplete to restore trading, but far current enough to identify fake claims.

My last deposit was made about an hour before this happened... Does this mean the info I can provide about that deposit is useless in making my claim?

And its the last deposit specifically the claim form asks for...

[Coinoisseur]

You are describing financial regulations, for that you should deal with financially regulated companies. I personally have no wish to see BTC taken over by any government's financial framework. It's a tough problem, imo this is a great opportunity to see if open sourcing security completely would actually be effective. If the whole process is completely transparent then people can decide for themselves whether to use a BTC service.

Reason there isn't a consolidated resource is because people and companies make a lot of money in the computer security sector. You can find a lot of good information piecemeal but the "how" of putting it all together has extra value from the hoarding of that knowledge.

So how about starting a wiki on the framework for recoverable and resilient systems? For instance, binlogging to an encrypted disk on a server in a different datacenter, because that would fix issues with deleted databases. And HSM devices or some equivalent for storing private keys. And daily and hourly backups to systems outside of the core network. Encrypted of course.
What other ideas do people have?

Very true. And, the "how" often changes so rapidly that maintaining a comprehensive resource on it is not workable. What I want to see is a list of minimum standards that should be expected from businesses dealing with our money. Simply a framework of technologies that you have to stir together in the right proportions, and a list of potential consequences of omitting one or more of them.

[rjk]

So how about starting a wiki on the framework for recoverable and resilient systems? For instance, binlogging to an encrypted disk on a server in a different datacenter, because that would fix issues with deleted databases. And HSM devices or some equivalent for storing private keys. And daily and hourly backups to systems outside of the core network. Encrypted of course.
What other ideas do people have?

Reason there isn't a consolidated resource is because people and companies make a lot of money in the computer security sector. You can find a lot of good information piecemeal but the "how" of putting it all together has extra value from the hoarding of that knowledge.

Very true. And, the "how" often changes so rapidly that maintaining a comprehensive resource on it is not workable. What I want to see is a list of minimum standards that should be expected from businesses dealing with our money. Simply a framework of technologies that you have to stir together in the right proportions, and a list of potential consequences of omitting one or more of them.

You are describing financial regulations, for that you should deal with financially regulated companies. I personally have no wish to see BTC taken over by any government's financial framework. It's a tough problem, imo this is a great opportunity to see if open sourcing security completely would actually be effective. If the whole process is completely transparent then people can decide for themselves whether to use a BTC service.

Not even government based, but just a wiki somewhere.

Many of the FIPS-140-2 guidelines are extremely applicable though.

[JusticeForYou]

Well I've written ISO ISM manuals in the past.

So why not a wiki?

Start ISO 31337 and a Bitcoin ISM

I'm sure crowd sourced contributions that are voted on would come up with something pretty reasonable.

[BadBitcoin (James Sutton)]

Site note:

You can safely delete Bitcoinica from your Google Authenticator app.

The GA keys were stored in plain text, so we can't use it as an authentication method. In OTP authentication, we have to use the same key that was added to your smartphone to verify your OTP.

'

zhou, is there any way to get updated information on the progression of these claims without having to sift through this forum thread every few hours?

[Otoh]

YOU BETTER SOMEHOW RETRIEVE THE DB FROM THE HACKER OR THIS IS GOING TO END UP IN DISASTER!

It's not a disaster already?

I mean insolvency style disaster. 18.5k BTC is nothing.

What are they going to do if (and it is not really "if" – when it comes to money, people will do it if they can get away with it) there are a few hundred cunning people like me who thought the same on 20th of May? Read my IRC log.

What are they going to do once the deposit claims total 500k BTC, most of them being from 100+ points users?

There are only so many funds, and the fight over them is ON.

Better find a way to retrieve the database before they go broke, no?

We have all kinds of accounting reports. They are outdated and incomplete to restore trading, but far current enough to identify fake claims.

Tihan is a careful person and he keeps all the logs when he runs his stats program.

Submitting false claims will only result in delay of fund returns. The point system I proposed is used to save time and effort for people with accurate records. There isn't any reason to verify people with verified Bitcoinica account and only 1 BTC balance (and our record proves so). We should just refund the customer in full immediately.

The claim process involves subjective judgements, and that's why I request for take-over. I'm obviously more familiar with the user base and I should provide as much assistance as I can.

I have requested again for takeover in the Skype group and provided my working plan. If they approve within 8 hours from now I'll probably start refunding customers by early next week and finish by the end of next week.

zhoutong
VIP
Hero Member
******
Founder, CEO
& hopes to be the
Official Receiver
of Bitcoinica  

Bitcoinica Memorial Day anyone, when we remember those brave & contagious souls who risked their all in a dodgy margin trading scheme, we recall those funds that were for ever lost, mislaid or indefinitely detained without due process, we vow never to repeat such a madness ever again, or at least not until Kronos.io opens for business next week

[rdponticelli]

Can anybody involved setup a communication thread where we can have some information without so much noise?

Site note:

You can safely delete Bitcoinica from your Google Authenticator app.

The GA keys were stored in plain text, so we can't use it as an authentication method. In OTP authentication, we have to use the same key that was added to your smartphone to verify your OTP.

'

zhou, is there any way to get updated information on the progression of these claims without having to sift through this forum thread every few hours?

Please?

This thread has some fun, I can't deny it. Is nice to read it sometimes. But it would be better if you start something like Bitcoinica Claim Process, or something alike, more focused on updates for those trying to know how everything's going, don't you think?

[hazek]

If there is a lesson to be learned here, it is that you can't make the world a better place in the context of a monetary system. The only wealth we have is our relationships with our fellow human beings. Any technological advancement that is outside the context of making such relationships healthier works to degrade that wealth. Zhou is making the right call in recognizing that he cannot pursue his desires to make society better while working in a sick game of token exchange.

looool Nothing like a Zeitgeister to top this thread off. 

[repentance]

Please?

This thread has some fun, I can't deny it. Is nice to read it sometimes. But it would be better if you start something like Bitcoinica Claim Process, or something alike, more focused on updates for those trying to know how everything's going, don't you think?

It's been explicitly stated several times that Bitcoinica Consultancy alone is handling the claims process, so perhaps the request for a dedicated thread needs to be made of them - even though they seem totally unable to communicate in a timely and comprehensible manner.  It would be valuable if they listed specific times when people can expect updates on the process.

I also notice that the question of whether this intrusion has been reported to law enforcement remains unanswered.  There is no reason whatsoever for a legitimate enterprise not reporting the theft of its database, regardless of the contents of that database.  In the past, there have been investigations into and charges laid over the theft of in-game items in virtual worlds - it's not necessary to define Bitcoin as a currency or a commodity in order to determine both that it has value and ownership.  That the operators of Bitcoinica are willing to reimburse customer losses doesn't mean that the theft shouldn't be formally investigated.  

[Mageant]

No database backups. Sorry for avoiding the question.

OMG.

The first rule of computer using is that you *always* make backups. You backup early and you backup often, on-site and off-site.

I learned that the hard way in the early years of my 30-year computer programming career. If you don't do this then eventually you can get a *really big problem* like Bitcoinica has now.

[repentance]

No database backups. Sorry for avoiding the question.

OMG.

The first rule of computer using is that you *always* make backups. You backup early and you backup often, on-site and off-site.

I learned that the hard way in the early years of my 30-year computer programming career. If you don't do this then eventually you can get a *really big problem* like Bitcoinica has now.

It's still extremely bizarre that Rackspace had no way to log the hacker out and that he was still able to delete the emergency backup in spite of the servers supposedly being suspended.  That's a huge security flaw for a hosting service to have and you do have to wonder whether the hacker was aware of that "hidden feature".  Whatever mistakes were made by Bitcoinica were certainly compounded by the inability of Rackspace to totally lock down the compromised servers.

Zhou, I notice that you are focusing primarily on what is technically possible.  For a whole lot of reasons, the claims process must also have integrity from an accounting point of view.  The principals have little choice but to assume that the manner in which they process user claims may be the subject of legal action in the future and to ensure that the process complies with recognised business and accounting standards (in fact, the process should really be independently audited).  While your proposals have merit, they need to be considered in a broader business context and it would be foolish of the principals to implement them without first obtaining professional advice.

[Otoh]

No database backups. Sorry for avoiding the question.

OMG.

The first rule of computer using is that you *always* make backups. You backup early and you backup often, on-site and off-site.

I learned that the hard way in the early years of my 30-year computer programming career. If you don't do this then eventually you can get a *really big problem* like Bitcoinica has now.

It's still extremely bizarre that Rackspace had no way to log the hacker out and that he was still able to delete the emergency backup in spite of the servers supposedly being suspended.  That's a huge security flaw for a hosting service to have and you do have to wonder whether the hacker was aware of that "hidden feature".  Whatever mistakes were made by Bitcoinica were certainly compounded by the inability of Rackspace to totally lock down the compromised servers.

Zhou, I notice that you are focusing primarily on what is technically possible.  For a whole lot of reasons, the claims process must also have integrity from an accounting point of view.  The principals have little choice but to assume that the manner in which they process user claims may be the subject of legal action in the future and to ensure that the process complies with recognised business and accounting standards (in fact, the process should really be independently audited).  While your proposals have merit, they need to be considered in a broader business context and it would be foolish of the principals to implement them without first obtaining professional advice.

para 1 - The only person involved it seems with the slightest understanding of even kinder garden level of security was the hacker here

para 2 - Indeed, why didn't they just do as in the Bond films & pull the mains plug, internet cable, cut some wires etc

para 3 - So now 90% of sequestrated funds will go to lawyers, accountants, receivers, loss adjusters, etc & the process will take years, really just throw it to Zhou Tong to fix as best he can, it won't be perfect but at least he will try & do the right thing by everyone & fast then move on

[repentance]

para 3 - So now 90% of sequestrated funds will go to lawyers, accountants, receivers, loss adjusters, etc & the process will take years, really just throw it to Zhou Tong to fix as best he can, it won't be perfect but at least he will try & do the right thing by everyone & fast then move on

One of Tihan's partners in CoinLab already spoke publicly about how raising VC for their projects was made more difficult by the fact that they involved Bitcoin.  If Bitcoin businesses continue to be seen as entities which just do whatever they want when something goes wrong rather than following established business practice, they will continue to have problems attracting venture capital.

It should not take an accountant (and there's already one associated with Bitcoinica and it's FSP, even if he might not have anticipated having to actually do something in relation to the business beyond setting it up) very long to communicate to Bitcoinica Consultancy the essential elements which must be considered when processing and disbursing claims.  This is not a business whose financial practices are not subject to external scrutiny - they are a registered financial services provider and that means they can't just do whatever the hell they want in terms of financial activity and accounting practices.

Messes like this one happen in part because many Bitcoin enterprises start out as one man operations and when they expand appropriate professional standards are not applied to their operations - areas in which the founder has little expertise often get ignored until something goes wrong.  Bitcoina's technical security was inadequate.  It should not compound an already bad situation by implementing a claims process which is also inadequate and which has no independent oversight.

[stochastic]

It sounds like there were too many people in charge at Bitcoinica.  Zhou kinda operated it, there was some main owner, and now some Bitcoin Consultancy general partners.  I am always against forming partnerships for this reason.  It is like getting married.  Instead of giving part of the company to someone, just give part of the profits.

Make one person in charge and then delegate other responsibilities to everyone else.  Don't let everyone in the company make public statements about the company.