Bitcoin Forum

Every exchange and wallet services (or actually anyone running a service with lots of coins) should only keep only a fraction of their bitcoin reserves available for immediate withdrawal - just enough to provide day-to-day liquidity.

The bulk of coins should be stored in multiple encrypted offline wallets (with multiple backups and a nice big gap of air between these wallets and the internet). These offline coins would not be available for immediate withdrawal, but they would be safe!

Coins could be sent to the offline wallets any time, and coins could be returned to the pool of available coins manually as needed. The concept is analogous to a bank vault. Most of the cash at your bank is not sitting in the till in front of the teller, but is securely stored in a vault which can only be accessed at certain times under heavy security.

Perhaps you think this doesn't need to be said, but some services have already lost all their coins by not doing this!

I also think that users of any such site should also have the option of storing a fraction of their coins in such an offline wallet. You could even give each user their own "offline address" where they could securely store coins for the long term, and they could see the coins in block explorer. Bringing the coins back online would require extra verification steps and built-in delays. This is analogous to a safety deposit box, and you could even call it that.

I'm sending PMs to the exchange and wallet reps, asking them to comment on this thread.

Well, I guess it's obvious enough, but yes, we do already keep a large part of the bitcoins in offline wallets, and also have fake offline wallets stored in various places which I would not disclose (each one is encrypted too, of course, but let's avoid useless troubles).

The good thing is we can add bitcoins to those wallets without accessing them, we just keep a list of addresses (signed) and send only when needed.

It SHOULD be obvious I suppose, but from what I understand that Polish exchange (bitomat) lost everything! Some of that loss could have been avoided with a strategy like this.

Obviously this doesn't help if the bitcoin service is fraudulent (MyBitcoin?), but any bitcoin wallet service who wants some degree of legitimacy needs to be doing this.

MagicalTux: Any plans to allow individual users to move their coins offline as described above?

That would be a pain, both to explain to users, and to implement. Plus actually, we can say that about 98% of all funds deposited by users are put offline. We have a handful of very big depositors who know they cannot withdraw their full balance in one go as we don't keep that much online.

Anyway we always keep only 2% of the balance online, which means that even in the case something happens to the online coins we could still guarantee 98% of users funds (we intend to write our ToS with this in mind, however it's not easy to put together). Of course we are doing everything possible to avoid this to ever happen.

This seem to me like a good idea. Its just that it is no fractional reserve, its just having two accounts.

We can't keep any percentage of coins offline like that..   We can't allow for a bank run ..    

We'll have to think about this as it does make sense, but can be a nightmare to the end users if one or two of them have large numbers of coins and want to withdraw...  it could "lock up the whole system" for a while...  but at the same time I do see it's security benefits.  .  Thanks for the advice.

Hello all,
That is awesome that MtGox does the offline storage, and 98%...sweet. We do the same thing and exercise the ability to send to wallets offline. At dacoinminster's request, I will throw my 2 Bit's,
1. It's not actually fractional reserve, It is a delayed withdraw as a result of security measures.
2. Our exchange uses MtGox regularly, and we have different security concerns because the nature of our businesses is truly different.
3. We all see the future of BitcoinBanking involving a set withdraw time window for someone to withdraw from a otherwise offline wallet. Meaning we keep it offline, You request a withdraw, We have to make it available online and we will do so for X amount of time before we take it back down.
4. If any exchanges are really keeping all there "eggs" in one basket still... I wouldn't use those exchanges just as a statement (and obviously because I want my coins protected).
5. Our system actually requires to keep most funds in USD spread among a lot of accounts and in hand. I keep a couple hundred BTC in my wallet and MtGox etc.  just to avoid waiting for more than 1 confirmation when dealing with my regulars. But I do have The offline wallets on a Nice, tough, external HD. '
Our Banking system that is in the works basically has been planned around the safety deposit box theory you posted above. Easy to deposit to your offline wallets but you must schedule a withdraw time window, and has a little LR$ mixed in in that you can have a accessible wallet with just the funds needed always online mitigating your risk for hack.

Again I want to clarify that My Exchange isn't Person2Person and we are required to hold cash rather than Bitcoins, so these issues arent as relevant to my particular exchange.


To the post above, I/m pretty sure it would only "lock the system" for about 6 confirmations. So people wont freak out I dont think. Again it's not actually Fractional Reserve because they are keeping all the coins, It is just a delayed withdraw system/

DCM,
      Very good point, but for newer exchanges our wallets see a lot of activity day-to-day.  For MT.Gox 2% of the wallet is sufficient to cover daily activity, for us it is more like >30%.  
As we move forward, this will of course be a top priority.

In the meantime, to stay ahead of natural / man-made disasters and hackers we have following measures in place:

1) Secure data center designed to survive Cat 5 hurricanes
2) Connectivity to three telco backbones
3) Redundant power with 2 Caterpillar diesel generators to support the data center power
4) Physical security for servers
5) Well defined chain of command and separate ownership for Database and Wallet
6) Multiple, offsite backups
7) Nightly security audits
8) Daily D-Dos simulations
9) Scheduled White-hat penetration tests


Hope this helps,
     Keyur

Hey CampBX,
Are your servers on shore? or off shore?
Sounds like a nice facility, if it's offshore I would love to use their services.

Now that Mtgox has confessed to keep the coins offline (not necessarily a bad thing) what are you continuity of operations plans? What happens if you die or are jailed? How is someone going to restore those funds in the event you can't?

We have several strategies to ensure protection of our user's Bitcoin and other assets.  

- multiple encrypted offline wallets
- online wallets are backed up at least hourly
- backups are encrypted and on separate physical servers
- backups are stored in at least two other different physical locations
- backups are stored in different legal jurisdictions (countries)
- ACL and other internal security policies

Unfortunately I cannot provide this kind of information without creating useless risks for the stored coins. As we grow we will create more ways of ensuring continuity on which we will be able to communicate openly.

I have to agree with Magical Tux...  we have a process in place as well to provide continuance... but there is no way that it would make sense to publish them ... 

Instawallet uses offline storage as well (about 75 % of all funds are currently offline). Most people follow the "not a bank, just spare change" rule so it's not much to begin with, which is how it should be. Just to reiterate: Instawallet is a spare time project of mine, does not offer high security and is more a show-case platform to make Bitcoin more convenient. If there ever is a "bank run" on Instawallet, then it might take a day or two until I move things out of offline storage.

How possible is it to have each account holder's coins associated with a single, separate address/key? (I'm sure it'll be a PITA, but is it technically feasible?) This way, once someone opens an account, you can mail (encrypted file attachment, or mailed on paper) then the private key associated with their account, with instructions to make copies and back it up. Should anything happen to the system (crash, hack, government raid, zombie attack, whatever), users will still be able to manually import their private keys and get their coins off the block chain. Your businesses will still be able to continue operating and providing the same services they do now (easily-accessible secure banking, exchange, whatever).
Likewise, this will shift some of the liability onto the users, in case you are worried about insurance or legal issues.

This is a good jumping point,
but I was thinking what if they pull the coins early and blame us for whatever reason?

OP pointed me to this thread, but if you would check our sourcecode (which is online BTW), then you can see that we already do this,

(if on line 40)
https://gitorious.org/intersango/intersango/blobs/master/cron/verify_withdrawals_bitcoin.php

Keeping all your funds online is insane. Keeping your backups on the same server is absurd.

Do you mind disclosing what % of britcoin/intersango funds are offline?

Damn, good point. Will have to think about how to mitigate that...

We project withdrawals and it changes based on our projection. Usually around 90%

Today, the Bitcoin world is a lot like the initial web.  Anyone with a little skill can create an online service.  Since Bitcoin is so new, its very hard for the consumer to tell who has acumen to provide lasting, reliable service.

ExchB takes its responsibility to its customers seriously.  We look at the recent issue of securing wallet.dat files as just a facet of providing reliable service.  Our staff has over two decades of experience in providing Internet services which includes dealing with computer security, the design, implementation, execution, and support of systems with enterprise level reliability and scalability, and to cover the business side of things we have very experienced executive management.

We strive to show a pattern of excellent service:

Our terms of service are clearly stated.  Additionally, for customers looking for a US-based service, the jurisdiction for legal disputes is California.

ExchB provides a telephone contact number.

ExchB was the first Bitcoin exchange to not only cut off Dwolla deposits but also Dwolla withdrawals.  We are currently working on adding Paxum support, but we feel their current agreements are not yet to the level needed by Bitcoin exchanges.  Even after coming to an agreement with Paxum, we plan to exercise due caution with our customer funds held with them.

Yesterday ExchB added free two-factor authentication.  We deliberately chose a solution that has instant setup and does not involve a 3rd party service or a bunch of confusing options.  Looking forward, we are planning on further enhancing this service.

As for our wallet.dat, at ExchB, we do only keep a fraction of our Bitcoin deposits available for immediate withdrawal. From there we have offline wallets secured by encryption and physically. It's a balance to find the right amount to keep available for transfers but as our deposits have been growing we've only had to do a reverse transfer once so we think we're doing the split fairly optimally.

In terms of further wallet and site security, we run multiple geographically-redundant encrypted backups and test them regularly.

We do have plans for many wallet features including a visible wallet which will allow customers to be able to verify their balance using block explorer and an offline wallet service which will only allow delayed access to funds.

I hope this addresses your wallet.dat concerns and some of the larger concerns you may have about ExchB and Bitcoin exchanges in general.

I don't think fractional reserve is an accurate name for this. What we're discussing here is more like storing the funds in a high security vault in the back of the bank than the fractional reserve systems used for physical cash.

Yes, I intentionally chose a thread title that sacrificed some accuracy in favor of being controversial and getting more people to read the thread. It's a bad habit of mine. One of many.

In order to make the title a teensy bit less misleading, I added quotes around "fractional reserve" in post #1.

i didn't really want details of what the plans are . I just want to make sure they exist. It would be very bad if only a few people had access and they all die in a plane crash or something.

As long as you can confirm there is a in fact a plan , that is good enough.

Lol, +1.

Hi There,

In response to a request I received via email asking Vekja.net to respond on this thread, here is our response:

We maintain the bulk of the deposited funds in an offline account. We always have. I am personally the most security conscious person, by orders of magnitude, that I know of. The ratio of how much is offline is manually maintained by an administrator. In the event a 'run' occurs, the user gets an error message and the transaction immediately is marked as failed. We receive an email. An administrator then investigates to see if nothing is amiss, and if so, I can top up the online funds from one of the offline wallets we maintain.

The offline 'wallets' are stored off-site and encrypted. They are backed up (encrypted) in multiple geographical locations as well. They are never connected to the internet, and the wallets are never fed into a bitcoind that is connected to the internet -- just in case a vulnerability exists in the official bitcoin client.

Thufir

Just curious, what would happen if you were to get hit by a bus? I'm not being cruel or insensitive here, but I'd be reassured if there are procedures and people in place to keep the site operational or at least allow everyone to withdraw if something where to happen to you personally.  would the secret hiding places and encryption keys of all the bitcoin offline wallets would be lost forever?  Would someone in your family inherit all our bitcoin deposits and we'd be left high and dry like mybitcoin.com customers?

Anyone ever watched the show "Dragon's Den" in CA/UK or "Shark Tank" in the US?  This 'what if you get hit by a bus' factor is an important thing to consider before investing in a company for the venture capitalists on that show, and I think it applies to bitcoin businesses especially so.   

I actually ignored this thread because of the title until I was asked to post here. TradeHill stores a lot of coins offline in a wallet that's not connected to the net. As funds rise to a certain level we move them over. We also take a lot of other measures that we will be posting about soon. We've been doing this since the beginning. Bitcoin provides new vulnerabilities but at the same times provides opportunities for security that traditional institutions don't have. We should take advantage of those and use it to minimize the damage if something catastrophic does occur.

Jered

Hello,

We also keep reserves on different places (online and offline) distributing the risk evenly.
I support the other exchanges in not sharing additional detailed information as this question is directly towards security and any public details may very well compromise any security measures.

Here is the official statement from VirWox. I guess now we know why they never post here.


I fully understand and appreciate why sites don't want to discuss their security measures in detail, but I think it's a great way to build trust to be as transparent as possible.

Thanks. I think I've gleaned enough hits on this thread by trickery. New thread title is:

"Open letter to online exchanges and wallets: store coins offline!"

Wondering how VirWox can accept
1. Paypal, which isn't even a form of payment anymore.
2. PaySafeCard, Which takes roughly 40% of the total transaction cost.

Probably by charging the CRAZY HIGH fees they do. The one time I used them, I spent $20.18 worth of Linden Dollars to buy ~$16 worth of Bitcoin :(