Bitcoin Forum

Some people today have faced some serious financial pain, and I have the utmost sympathy for  them, and respect that those who run services on linode that have been stolen from are eating the costs themselves.

In the most non-gloating way that I can say this: I hosted a bitcoin service on linode and when I heard the news of the hack, I was completely unconcerned.

Now, granted coinsmack's losses could have been covered by the change floating around in some people's couches. But even if the service was handling thousands of bitcoins, I still would be fine hosted on linode even if someone roots me.

The reason why that is the case is that I don't run bitcoind on web servers. I consider it just a matter of time for a web server to be compromised, as there are just so many vectors of attack.

So the way I set it up is that I generated 10000 keypairs on a local machine behind a firewall and with no other services turned on. It runs bitcoind with local rcp calls only. The public addresses are copied up to the webserver, and the webserver uses blockexplorer to check balances. Based upon the logic of the site as far as what needs to be paid out to what addresses, an admin page is generated that contains the data that is fed into the local bitcoind to handle the transactions (it is really all handled in just one massive transaction).

This is manually initiated, so my monkey brain can take a quick scan of things and make sure things look alright before pressing the big red shiny button. Where the money goes and how much goes where is still determined by the webserver, but the transaction only happens outside of the webserver and by a manual process.

Some people have asked why bitcoind on the hacked sites was not encrypted. This would indeed have saved them in this case, but if the site was hacked in another way where the server stayed online, and bitcoind had already decrypted the wallet so it could take transactions, that would still have resulted in the same loses.

Trusting a webserver to store your wallet is a dangerous thing as is shown time and time again.

Smart.  8)

+1.  I strongly recommend keeping wallets of any significant value in a separate high-security location and retrieving queued transactions from your web server.

Is this really an honest comparison?  The practices of those who got hit are likely quite similar.  In all cases, the coins stolen were those in the "hot wallet" -- coins needed for immediate disbursement.  If you don't run the kind of business that needs to support immediate disbursement, using an offline address is a tautology.  The issues only arise when you need to support immediate disbursement. 

I find the need for immediate disbursement to be illusory.  I for one would be more comfortable with a business that only allowed me to withdraw a trivial amount immediately, and required cursory manual review for larger amounts.  I would not be bothered by having to wait 6 hours to withdraw 43000 BTC (hypothetical I suppose, since I'm not withdrawing like that with any regularity), but would expect that I could immediately withdraw (for example) 43 BTC just in case I wanted to make a payment with my account.

I think 43000 BTC is simply too much to have on a hot wallet.

Yes Mike, but when your service has thousands of customers, 43000 BTC doesn't give much instant spendability per user. 

I have no idea what Bitcoinica volumes are like but it surely seems like far too much. I think a better way to handle it would be a priority queueing mechanism. Small amounts could be disbursed immediately from the hot wallet, larger amounts could trigger replenishment and really wacky amounts could just be relayed for manual auditing. Combine this with automated replenishment of the hot wallet at suitable intervals. Also, it would be easy to modify the trigger levels based on hot wallet balance to keep flows steady.

One mechanism for this is pre-canned offline transactions that are submitted at intervals from another system. They can't be modified after creation and I think there is even a mechanism for post-dating. A script on the offline system could be used to generate a number of them for some reasonable time period so you aren't bothered with manual intervention unless the expected overall volume increases substantially. This would be quite simple to setup.

Well, there's likely all sorts of ways to secure things but the first step is realizing that out-of-the-box use isn't adequate for online high value wallets.

If a service has an employee, they can during business hours fetch pending payouts from the websever, and confirm that the offline wallet should make the payout. This person should have the very limited yes/no power.

I think even for trading platforms an intern doing this once every couple hours would be fine. Who needs immedate payout?

Not even when you consider flow of incoming funds into the hot wallet? And especially when it's a service where most transactions are just private journal entries in a database, not BTC withdrawals?