List of Bitcoin Companies with Adequate Backup--Please Submit

[Phinnaeus Gage]

ssaCEO of StrikeSapphire is the first Bitcoin related company to publically state that their site, one that deals with people's bitcoin, has an adequate backup system in place, protecting their user's funds.

We're not an exchange, but given that we deal with people's Bitcoins we do have an obligation to state this: We have always maintained 1) hourly database backups to a second data center, 2) daily offline backups, 3) a hot wallet stored in a third datacenter, on a dedicated server, and 4) offline wallet storage of all funds other than petty cash. Furthermore, everything we run other than our blog is on offshore dedicated servers at datacenters with casino-grade physical security measures, NOT on VPS. A hacker who accessed one of our dedis would find our hot wallet basically empty and our user passwords hashed. At most we'd lose a hundred bucks or so.

We don't have anything near the volume of Bitcoinica. We've got about 1000 users. When we launched, and started paying for the servers involved in this elaborate setup, we had no users. There's no doubt the added security has come at a cost that dug into our bottom line. But what's the alternative? Hosting on a VPS somewhere and waiting for disaster? You don't screw around with cutting costs on security; a wise guy once told me it's better to be "insurance poor" than temporarily rich and waiting for the other shoe to drop. One of the dumbest things I've done in recent memory was send some of our first positive revenues into a Bitcoinica account. I would never have imagined the security there would be more lax than ours, but it's my fault for not doing more research. I accept that.

There is no formulaic way yet of definitively securing a site that deals in BTC, but having lots of backups and dedicated servers seems like kind of a no-brainer place to start. It's a shame all the "security experts" floating around the Bitcoinica scene couldn't afford an extra $200 a month for a dedicated / offline backup solution. We don't make the kind of money they do, and we trusted them with our profit; if I'd known they were so poverty-stricken I would have paid for it myself goddamn it.

Post #2 of this thread will consist of a list of all Bitcoin related sites that publically state on this thread that they, too, have an adequate backup system in place that maintains all data needed to protect its user's assets, investments or funds.

Readers of this thread are encouraged to request from operators of exchanges, and the like, a public statement that their data is properly backed up. Also, if you know as a fact that this information is provided on their website(s), please submit that as well, for that'll be proof enough to include them on the list.

I now advise that anybody having BTC funds (or possible fiat) held in any company not listed in Post #2 of this thread, consider removing your funds from that entity ASAP. I'm not planning on maintaining a list of entities that don't disclose their backup practices, but I will reserve the right to change my mind, adding said list to Post #2 of this thread.

~Bruno~

[Phinnaeus Gage]

List of companies with an adequate backup system of their valuable data in place.

• StrikeSapphire
• Crypto X Change
• BitInstant
• Casascius Physical Bitcoins
• Mr. Bitcoin
• Bitstamp

List of companies who have not provided a statement that they have an adequate backup system in place protecting their clients funds, and must now prove otherwise to get removed from this list and be placed on the list above.

For this reason alone, I recommend to anyone who use the following sites to discontinue that practice ASAP.

• QwickBit

[stochastic]

List of companies with an adequate backup system in place.

• StrikeSapphire

This is reassuring.

[Phinnaeus Gage]

List of companies with an adequate backup system in place.

• StrikeSapphire

This is reassuring.

Thanks, Stoch! We'll soon find out which agents have their client's best interest at heart, and which agents...

~Bruno~

NOTE: I've edited the bold statement since this post. See revision in Post #2.

[cryptoxchange]

Hi Bruno,

For Crypto X Change we have spared no expense on security, hardware or backups.

Automated Backups are taken Hourly, Daily & Monthly and are taken Off Site.

We perform manual backups also for DB's, Wallets and Data. This Data is also Taken Off Site.

We have a top level Cisco Hardware firewall in place.

Dedicated Servers only, with RAID 10 SAS 15K RPM Arrays for maximum performance and reliability

Data Center is the most secure in Australia by far ( Location & Name Will NOT be released for security reasons )

Access to our systems are absolutely limited to 2 people within our company via very secure means, code updates by our programmers are installed by one of the 2 people who have sole access to our systems.

We have services separated to each dedicated server, enabling maximum security

Our site is PenTested often, any and all updates are also checked prior to being installed on the production servers.

Development servers are also locked down to only 2 people also.

Our company has staff that have been in the IT industry for nearly 20 years with real world experience and working and servicing some of the largest government networks in Australia, we take security very seriously.

Our Exchange is a continuous work in progress, improving all the time, building new features. There are still features un released from when we opened the exchange, we have had our problems, slow trade engine, slow API BUT never any security breaches at all, or any issues with human error and data.

Before opening, we asked the community to do their best to hack our exchange, and there were Zero breaches.

We hope this explains our security, if anyone has any further questions please do ask! If we are slow to reply here we appologise, we have a lot of development going on and new services coming out shortly.

Regards

Crypto X Change Team

[stochastic]

List of companies with an adequate backup system in place.

• StrikeSapphire

This is reassuring.

Thanks, Stoch! We'll soon find out which agents have their client's best interest at heart, and which agents...

~Bruno~

I would like to see these services doing security audits by professionals and getting some hacking insurance.

[cryptoxchange]

List of companies with an adequate backup system in place.

• StrikeSapphire

This is reassuring.

Thanks, Stoch! We'll soon find out which agents have their client's best interest at heart, and which agents...

~Bruno~

I would like to see these services doing security audits by professionals and getting some hacking insurance.

Crypto X Change is PenTested and Audited externally by professionals.

[Phinnaeus Gage]

Hi Bruno,

For Crypto X Change we have spared no expense on security, hardware or backups.

Automated Backups are taken Hourly, Daily & Monthly and are taken Off Site.

We perform manual backups also for DB's, Wallets and Data. This Data is also Taken Off Site.

We have a top level Cisco Hardware firewall in place.

Dedicated Servers only, with RAID 10 SAS 15K RPM Arrays for maximum performance and reliability

Data Center is the most secure in Australia by far ( Location & Name Will NOT be released for security reasons )

Access to our systems are absolutely limited to 2 people within our company via very secure means, code updates by our programmers are installed by one of the 2 people who have sole access to our systems.

We have services separated to each dedicated server, enabling maximum security

Our site is PenTested often, any and all updates are also checked prior to being installed on the production servers.

Development servers are also locked down to only 2 people also.

Our company has staff that have been in the IT industry for nearly 20 years with real world experience and working and servicing some of the largest government networks in Australia, we take security very seriously.

Our Exchange is a continuous work in progress, improving all the time, building new features. There are still features un released from when we opened the exchange, we have had our problems, slow trade engine, slow API BUT never any security breaches at all, or any issues with human error and data.

Before opening, we asked the community to do their best to hack our exchange, and there were Zero breaches.

We hope this explains our security, if anyone has any further questions please do ask! If we are slow to reply here we appologise, we have a lot of development going on and new services coming out shortly.

Regards

Crypto X Change Team

Thanks, CXE. You're now on the list. Keep 'em comin' all.

~Bruno~

[Phinnaeus Gage]

So far, we only have two. Any more?

~Bruno~

[Yankee (BitInstant)]

While Bitinstant holds no Bitcoin, we haveexcellent security measures in place to protect our customers privacy and security.

Taken from: https://www.bitinstant.com/security

Basic principles

Due to the nature of our business, security is very important and we have a very unique approach to the issue, defined by the following principles:

Transparency
Contrary to popular belief, the most secure system is one where everyone can see every aspect of it and still can not break in. Imagine a safe where even with the blueprints you can not find a way to get it open. This principle is one reflected in the history of information security and cryptography. Open-source and free software systems have a track record of rapidly fixing security issues with systems such as OpenBSD having a reputation for heavy security. Cryptographic algorithms such as ECDSA (on which bitcoin is heavily based) are open for inspection by all, and it is this which gives them strength. For more on the principle of transparency and why full disclosure is the ideal we refer you to this excellent essay by Bruce Schneier: Full Disclosure of Security Vulnerabilities a 'Damned Good Idea'.

Proactivity
Our second principle is that of proactivity, taking an active rather than responsive approach to potential security threats. Rather than responding after an incident we take active measures to audit our system and our business practices to find any potential threats before they occur. This includes even theoretical and obscure threats, as we assume any attacker targeting us will explore all possible avenues of attack. In practice, this principle means regular audits on at least a daily basis using vulnerability scanners from multiple different vendors in addition to analysis of system log files for anomalious entries. We also conduct reviews of changes to the filesystem on any of our servers and flag up automatically any changes outside of those specifically defined as normal and authorised.

Awareness
We keep track of multiple sources of information about the current state of our systems as well as external reports of security issues with the software we use. Our support system automatically creates tickets for any potential security issues discovered by automated scans and audits as well as for any advisories on public mailing lists. We treat these support tickets with high priority. On top of this, we track multiple metrics for performance, potential attack attempts and other measures.

Paranoia
Encrypt everything, use default deny on all firewalls, lock down ACLs and filesystem access, restrict syscalls on daemon processes and trust nobody. We assume that 24/7, somebody is actively trying to break into our system and is desperate to do so. This assumption assures that in the scenario where the bad guys are not so active our system stands up.

Realism
Realistically we are humans and not perfect security-hardening machines, therefore we assume that there's always something we've missed - this is why we use outside vendors to do scans and audits, but in keeping with the principle of paranoia we also assume outside vendors are full of bad guys or employees that can't be trusted. We also do audits again after system configuration changes to ensure that nothing is missed.

Specific practices

Multiple vendor security scanning
We currently use the following software and services to audit our systems: Nessus, Nikto, snort IDS, checksecurity, TrustGuard PCI scanning, HackerTarget, nmap, tripwire. These tools are run on a daily basis where possible and any flaws found are dealt with immediately after audit. In addition to manual checks, our servers have a daily cronjob that provides up to date information and metrics relating to performance and security. To prevent a possible attacker from compromising a system and covering their tracks, these reports are emailed to our staff and backed up in an encrypted form onto amazon S3. Hashes are stored of logfiles using different algorithms (MD5 and SHA512) and regular checks are made that no alterations have been made.

Configuration backups
On a regular basis we make backups of all servers at the disk level (raw block device dumps) and store them in order to allow for easily reverting in the event of a compromised system. Should any of our systems end up compromised, our policy would be to estimate the date of first breakin and then restore the backup taken at least 1 month previous to that in a controlled environment (no external network access except for SSH from our staff). We would then install the latest security patches and close whatever hole enabled the initial compromise before running all auditing tools at our disposal in addition to a manual check of all operating system binaries for potential hidden rootkits.

Untainted logs
Every quote, every transaction stage, every completed transaction - all of this information is logged to multiple locations with cryptographic hashes used to verify it is untainted. Our systems are not able to do anything other than write to the transaction log and we keep the entire log archived. Physical backups to removeable media are also made on a regular basis and all copies of the transaction log are encrypted with the key known only to our 2 founding partners. Should every other security somehow fail, we will always be capable of rebuilding our entire service using this information - this is our key asset, and it is also your key asset - customer transactions.

Source code management and end user software
Like any online service, our software is what powers everything. For every piece of code, we ask whether it would be safe to release that code to the general public or if doing so would open up security holes, and if releasing it would open up security holes then we rewrite it. As a service provider whose business is selling a service rather than the software powering it, we do not require our customers to run any specific software other than a modern web browser (we do not even require javascript - our site will generally work without it, although we do recommend it is enabled for ease of use). As we do not require our customers to run any software we restrict our attention to the server side and to the small class of potential attacks against javascript and HTML (XSS and injection attacks). Our customers are free to examine and audit any javascript or HTML sent by our webservers and ensure there are no security flaws, and if any flaws are found we encourage customers to inform support.

Default deny
Any of our systems has a firewall installed with the default being to deny any and all connections. This host firewall is in addition to any upstream firewall provided by our hosting provider (to control for the scenario in which our hosting provider is compromised). The same "default deny" philosophy applies to filesystem privileges and ACLs. Whenever a system component can be configured to deny everything by default, we do so and whitelist specific authorised uses.

System log anomaly monitoring
During regular operation there are various log entries written by a linux server. These log entries fall into 3 basic categories as regards security. Legitimate use, "background noise" and anomalies. We define "background noise" as the fruitless attempts by botnets and individuals with port scanners to try and break into any IP host they come across, usually in the form of dictionary attacks. Background noise is mostly an annoyance but we do not take the risk and so IP blocks from which these kinds of attempts originate are automatically blocked by our firewall. Anomalies are targeted attempts or requests sent to our servers which are neither part of normal usage or background noise. We assume all anomalies to be an attempt to break in and block IPs sending such requests before analysing the requests to look at what exploits (if any) were attempted. We assume that such attempts are successful until conducting audits to verify they were not.

Regular key changing
On a regular basis, all keys and passwords are changed and new ones are generated randomly with a high-quality entropy source. Specifically, the entropy source used is static noise from a soundcard blended with prices of random stocks on public markets on random dates before being passed into a hash function and fed to /dev/random on a linux server.

No production configuration in version control
We use the subversion version control system to develop code, but we make use of the .dist design pattern, wherein configuration files are named (for example) "importantstuff.cfg.dist" rather than "importantstuff.cfg". We then set subversion to ignore plain .cfg files and perform regular audits to ensure they never make their way into version control. When a code change requires merging an old configuration file, this is done by hand. By following this pattern we ensure that should our SVN repository be compromised it will not lead to disclosure of important server configuration information.

Treating all errors as potential security flaws
Our web application code logs HTTP 500 errors caused by exceptions or other problems in the underlying code and these logs are emailed to support in addition to generating support tickets that are treated with a high priority. This also allows us to respond rapidly in case of problems affecting customer transactions and in theory resolve issues before the customer even has time to contact support seperately.

[BitcoinOPX]

Hi Bruno,

I think this is a great idea. BitcoinOPX has just opened for testing, but we already have very secure procedures in place. We perform hourly offsite backups of all data. This data is then copied to a third location for redundancy.

A key thing to note is we don't use a "hot wallet" which has been at the center of security breaches. We do not host bitcoins online. Instead, we approve withdrawals in about one hour. We use the Armory cold storage method to store bitcoins which is extremely secure.

We also host a Security page on our site transparently describing how we, and the escrow service we work with, secure bitcoins.

[ssaCEO]

Just want to say thanks for the distinction. I wish this information had been available a couple months ago.

[R-]

Question to Phinneas: How do we determine that the owner of the said company is telling the truth? Perhaps we should specific that certain evidence is required in orer to prove one has adequate backups?

[ribuck]

More important than a self-authored "adequate backup" claim, would be a statement from each business indicating:

1. Whether the business guarantees that it will "make good" any customer balances after a catastrophic failure
2. How the business will do so
3. Who is the person or entity responsible for seeing that this guarantee is carried out

[Phinnaeus Gage]

Question to Phinneas: How do we determine that the owner of the said company is telling the truth? Perhaps we should specific that certain evidence is required in orer to prove one has adequate backups?

I was waiting for this question. This thread is not meant to have Bitcoin related companies offer up solid proof that they have adequate backup, but simply have them state publically that they do, offering up any proof that they desire. Their word would be sufficient for this thread, but be forewarned, if later down the road we find out that any entity who claimed otherwise, but proven false, should be immediately labeled--no exceptions. Moreover, this thread is meant to find out exactly which companies don't have an adequate backup system in place.

If no backups, remove your funds ASAP.

That said, who's next to publically claim they have what it takes to protect each and every Bitcoiner?

~Bruno~

[Phinnaeus Gage]

More important than a self-authored "adequate backup" claim, would be a statement from each business indicating:

1. Whether the business guarantees that it will "make good" any customer balances after a catastrophic failure
2. How the business will do so
3. Who is the person or entity responsible for seeing that this guarantee is carried out

Possibly reasonable, but looking forward to what the community has to say on your proposal, ribuck.


I'm adding Yankee of BitInstant to the list.

~Bruno~

[repentance]

More important than a self-authored "adequate backup" claim, would be a statement from each business indicating:

1. Whether the business guarantees that it will "make good" any customer balances after a catastrophic failure
2. How the business will do so
3. Who is the person or entity responsible for seeing that this guarantee is carried out

It might make people feel good to have businesses make such a statement, but ultimately such reassurances are pretty meaningless.  The community has shown itself to have significant aversion to any kind of externally supervised recovery process when Bitcoin enterprises fail and it's also been unwilling to use legal processes to recover funds.  All the promises in the world are meaningless when there is neither a means nor an inclination to enforce them.

If Bitcoin businesses were to be brutally honest about what would happen in the event of a catastrophic event, it's likely that few of them would have the capacity to fully cover any losses and that some would lack the capacity to even partially cover them.

[ribuck]

More important than a self-authored "adequate backup" claim, would be a statement from each business indicating:

1. Whether the business guarantees that it will "make good" any customer balances after a catastrophic failure
2. How the business will do so
3. Who is the person or entity responsible for seeing that this guarantee is carried out

It might make people feel good to have businesses make such a statement, but ultimately such reassurances are pretty meaningless.  The community has shown itself to have significant aversion to any kind of externally supervised recovery process when Bitcoin enterprises fail and it's also been unwilling to use legal processes to recover funds.  All the promises in the world are meaningless when there is neither a means nor an inclination to enforce them.

Social pressure isn't the answer to everything, but it can solve a lot of problems that "externally supervised enforcement" will never solve. And even the act of making a public statement will cause all but the most sociopathic to be more aware of the consequences of their actions, and perhaps to take more care.

[repentance]

Social pressure isn't the answer to everything, but it can solve a lot of problems that "externally supervised enforcement" will never solve. And even the act of making a public statement will cause all but the most sociopathic to be more aware of the consequences of their actions, and perhaps to take more care.

I can't think of any instance so far where social pressure has led to enterprise users recovering funds.  In a couple of instances user balances have been repaid because another company has rescued a business (Mt Gox with Bitomat and the Coinlab guys with Bitcoinica).  TradeHill made the decision to close its doors when it could no longer afford to absorb losses and while it still had the capacity to honour user deposits.  Most other businesses have either simply failed with no return to users or have made a payment to users based on what they claim to have available and made no promises to repay any remaining balances in the future. 

[hazek]

And this ladies and gentlemen is how a market regulated by strictly market consumers (i.e. a free market) regulates itself. Isn't it beautiful?