Sorry it's taken a little longer than I'd hoped to post this. This weekend was absurd in the amount of work we had to do to get the site back up and stable. We wanted to post this as things are nearing a final resolution.
What happened:
On Friday, March 21st, someone found an exploit on our site and used it to change the passwords of users, log in as them, and withdraw their funds.
What was lost:
After all accounting was done, only three users lost coins. A total of about 81k BTCS was taken, and 7.7 million KARM. One user had SLR sold for BTC in an attempt to withdraw that, but our BTC security kicked in and locked down all withdrawals, so nothing was lost.
What we did wrong:
We failed to secure a section of our website properly. Hackers are smart. Sometimes smarter than we are. They used an exploit that was possible because we had some errors in our code that did not properly validate the input before processing what was sent. This was used to change the passwords of some users.
What we did right:
Our BTC security kicked in and halted all withdrawals. It's how we avoided losing an attempted 15 BTC in withdrawals - the website accepted the withdrawals and passed them to the withdrawal daemon, which using security we're not about to explain (sorry hackers - no inside info here) left the withdrawals in a queue until it could be manually approved. A "BTC withdrawal issue/delay" notice was posted on the site to alert users that there would be a delay in processing their withdrawals. We immediately investigated the situation as it was further brought to our attention by a user who noticed pending BTCS withdrawals he did not initiate.
We hit the killswitch which disconnects the wallet servers and database from the network, and throws up the "Down, back soon" page.
A total of 12 accounts were logged into, and 4 were actually effected, before we stepped in and stopped the issue.
As soon as we could take a breath, we posted a notice in our blog and CC:ed it to Twitter and Facebook. Notice was posted within 2 hours of us knowing what happened. Later that night, another, more in-depth notice was posted. To let everyone know, and to be forthright and honest. Something we insist upon.
What we have done so far:
Over the weekend, all users who lost coins were contacted and the situation explained. We're in the process of attempting recovery of those coins, and by 10PM EDT on 3/25/14 we will have some reply as to that status.
We combed the code, line by line, three times, for other possible exploits. The problem that allowed the hack in the first place has been fixed. We saw no other possible holes. We also installed an extremely sensitive firewall system that watches the web server. At the first hint of non-normal traffic, the offending client is banned from the site through multiple means. White hat hackers who want to help find holes? Don't. You'll be banned with the very first hit you make that doesn't fit normal AllCrypt.com usage patterns. Again, I won't say we're hack proof, but we're 10x as stringent with security now.
What we plan to do going forward:
We've set up other watching systems and auditing which get reported to us multiple times throughout the day, so if in the event of another issue, we can stop it before anything else happens. Our firewall system banned 7 scans yesterday. Not sure if it was the original hacker trying again, or just routine web scans, but they won't be accessing the site.
We are committed to security and transparency. We want you to trust us, not out of some blind faith that we've seen lately for some other exchanges that have gone dark with nary an update and only vague promises, but because you know that we will act, swiftly. Our security is strong, and is now even stronger. And we pledge to communicate all we can as soon as we can.
In a way, I'm happy this break happened while we were so young. We'd only been open 3 weeks when it happened. It was a testament to our security that so little was lost, at how fast we reacted, and it helped us close a hole we didn't know was there. It helped us to tighten existing security.
No one wants to get hacked - especially days after making security statements like we have (Stating "No, we're not hack proof" - aah the irony!), but in a way, I am personally thankful, that the loss was so small, and that we are stronger than we were before.
About those lost coins:
Like I've alluded to before - something is in the works regarding those lost coins. I've pledged an answer of some sort by 10PM tonight. Stay tuned.
For the full story, see our blog at
http://www.AllCrypt.com/blog
Show Posts
