Amazon Cloud Outage Causing Major Issues at Some Crypto Exchanges

[asayoyaasa]

Why something like this never happens when I'm using an exchange lol.

apparently of order book data from an unnamed exchange that a customer had posted in the BitMax exchange’s official Telegram channel. It suggests traders have been able to buy 45 bitcoin for less than $1 (in the tether stablecoin, USDT). The price of bitcoin at time of writing should be around $10,190, according to CoinDesk’s global average price index.

https://www.coindesk.com/amazon-cloud-outage-causing-major-issues-at-some-crypto-exchanges

[1715232569]

1715232569

[1715232569]

1715232569

[1715232569]

1715232569

[1715232569]

1715232569

[mk4]

Anyone here is handling or has handled trading bots that has any idea on what actually specifically caused some accounts to be able to buy sub $1 bitcoins? Was it a problem with the exchange that's caused by the AWS downtime? Or was it a problem with the trading bots caused by the AWS downtime, that ended up making some bots put in sub $1 sell offers? Or something else? Couldn't wrap my head around it.

[DdmrDdmr]

It’s pretty incredible that the problems with AWS caused various exchanges to, at some point, manage to sell BTC for an extremely low value. Perhaps the software on some of the exchanges does not manage (timeout) errors properly, and these unexpected and often non-tested situations can lead to fuzzy situations that were not planned for.

[Jating]

It’s pretty incredible that the problems with AWS caused various exchanges to, at some point, manage to sell BTC for an extremely low value. Perhaps the software on some of the exchanges does not manage (timeout) errors properly, and these unexpected and often non-tested situations can lead to fuzzy situations that were not planned for.

And that is very interesting in crypto exchanges can't cope with AWS errors and will have to seriously consider updating their applications because people and the company itself will loss money this way. I even saw CZ tweeting about it as well (500 error messages) and it did had a dire impact on them as it affecting their customers withdrawals.

I also remember that we have a similar issues that impacted some exchanges last month when Cloudfare went down, if I'm not mistaken.

[romero121]

It’s pretty incredible that the problems with AWS caused various exchanges to, at some point, manage to sell BTC for an extremely low value. Perhaps the software on some of the exchanges does not manage (timeout) errors properly, and these unexpected and often non-tested situations can lead to fuzzy situations that were not planned for.

Months back similar incident got posted on an article. This is getting to be a common issue which needs to be taken into consideration and rectified at the shortest. More the delay will cause situations more worse to the exchange, because there is difference between $1 and the real price of bitcoin.

[o_e_l_e_o]

Could the users in question withdraw though? I would imagine most exchanges have some sort of process in place to manually review large withdrawals, and if you tried to withdraw 45 BTC, which is currently around half a million dollars, it would need some input by an actual human at the exchange before it would process. Not to mention that many exchanges probably don't keep that much in their automated hot wallets, and would need to transfer from a cold wallet to cover it.

Would the exchange reverse these transactions as they are due to a technical fault?

[DdmrDdmr]

Some of the allegedly affected exchanges have gone ahead and temporarily suspended the trading serviced for some of the accounts:

Due to abnormal market fluctuations, BitMax.io has temporarily suspended trading service for accounts which had traded during 1:00 a.m. - 3:30 a.m. EDT, Aug 23rd, 2019. Trading of these accounts will be resumed upon completion of confirmation and follow-up measures.

See https://bitmaxhelp.zendesk.com/hc/en-us/articles/360034272414

Kucoin announced the following:

Due to the overheating of part of our chassis in the machine room we deployed in AWS, Tokyo, part of our services might become unavailable. The engineering operation team is currently deploying relevant resources of high availability across regions to deal with any possible emergencies that might happen. Some services might be effected during the deployment.



see https://www.kucoin.com/news/en-kucoin-system-deployment-notice

[barbara44]

If they were able to buy the coin for $1, then who sold it, and on which transaction was it recorded? I am very sure that it is the bot error or a programming error on the exchange that made the record that way and I am sure that it will be corrected and if those people had sold it, I really doubt if it will affect anyone in the market. It would just probably make the market cap to reduce which will not even have any significant effect on the value that much.

That is just my assumption though, because I have never witnessed or experienced such thing before for me to know how it is being resolved, I think that it may likely affect the exchanges’ wallet, but I am sure before they get to remove it, the value would have been adjusted back to the real one.

[bL4nkcode]

If they were able to buy the coin for $1, then who sold it, and on which transaction was it recorded?

I'm sure it's on the exchange bot issue which was affected by the AWS downtime, no one would try to sell such rate, not unless if he/she offers me such rate to sell I would not say no to him lol.

[hatshepsut93]

I know that almost everything is in the cloud today, but I think critical applications like exchanges should have their own physical servers, it's just better for security and safety, and it's in the spirit of Bitcoin. This incident is only one example of disadvantages of cloud, for example when Spectre and Meltdown vulnerabilities were revealed, some services that use clouds could be vulnerable. As another example, Github banned Iranian users because of the sanctions - similarly, cloud providers can be easy forced by governments to drop any Bitcoin products if they decide to ban Bitcoin.

[jseverson]

I know that almost everything is in the cloud today, but I think critical applications like exchanges should have their own physical servers, it's just better for security and safety, and it's in the spirit of Bitcoin.

Bolded part is entirely dependent on the exchange in question. How many exchanges would have the capability and budget to secure their servers more than a giant like Amazon, which actually specializes and pours resources in it? Most of them can't even keep their wallets safe (a core part of their business, unlike server maintenance), so I'm sure the number would be quite disappointing. It might suck, but there's a reason it's the norm.

Either way, I don't think there's really any harm done in this case. I'm pretty sure there aren't any exchanges out there which has a fully automated withdrawal process, so it's just going to be a short downtime and some rollbacks in affected accounts. "Not your keys, not your Bitcoin" also aptly applies here lol.

[Kakmakr]

Most of these transactions are actually done on the exchanges internal ledgers and they should be able to "reverse" these transactions, if the user did not withdraw those coins to an external wallet. Unfortunately if the exchanges did not pick this up and they allowed the withdrawal, then they must accept the damages and then report this to the legal authorities to deal with this matter.

It still come down to fraudulent behavior, if you used a loophole in their system to exploit the service and then benefit from that. The customer cannot deny that they did not do this, without noticing that there was a flaw in their system. 

[Red-Apple]

Most of these transactions are actually done on the exchanges internal ledgers and they should be able to "reverse" these transactions, if the user did not withdraw those coins to an external wallet. Unfortunately if the exchanges did not pick this up and they allowed the withdrawal, then they must accept the damages and then report this to the legal authorities to deal with this matter.

It still come down to fraudulent behavior, if you used a loophole in their system to exploit the service and then benefit from that. The customer cannot deny that they did not do this, without noticing that there was a flaw in their system. 

this wasn't an exploit in their system that anybody abused though. this was normal orders executed by normal people as soon as they saw a better price. the fact that the platform was buggy and was showing them that better price is not their fault. i would go as far as saying reversing any of those orders is not ethical.
if someone sold their bitcoins at that low price then it is their own fault for not being more careful, it is not the buyers fault that they made a mistake.

[o_e_l_e_o]

i would go as far as saying reversing any of those orders is not ethical.

It depends entirely on the exchange in question's Terms and Conditions. Many of the larger crypto exchanges have a clause in place (much like the vast majority of fiat/stock/share exchanges), which specifies that "clearly erroneous" trades, transactions and orders can be modified, cancelled or reversed. For example:

Coinbase:

Orders which under the circumstances involve an obvious error with respect to price, quantity, or other parameters - a ‘clearly erroneous transaction’.

Binance:

Orders which involve an obvious error with respect to price, quantity, or other parameters – a ‘Clearly Erroneous Transaction’.

If something like this was to happen on a big exchange, it's likely that the withdrawal of half a million dollars worth of BTC wouldn't take place without manual review by an employee first, at which point it would be clear what had happened and the transaction would be likely to be reversed.

Small exchanges (which are more likely to be affected by an error such as this) are less likely to have thorough and all encompassing Terms and Conditions, and so may not have a clause such as this. However, they are also less likely to have half a million dollars in their hot wallets to process an automated withdrawal, so again, would need manual review. Whether or not it is ethical for exchanges which do not say "trades may be reversed" in their terms to reverse "clearly erroneous" trades is up for debate.

[eaLiTy]

That is just my assumption though, because I have never witnessed or experienced such thing before for me to know how it is being resolved, I think that it may likely affect the exchanges’ wallet, but I am sure before they get to remove it, the value would have been adjusted back to the real one.

From what i understand it is a software glitch and really bitcoin are not being sold by anyone but the system updates the balance and if the exchange do not identify the issue and the user withdraws the amount then there is nothing they can do, these events happened in the past and some users have filed a complaint because they were unable to withdraw the amount they held in their account. These are rare incidents though to happen and hopefully exchanges will sort these out completely.

[kaya11]

That is just my assumption though, because I have never witnessed or experienced such thing before for me to know how it is being resolved, I think that it may likely affect the exchanges’ wallet, but I am sure before they get to remove it, the value would have been adjusted back to the real one.

From what i understand it is a software glitch and really bitcoin are not being sold by anyone but the system updates the balance and if the exchange do not identify the issue and the user withdraws the amount then there is nothing they can do, these events happened in the past and some users have filed a complaint because they were unable to withdraw the amount they held in their account. These are rare incidents though to happen and hopefully exchanges will sort these out completely.

So the traders that have nothing to do with the glitch will pay their mistakes? Did it ever happen? The ones who bought the BTC with that price? Is he able to withdraw the coins? I have never imagine that such case existed.