Buyer Beware. Proposal for a non high-frequency manipulatable exchange.

[Isosceles]

I placed several buy orders on MtGox last night which were clearly above market, inside plenty of bid orders as shown on MtGoxLive. I was amazed that I didn't get filled, and instead it moves the market to my offer. It is as if someone is seeing the order flow before orders are processed, and cancelling their orders on the order book before they're lifted. (MtGoxLive updated to show my bid, so it hadn't stalled)

It may be a high frequency trading expert has cornered the bitcoin market. This is very bad for everyone, as it means the bid / ask spread will be much wider than you think when anyone trades. At the moment, most people are ignorant of the cost of HF trading manipulation because it's so hidden. eg. http://ftalphaville.ft.com/blog/2011/12/19/797701/alphachat-sal-arnuk-on-high-frequency-trading/ It's effectively using inside information to front run your trade.

Here's a proposal to make it harder : exchanges only allow trading every 30 seconds. You place orders whenever, they get filled using an auction mechanism every 30 seconds where the market gets filled the best bid / ask price below/above their order. Think of it like decimalisation of time. It should also help reduce volatility. Please can someone make this? I'll pledge some BTC, I'm tired of paying huge BTC transaction fees.

[1715281680]

1715281680

[1715281680]

1715281680

[1715281680]

1715281680

[DarkEmi]

I dont think mtgox has any incentive to run shadowy business when they are making bank with a clean exchange, thus what happened to you was probably a mere coincidence

And yes, allowing some "insiders" to get market data and using it before executing trades would be very close to theft.

Edit : However i would love to see removed the option to trade for 0.00001 btc or wathever which currently allow bots to spam the trade history, it is annoying and useless

[paraipan]

I placed several buy orders on MtGox last night which were clearly above market, inside plenty of bid orders as shown on MtGoxLive. I was amazed that I didn't get filled, and instead it moves the market to my offer. It is as if someone is seeing the order flow before orders are processed, and cancelling their orders on the order book before they're lifted. (MtGoxLive updated to show my bid, so it hadn't stalled)

It may be a high frequency trading expert has cornered the bitcoin market. This is very bad for everyone, as it means the bid / ask spread will be much wider than you think when anyone trades. At the moment, most people are ignorant of the cost of HF trading manipulation because it's so hidden. eg. http://ftalphaville.ft.com/blog/2011/12/19/797701/alphachat-sal-arnuk-on-high-frequency-trading/ It's effectively using inside information to front run your trade.

Here's a proposal to make it harder : exchanges only allow trading every 30 seconds. You place orders whenever, they get filled using an auction mechanism every 30 seconds where the market gets filled the best bid / ask price below/above their order. Think of it like decimalisation of time. It should also help reduce volatility. Please can someone make this? I'll pledge some BTC, I'm tired of paying huge BTC transaction fees.

+1, i vote for a no-HFT exchange, we had a few threads talking on the subject already, i pledge some btc too

[Tril]

This is a good question.  We know someone is placing many tiny bids and asks at 0.0001 increments,
and can get notified when they are bought instantly over socketio.  The question is, while mtgox is processing
a trade, do you have time between receiving notification one of your 0.0001 orders was filled,
to edit the order book and remove a larger order right behind it?

[Serge]

HFT is inevitable

[DarkEmi]

This is a good question.  We know someone is placing many tiny bids and asks at 0.0001 increments,
and can get notified when they are bought instantly over socketio.  The question is, while mtgox is processing
a trade, do you have time between receiving notification one of your 0.0001 orders was filled,
to edit the order book and remove a larger order right behind it?

If they have time this is a bug from mtgox. The issue is not with HFT which provides liquidity to the market.

And also I dont think HFT has any edge for bitcoin, usually HFT is used for arbitrage opportunities...

[Steve]

As far as I know, all of the bitcoin exchanges have continuous matching engines.  I do think a discrete matching engine would be a valuable service, though I think 30 seconds is overkill.  You could operate a discrete engine on a 1 second basis and all but eliminate any advantage that a trader with a low latency connection would have.  Maybe 5 seconds if you are concerned that people in rural Africa on dial up modems might be at a disadvantage.

However, a continuous matching engine shouldn't behave in the way that the OP described unless it's compromised in some way or you just got extremely unlucky.  The buy order should have matched against standing sell orders right away.  If those sell orders were removed ahead of your buy order reaching the matching engine, then either it was a stroke of bad luck, or someone is able to view the incoming orders before they are placed with the matching engine and place or remove other orders before yours (obviously not a good thing).

[Fireball]

You misunderstand how the exchange works and start blaming HFT without understanding how it works.

If you see sell orders, e.g.

3.700 - z
3.600 - y
3.500 - x

and you place a buy order at 4.0 $ per 1 BTC for N bitcoins, then your order eats all those starting from the cheapest (3.5 in our example) to buy N bitcoins, as you asked.

Obviously, if you consumed those sell orders, and noone else is selling at those prices (they even didn't have time to put their new orders!), you start seeing orders above (e.g. 3.800, 3.900, 4.000, etc).

It's as simple as that.


Now about HFT.
First of all, Mt.Gox is not suitable for HFT at all, but there are exchanges that will be suitable (like my derivatives market one).
Second, why HFT is good. A usual, "manual" trader would not even notice the subtle difference where HFT play. They are talking about positions aimed to get 0.000x points, while keeping open position for a couple of seconds. So it introduces very small rate drift, which, I will repeat myself, is unnoticeable to the manual trader. It's only important for algo-traders.
Third. HFT and algo players provide additional liquidity to the market. That means, that when you just consumed those "cheap" sell orders as in example above in absence of HFT players, if they were present - it would not be so easy. There would be more bid/ask offers, and this is a key factor in a stably going rate.

More depth in the market - better. Market must be free, and provide equal access for everyone. Then there will be no "losers" or "winners", everyone gets what she wants.

[Fireball]

I didn't tell another reason, maybe Mt.Gox just have a bug in the implementation.

In the trading engine I am developing, this just can't happen by nature of implementation, but it aims for HFT trading because as I said - the market must be free, no artificial limitations should be added, and everyone should be in the same conditions (thus have same access time).

[paraipan]

... everyone should be in the same conditions (thus have same access time).

exactly, ppl will wake up sooner than you think regarding trading bots.
I only want equal conditions with all other traders without some real fast bots interfering with the market. The best analogy are on line games where some players use scripts for better aiming or see through walls, like OGC in HalfLife. Yes, I'm completely aware that we have some real smart individuals around that write those scripts and use them. That doesn't mean they can't use their creativity edge for other purposes when bitcoiners will let them understand that cheats are not wanted.
When a real alternative to mtgox platform surfaces we will see how bots get along when no humans are left to play their game.

[PrintCoins]

I disagree. I think bots are exceptional at rapidly converging on a correct price.

Also, don't think of a bot as a bad thing if you are a trader. A bot is an algorithm written by someone who has to predict how to predict what way a price is going. If you ever played an RTS game against a bot, you will realize they don't have any intuition or smarts. They will always do the same thing over and over. Predict what the bots will do, and you win. Good luck trying to predict what the humans will do.

[Fireball]

I disagree. I think bots are exceptional at rapidly converging on a correct price.

Also, don't think of a bot as a bad thing if you are a trader. A bot is an algorithm written by someone who has to predict how to predict what way a price is going. If you ever played an RTS game against a bot, you will realize they don't have any intuition or smarts. They will always do the same thing over and over. Predict what the bots will do, and you win. Good luck trying to predict what the humans will do.

One more sane and smart person in the thread :-) Indeed bots are useful, and anyway could be outperformed by human if needed.

[paraipan]

I disagree. I think bots are exceptional at rapidly converging on a correct price.

Also, don't think of a bot as a bad thing if you are a trader. A bot is an algorithm written by someone who has to predict how to predict what way a price is going. If you ever played an RTS game against a bot, you will realize they don't have any intuition or smarts. They will always do the same thing over and over. Predict what the bots will do, and you win. Good luck trying to predict what the humans will do.

One more sane and smart person in the thread :-) Indeed bots are useful, and anyway could be outperformed by human if needed.

Agree with you ppl, bots are good but not in all situations. Sooner or later we will have a "humans only" exchange, i'm sure about that, because we have real bad "response times" when compared to automated programs and feel a lot more comfortable trading with equals.

[PrintCoins]

I disagree. I think bots are exceptional at rapidly converging on a correct price.

Also, don't think of a bot as a bad thing if you are a trader. A bot is an algorithm written by someone who has to predict how to predict what way a price is going. If you ever played an RTS game against a bot, you will realize they don't have any intuition or smarts. They will always do the same thing over and over. Predict what the bots will do, and you win. Good luck trying to predict what the humans will do.

One more sane and smart person in the thread :-) Indeed bots are useful, and anyway could be outperformed by human if needed.

Agree with you ppl, bots are good but not in all situations. Sooner or later we will have a "humans only" exchange, i'm sure about that, because we have real bad "response times" when compared to automated programs and feel a lot more comfortable trading with equals.

You are sure about that? Let me know when it appears. I would like to set up my bot on it. It will just trade at whatever the exchange rate is on gox. If you try to filter it out with a captcha, there is a fine automated turk that will help me with that. The irony of using a human to serve a machine to serve a human to undercut other humans is too sweet to miss.

If you really want a human only exchange, you need to go here:
https://en.bitcoin.it/wiki/In_Person_Traders
I suspect that they all just use the bot derived price from their cell phones though.

[paraipan]

I disagree. I think bots are exceptional at rapidly converging on a correct price.

Also, don't think of a bot as a bad thing if you are a trader. A bot is an algorithm written by someone who has to predict how to predict what way a price is going. If you ever played an RTS game against a bot, you will realize they don't have any intuition or smarts. They will always do the same thing over and over. Predict what the bots will do, and you win. Good luck trying to predict what the humans will do.

One more sane and smart person in the thread :-) Indeed bots are useful, and anyway could be outperformed by human if needed.

Agree with you ppl, bots are good but not in all situations. Sooner or later we will have a "humans only" exchange, i'm sure about that, because we have real bad "response times" when compared to automated programs and feel a lot more comfortable trading with equals.

You are sure about that? Let me know when it appears. I would like to set up my bot on it. It will just trade at whatever the exchange rate is on gox. If you try to filter it out with a captcha, there is a fine automated turk that will help me with that. The irony of using a human to serve a machine to serve a human to undercut other humans is too sweet to miss.

If you really want a human only exchange, you need to go here:
https://en.bitcoin.it/wiki/In_Person_Traders
I suspect that they all just use the bot derived price from their cell phones though.

I will you can bet on that but be warned that it will have a 1 transaction/minute limit enabled for every trader/ip, so your bot will trade just like a human without taking advantage of sub-second orders on the platform  
the only real advantage you will have is watch some movies or go out with you girlfriend while tirelessly trading bitcoins, nothing more

if you don't get my point from this post i really give up for not being able to explain what i really mean.

[PrintCoins]

IP block - ha! There are world-wide proxy pools for me to dive into. The water is great and it makes my metallic skin glisten like the chrome on a classic automobile. I'll grab 120 of them and make trades every half second on average. Resistance to the swarm is useless. You will be assimilated.

[paraipan]

meh, whatever 

[cbeast]

I believe that HFT pump and dump bots cause the instability of Bitcoin price. Why don't we just make Bitcoin a decentralized fiat currency with greater stability?

[paraipan]

I believe that HFT pump and dump bots cause the instability of Bitcoin price. Why don't we just make Bitcoin a decentralized fiat currency with greater stability?

good question... but how ?

[the joint]

I dont think mtgox has any incentive to run shadowy business when they are making bank with a clean exchange, thus what happened to you was probably a mere coincidence

And yes, allowing some "insiders" to get market data and using it before executing trades would be very close to theft.

Edit : However i would love to see removed the option to trade for 0.00001 btc or wathever which currently allow bots to spam the trade history, it is annoying and useless

I actually think the back-and-forth bids of .00001 BTC might be a data-gathering technique to help inform the bot of what to do next.

[cbeast]

I believe that HFT pump and dump bots cause the instability of Bitcoin price. Why don't we just make Bitcoin a decentralized fiat currency with greater stability?

good question... but how ?

Even though it is decentralized, Bitcoin does have an authority with the ability to impose its price. Math itself can set the price. I've proposed this several times before that we should statistically analyze the variance of price and predict its value as a mean. A reported price this way will serve as a psychological dampener to the volatility.

[Herodes]

mtGox is know to not always provide accurate data through their API. The last months, there's been several glitches when trades has been done far over or far below current market value. Also, I don't think mtgoxlive.com is your fastest source for mtGox information. I once watched mtgoxlive.com next to a feed using the websocket, and the websocket feed was always faster.

Also when making assumptions, accusations or suggestions, it never hurts to be more technically accurate. What you 'feel' happened is hardly any real evidence. But for the record, I've had lot's of incidences that seemed 'weird' with mtGox trading the last months, but I won't complain about it unless I can back it up with hard facts. Human error may often be the cause of a percieved glitch as well.

I'd add to that I'd also like the extremely small trades to not be allowed either. Perhaps 0.1 BTC should be the smallest allowed amount for a mtGox trade?

As for bots, I see nothing wrong with them, they're just like humans, only thing is that they have more stamina, don't get tired and are 100% diciplined 100% of the time.

[Isosceles]

Lots of great posts, lots of interest. To summarize, my goal is to make trading BTC fairer, more stable, and less expensive.

I strongly suspect both MtGox and Bitcoinica's matching engines have a weakness. For example,
 - From my Bitcoinica outage experience yesterday (previous post), I was able to cancel Bitcoinica orders whilst the matching engine is busy.
 - Similar outages have been observed at MtGox, e.g. the infamous time when the price dropped to 0.00001 took several hours. The MtGox engine was filling one super order. I could make & cancel new orders as it was being filled. This is a critical weakness.

This is how I expect the HFT manipulates the market :
1) Spam exchanges with lots of small orders or trades. Keep their matching engines very busy to slow down orders.
2) Place large market orders up and down to clear out the existing static order book
3) Place your own orders to fill up the book. You now control the price.
4) When a significant buy order is placed by another person, (e.g. buy @ $4, best offer is $3.5), the matching engine will start to fill the trade. This will take time, especially if you've populated the order book with 1000 tiny orders.
5) Knowing that a big order is in progress, you can cancel your outstanding orders in the book, forcing the other person to pay a much higher price than they expected to trade. If the exchange forced you to keep the existing order, you could withdraw funds instead (which is probably not blocked on the matching engine thread).
6) After their order is filled, you can push the price down immediately by adding more (fake) sell orders to the book.

They could also get a timing advantage by being closer to the exchange, or by delaying/DOSing the market data that other participants rely on (MtGoxLive, MtGox websockets, etc)

BTW, calling it HFT is a poor comparison to the real-world HFTs, but my manual trading lag time is probably 10-30 seconds including human reaction, data entry & internet lag time

Auction Exchange : The advantage of choosing a longer time before the auction occurs is to build up a bigger at-the-money order book. Assuming the exchange is non-profit (low transaction fees was the main idea of bitcoin, remember?), a larger number of filled orders would be crossable, so you could fill large orders without moving the market and people would have more confidence in the value of the currency. I would prefer to buy a large number of BTC with a super-long auction than drip-feed it in to MtGox. One additional restriction, the auction order book can't be visible before its executed. Otherwise it would lose its value.
Another way of looking at it, what do you gain if you were only able to trade every ~10 seconds? What would you lose? If you can't value it, you're the free lunch. Same reason Facebook is free.

Question : Does MtGox use PHP for its matching engine as well as its API? It should be written in C, with someone dedicated looking for timing arbitrage problems like this.

@Fireball : The advantage you advocated for is provided by automated market making, not just HFT. Automated market making would also be available for a discrete time auction exchange. HFT can destroy for profit too, e.g. Nanex - The Disruptor http://www.nanex.net/Research/EMini2/EMini2.html

@cbeast : Worship of financial maths will get you as far as CDOs and mark-to-model. When the model fails, you'll need real people to untangle the expensive mess.

[DeathAndTaxes]

You are sure about that? Let me know when it appears. I would like to set up my bot on it. It will just trade at whatever the exchange rate is on gox. If you try to filter it out with a captcha, there is a fine automated turk that will help me with that. The irony of using a human to serve a machine to serve a human to undercut other humans is too sweet to miss.

There are ways around that.  Issue a one time password device which displays the current one time passwords.  Randomly with a x% chance per trade input you need to enter the passcode.  Of course you could give the one time password device to your human turk ... of course the codes from the same device allows changing withdraw bank accounts, changing login password, and withdawing USD and BTC.

Of course Mt. Gox (or any exchange) would never do that.  They make money off volume.  More volume = more money.  They aren't going to do anything which may reduce volume.  Just pointing out that it is possible to put into place bot countermeasures if one wanted too.

[markm]

I am setting up my exchange to treat each asset (currency, coin, etc) as integer.

That not only eliminates the offers of 0.001 BTC, it also forces larger offers for larger numbers of decimals of price granularity. For example you can offer 1 BTC for 1, 2, 3, 4, 5 etc of something else, but to do fractions you have to make a larger offer, such as 10 BTC for 11, 12, 13 etc of something else, or 100 BTC for 101, 102, 103 etc of something else.

Comments on that idea are welcome. It is what I am actually currently doing but will anyone actually use such an exchange?

Its engine seems to tick every few seconds by the way; I suppose the rate of ticking could be adjusted too if need be.

-MarkM-

[Steve]

I think automated trading is good, but what's not desirable is the arms race for better and better connectivity and hardware to enable an ever increasing frequency of trading.  When I think about the big trading companies pouring tons of money into high speed hardware co-located with the matching engines…and paying very high rental rates for colo space next to exchanges, all I think about is waste.  They are pouring a lot of money into gaining an advantage over the competitors simply because of a design decision in the matching engine (continuous matching).  With a discrete time matching engine, you can eliminate the advantage to be had from high speed, low latency connections to the matching engine.  Make the time period long enough (and I think 1 second is plenty long enough) and you level the playing field for trading bots.  They don't need to be co-located near the matching engine because it won't give them any special advantage.  Some people might say that this wouldn't work because the automated traders couldn't make as much money, however, when the average traders realize that it's better for them to trade in a pool where there's a more level field, all the people that the big HFT take advantage of to make their money will simply be gone.  The exchange offering discrete trading will have an advantage over continuous exchanges because they'll attract more traders interested in the better designed platform.

Also, one thing that might be beneficial is if the tools and algorithms for automated trading were more readily available.  This would allow more people to throw a little extra money they have into a bot that essentially helps bring stability to the price of bitcoin.

[DeathAndTaxes]

You are right Steve but exchanges get paid on volume.  As HFT firms try to out bankrupt each other they rack up massive volume and that means lots of money for the exchanges.  So exchanges while they could level the playing field have absolutely no incentive to do so.

[tvbcof]

To the limited extent that I understand things, I'm all for (or at least, not against) high frequency trading, but front-running is an entirely different matter.

One of the things I would be looking for in an exchange is that they allow orders to match before printing them to tape.  I sense that exchanges are sensitive about the time it takes for orders to be withdrawn and their book engines are optimized for this, but (IMHO) it would be at least negligent to provide actionable information on in-the-money buys in time for sells to be retracted (or vice versa).

[Fireball]

I think automated trading is good, but what's not desirable is the arms race for better and better connectivity and hardware to enable an ever increasing frequency of trading.

But it's their race, which does not affect non-HFT players that much. HFT traders affect the rate, but they don't define it. Actual demand and offer define the rate.