Bitcoin Forum

<MagicalTux> sturles: if you need realtime data, use socket.io
<molec> MagicalTux, "realtime" ??? ;-)
<MagicalTux> molec: socket.io feed is actually realtime
<molec> MagicalTux, so it's the engine that can't cope?
<MagicalTux> right now, yep
<MagicalTux> because it needs to do too many checks
<MagicalTux> we are working on changing the way all this works
<MagicalTux> but it needs a *lot* of changes
<molec> MagicalTux, I think you're doing something wrong. I can't see why it's hard... what's the reason?
<molec> MagicalTux, might be my ignorance, but I did quite a few dbs. I might be overlooking some problem,... maybe security-related?
<MagicalTux> molec: we need to change the way users place orders to solve this
<molec> MagicalTux, in what way?
<MagicalTux> disallow users to place orders they can't fill
<molec> MagicalTux, +1. oh...
<molec> MagicalTux, it's dawning on me
<MagicalTux> and disallow users to withdraw balance required for open orders
<molec> MagicalTux, you have to trickle down the whole order list to set invalid/open status
<molec> MagicalTux, go do it!
<MagicalTux> right now each time a user balance change (trade, etc) we need to recheck all his open orders
<molec> MagicalTux, I see. and that gives you concurrency issues like hell
<MagicalTux> we're working on solving this, but changing the way the site works need to be done right
<MagicalTux> so we're doing a lot of tests first
<molec> MagicalTux, I agree. don't haste
<MagicalTux> well
<MagicalTux> we need to haste, because current volumes cannot be handled anymore by current engine
<MagicalTux> we made it faster (no more 30min delays), but it still has troubles with large moves

Does it also mean that ask side is also working the same way? So this is also bearish?

With the transition of North American customers to Coinlab, are they still going to use the same trading engine (in Tokyo?)? If all these orders will be done on a new engine in the US, this could drastically reduce the pressure on the current system. If so, when will this transition happen?

The situation I described had bearish result (rising market, no more auto-opening of bids => perceptively less bids). The other way around has bullish result: in a crash or falling market no more auto-opening of asks => ask side looks less impressive.

So if I judge this right, it could dampen the moves (up and down).

But keep in mind: this is mainly by perception. If people judge this correctly, it should not have much impact at all.

EDIT: just thought again: of course the order-book will be thinner, maybe this could mean that rebounds will be even quicker.

Last night I finally understood what's with the gox engine lag:

I think that with proper data structures, one can link together the bids and asks of single user and immediately update "reserve" for the bid/ask after sell or buy. Funding and withdrawing are infrequent compared to trades. The structure can be held in memory, with commits to permanent storage. This should handle hundreds of trades per second, not a few, like it is now.

Of course, this requires to avoid using normal relational databases, queries upon them, PHP code in server, etc.

Also, the socket with trade data is sometimes overloaded, too. I am not a TCP/IP expert, but I cant see problem with cloning the data between multiple (in order of thousands) sockets open. This would solve all those disconnects on ClarkMoody, MtGoxLive, etc.

An exchange, which cannot provide reliable and non-lagged data for its users, is .. unusable. Like that graph at the top of mtgox.com. Isn't it ironic that to trade at MtGox, or just watch it, you need to use third party tools?

We need to pay attention especially to a rapid price movement scenario, when people will be trying to issue orders desperately, and may never bother to check on their balances, and Gox's UI irresponsiveness only adds salt to the injury, as people will try to issue more orders when they can't see their orders showing on the list. The more orders people issue, the greater the lag, which would make people more anxious and making more vain efforts, which in turn causes even more lag. This vicious closed-feedback loop causes the ridiculous 10-minutes lag we saw on a weekly basis.

An exchange, which cannot provide reliable and non-lagged data for its users, is .. unusable. Like that graph at the top of mtgox.com. Isn't it ironic that to trade at MtGox, or just watch it, you need to use third party tools?

Absolutely agree with you. I've been long enough to know that bitcoin and exchanges are not the same thing but I can't help to think that this market got too big for what the infrastructure really is. It's just not there yet.

I can't imagine them running 2 markets. That doesn't make sense.

They can (and should) just maintain a central order book where all orders present are sure do be fully covered (see my previous comment), and have the two "brokers" independantly insert orders into it.

Re-Calculating an accounts open-orders after each transaction is just NOT that big of a deal.  If the site was designed by anyone with even the remotest sense of distributed computing, it would be such a non-issue that this discussion wouldn't even be taking place.  A properly designed trading system could handle the current load of bitcoin trading and still twiddle it's electronic fingers 75% of the time.

If I just did a calculation with my ass, it's gonna make the complexity roughly O(m*log(n)) instead of O(log(n)), and m could be easily larger than log(n).

I suspect there are many badly-programmed bots relying on this out there.

Are there people actively taking advantage of this deficiency? Gox is  lagging even if there are only several orders being placed in total, WTF?

Not allowing users to place an order if they do not have the funds at the time the order is placed seems wrong to me. The order needs the funds only when it is executed, not before. Consider that Mt.Gox is there to fulfill its users' needs, not the other way around. Provided, of course, that the users pay appropriately for the service.

The user could have a combination of orders that works, even though not all orders are funded beforehand. There could be sufficient funds coming in before an order is executed. I see no need to check funding before an order is actually executed. I would not complain if they did that though. They can do what they like, as long as they don't alienate their users.

I do see a need though for a fixed base fee for transactions, like every other online broker has. If necessary, there could even be a base fee for placing an order or a punishment fee for trying to execute an unfunded order. It seems that Mt.Gox is wasting their limited computer power on lots of tiny nuisance orders. The whole bitcoin system is not designed for large numbers of tiny transactions. There is a block size limit, for example. Bitcoin is not a micro-payment system. This will become ever more obvious as bitcoin trade volume grows.

I see the need to check funding of order before they are displayed in the public order book. Unfunded order should not be in the book because they cannot be fulfilled.

well in stock and forex u cant advertise that you want to buy or sell a currency or a stock if you dont freaking owned it first

Not familiar with Gox.

Over at virtex, every bid or ask that you place puts that money into escrow. so if you don't have the money (ex. if I want to buy right now, but all my money is tied up in lower bids) I have to cancel some bids to free up the money.

From the sounds of it, this is not the way it works at Gox?

Is the escrow system just dumb? or is it something that Gox will have to implement to get around this problem? […]

what about naked short selling?

you cant do that no broker allow you to do that