[MtGox] -- negative amount while initializing market image

[xand]

Sorry for posting here, but it's the only section I'm allowed to.

When working with MtGox market depth I get some positions with negative amount.
How is that possible?

My initilization protocol is the following:

1. I start listening for incoming messages via SocketIO.

2. I buffer all messages and I wait for about 2 minutes, after that I have a collection of buffered messages.

3. I download market depth from https://mtgox.com/api/1/BTCUSD/fulldepth and I process it.

4. After that I process all the messages from step 2 comparing the timestamp for position. If the timestamp from step 2 message is lower than the one from step 3 I just ignore the message.

Anyway, after doing the initialization I still having some positions with negative amount.

For example I see the following log:

194214,452|ERROR|com.xand.bt.market.MarketStore:66|Level (ASK) for price 1271752 has negative volume!!!

So, I go to my market log and see what happened to price 1271752 on ASK leg.

I see that on step 2 I received the following message:

[marketLabel=MTGOX; direction=ASK; price=1271752; volume=-5956000; time=29/07/2012 19:40:44,391]

But I didn't receive that position when on step 3.

Ho is that possible?

[1715197514]

1715197514

[1715197514]

1715197514

[1715197514]

1715197514

[adubs]

I'm having the exact same problem, did you ever get this resolved?

Also of note, i get updates from the socket connection that have a amount of 0, which doesn't make any sense to me...

[gllen]

I had the same issue about a month ago when I tried working with the socket io feed.

I just gave up after a while. At least when I was working on it, the timestamps on the depth were completely out of whack with the timestamps from socketio and there seemed to be no good way to ensure your live book is 100% accurate.

[davout]

I'm interested in your potential findings

[adubs]

related: what's the difference between https://mtgox.com/api/1/BTCUSD/depth and https://mtgox.com/api/1/BTCUSD/fulldepth ?

what does fulldepth show that depth doesn't?

[adubs]

also, i'm slowing piecing together each of the bits in this feed. from https://bitcointalk.org/index.php?topic=61293.0

so the timestamp on the http api is for the most recent order remaining on the book at that price. That explains the discrepancy (of volumes changing w/o the timestamp changing).

[adubs]

My solution is to use /depth instead of /fulldepth, and to use the amount_int fields instead of the floating point fields for the realtime depth.

my current strategy is to:
1. connect to realtime feed and cache messages

2. load data from /depth

3. replay data from #1 and update my local store of the depth

4. new realtime messages update the store just like #3

5. i continue to refresh /depth every 15s to verify that my local cache matches the REST depth, and i'm 100% in sync now. there are some small issues where the /depth will see a change before the realtime feed, or vice-versa, but the realtime feed always catches up and things are back in sync w/in 10s (even much sooner)

thanks for this thread + forums! i'll go back to lurking now  

[edit] I'm using the total_amount_int fields, not the amount_int fields. and i just update the record entirely for that price - not bothering trying to sum with amount_int. There were too many times when the message would give 0 for the amount_int, even though /depth would say that the amount changed around that timestamp.

[xand]

Sorry guys, I was out of the town so I didn't see the replies.

Of course it's more accurate to use _int fields instead of float.

I will try the adubs's solution.