Bitcoin Forum
December 04, 2016, 04:18:19 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 [19] 20 21 »  All
  Print  
Author Topic: [BETA] MTGox websocket API, testers wanted  (Read 72199 times)
pescador
Newbie
*
Offline Offline

Activity: 14



View Profile
April 28, 2013, 09:02:40 PM
 #361

Are you talking about trades in other currencies or about all trades? Because I'm pretty sure that you have to update your depth tables when a trade message comes in as is explained here https://bitcointalk.org/index.php?topic=5855.msg1636817#msg1636817. I do that and it seems to be working fine. I only don't know what to do with trades in other currencies.

I'm already using the trick to use ticker messages to fix the tables when they appear to be broken, but that doesn't happen a lot.
1480825099
Hero Member
*
Offline Offline

Posts: 1480825099

View Profile Personal Message (Offline)

Ignore
1480825099
Reply with quote  #2

1480825099
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480825099
Hero Member
*
Offline Offline

Posts: 1480825099

View Profile Personal Message (Offline)

Ignore
1480825099
Reply with quote  #2

1480825099
Report to moderator
1480825099
Hero Member
*
Offline Offline

Posts: 1480825099

View Profile Personal Message (Offline)

Ignore
1480825099
Reply with quote  #2

1480825099
Report to moderator
1480825099
Hero Member
*
Offline Offline

Posts: 1480825099

View Profile Personal Message (Offline)

Ignore
1480825099
Reply with quote  #2

1480825099
Report to moderator
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
April 29, 2013, 05:49:31 AM
 #362

strange I currently use only depth and ticker messages. and did not have problems with orderbook state.

on my opinion there is not need to use trade messages because some trades can be from orders which are not in orderbook but executed before getting into orderbook.

but probably I missed something. It would be good to hear others

prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
April 29, 2013, 09:15:44 AM
 #363

Are you talking about trades in other currencies or about all trades? Because I'm pretty sure that you have to update your depth tables when a trade message comes in as is explained here https://bitcointalk.org/index.php?topic=5855.msg1636817#msg1636817
I did some further experimenting and debugging and it seems this old posting of mine is wrong. There *will* be a depth message immediately following the trade message, so theoretically it would be enough to use only depth messages to update the orderbook.

But in goxtool/goxapi I still update the orderbook already in the trade message, so a hypothetical goxtool bot that would rely on the *new* orderbook state *after* the trade has been executed would already find the orderbook in the new updated state from inside the slot_trade() slot. Since I only ever use the total_vol of the depth messages and ignore the volume difference in my implementation the depth messages that immediately follow a trade message will simply result in no change to the orderbook at all.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
pescador
Newbie
*
Offline Offline

Activity: 14



View Profile
April 29, 2013, 12:04:02 PM
 #364

Thanks you both for your answers!
alpet
Legendary
*
Offline Offline

Activity: 1535


View Profile WWW
May 04, 2013, 12:34:43 PM
 #365

How can make a subscription to my bot trades? Requesting for every order private/order/result is slow bad solution. I need recieve trades with tid, oid, price and volume fields, as soon as possible after order matched partally or full.

Cryptsy, BFL and Bitsyncom - scammers, don't work with them!
Плавайте поездами Аэрофлота.
prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
May 04, 2013, 03:11:35 PM
 #366

How can make a subscription to my bot trades? Requesting for every order private/order/result is slow bad solution. I need recieve trades with tid, oid, price and volume fields, as soon as possible after order matched partally or full.
There will be a trade message on the private channel (see streaming API docs) but I'm not sure if it has the order ID.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
alpet
Legendary
*
Offline Offline

Activity: 1535


View Profile WWW
May 05, 2013, 11:42:34 AM
 #367

There will be a trade message on the private channel (see streaming API docs) but I'm not sure if it has the order ID.

It is anonymous data stream, without orders association. So bad, protocol provoke for many additional requests Sad

Cryptsy, BFL and Bitsyncom - scammers, don't work with them!
Плавайте поездами Аэрофлота.
prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
May 05, 2013, 06:06:11 PM
 #368

There will be a trade message on the private channel (see streaming API docs) but I'm not sure if it has the order ID.

It is anonymous data stream, without orders association. So bad, protocol provoke for many additional requests Sad

You can also trigger your actions in the user_order message for order removal, this message will be fired when the order is completely filled or the last remaining part of it is filled (and therefore the order is removed). Problem here is: it will also fire when you cancel the order. So you need some additional code that knows what was going on before. If your bot is the only thing that ever cancels orders then will know the reason for the remove message but if you also cancel orders manually or through other bots running on the same account then you need somethng more sophisticated, then you need to keep track of the private trade messages too and if a trade message at the same price preceded the the order remove than it is a filled order, otherwise its a cancel or something like that.

The API clearly was not designed by someone who has ever written a trading bot before, its missing needed information in a lot of places and forces you to do cumbersome workarounds to detect certain events or keep track of the information you need, and at the same time all the json messages are bloated with totally useless redundant data that nobody ever uses.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
alpet
Legendary
*
Offline Offline

Activity: 1535


View Profile WWW
May 06, 2013, 06:41:27 AM
 #369

The API clearly was not designed by someone who has ever written a trading bot before, its missing needed information in a lot of places and forces you to do cumbersome workarounds to detect certain events or keep track of the information you need, and at the same time all the json messages are bloated with totally useless redundant data that nobody ever uses.

I Agree, MtGox API need strongly to improve. Current realisation can be declared only as "not real-time". Extreme event lags! Some time required tens seconds, from send order to retrive all trades, what make API not usable for fast trailing stops and many others. Since my trading program cannot normal work with MtGox, I just switch to Icbit.

Cryptsy, BFL and Bitsyncom - scammers, don't work with them!
Плавайте поездами Аэрофлота.
yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
May 27, 2013, 09:18:59 PM
 #370

I'm trying to subscribe to "lag" channel on socketIO:

Tried sending this:

{
    "channel":"85174711-be64-4de1-b783-0628995d7914",
    "op":"subscribe"
}


and without quotes around channel ID like this:

{
    "channel":"85174711-be64-4de1-b783-0628995d7914",
    "op":"subscribe"
}


But whatever I try there is no response, and certainly no subscription success response.

My Socket.IO TX framing is correct, if I break the framing the mtgox server drops connection.

I'm not able to subscribe/unsubscribe any channels dynamically.

To add/remove currencies I have to reset connection with new list of currency codes in the initial HTTP exchange.

I wonder what am I doing wrong?

genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
June 01, 2013, 04:09:10 PM
 #371

Hi, all!

We at bitcoin-analytics.com support local state of the orderbook and update it listening depth messages and fix obvious invalid state of orderbook ( when ask price is less than bid price) using ticker messages.

While fixing with ticker messages helps, it seems that it is not enough. Invalid orderbook state occures because some depth mesages does not get to the client from server. It is not so obvious if only you do not maintain history of the orderbook volume which we do on bitcoin-analytics.com.

What we noticed a repeting pattern like on the screenhot.



It looks like the volume increased steadily during several hours and then abruptly reduced. This happend exact at the moment when orderbook was just reinitialized.

The problem as we see it is the following: during the long period of time some depth messages does not come to client but do not invoke fixing procedure cause those lost messages does not break the validity of the orderbook state. But those lost messages did not remove the orders which they had to remove or change. During the period of time new orders were added to the orderbook. Thus the volume was increased while errors of lost messages were accumulated.

Did anyone noticed something similar. And does anyone has any ideas about this problem? How is it possible to detect those lost messages orders and fix them?

thanks

Clark
Hero Member
*****
Offline Offline

Activity: 540


So much code.


View Profile WWW
June 03, 2013, 04:51:56 PM
 #372


Did anyone noticed something similar. And does anyone has any ideas about this problem? How is it possible to detect those lost messages orders and fix them?


Well there was a $1M bid wall that got removed right before the selling started yesterday. I don't believe that large drop was an error.

PGP KEY | 1Bitcoin3Tg2KWyAq3wzivdqwYqGwKYaGd
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
June 03, 2013, 05:35:38 PM
 #373

that is the question,

there was no any corresponding trading at that time.


molecular
Donator
Legendary
*
Offline Offline

Activity: 2128



View Profile
June 03, 2013, 09:54:50 PM
 #374

Hi, all!

We at bitcoin-analytics.com support local state of the orderbook and update it listening depth messages and fix obvious invalid state of orderbook ( when ask price is less than bid price) using ticker messages.

While fixing with ticker messages helps, it seems that it is not enough. Invalid orderbook state occures because some depth mesages does not get to the client from server. It is not so obvious if only you do not maintain history of the orderbook volume which we do on bitcoin-analytics.com.

What we noticed a repeting pattern like on the screenhot.



It looks like the volume increased steadily during several hours and then abruptly reduced. This happend exact at the moment when orderbook was just reinitialized.

The problem as we see it is the following: during the long period of time some depth messages does not come to client but do not invoke fixing procedure cause those lost messages does not break the validity of the orderbook state. But those lost messages did not remove the orders which they had to remove or change. During the period of time new orders were added to the orderbook. Thus the volume was increased while errors of lost messages were accumulated.

Did anyone noticed something similar. And does anyone has any ideas about this problem? How is it possible to detect those lost messages orders and fix them?

thanks

I've been having massive problems keeping order book synced in my client, there just seem to be lost messages. Never got it right.

Last resort: just pull full-depth 5 times per hour (magicaltux allows that frequency) and sync your orderbook

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
June 04, 2013, 08:43:46 AM
 #375

I agree, and wanted to add that this noticed affect is further suspicious becuase it is visible for all 8 currecly orderbooks at the same moment.

Just to remind that we support state of 8 currency orderbooks in the sime time. And this effect can be seen for all of them synchronosouly.

Needless to say that for such currencies like GPB or RUB here are significantly less trading activity which could be account for those simultanous volume decreases.

hm

Eisenhower34
Legendary
*
Offline Offline

Activity: 898



View Profile
June 04, 2013, 09:08:07 AM
 #376

Most of you are writing a trading bot, so Im sorry, but why do you all need full-depth? depth/fetch works fine and you can pull the data like every 30sec...
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
June 04, 2013, 09:09:23 AM
 #377

see bitcoin-analytics.com

prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
June 04, 2013, 01:09:33 PM
 #378

I agree, and wanted to add that this noticed affect is further suspicious becuase it is visible for all 8 currecly orderbooks at the same moment.

MtGox has only one orderbook. Every order you add in one currency will also appear in every other currency (bids will appear 2.5% lower in other currencies, asks will appear 2.5% higher in other currencies).

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
June 04, 2013, 02:19:14 PM
 #379

As long as gox provides separate http requests for full orderbooks for different currencies and allows socket.io subscribtion to multiple currenclies sending depth messages for each currecy in a separate channel this allows us to support orderbook state for each currency separately as if they are independent.

Observed lost depth messages effect is visible not only for one crrency orderbook but for all of them synchronously. We do not know what is the cause of the problem. That fact that all separate currency orderbooks are produced from the one majore orderbook is understood. The thing is that no matter is there only one major orderbook or not, the mechanism of depth messages itself is broken and not all depth messages reach the client.

This is suported by the obvious neccessity of fixing invalid orderbook state (when best bid price is greater than best ask price) against ticker messages, but also by observing the effect of accumulated old (not removed) orders which lead to increase in orderbook volume through time and significant descrease immediately after orderbooks state was reinitialized.


prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
June 04, 2013, 07:15:25 PM
 #380

We do not know what is the cause of the problem. That fact that all separate currency orderbooks are produced from the one majore orderbook is understood.

I mentioned this only as an explanation of why the problem is observed synchronously in all currencies. MT once mentioned in #mtgox that it might be due to occasional database rollbacks that can happen *after* the depth message has been sent out already (its sending out the depth mesages before the database commit happens and no inverse depth mesage is sent when it must be rolled back for whatever reason).

I am observing the same problem in goxtool and have no solution for it. Also fulldepth is cached a few minutes by the server, so simply reloading fulldepth every x minutes will also lead to some sort of inconsistency, especially near the top of the book where most of the action is and where its most important for traders and bots. I have no idea how to properly and cleanly fix this mess client side, probably nothing can be done about it at all.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 [19] 20 21 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!