nitrous (OP)
|
|
January 07, 2014, 03:58:26 PM |
|
after searching around a bit (Wikipedia... where I should have started...) I understood your explanation a bit better, what I did not get is that URL does not get encrypted as well when using SSL...
thanks for your explanations !
No problem The URL should also get encrypted, but it is also stored in server logs. It is certainly possible to work around all these things, but with all the things you'd have to work around you'd probably miss something or introduce a bug. Simply put, it's unwise to use GET when passing secure information, and far easier to use POST, whose attributes make perfect sense for use with SSL. Hmm, I haven't used those APIs but if you're right, that's really surprising. I wouldn't expect such large companies to use GET for secure requests, especially Amazon where money is involved. EDIT: Another important point, though obviously not a factor with a simple text API, is that on a webpage any third-party resources that are accessed will be sent a referrer url, which is the url of the webpage (query string and all!).
|
|
|
|
wilfried
Sr. Member
Offline
Activity: 288
Merit: 250
ManualMiner
|
|
January 08, 2014, 07:47:17 PM |
|
hy nitrous!
what do you consider the best method to always get the last trade (every latest trade)?
i tried (python):
get the latest time
letztertid=str(int(time.time()*1000000));
fetch trades since that
x=req('BTCUSD/money/trades/fetch?since='+jetzt, {}, True)
take the latest trade from that result and repeat.
but its lagging 3-4 trades. if i try something like that
letztertid=str(int(time.time()*1000000-864000));
it shows one lates trade and then lags some trades and then shows another latest trade.
i also tried to start with time*1000000, take the tid of the latest trade and fetch all trades since that, but that doesnt return no data (i think on your website you describe this as beeing some kind of ddos protection).
what is the minimum x i can subtract from time? thx!
|
|
|
|
nitrous (OP)
|
|
January 09, 2014, 12:17:18 PM |
|
hy nitrous!
what do you consider the best method to always get the last trade (every latest trade)?
i tried (python):
get the latest time
letztertid=str(int(time.time()*1000000));
fetch trades since that
x=req('BTCUSD/money/trades/fetch?since='+jetzt, {}, True)
take the latest trade from that result and repeat.
but its lagging 3-4 trades. if i try something like that
letztertid=str(int(time.time()*1000000-864000));
it shows one lates trade and then lags some trades and then shows another latest trade.
i also tried to start with time*1000000, take the tid of the latest trade and fetch all trades since that, but that doesnt return no data (i think on your website you describe this as beeing some kind of ddos protection).
what is the minimum x i can subtract from time? thx!
I'm not quite sure what behaviour you're seeing... Visiting BTCUSD/money/trades/fetch, the last few trades had the following tids: 1. 1389268906937622 2. 1389268907156855 3. 1389268907286795 4. 1389268919402914 5. 1389268919674069
Then going to fetch?since=1389268906937622, ie. the first one, gives: 2. 1389268907156855 3. 1389268907286795 4. 1389268919402914 5. 1389268919674069 6. 1389269081502565 7. 1389269099578993 8. 1389269100231335 9. 1389269128381212
It returned all trades after the tid I specified, as expected. If I subtract 1 (this should be the minimum you need to use for x) from tid#1, fetch?since=1389268906937621, then the results also include the first trade as well: 1. 1389268906937622 2. 1389268907156855 3. 1389268907286795 4. 1389268919402914 5. 1389268919674069 6. 1389269081502565 7. 1389269099578993 8. 1389269100231335 9. 1389269128381212
Also, you probably shouldn't use time.time() as it's very possible there won't be any trades that recent. On your first request, just call BTCUSD/money/trades/fetch instead, then start using the last trades from the last results set. Anyway, I don't seem to be seeing your problem, all the results I got were exactly as I expected, what do you mean by lagging 3-4 trades?
|
|
|
|
wilfried
Sr. Member
Offline
Activity: 288
Merit: 250
ManualMiner
|
|
January 09, 2014, 01:26:16 PM |
|
hy! thx for your help, i will test that out with -1. i was comparing bitcoinwisdom with the output of my script and as my script only displayed the last tid, bitcoinwisdom showed some trades in between that. but with -1 i didnt yet experiment.
|
|
|
|
wilfried
Sr. Member
Offline
Activity: 288
Merit: 250
ManualMiner
|
|
January 12, 2014, 07:13:04 PM |
|
once=str(int(time.time()*1000000-1)) works like a charm next thing i need is the orderbook. all i found is /money/depth/fetch but i think i might run into ddos protection, cause what i want to do, is to closely watch the orderbook max 6 btc away from the last price, so i would have to update it at least after each new trade. websocket isnt for me.. do you have a tip for me? thx!
|
|
|
|
MusX
|
|
January 12, 2014, 07:41:32 PM |
|
once=str(int(time.time()*1000000-1)) works like a charm next thing i need is the orderbook. all i found is /money/depth/fetch but i think i might run into ddos protection, cause what i want to do, is to closely watch the orderbook max 6 btc away from the last price, so i would have to update it at least after each new trade. websocket isnt for me.. do you have a tip for me? thx! there is a 'quote' method on mtgox - check it - as I remember it works only one way, not sure which thread about mtgox market depth (much more than 6btc): https://bitcointalk.org/index.php?topic=194868
|
|
|
|
nitrous (OP)
|
|
January 15, 2014, 01:12:00 PM |
|
once=str(int(time.time()*1000000-1)) works like a charm next thing i need is the orderbook. all i found is /money/depth/fetch but i think i might run into ddos protection, cause what i want to do, is to closely watch the orderbook max 6 btc away from the last price, so i would have to update it at least after each new trade. websocket isnt for me.. do you have a tip for me? thx! Glad it works Sorry, unfortunately the only way to maintain a realtime copy of the orderbook is with the websocket - there is a maximum of 5 requests per hour for /money/depth/fetch, so it is simply impossible to call it any more often than once per 12 minutes, which I assume is not good enough for your purposes. If you can't use the websocket, you may want to reconsider your timescale or strategy. (What is wrong with using the websocket though? I've successfully used the websocket in python using the websocket-client package). The thread posted by MusX says the same thing.
|
|
|
|
|
wilfried
Sr. Member
Offline
Activity: 288
Merit: 250
ManualMiner
|
|
January 18, 2014, 07:30:57 PM |
|
ok, here it is for all python-noobs as me - for starters: #!/usr/bin/python import websocket import thread import time
def on_message(ws, message): print 'ola, messages incoming'; print message; print 'endofmessagedude';
def on_error(ws, error): print 'ola, error incoming:'; print error;
def on_close(ws): print "### closed ###";
def fuckitin(ws): def run(*args): #for i in range(5): # time.sleep(1) # ws.send("1") #ws.send("Hello %d" % i) #time.sleep(1) #ws.send(''); ws.close() print "thread terminating..." thread.start_new_thread(run, ())
if __name__ == "__main__":
websocket.enableTrace(True) ws = websocket.WebSocketApp("wss://websocket.mtgox.com:443/mtgox?Currency=USD", on_message = on_message, on_error = on_error, on_close = on_close) #ws.on_open = fuckitin
ws.run_forever()
how do i tell him which channel i want?
|
|
|
|
nitrous (OP)
|
|
January 19, 2014, 12:27:08 AM |
|
ok, here it is for all python-noobs as me - for starters: #!/usr/bin/python import websocket import thread import time
def on_message(ws, message): print 'ola, messages incoming'; print message; print 'endofmessagedude';
def on_error(ws, error): print 'ola, error incoming:'; print error;
def on_close(ws): print "### closed ###";
def fuckitin(ws): def run(*args): #for i in range(5): # time.sleep(1) # ws.send("1") #ws.send("Hello %d" % i) #time.sleep(1) #ws.send(''); ws.close() print "thread terminating..." thread.start_new_thread(run, ())
if __name__ == "__main__":
websocket.enableTrace(True) ws = websocket.WebSocketApp("wss://websocket.mtgox.com:443/mtgox?Currency=USD", on_message = on_message, on_error = on_error, on_close = on_close) #ws.on_open = fuckitin
ws.run_forever()
how do i tell him which channel i want? There are different ways depending on the type of channel, try sending the following: { "op": "mtgox.subscribe", "channel": "channel-id" }
where channel-id is one from http://data.mtgox.com/api/2/stream/list_public. You can do this as a python object, and then use json.dumps (import the json module). Have a look here for more information on websocket methods :- https://en.bitcoin.it/wiki/MtGox/API/Streaming. There's some conflicting information, but it should be helpful.
|
|
|
|
wilfried
Sr. Member
Offline
Activity: 288
Merit: 250
ManualMiner
|
|
January 19, 2014, 12:29:34 PM |
|
thx again, your tips are really helpful in principle i can do what i want now. i can subscribe to any channel listed on https://mtgox.com/api/2/stream/list_publicwith this: message={ "op":"mtgox.subscribe", "channel":"trade.lag" } ws.send(json.dumps(message))
or i can have a single channel opened by just modifying the uri and not doing any send: ws = websocket.WebSocketApp("wss://websocket.mtgox.com:443/?Channel=trade.BTC", on_message = on_message, on_error = on_error, on_close = on_close)
(btw that was in the comments of prof7bits goxapi.py, line 1137, https://github.com/prof7bit/goxtool/blob/master/goxapi.py) so, the ws: uri is important, if calling wss://websocket.mtgox.com:443/mtgox?Currency=USD" i can subscribe to additional channels and always get btc.depth and btc.trades but cant unsubscribe to btc.depth and btc.trades. if using wss://websocket.mtgox.com:443 i get "{\"message\":\"Now online (no channels)\",\"op\":\"remark\"}" and can subscribe to any channel. so i think unsubscribe may be only usable to unsubscribe channels one subscribed but not for avoiding messages like btc.trades that result from calling wss://websocket.mtgox.com:443/mtgox?Currency=USD"
|
|
|
|
nitrous (OP)
|
|
January 19, 2014, 02:35:52 PM |
|
so, the ws: uri is important, if calling wss://websocket.mtgox.com:443/mtgox?Currency=USD" i can subscribe to additional channels and always get btc.depth and btc.trades but cant unsubscribe to btc.depth and btc.trades.
if using wss://websocket.mtgox.com:443 i get "{\"message\":\"Now online (no channels)\",\"op\":\"remark\"}" and can subscribe to any channel.
so i think unsubscribe may be only usable to unsubscribe channels one subscribed but not for avoiding messages like btc.trades that result from calling wss://websocket.mtgox.com:443/mtgox?Currency=USD"
To unsubscribe, try using { "op": "unsubscribe", "channel": "id" }
If I remember correctly, I think the id has to be the long guid code instead of the channel name when you unsubscribe.
|
|
|
|
wilfried
Sr. Member
Offline
Activity: 288
Merit: 250
ManualMiner
|
|
February 01, 2014, 10:48:54 AM |
|
hy, when receiving ticker data by websocket, is there another possibility to get the volume of the last_local trade other than taking the difference of the actual volume to the subsequent volume? thx!
|
|
|
|
BitcoinPappi
Legendary
Offline
Activity: 915
Merit: 1005
|
|
February 13, 2014, 11:07:42 AM |
|
Hi Guys Guidance on Calculating Depth Or Order Book. From what I understand so far. 1. A initial HTTP request has to be made to get the full order book ( https://data.mtgox.com/api/2/BTCUSD/money/depth/fetch) 2.After receiving the full order book. You can then calculate the changes using websocket updates a soon below. My objective is to obtain and keep track of the order book, what is the best way to do this. Are the above steps the correct way ? Are there some factor or calculations that need to be made to get the actual volume. "total_volume_int" is too large and "volume" is deprecated. {
"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe", "channel_name":"depth.BTCUSD", "op":"private", "origin":"broadcast", "private":"depth", "depth":{ "price":"511.55897", "type":1, "type_str":"ask", "volume":"10.5", "price_int":"51155897", "volume_int":"1050000000", "item":"BTC", "currency":"USD", "now":"1392264454601642", "total_volume_int":"1050000000" }, "stamp":1392264454602040
}
|
|
|
|
wilfried
Sr. Member
Offline
Activity: 288
Merit: 250
ManualMiner
|
|
February 13, 2014, 11:41:15 AM |
|
i use vol_int: if its 0 i delete the order from my db, if its not 0 i replace every order where price matches; but you need to delete trades from the orderbook too
|
|
|
|
BitcoinPappi
Legendary
Offline
Activity: 915
Merit: 1005
|
|
February 13, 2014, 02:26:42 PM |
|
i use vol_int: if its 0 i delete the order from my db, if its not 0 i replace every order where price matches; but you need to delete trades from the orderbook too
Thanks. Can you tell me what volume_int represents. Looking on other charts and third party order book data like bitcoincharts. The value that comes back is way too large to be the volume at that single price. I am missing something in understanding what is being returned
|
|
|
|
BitcoinPappi
Legendary
Offline
Activity: 915
Merit: 1005
|
|
February 14, 2014, 01:03:49 AM |
|
i use vol_int: if its 0 i delete the order from my db, if its not 0 i replace every order where price matches; but you need to delete trades from the orderbook too
Thanks. Can you tell me what volume_int represents. Looking on other charts and third party order book data like bitcoincharts. The value that comes back is way too large to be the volume at that single price. I am missing something in understanding what is being returned So volume_int and total_volume_int seem to be bigger by a factor of 1E8 and price_int bigger by factor of 1E6 . Now that the numbers make sense, I can work on implementation
|
|
|
|
BitcoinPappi
Legendary
Offline
Activity: 915
Merit: 1005
|
|
February 14, 2014, 03:39:55 AM |
|
For some reason I get the following when i try to subscribe to channels. I get unknown command "op" :s {"debug":{"data":{"channel":"d5f06780-30a8-4a48-a2f8-7ed181b4a13f","op":"subscribe"},"data_raw":"{\"channel\": \"d5f06780-30a8-4a48-a2f8-7ed181b4a13f\", \"op\": \"subscribe\"}","op":"client","uuid":"4a1f761a-4f50-4a21-91ff-e9037a2fa91f"},"message":"Unknown command","op":"remark","success":false}
|
|
|
|
BitcoinPappi
Legendary
Offline
Activity: 915
Merit: 1005
|
|
February 14, 2014, 03:53:30 AM |
|
For some reason I get the following when i try to subscribe to channels. I get unknown command "op" :s {"debug":{"data":{"channel":"d5f06780-30a8-4a48-a2f8-7ed181b4a13f","op":"subscribe"},"data_raw":"{\"channel\": \"d5f06780-30a8-4a48-a2f8-7ed181b4a13f\", \"op\": \"subscribe\"}","op":"client","uuid":"4a1f761a-4f50-4a21-91ff-e9037a2fa91f"},"message":"Unknown command","op":"remark","success":false}
So it appears that you can only unsubscribe from channels when using the wss://websocket.mtgox.com:443/mtgox endpoint. Just unsubscribe from the channels your not interested in vs choosing which one to subscribe to since it will give the exception above.
|
|
|
|
|