|
Title: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 16, 2012, 07:13:53 AM So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info.
The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed. It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended. Also what is the forum code to make my links like this (html example): Code: <a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a> Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: Revalin on February 16, 2012, 07:26:16 AM Step 1: Fetch https://mtgox.com/api/0/data/getDepth.php?Currency=USD
Step 2: save it somewhere. Repeat as desired. For the link: Code: [url=https://bitcointalk.org/index.php?topic=6019.0]sierrachartfeed-0.4[/url] Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 16, 2012, 07:35:01 AM Thanks, but how do I expand the range?
Also, I would like to automate this but maybe that is too annoying of a question to ask. If noone wants to help with that I understand, I will return when I have messed around in the commandline and with some scripts. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: Revalin on February 16, 2012, 08:05:21 AM You don't. You get the depth as of now, and only a slice around the current price.
To automate it on Unix, create a cron job like this: Code: * * * * * curl https://mtgox.com/api/0/data/getDepth.php > `date +mtgox-%F--%T` That will dump it to a new file once a minute. Better find another way to store it before you get buried in files. :) Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 16, 2012, 08:15:50 AM I don't think I emphasized my ignorance enough. Thank you for your help though. Once I get to the point where I know what you are talking about, it will be useful.
Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: Revalin on February 16, 2012, 08:36:48 AM Sooner or later you'll have to learn a lot before you can do something useful with that much data. :)
But let me give you some leads to explain what that last thing meant. Cron is the task scheduler for Unix. Lots about that here: https://en.wikipedia.org/wiki/Cron The * * * * * means to run something every minute. curl is a simple http client. It grabs the web page and spits it to stdout. https://en.wikipedia.org/wiki/CURL (https://en.wikipedia.org/wiki/CURL) > means to redirect stdout to a file. putting stuff in backticks ` ` means to run the command inside and then put the result on the command line. running "date +mtgox-%F--%T" outputs: mtgox-2012-02-16--00:30:53 So the actual command that gets run looks like: curl https://mtgox.com/api/0/data/getDepth.php > mtgox-2012-02-16--00:30:53 To set up the cron job, run "crontab -e". An editor will come up, and you paste in the line: * * * * * curl -q https://mtgox.com/api/0/data/getDepth.php > `date +mtgox-%F--%T` (I added a -q so curl won't output progress messages on stderr; otherwise they would be emailed to you with every run) Go install Linux somewhere (you can install it in a VirtualBox VM if you don't have a spare computer), try it out, and let me know if you get stuck. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 17, 2012, 10:58:33 AM So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info. The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed. It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended. Also what is the forum code to make my links like this (html example): Code: <a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a> What kind of analysis do you want to do? Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 17, 2012, 12:02:04 PM So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info. The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed. It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended. Also what is the forum code to make my links like this (html example): Code: <a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a> What kind of analysis do you want to do? I'm not positive yet. I just love input/data. In general, look for patterns in the behavior of the price and order book before large movements. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: TTBit on February 17, 2012, 01:06:35 PM Sooner or later you'll have to learn a lot before you can do something useful with that much data. :) But let me give you some leads to explain what that last thing meant. Cron is the task scheduler for Unix. Lots about that here: https://en.wikipedia.org/wiki/Cron The * * * * * means to run something every minute. curl is a simple http client. It grabs the web page and spits it to stdout. https://en.wikipedia.org/wiki/CURL (https://en.wikipedia.org/wiki/CURL) > means to redirect stdout to a file. putting stuff in backticks ` ` means to run the command inside and then put the result on the command line. running "date +mtgox-%F--%T" outputs: mtgox-2012-02-16--00:30:53 So the actual command that gets run looks like: curl https://mtgox.com/api/0/data/getDepth.php > mtgox-2012-02-16--00:30:53 To set up the cron job, run "crontab -e". An editor will come up, and you paste in the line: * * * * * curl -q https://mtgox.com/api/0/data/getDepth.php > `date +mtgox-%F--%T` (I added a -q so curl won't output progress messages on stderr; otherwise they would be emailed to you with every run) Go install Linux somewhere (you can install it in a VirtualBox VM if you don't have a spare computer), try it out, and let me know if you get stuck. Thanks for doing this. I can read the man pages, but this is easier for me to understand. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 17, 2012, 01:07:03 PM So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info. The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed. It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended. Also what is the forum code to make my links like this (html example): Code: <a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a> What kind of analysis do you want to do? I'm not positive yet. I just love input/data. In general, look for patterns in the behavior of the price and order book before large movements. You should work with R (http://cran.r-project.org/bin/windows/base/) then. It was designed for analyzing large data sets. Once you open it up you have to install two packages. (I have not tested this in Windows, only in Linux and Mac). Code: install.packages("RCurl")To inspect the data you can do a lot of different things in R. There are a lot of user guides as R is an open source community. But to get started you can call the bid and ask objects by typing in R: Code: y$bid EDIT: fixed the typ. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 17, 2012, 01:18:11 PM Ah, excellent I have been messing around with R on and off over the last few months and have access to a few people with lots of experience. Thanks, I will let you know how it goes.
Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 17, 2012, 01:25:45 PM Ah, excellent I have been messing around with R on and off over the last few months and have access to a few people with lots of experience. Thanks, I will let you know how it goes. I translated Bitcoinica's whole API into R. You can get the code here (https://github.com/ktdservices/Rbitcoinica). I have been having trouble getting authentication to work in R. Let me know if those people with R experience are able to help. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 17, 2012, 01:43:49 PM Will do. I need to mess around for a bit though. By authentication you mean user and pass verification?
Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 17, 2012, 01:52:53 PM Will do. I need to mess around for a bit though. By authentication you mean user and pass verification? Yes, but MtGox uses a new authentication that requires a public and private key. It is not basic authentication like Bitcoinica uses. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 17, 2012, 02:06:37 PM How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob.
Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: Revalin on February 17, 2012, 06:51:25 PM How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob. Code: * * * * * curl -q https://mtgox.com/api/0/data/ticker.php > `date +mtgox-ticker-%F--%T` +1 for R. The language itself isn't anything special; languages like Python, Ruby and Perl are much more fun. The reason for R is it has an immense library of high-quality statistics functions. I suggest the rkward GUI to get started. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 17, 2012, 10:35:38 PM How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob. Code: * * * * * curl -q https://mtgox.com/api/0/data/ticker.php > `date +mtgox-ticker-%F--%T` +1 for R. The language itself isn't anything special; languages like Python, Ruby and Perl are much more fun. The reason for R is it has an immense library of high-quality statistics functions. I suggest the rkward GUI to get started. R also has libraries for financial indicator and strategy testers. See quantmod (available on CRAN) and quantstrat (http://blog.fosstrading.com/2011/08/introduction-to-quantstrat.html) (available on R-Forge). Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 18, 2012, 01:44:33 AM install.packages("RCurl") install.packates("RJSONIO") require(RCurl) require(RJSONIO) #### Make a function to get the MtGox depth, returns a list of the ask and bids in a data frame format MT.BTCUSDdepth = function ( ) { tickMTGXurlBTCUSD = "https://mtgox.com/api/0/data/getDepth.php?Currency=USD" y = fromJSON(getURLContent( tickMTGXurlBTCUSD, ssl.verifypeer = FALSE, useragent = "R") ) ask = as.data.frame(matrix(unlist(y$ask),ncol=2,byrow=T)) colnames(ask) = c("ask","amount") bid = as.data.frame(matrix(unlist(y$bid),ncol=2,byrow=T)) colnames(bid) = c("bid","amount") list(ask=ask,bid=bid) } y = MT.BTCUSDdepth() #returns the depth and stores it in the object 'y' BTW, I figured out why I had to install the RJSONIO package manually Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 18, 2012, 02:22:05 AM install.packages("RCurl") install.packates("RJSONIO") require(RCurl) require(RJSONIO) #### Make a function to get the MtGox depth, returns a list of the ask and bids in a data frame format MT.BTCUSDdepth = function ( ) { tickMTGXurlBTCUSD = "https://mtgox.com/api/0/data/getDepth.php?Currency=USD" y = fromJSON(getURLContent( tickMTGXurlBTCUSD, ssl.verifypeer = FALSE, useragent = "R") ) ask = as.data.frame(matrix(unlist(y$ask),ncol=2,byrow=T)) colnames(ask) = c("ask","amount") bid = as.data.frame(matrix(unlist(y$bid),ncol=2,byrow=T)) colnames(bid) = c("bid","amount") list(ask=ask,bid=bid) } y = MT.BTCUSDdepth() #returns the depth and stores it in the object 'y' BTW, I figured out why I had to install the RJSONIO package manually You can also use rjson. Either one is fine. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 18, 2012, 02:25:26 AM How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob. Code: MT.BTCUSD = function ( ) {This will return a data frame Code: high low avg vwap vol last_all last_local last buy sell If you ever make some money off of this, leave a tip. Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 18, 2012, 02:35:02 AM yea I figured that out. Right now I am trying to get the grab ticker "command" to repeat once a minute. Revalin's code didn't work for me. Then figure out how to only display the last price from the ticker.
Do you know why R crashes if I don't include useragent= "R"? *well I should say I don't really get the make.dataframe part yet Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 18, 2012, 02:54:42 AM yea I figured that out. Right now I am trying to get the grab ticker "command" to repeat once a minute. Revalin's code didn't work for me. Then figure out how to only display the last price from the ticker. Do you know why R crashes if I don't include useragent= "R"? *well I should say I don't really get the make.dataframe part yet Yea that is interesting about it crashing. There is a short thing about it on the RCurl FAQ (http://www.omegahat.org/RCurl/FAQ.html). Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 18, 2012, 03:01:35 AM Right now I am trying to get the grab ticker You need to set up a cron. I don't know how to do that in Windows. But in unix-like system it is easy. First you need to make an Rscript. This is just a file saying what you want R to do in a .R format. It would be cool to have it make a tweet with the current ticker. There is the twitteR package. I was playing around with this before. Also, you could try just have R run in a loop. Have R run the ticker function, print out the data, then have it wait x seconds and do it again. I think adding a tweet or email using a cron job is better. Let me know if you need help with the cron setup. to do the last price: Code: x <- MT.BTCUSD() Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: stochastic on February 20, 2012, 11:56:44 AM Here is a site that explains and also makes a cron job:
http://htmlbasix.com/crontab.shtml Title: Re: I Need a Newbs Noob Guide to Accessing the Mt Gox API Post by: bb113 on February 22, 2012, 08:55:05 AM Thanks, working on this in my spare time so it may be a bit. I will report back.
|