Bitcoin Forum
November 23, 2017, 03:00:30 PM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: I Need a Newbs Noob Guide to Accessing the Mt Gox API  (Read 4418 times)
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 16, 2012, 07:13:53 AM
 #1

So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info.

The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed.

It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended.

Also what is the forum code to make my links like this (html example):
Code:
<a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a>
Join ICO Now A blockchain platform for effective freelancing
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1511449230
Hero Member
*
Offline Offline

Posts: 1511449230

View Profile Personal Message (Offline)

Ignore
1511449230
Reply with quote  #2

1511449230
Report to moderator
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
February 16, 2012, 07:26:16 AM
 #2

Step 1: Fetch https://mtgox.com/api/0/data/getDepth.php?Currency=USD

Step 2: save it somewhere.

Repeat as desired.


For the link:
Code:
[url=https://bitcointalk.org/index.php?topic=6019.0]sierrachartfeed-0.4[/url]

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 16, 2012, 07:35:01 AM
 #3

Thanks, but how do I expand the range?

Also, I would like to automate this but maybe that is too annoying of a question to ask. If noone wants to help with that I understand, I will return when I have messed around in the commandline and with some scripts.
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
February 16, 2012, 08:05:21 AM
 #4

You don't.  You get the depth as of now, and only a slice around the current price.

To automate it on Unix, create a cron job like this:

Code:
* * * * * curl https://mtgox.com/api/0/data/getDepth.php > `date +mtgox-%F--%T`

That will dump it to a new file once a minute.  Better find another way to store it before you get buried in files.  Smiley

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 16, 2012, 08:15:50 AM
 #5

I don't think I emphasized my ignorance enough. Thank you for your help though. Once I get to the point where I know what you are talking about, it will be useful.
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
February 16, 2012, 08:36:48 AM
 #6

Sooner or later you'll have to learn a lot before you can do something useful with that much data.  Smiley

But let me give you some leads to explain what that last thing meant.

Cron is the task scheduler for Unix.  Lots about that here: https://en.wikipedia.org/wiki/Cron

The * * * * * means to run something every minute.

curl is a simple http client.  It grabs the web page and spits it to stdout.  https://en.wikipedia.org/wiki/CURL

> means to redirect stdout to a file.

putting stuff in backticks ` ` means to run the command inside and then put the result on the command line.

running "date +mtgox-%F--%T" outputs:  mtgox-2012-02-16--00:30:53

So the actual command that gets run looks like:

curl https://mtgox.com/api/0/data/getDepth.php > mtgox-2012-02-16--00:30:53

To set up the cron job, run "crontab -e".  An editor will come up, and you paste in the line:

* * * * * curl -q https://mtgox.com/api/0/data/getDepth.php > `date +mtgox-%F--%T`

(I added a -q so curl won't output progress messages on stderr; otherwise they would be emailed to you with every run)

Go install Linux somewhere (you can install it in a VirtualBox VM if you don't have a spare computer), try it out, and let me know if you get stuck.

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
stochastic
Hero Member
*****
Offline Offline

Activity: 532


View Profile
February 17, 2012, 10:58:33 AM
 #7

So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info.

The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed.

It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended.

Also what is the forum code to make my links like this (html example):
Code:
<a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a>

What kind of analysis do you want to do?

Introducing constraints to the economy only serves to limit what can be economical.
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 17, 2012, 12:02:04 PM
 #8

So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info.

The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed.

It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended.

Also what is the forum code to make my links like this (html example):
Code:
<a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a>

What kind of analysis do you want to do?

I'm not positive yet. I just love input/data. In general, look for patterns in the behavior of the price and order book before large movements.
TTBit
Legendary
*
Offline Offline

Activity: 1136


View Profile
February 17, 2012, 01:06:35 PM
 #9

Sooner or later you'll have to learn a lot before you can do something useful with that much data.  Smiley

But let me give you some leads to explain what that last thing meant.

Cron is the task scheduler for Unix.  Lots about that here: https://en.wikipedia.org/wiki/Cron

The * * * * * means to run something every minute.

curl is a simple http client.  It grabs the web page and spits it to stdout.  https://en.wikipedia.org/wiki/CURL

> means to redirect stdout to a file.

putting stuff in backticks ` ` means to run the command inside and then put the result on the command line.

running "date +mtgox-%F--%T" outputs:  mtgox-2012-02-16--00:30:53

So the actual command that gets run looks like:

curl https://mtgox.com/api/0/data/getDepth.php > mtgox-2012-02-16--00:30:53

To set up the cron job, run "crontab -e".  An editor will come up, and you paste in the line:

* * * * * curl -q https://mtgox.com/api/0/data/getDepth.php > `date +mtgox-%F--%T`

(I added a -q so curl won't output progress messages on stderr; otherwise they would be emailed to you with every run)

Go install Linux somewhere (you can install it in a VirtualBox VM if you don't have a spare computer), try it out, and let me know if you get stuck.

Thanks for doing this. I can read the man pages, but this is easier for me to understand.

good judgment comes from experience, and experience comes from bad judgment
stochastic
Hero Member
*****
Offline Offline

Activity: 532


View Profile
February 17, 2012, 01:07:03 PM
 #10

So I would like to start sampling and saving the mt gox order book and price info for analysis. I am able to gather the trade history using slush's sierrachartfeed script, but no orderbook info.

The most complicated thing I have ever programmed was a perl script that asked for username and password then attempted to play blackjack with you and failed.

It would be most convenient if this would work on windows xp, but I can install some linux distro if recommended.

Also what is the forum code to make my links like this (html example):
Code:
<a href=https://bitcointalk.org/index.php?topic=6019.0>sierrachartfeed-0.4</a>

What kind of analysis do you want to do?

I'm not positive yet. I just love input/data. In general, look for patterns in the behavior of the price and order book before large movements.

You should work with R then.  It was designed for analyzing large data sets.

Once you open it up you  have to install two packages. (I have not tested this in Windows, only in Linux and Mac).

Code:
       install.packages("RCurl")
        install.packages("RJSONIO")
        # install.packages("rjson") #also an option
        require(RCurl)
        require(RJSONIO)
        # require(rjson)

        #### Make a function to get the MtGox depth, returns a list of the ask and bids in a data frame format
        MT.BTCUSDdepth = function ( ) {
             tickMTGXurlBTCUSD = "https://mtgox.com/api/0/data/getDepth.php?Currency=USD"
             y = fromJSON(getURLContent( tickMTGXurlBTCUSD, ssl.verifypeer = FALSE, useragent = "R") )
             ask = as.data.frame(matrix(unlist(y$ask),ncol=2,byrow=T))
             colnames(ask) = c("ask","amount")
             bid = as.data.frame(matrix(unlist(y$bid),ncol=2,byrow=T))
             colnames(bid) = c("bid","amount")
             list(ask=ask,bid=bid)
             }

         y = MT.BTCUSDdepth() #returns the depth and stores it in the object 'y'



To inspect the data you can do a lot of different things in R.  There are a lot of user guides as R is an open source community.  But to get started you can call the bid and ask objects by typing in R:

Code:
         y$bid
          y$ask

EDIT:  fixed the typ.

Introducing constraints to the economy only serves to limit what can be economical.
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 17, 2012, 01:18:11 PM
 #11

Ah, excellent I have been messing around with R on and off over the last few months and have access to a few people with lots of experience. Thanks, I will let you know how it goes.
stochastic
Hero Member
*****
Offline Offline

Activity: 532


View Profile
February 17, 2012, 01:25:45 PM
 #12

Ah, excellent I have been messing around with R on and off over the last few months and have access to a few people with lots of experience. Thanks, I will let you know how it goes.

I translated Bitcoinica's whole API into R.  You can get the code here.

I have been having trouble getting authentication to work in R.  Let me know if those people with R experience are able to help.

Introducing constraints to the economy only serves to limit what can be economical.
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 17, 2012, 01:43:49 PM
 #13

Will do. I need to mess around for a bit though. By authentication you mean user and pass verification?
stochastic
Hero Member
*****
Offline Offline

Activity: 532


View Profile
February 17, 2012, 01:52:53 PM
 #14

Will do. I need to mess around for a bit though. By authentication you mean user and pass verification?

Yes, but MtGox uses a new authentication that requires a public and private key.  It is not basic authentication like Bitcoinica uses.

Introducing constraints to the economy only serves to limit what can be economical.
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 17, 2012, 02:06:37 PM
 #15

How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob.
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
February 17, 2012, 06:51:25 PM
 #16

How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob.

Code:
* * * * * curl -q https://mtgox.com/api/0/data/ticker.php > `date +mtgox-ticker-%F--%T`

+1 for R.  The language itself isn't anything special; languages like Python, Ruby and Perl are much more fun.  The reason for R is it has an immense library of high-quality statistics functions.  I suggest the rkward GUI to get started.

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
stochastic
Hero Member
*****
Offline Offline

Activity: 532


View Profile
February 17, 2012, 10:35:38 PM
 #17

How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob.

Code:
* * * * * curl -q https://mtgox.com/api/0/data/ticker.php > `date +mtgox-ticker-%F--%T`

+1 for R.  The language itself isn't anything special; languages like Python, Ruby and Perl are much more fun.  The reason for R is it has an immense library of high-quality statistics functions.  I suggest the rkward GUI to get started.

R also has libraries for financial indicator and strategy testers.

See quantmod (available on CRAN) and quantstrat (available on R-Forge).

Introducing constraints to the economy only serves to limit what can be economical.
bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
February 18, 2012, 01:44:33 AM
 #18



        install.packages("RCurl")
        install.packates("RJSONIO")
        require(RCurl)
        require(RJSONIO)

        #### Make a function to get the MtGox depth, returns a list of the ask and bids in a data frame format
        MT.BTCUSDdepth = function ( ) {
             tickMTGXurlBTCUSD = "https://mtgox.com/api/0/data/getDepth.php?Currency=USD"
             y = fromJSON(getURLContent( tickMTGXurlBTCUSD, ssl.verifypeer = FALSE, useragent = "R") )
             ask = as.data.frame(matrix(unlist(y$ask),ncol=2,byrow=T))
             colnames(ask) = c("ask","amount")
             bid = as.data.frame(matrix(unlist(y$bid),ncol=2,byrow=T))
             colnames(bid) = c("bid","amount")
             list(ask=ask,bid=bid)
             }

         y = MT.BTCUSDdepth() #returns the depth and stores it in the object 'y'



BTW, I figured out why I had to install the RJSONIO package manually
stochastic
Hero Member
*****
Offline Offline

Activity: 532


View Profile
February 18, 2012, 02:22:05 AM
 #19



        install.packages("RCurl")
        install.packates("RJSONIO")
        require(RCurl)
        require(RJSONIO)

        #### Make a function to get the MtGox depth, returns a list of the ask and bids in a data frame format
        MT.BTCUSDdepth = function ( ) {
             tickMTGXurlBTCUSD = "https://mtgox.com/api/0/data/getDepth.php?Currency=USD"
             y = fromJSON(getURLContent( tickMTGXurlBTCUSD, ssl.verifypeer = FALSE, useragent = "R") )
             ask = as.data.frame(matrix(unlist(y$ask),ncol=2,byrow=T))
             colnames(ask) = c("ask","amount")
             bid = as.data.frame(matrix(unlist(y$bid),ncol=2,byrow=T))
             colnames(bid) = c("bid","amount")
             list(ask=ask,bid=bid)
             }

         y = MT.BTCUSDdepth() #returns the depth and stores it in the object 'y'



BTW, I figured out why I had to install the RJSONIO package manually

You can also use rjson.

Either one is fine.

Introducing constraints to the economy only serves to limit what can be economical.
stochastic
Hero Member
*****
Offline Offline

Activity: 532


View Profile
February 18, 2012, 02:25:26 AM
 #20

How do I alter the mt gox script to give me ticker price as well? Like I said I am a newb's noob.

Code:
MT.BTCUSD = function ( ) {
tickMTGXurlBTCUSD = "https://mtgox.com/api/0/data/ticker.php"
y = fromJSON(getURLContent( tickMTGXurlBTCUSD, ssl.verifypeer = FALSE, useragent = "R") )
y = do.call(rbind,lapply(y,as.data.frame))
return(y)
}

MT.BTCUSD()


This will return a data frame

Code:
       high    low      avg     vwap    vol last_all last_local    last     buy  sell
ticker 4.74 4.2611 4.447009 4.453705 101154  4.46499    4.46499 4.46499 4.46218 4.464

If you ever make some money off of this, leave a tip.

Introducing constraints to the economy only serves to limit what can be economical.
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!