Bitcoin Forum

Bitcoin => Project Development => Topic started by: jwest411 on August 07, 2013, 10:20:19 PM



Title: Help Making GET Request on BitStamp API
Post by: jwest411 on August 07, 2013, 10:20:19 PM
Hi everyone,
I am trying to make a GET request to pull the ticker from the bitstamp api, but I am getting a 403 forbidden server error. This is surprising because you are not supposed to need any credentials to access the ticker. Here is a snippet of the code I am using:


         String path  ="https://www.bitstamp.net/api/ticker/";
        try{
           URL queryUrl = new URL(path);
             HttpsURLConnection connection = (HttpsURLConnection)queryUrl.openConnection();
      
            connection.setRequestMethod("GET");

           BufferedReader reader = new BufferedReader( newInputStreamReader(connection.getInputStream()));
        }catch (Exception e){
               System.out.println("exception in querying BS");
           }
            
I am getting the error when I try to construct the input stream of course. I know that I have been verified and have turned on API access on my account. Does anyone know of any other reason why I might be getting the 403 response? Or maybe see an error in my code? This is driving me crazy!

I would really appreciate your wisdom!
Thanks,
Jon


Title: Re: Help Making GET Request on BitStamp API
Post by: solex on August 07, 2013, 10:25:07 PM
What happens if you try ""https://www.bitstamp.net/api/order_book/" instead...


Title: Re: Help Making GET Request on BitStamp API
Post by: jwest411 on August 07, 2013, 10:59:50 PM
I tried using order_book, and I am still getting a 403. Well, technically the code I posted catches the exception.

here are some more potentially useful facts:
1) The error stream is null, server response message is "Forbidden", and permissions: ("java.net.SocketPermission" "www.bitstamp.net:80" "connect,resolve")
2) I am getting the same exact responses when I am doing a POST request with "balance" method (with user and password params included in post)

hmm...


Title: Re: Help Making GET Request on BitStamp API
Post by: BitPirate on August 08, 2013, 12:45:57 AM
I tried using order_book, and I am still getting a 403. Well, technically the code I posted catches the exception.

here are some more potentially useful facts:
1) The error stream is null, server response message is "Forbidden", and permissions: ("java.net.SocketPermission" "www.bitstamp.net:80" "connect,resolve")
2) I am getting the same exact responses when I am doing a POST request with "balance" method (with user and password params included in post)

hmm...

Should be port 443 right?


Title: Re: Help Making GET Request on BitStamp API
Post by: jwest411 on August 08, 2013, 01:08:22 AM
BitPirate,
I looked it up and I see that port 443 means an ssl connection so that makes sense because I know that is the required type of connection... although I already thought I was connected over ssl since I used httpsurlconnection. How do I check what port I am on? Is that "80" in the permissions my current port? I will look up how to set up a proper ssl connection in the meantime.

I have a good feeling about your response!

edit: it appears that HttpsURLConnection does in fact use port 443 by default.

another edit: I tried using  System.setProperty("https.proxyPort", "443");
but to no avail. I am still getting "www.bitstamp.net:80" in the permissions though-- is there something else I have to do to set the server port to 443?


Title: Re: Help Making GET Request on BitStamp API
Post by: adamstgBit on August 08, 2013, 03:37:05 AM
could it be your importing com.sun.net.ssl.HttpsURLConnection when you really want javax.net.ssl.HttpsURLConnection??


Title: Re: Help Making GET Request on BitStamp API
Post by: Dunkelheit667 on August 08, 2013, 03:48:01 AM
Well, you have to add some header information (User-Agent etc.), try to emulate a browser... Guess it's some kind of DDoS protection. Wired for an API, however, it works. ;)


Title: Re: Help Making GET Request on BitStamp API
Post by: jwest411 on August 08, 2013, 04:23:13 AM
@adamstgBit, good point and thanks for the suggestion, though I was already using the javax package.

Well, you have to add some header information (User-Agent etc.), try to emulate a browser... Guess it's some kind of DDoS protection. Wired for an API, however, it works. ;)
@Dunkelheit667-Thank you so much! After I added the header information it worked. Though I am not exactly sure why... what other values could I have entered as "content-type" and "user-agent" in the header fields? Anywways, you are a godsend!


Title: Re: Help Making GET Request on BitStamp API
Post by: Dunkelheit667 on August 08, 2013, 04:58:10 AM
@Dunkelheit667-Thank you so much! After I added the header information it worked. Though I am not exactly sure why... what other values could I have entered as "content-type" and "user-agent" in the header fields? Anywways, you are a godsend!

You're welcome! Glad it worked with just adding some header information. :) Same for me, not sure why it works... But have seen these 403s on many pages. As mentioned, guess it's something like a DDoS protection or keeping bots outside. Even if this would not make much sense for an API in this particular case (and Bitstamp gave some information and rules how to use the API). As long as the server will not receive (enough) header information, it just throws a 403. ::)

You will find additional header fields there: http://tools.ietf.org/html/rfc2616#section-14

One additional hint. Your script will work today and in some weeks, you might get a 403 again. Just add additional or different fields/values. It's always a bit playing around. Which fields/values the server will expect/accept today/tomorrow...


Title: Re: Help Making GET Request on BitStamp API
Post by: jwest411 on August 08, 2013, 03:54:15 PM
Thanks, I will keep that in mind. Also, just for anyone else who might have this problem, the header fields I added are
connection.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
        connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; Java Test client)");

I hope anyone else who has this problem sees this thread!
-Paul


Title: Re: Help Making GET Request on BitStamp API
Post by: jwest411 on August 08, 2013, 06:26:59 PM
update/warning:
I was just banned from Bitstamp for using these header lines! So watch out because the header lines posted above are NOT a good solution! I suppose using another one of the possibilities given by DunkelHeit667 might be better.

On that note, how do I get un-banned from BitStamp?


Title: Re: Help Making GET Request on BitStamp API
Post by: Dunkelheit667 on August 08, 2013, 07:15:40 PM
On that note, how do I get un-banned from BitStamp?

Guess simply getting a new IP should solve this.

Bitstamps suggests:

Quote
Request limits

Do not make more than 600 request per 10 minutes or we will ban your IP address.


Title: Re: Help Making GET Request on BitStamp API
Post by: jwest411 on August 08, 2013, 08:36:11 PM
Yeah its possible that was the cause, though surprising... I was using a POST to set a buy limit order in a timer loop (bad idea in retrospect), but I didn't have any funds in my account at the time so I thought it would just give me an "insufficient funds" or something, plus I really doubt I made over 600 requests. But either way, I will change my ip address and see what happens. Do you think I will need to change my "user-agent"?


Title: Re: Help Making GET Request on BitStamp API
Post by: Dunkelheit667 on August 08, 2013, 08:51:38 PM
Not sure if the mentioned figures are still the actual limit. Just noticed as well there is a limit. ;) Requires some playing to test this frontier. I'm fine with eight API requests per hour (simply obtaining actual exchange rates). Nope, in case your IP was banned, there will be no need to change the User-Agent.


Title: Re: Help Making GET Request on BitStamp API
Post by: daybyter on August 09, 2013, 06:21:59 PM
Oh...another Java coder! Cool! Welcome!

Take a look at:

https://github.com/ReAzem/cryptocoin-tradelib/blob/master/modules/bitstamp/src/de/andreas_rueckert/trade/site/bitstamp/client/BitstampClient.java

It's not complete yet, but it might help you...


Title: Re: Help Making GET Request on BitStamp API
Post by: mazi on August 19, 2013, 10:04:11 AM
Have you tried https://github.com/timmolter/XChange ? It provides a unified interface to several Bitcoin exchanges; some of the issues mentioned here are already solved in XChange.

Here's a Bitstamp ticker demo:
https://github.com/timmolter/XChange/blob/develop/xchange-examples/src/main/java/com/xeiam/xchange/examples/bitstamp/marketdata/TickerDemo.java

Hope this helps!