Bitcoin Forum
May 11, 2024, 01:15:11 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
  Home Help Search Login Register More  
  Show Posts
Pages: « 1 2 3 [4]
61  Alternate cryptocurrencies / Altcoin Discussion / Re: Litecoin Sierrachart feed [v0.1.2] (BTC-e.com Market Data) on: May 11, 2013, 02:05:19 PM

Hi and thanks for your work! There's a couple problems in your CSV feeds:

In the BTCUSD csv, there's quite a few occurrences like this one throughout the feed (just do a search for "meta"):

12/31/1969,20:00:00,/>,
"<meta name=""title"" content=""Биржа BTC""/>",,,
"<meta name=""keywords"" content=""BTC",н BTC,,

and a little after that, the date format suddenly changes from:
4/14/2013
to this:
2013/04/14

Same thing in your LTCBTC feed:

it goes from space-separated data formatting like this (line 576696):
4/9/2013   7:27:23   0.02439   51.8273
to comma separated data, with a different date format like this (line 576697):
2013/04/09,07:27:27,0.02416,0.356857

Other than that, great job! Smiley
62  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 11, 2013, 12:49:41 PM
The API's the bot currently uses allow it to specify from which time we want the chunk, so the first trade in each reply is always "on time". But we can't specify how big the chunk should be (in our case we only need it to be 1 trade).

That's my point: since the responses are now larger, do they overlap the requests? If so, you're better off parsing the response rather than making a now obsolete request for some data you already received.
I probably should have been clearer Smiley
63  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 11, 2013, 12:32:55 PM
But the problem with fetching historical trade data at given points in time with MtGox API is that the bot only wants ONE trade (the first one) from each chunk of trades. The rest of the data is usually not relevant since we need samples at the correct points in time (if the sample interval is very short, there COULD be more than one useful trade in the chunk, but in most cases there are probably more than 1000 trades between each sample time, so only the first trade is useful. So in most cases we have to make one call for each sample). If there was a way to specify how many trades to fetch with each call, I would set it to 1, but as far as I know that's not possible with any API version...

But caching data should cure some of the problem. Then you could at least restart the bot/browser without a new storm of calls to MtGox. But as you said, it's probably ok - while developing, I need to restart the bot all the time, causing 144 calls to be made, and I don't think I've been blocked...

Good point. But if the bot expects each request to be roughly equivalent to one sample and uses only one price from each request (looks like it's using the opening price from each sample/response), then that seems like a pretty rigid and ineffective way to go.

So maybe instead of making n requests and pick the opening price from each, it would be more effective to use all those trades to build your own candles from it? Take a look at goxtool's slot_fullhistory() and OHLCV() functions here https://github.com/prof7bit/goxtool/blob/master/goxapi.py as this is pretty much the way it does it, building its own Open-High-Low-Close-Volume candles from all the fetched history. That could give you a relative independence from MtGox's response size since you'd be fetching for a total history duration rather than making a precise number of requests in which only one price is picked from each. Does that make sense?
64  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 11, 2013, 10:54:32 AM
Yes, 144 is from the original bot, and I have not dared to change or question it Smiley
However, I think the number of samples should be the same no matter what the sample interval is (the bot fetches 144 samples, not 144 hours of samples). The EMA-vaules are calculated from the samples - independent of the interval, so the accuracy should be exactly the same as the original bot for any sample interval.

Caching samples would probably be a good idea!
Because calling the MtGox API repeatedly in a short time could mabe get you blocked by their DDoS-protection...

I see. I completely agree that the history size should be at least 144 x the sample size, but what i meant is that since you're getting 10x as much data for each request when using the API v2, you probably don't need to fetch as many times as the original bot did when using the API v0. In that regard, obtaining 144 samples' worth of data does not necessarily mean having to make 144 requests to the API.

In my experience, the MtGox API is more defensive against the time between each request (hammering will get you banned for a while) than to the actual number of requests itself. In the case of the bot, i think the time it takes to parse the json and fetch the next request should be enough timeout. You could always add a small timeout in between requests if i'm wrong, but since it seems to work as-it-is now, i think you should be okay with an even smaller number of requests for the same amount of data.
65  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 11, 2013, 09:33:19 AM
Yes, the bot uses the "since"-parameter, and it will make one fetch for each sample interval back in time to calculate EMA. Total 144 calls when starting the bot or changing sample interval (EMA-values need to be recalculated). This many calls may be a problem in itself since I think MtGox's (or Couldflare's) DDoS-protection might kick in and block your IP when making so many calls in a short time.

But the problem with API v1 and v2 is that it seems that the size of the chunks has been increased from 100 to 1000.

For instance: https://data.mtgox.com/api/2/BTCUSD/money/trades/fetch?since=1354250571691361 will return 1000 trades
And: https://data.mtgox.com/api/0/data/getTrades.php?Currency=USD&since=1354250571691361 will return 100.

I've taken a quick look at the code, and a few things come to mind. First, that 144 is pretty arbitrary: since EMA is a recursive calculation, its accuracy is very dependent on the number of samples. After a certain point it becomes less relevant since the further you go back, the less the weight given to the data, but 200 * whatever timeframe you're trading is a generally accepted sample size. I remember someone complaining that the initial sample size was not accurate enough so Piotr increased it to 144 which should be an ok compromise, but it won't hurt if you have more.

So you could either keep making these 144 requests of 1000 trades and then divide that by your trading sample size, which would give you about 10 times more candles to back your calculations on, or you can make that 144 dynamic by making requests with a timestamp equal to (number of samples required for accuracy x trading sample size) instead of this fixed 144. In other words, fetch only as much as you need according to your sample size instead of making an arbitrary number of requests. This fixed value made sense when the original bot only dealt with H1 trading and needed 144H of backdata, but not so much now that you allow users to define the trading candle size.

Also, this history and its subsequent updated candles should probably be cached into localstorage until next launch. Since for a 60m candle you're fetching about 5 days worth of history, chances are that some of that data will still be valid on next bot launch, and that would also prevent from refetching everything upon a simple change of parameters.
66  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 10, 2013, 09:26:47 PM
Hmmm - I just got API v2 working here BUT there's a bit of a problem fetching trades with API v1 and API v2 - MtGox will return 1000 trades instead of 100 (with API 0), so the data traffic will increase 10-fold (and it will update a lot slower!) :/
Unfortunately there doesn't seem to be any parameter to specify how many trades we want to fetch (we only need ONE sample for each interval!) The docs for API v2 is quite limited however - anyone know if there's a way to fetch single historical trades using API v2?

The reason it's replying with a lot of trades is that in API v2, it will return the last 24h history by default if you don't specify anything and just fetch /money/trades

You can add a ?since= parameter at the end of the url to have it return only trades since that order id. Also, if you do that it'll then reply in chunks of 100 trades like you want.

To quote this post https://bitcointalk.org/index.php?topic=150786.msg1690500#msg1690500

since is in time units*1000000
to start x hours ago you can calculate
   since = int(time() - x*60*60)*1000000
   url = "http://data.mtgox.com/api/1/BTCUSD/trades?since=" + str(since)

You will get batches of 100 trades, so you need to keep calling
with since set to highest tid in previous batch

Hope this helps.
67  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 08, 2013, 09:36:57 PM
Ohhh, you're right!
I didn't get all the files in the commit!
It's fixed now - thank's for telling me!

No pb, glad to see i'm not completely crazy Smiley Nice use of Sparkline btw.
68  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 08, 2013, 08:15:28 PM
Just released a new version of my (free) trading bot modification: https://github.com/TobbeLino/GoxTradingBotTobli

It now also has a chart!

Yeah, but where? Smiley
I just upgraded and i don't see it...

There's an icon in the popup-window (in the upper right corner)!

Sorry but i don't see it... are you sure you've commit the right files? i only see the sparkline & jquery js, and 2 png in your last commit, but no other files such as popup.html or popup.js

Also, small typo in the popup.html:
Buy threshold: Sell threshold= instead of Buy threshold: Sell threshold:

hope this helps, and thanks for all your work!
69  Economy / Trading Discussion / Re: Best Source for tick data on: May 08, 2013, 07:10:38 PM
You can take a look at this thread:
https://bitcointalk.org/index.php?topic=196834.0

Or you can use a trial version of Sierrachart to download and export its historical tick data into csv yourself...
70  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 08, 2013, 05:50:15 PM
Just released a new version of my (free) trading bot modification: https://github.com/TobbeLino/GoxTradingBotTobli

It now also has a chart!

Yeah, but where? Smiley
I just upgraded and i don't see it...
71  Economy / Trading Discussion / Re: Chrome Browser extension: MtGox trading bot on: May 08, 2013, 05:39:42 PM

Yes, probably was some problem delivering the data... Now it is working great! After a time of testing, I think you deserve a little donation Smiley... Thanks for your work!

Do you think that would be easy turning the extension into some independent html/javascript elements? In order to, for example, host the files on a local webserver and run it from any device, without the need of having chrome and this pluging on every computer/device! I tried with the original extension, but this seems to be a little more tricky (with progress fetch log, etc)...

I'm not sure how hard that would be to make it into a server version.
I can definitely see the advantage, and I agree it would be nice!
But since extensions are javascript-code, it's normally not suited for running on a server right away.
I'll have think about that, and how it could be done without too much hassle...

If you want to turn it into a standalone app, Node.js would be the way to go... Nice job BTW
72  Bitcoin / Bitcoin Technical Support / How to auth. with MtGox API v2 (node.js) ? on: April 19, 2013, 03:34:13 PM
Hello,

I can't authenticate with MtGox API (v2)
I use the following function :

Code:
var querystring = require('querystring'),
        https = require('https'),
        crypto = require('crypto');
 
function MtGoxClient() {
     this.key="my_api_key_here";
     this.secret ="mysecret_key_here";
}
 
MtGoxClient.prototype.query = function(path, args, callback) {
        var client = this;
        // if no args or invalid args provided, just reset the arg object
        if (typeof args != "object") args = {};
 
        // generate a nonce
        args['nonce'] = (new Date()).getTime() * 1000;
        // compute the post data
        var post = querystring.stringify(args);
        // compute the sha512 signature of the post data
         hash_data = path  + post;
        var hmac = crypto.createHmac('sha512',new Buffer(client.secret, 'base64'));
        hmac.update(hash_data);
        // this is our query
        var options = {
                host: 'data.mtgox.com',
                port: 443,
                path: '/api/' + path,
                method: 'POST',
                agent: false,
                headers: {
                        'Rest-Key': client.key,
                        'Rest-Sign': hmac.digest('base64'),
                        'User-Agent': 'Mozilla/4.0 (compatible; MtGox node.js client)',
                        'Content-type': 'application/x-www-form-urlencoded',
                        'Content-Length': post.length
                }
        };
        // run the query, buffer the data and call the callback
        var req = https.request(options, function(res) {           
                res.setEncoding('utf8');
                var buffer = '';
                res.on('data', function(data) { buffer += data; });
                res.on('end', function() { if (typeof callback == "function") { callback(JSON.parse(buffer)); } });
        });
        // basic error management
        req.on('error', function(e) {
                console.log('warning: problem with request: ' + e.message);
        });
        // post the data
        req.write(post);
        req.end();
};
 
var client = new MtGoxClient();
client.query('2/BTCUSD/money/info', {}, function(json) {       
        console.log(json);
});

and always get :
Code:
{ result: 'error',
  error: 'Identification required to access private API',
  token: 'login_error_invalid_rest_sign' }

What am I doing wrong?
after hours of trying, any help will reaaaaaaaally be appreciated Smiley
73  Bitcoin / Hardware / Re: List of Avalon ASIC batch 3 orders on: March 31, 2013, 12:39:27 PM
Hi, you can add my order as well:

User            Order number      3 or 4 modules      Total ordered     With ATX PSU (Y/N)         Date Paid              Shipper        Dest. Country
============================================================================================================
tagada          52xx                   4                       1                         N                               3/25                     EMS            FRANCE
74  Other / Beginners & Help / Re: Whitelist Requests (Want out of here?) on: March 26, 2013, 05:51:26 PM
Hi there,

Long time lurker, i'd like to be whitelisted so i can reply to a couple of threads about the batch #3 Avalon orders (amongst other things!).

Thanks!
Pages: « 1 2 3 [4]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!