Bitcoin Forum
March 29, 2024, 10:42:57 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 [21] 22 23 24 25 26 »
  Print  
Author Topic: SierraChart bridge - Realtime Bitcoin charts [v0.5] (MtGox, Intersango, ...)  (Read 142166 times)
arklan
Legendary
*
Offline Offline

Activity: 1778
Merit: 1008



View Profile
April 25, 2013, 08:40:09 AM
 #401

ah, got it. all working now though.

i don't post much, but this space for rent.
1711708977
Hero Member
*
Offline Offline

Posts: 1711708977

View Profile Personal Message (Offline)

Ignore
1711708977
Reply with quote  #2

1711708977
Report to moderator
BitcoinCleanup.com: Learn why Bitcoin isn't bad for the environment
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1711708977
Hero Member
*
Offline Offline

Posts: 1711708977

View Profile Personal Message (Offline)

Ignore
1711708977
Reply with quote  #2

1711708977
Report to moderator
1711708977
Hero Member
*
Offline Offline

Posts: 1711708977

View Profile Personal Message (Offline)

Ignore
1711708977
Reply with quote  #2

1711708977
Report to moderator
Mashuri
Full Member
***
Offline Offline

Activity: 135
Merit: 107


View Profile
April 25, 2013, 07:02:59 PM
 #402

Every attempt I make to download the whole history, it often gets through a month or two but always ends up crapping out with a 502 error.  This is in Win 7 Ultimate 64.  Tried running as admin, increasing the buffer and running in XP SP3 compatibility mode all to no avail.

I'm just going to leave this here:

http://we.lovebitco.in/schart/sierrachartfeed.exe (windows 32 bit standalone exe)

or

http://we.lovebitco.in/schart/sierrachartfeed.py (python 2.7 source, replace old file)

Yes, it's fixed in a big way.


NOTE: rename or delete your old C:\SierraChart\data\*.scid files if you've been using the broken sierrachartfeed version, or want older history than you previously retrieved.


Code:
Usage: sierrachartfeed.py [options]

Options:
  -h, --help            show this help message and exit
  -d DATADIR, --datadir=DATADIR
                        Data directory of SierraChart software
  -y, --disable-history
                        Disable downloads from bitcoincharts.com
  -p PRECISION, --volume-precision=PRECISION
                        Change decimal precision for market volume.
  -s SYMBOLS, --symbols=SYMBOLS
                        Charts to watch, comma separated. Use * for streaming
                        all markets.
  -l HISTORY, --history=HISTORY
                        Number of days of history to retrieve (default=30).

I set the default "symbol" to only be mtgoxUSD. The symbols are the file names for the exchange you can see here: http://api.bitcoincharts.com/v1/csv/ - for multiple exchanges, use the command-line option like:
sierrachartfeed -smtgoxUSD,bitfloorUSD,btceUSD

This is my own code, you can examine how I solved the problem of reading from old to new without an external database to sort and with 0 chance of missing or duplicate trades being downloaded (although it does inevitably waste some bandwidth).

Update 1: smarter download size adaptation, days of history option, actual source comments
Update 2: fixed duplicate trade bug I created in update 1

deepceleron
Legendary
*
Offline Offline

Activity: 1512
Merit: 1025



View Profile WWW
April 25, 2013, 07:18:19 PM
 #403

Every attempt I make to download the whole history, it often gets through a month or two but always ends up crapping out with a 502 error.  This is in Win 7 Ultimate 64.  Tried running as admin, increasing the buffer and running in XP SP3 compatibility mode all to no avail.

I can't make bitcoincharts not go down. http://www.tomshardware.com/forum/237521-49-what-gateway-error
Is there a particular exchange/currency that it is always getting or date you never get past when you see this error?

The history download will continue where it left off when the bridge program is restarted - the program SHOULD stop if it gets website errors, otherwise it will skip history downloading and get the live trade stream. After the live ticker data is added to the database with the current date and time, you will never get data older than that.
PuertoLibre
Legendary
*
Offline Offline

Activity: 1834
Merit: 1003


View Profile
April 26, 2013, 02:50:06 AM
 #404

Are there any Fibonacci plugins for Sierra charts? (Beyond the Fibonacci fan)
xeverse
Full Member
***
Offline Offline

Activity: 124
Merit: 100



View Profile
April 26, 2013, 06:55:54 AM
 #405

Are there any Fibonacci plugins for Sierra charts? (Beyond the Fibonacci fan)
Beyond? Like what? I've been thinking about phi spiral recently... Fibo fan is great fun, sometimes..
jones
Newbie
*
Offline Offline

Activity: 44
Merit: 0


View Profile WWW
April 26, 2013, 09:03:49 AM
 #406

I wonder if there is any way to get Sierra Chart to display Level II data like: Orderbook and Sales

For daytrading it is quite important to receive the most actual tick data possible. I was comparing the chart display on MTGOX USD to the chart @ http://bitcoin.clarkmoody.com/ and I had the impression it was lagging behind that website. I wonder which feed is more up2date: the one from bitcoincharts or the one SierraChart is feeding?

xeverse
Full Member
***
Offline Offline

Activity: 124
Merit: 100



View Profile
April 26, 2013, 10:25:00 AM
 #407

I wonder if there is any way to get Sierra Chart to display Level II data like: Orderbook and Sales

For daytrading it is quite important to receive the most actual tick data possible. I was comparing the chart display on MTGOX USD to the chart @ http://bitcoin.clarkmoody.com/ and I had the impression it was lagging behind that website. I wonder which feed is more up2date: the one from bitcoincharts or the one SierraChart is feeding?
If they implement trading i think there will be L2.
http://anonym.to/http://www.sierrachart.com/SupportBoard.php?ThreadID=361

Very essential. Yes a bit lagging indeed.. This bridge takes bitcoinchart data.
I've tried sierra's feed. It seemed even more lively to me than bitcoinity or perhaps clark.
No idea where do they get their data. But it's more "realtime'y" than the bridge for sure..
deepceleron
Legendary
*
Offline Offline

Activity: 1512
Merit: 1025



View Profile WWW
April 26, 2013, 02:11:37 PM
Last edit: April 26, 2013, 02:25:31 PM by deepceleron
 #408

I wonder if there is any way to get Sierra Chart to display Level II data like: Orderbook and Sales

For daytrading it is quite important to receive the most actual tick data possible. I was comparing the chart display on MTGOX USD to the chart @ http://bitcoin.clarkmoody.com/ and I had the impression it was lagging behind that website. I wonder which feed is more up2date: the one from bitcoincharts or the one SierraChart is feeding?

I just logged trade times vs actual UTC time I received them on my computer (I have sub-1-second accuracy using NTP). Here are the results:

Code:
rec date rec time timestamp delay
4/26/2013 13:49:21 13:49:18 3.6
4/26/2013 13:49:51 13:49:21 30.5
4/26/2013 13:49:51 13:49:24 27.6
4/26/2013 13:49:51 13:49:24 27.6
4/26/2013 13:49:51 13:49:29 22.6
4/26/2013 13:49:51 13:49:32 19.6
4/26/2013 13:49:51 13:49:35 16.6
4/26/2013 13:49:51 13:49:39 12.6
4/26/2013 13:49:51 13:49:42 9.6
4/26/2013 13:49:51 13:49:42 9.6
4/26/2013 13:49:51 13:49:45 6.6
4/26/2013 13:49:51 13:49:45 6.6
4/26/2013 13:50:21 13:50:02 19.7
4/26/2013 13:50:22 13:50:17 5.1
4/26/2013 13:50:22 13:50:18 4.1
4/26/2013 13:50:52 13:50:22 30.6
4/26/2013 13:50:52 13:50:22 30.6
4/26/2013 13:50:52 13:50:31 21.6
4/26/2013 13:50:52 13:50:33 19.6
4/26/2013 13:50:52 13:50:36 16.6
4/26/2013 13:50:52 13:50:39 13.6
4/26/2013 13:50:52 13:50:44 8.6
4/26/2013 13:50:52 13:50:48 4.6
4/26/2013 13:51:22 13:51:00 22.6
4/26/2013 13:51:22 13:51:03 19.9
4/26/2013 13:51:22 13:51:11 11.9
4/26/2013 13:51:52 13:51:24 28.7
4/26/2013 13:51:52 13:51:29 23.8
4/26/2013 13:51:52 13:51:31 21.8
4/26/2013 13:51:52 13:51:33 19.8
4/26/2013 13:51:52 13:51:36 16.8
4/26/2013 13:51:52 13:51:38 14.8
4/26/2013 13:51:52 13:51:41 11.8
4/26/2013 13:51:52 13:51:43 9.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:45 7.8
4/26/2013 13:51:52 13:51:47 6.0
4/26/2013 13:51:52 13:51:50 3.0
4/26/2013 13:52:24 13:51:52 32.0
4/26/2013 13:52:24 13:51:57 27.1
4/26/2013 13:52:24 13:51:57 27.1
4/26/2013 13:52:24 13:52:01 23.1
4/26/2013 13:52:24 13:52:02 22.1
4/26/2013 13:52:24 13:52:04 20.1
4/26/2013 13:52:24 13:52:08 16.1
4/26/2013 13:52:24 13:52:08 16.1
4/26/2013 13:52:24 13:52:11 13.1
4/26/2013 13:52:24 13:52:11 13.1
4/26/2013 13:52:24 13:52:13 11.1
4/26/2013 13:52:24 13:52:15 9.3
4/26/2013 13:52:24 13:52:17 7.3
4/26/2013 13:52:24 13:52:20 4.3
4/26/2013 13:52:24 13:52:22 2.3
4/26/2013 13:52:24 13:52:22 2.3
4/26/2013 13:52:24 13:52:23 1.3
4/26/2013 13:52:24 13:52:23 1.3
4/26/2013 13:52:54 13:52:24 30.3
4/26/2013 13:52:54 13:52:28 26.4
4/26/2013 13:52:54 13:52:28 26.4
4/26/2013 13:52:54 13:52:28 26.4
4/26/2013 13:52:54 13:52:28 26.4
4/26/2013 13:52:54 13:52:30 24.4
4/26/2013 13:52:54 13:52:34 20.4
4/26/2013 13:52:54 13:52:37 17.4
4/26/2013 13:52:54 13:52:43 11.4
4/26/2013 13:52:54 13:52:45 9.4
4/26/2013 13:52:54 13:52:46 8.4
4/26/2013 13:52:54 13:52:50 4.6
4/26/2013 13:52:54 13:52:52 2.6
4/26/2013 13:53:24 13:52:54 30.4
4/26/2013 13:53:24 13:52:56 28.8
4/26/2013 13:53:24 13:52:56 28.8
4/26/2013 13:53:24 13:52:58 26.8
4/26/2013 13:53:24 13:53:02 22.8
4/26/2013 13:53:55 13:53:31 24.1
4/26/2013 13:53:55 13:53:32 23.1
4/26/2013 13:53:55 13:53:32 23.1
4/26/2013 13:53:55 13:53:34 21.1
4/26/2013 13:53:55 13:53:36 19.1
4/26/2013 13:53:55 13:53:44 11.1
4/26/2013 13:53:55 13:53:46 9.1
4/26/2013 13:53:55 13:53:47 8.1
4/26/2013 13:53:55 13:53:48 7.1
4/26/2013 13:54:25 13:53:58 27.5
4/26/2013 13:54:25 13:54:09 16.8
4/26/2013 13:54:25 13:54:09 16.8
4/26/2013 13:54:25 13:54:09 16.8
4/26/2013 13:54:25 13:54:13 12.8
4/26/2013 13:54:25 13:54:21 4.8
4/26/2013 13:55:26 13:54:57 29.7
4/26/2013 13:55:26 13:55:00 26.7
4/26/2013 13:55:26 13:55:14 12.7
4/26/2013 13:55:26 13:55:18 8.7
4/26/2013 13:55:26 13:55:20 6.7
4/26/2013 13:55:26 13:55:23 3.7
4/26/2013 13:55:26 13:55:23 3.7
4/26/2013 13:55:55 13:55:38 17.9
4/26/2013 13:55:56 13:55:38 18.3
4/26/2013 13:55:56 13:55:38 18.3
4/26/2013 13:55:56 13:55:40 16.3
4/26/2013 13:55:56 13:55:40 16.3
4/26/2013 13:55:56 13:55:48 8.3
4/26/2013 13:55:56 13:55:48 8.3
4/26/2013 13:55:56 13:55:50 6.3
4/26/2013 13:55:56 13:55:52 4.3

The average delay time is 15 seconds. If you use clarkmoody, it gets its trades to your browser directly from mtgox websocket; you can compare it's console and see that the bitcoincharts data is pushed in approximately ~30 second chunks (which is adequate for charting 1-minute intervals, there is no trading in bridge-connected sierrachart).

I might take a look at getting data from MtGox again, which is how SC is getting their feed (but how much do they delay?...). It was the original method, but was switched due to problems:

Version 0.4 released, fixes non-working streaming from MtGox. It is now using socket interface to bitcoincharts instead of MtGox's socket.io.

Pros - sierrachart now supports live streaming of all bitcoin charts, not just MtGox.
Cons - there's currently implemented only polling between bitcoincharts and mtgox and chart updates are delayed 5 seconds in average. But tcatm is working on socket.io support on his side, so it will gets better soon.

Enjoy! :-)

deepceleron
Legendary
*
Offline Offline

Activity: 1512
Merit: 1025



View Profile WWW
April 26, 2013, 04:11:59 PM
 #409

on SC data feed since the data is properly formatted and you can use tick charts for example the only problem is they dont have data since day 1

With the exe version I just uploaded, data should be tick-accurate, although you need to set chart display appropriately (apparently the SCID can't set appropriate defaults).


Permanent fix for tick (adjust appropriately for other exchanges):
Go into Menu -> Tools -> Chart Settings. Pick "Edit Global Symbol Settings". Choose Service "SC Historical Data", and press New. Then edit on the "General" Tab:
-Symbol: mtgoxUSD
-Price Display Format: .00001
-Tick Size: 0.0000100
-Currency Value Per Tick: 0.0000100

Now in Menu -> Tools -> Chart Settings, "Price Display Format" and "Tick Size" is auto set to  ".00001"

Temporary Volume Fix until next SierraChart Launch:
Go into Menu -> Tools -> Chart Settings. Go to "Advanced Settings 2" tab and change "Volume/Open Int. Multiplier" to "0.01". Press Apply

See volume:
Menu -> Analysis -> Studies. select "Volume" and press "Add>>", and OK.

Looks like:


Bar analysis
Go into Menu -> Tools -> Chart Settings.
Set Bar Period to "Number Of Ticks", "Range Of Ticks", set the number, and do whatever analysis you want.
arklan
Legendary
*
Offline Offline

Activity: 1778
Merit: 1008



View Profile
April 26, 2013, 04:22:57 PM
 #410

oooh, useful changes. thanks for that.

i don't post much, but this space for rent.
xeverse
Full Member
***
Offline Offline

Activity: 124
Merit: 100



View Profile
April 26, 2013, 09:35:48 PM
 #411

With the exe version I just uploaded, data should be tick-accurate
Thank you! So it seems the delay is 30s on average..
myself
Legendary
*
Offline Offline

Activity: 938
Merit: 1000


chaos is fun...…damental :)


View Profile
April 27, 2013, 11:47:03 AM
 #412

@deepceleron nice fix Cheesy

Los desesperados publican que lo inventó el rey que rabió, porque todo son en el rabias y mas rabias, disgustos y mas disgustos, pezares y mas pezares; si el que compra algunas partidas vé que baxan, rabia de haver comprado; si suben, rabia de que no compró mas; si compra, suben, vende, gana y buelan aun á mas alto precio del que ha vendido; rabia de que vendió por menor precio: si no compra ni vende y ván subiendo, rabia de que haviendo tenido impulsos de comprar, no llegó á lograr los impulsos; si van baxando, rabia de que, haviendo tenido amagos de vender, no se resolvió á gozar los amagos; si le dan algun consejo y acierta, rabia de que no se lo dieron antes; si yerra, rabia de que se lo dieron; con que todo son inquietudes, todo arrepentimientos, tododelirios, luchando siempre lo insufrible con lo feliz, lo indomito con lo tranquilo y lo rabioso con lo deleytable.
arklan
Legendary
*
Offline Offline

Activity: 1778
Merit: 1008



View Profile
April 27, 2013, 11:50:05 AM
 #413

anyone else seeing the error message about mysql server failure using deepceleron's exe?

if so, anyone got a clue when it'll be sorted?

if not, uh... help?

i don't post much, but this space for rent.
myself
Legendary
*
Offline Offline

Activity: 938
Merit: 1000


chaos is fun...…damental :)


View Profile
April 27, 2013, 11:54:01 AM
 #414

anyone else seeing the error message about mysql server failure using deepceleron's exe?
confirmed

Los desesperados publican que lo inventó el rey que rabió, porque todo son en el rabias y mas rabias, disgustos y mas disgustos, pezares y mas pezares; si el que compra algunas partidas vé que baxan, rabia de haver comprado; si suben, rabia de que no compró mas; si compra, suben, vende, gana y buelan aun á mas alto precio del que ha vendido; rabia de que vendió por menor precio: si no compra ni vende y ván subiendo, rabia de que haviendo tenido impulsos de comprar, no llegó á lograr los impulsos; si van baxando, rabia de que, haviendo tenido amagos de vender, no se resolvió á gozar los amagos; si le dan algun consejo y acierta, rabia de que no se lo dieron antes; si yerra, rabia de que se lo dieron; con que todo son inquietudes, todo arrepentimientos, tododelirios, luchando siempre lo insufrible con lo feliz, lo indomito con lo tranquilo y lo rabioso con lo deleytable.
deepceleron
Legendary
*
Offline Offline

Activity: 1512
Merit: 1025



View Profile WWW
April 27, 2013, 11:58:59 AM
 #415

anyone else seeing the error message about mysql server failure using deepceleron's exe?

if so, anyone got a clue when it'll be sorted?

if not, uh... help?
All of http://bitcoincharts.com is currently down (the source of Sierrachartfeed data), replaced only with "Our MySQL server started to fail this night. We're migrating to postgres now, but this will take some time." This is a first in two years. This may also create a gap in history if they don't have a separate grabber, so anybody else that has some alternate mtgox API logging, KEEP IT GOING so that bitcoincharts can resurrect any missing data!
S3052
Legendary
*
Offline Offline

Activity: 2100
Merit: 1000


View Profile
April 27, 2013, 12:19:42 PM
 #416

since a few days, it really looks all screwed:

http://postimg.org/image/qjkddba5t/full/


1) the intraday data is displayed backwards (right to left from morning to evening and not how it should be from left to right)
2) it shows several times the same candles (i.e. 3x per day)
how can this be fixed?

arklan
Legendary
*
Offline Offline

Activity: 1778
Merit: 1008



View Profile
April 27, 2013, 12:23:40 PM
 #417

anyone else seeing the error message about mysql server failure using deepceleron's exe?

if so, anyone got a clue when it'll be sorted?

if not, uh... help?
All of http://bitcoincharts.com is currently down (the source of Sierrachartfeed data), replaced only with "Our MySQL server started to fail this night. We're migrating to postgres now, but this will take some time." This is a first in two years. This may also create a gap in history if they don't have a separate grabber, so anybody else that has some alternate mtgox API logging, KEEP IT GOING so that bitcoincharts can resurrect any missing data!

eww...

i don't post much, but this space for rent.
myself
Legendary
*
Offline Offline

Activity: 938
Merit: 1000


chaos is fun...…damental :)


View Profile
April 27, 2013, 12:27:54 PM
 #418

anyone else seeing the error message about mysql server failure using deepceleron's exe?

if so, anyone got a clue when it'll be sorted?

if not, uh... help?
All of http://bitcoincharts.com is currently down (the source of Sierrachartfeed data), replaced only with "Our MySQL server started to fail this night. We're migrating to postgres now, but this will take some time." This is a first in two years. This may also create a gap in history if they don't have a separate grabber, so anybody else that has some alternate mtgox API logging, KEEP IT GOING so that bitcoincharts can resurrect any missing data!
just take SC data feed and join it later

Los desesperados publican que lo inventó el rey que rabió, porque todo son en el rabias y mas rabias, disgustos y mas disgustos, pezares y mas pezares; si el que compra algunas partidas vé que baxan, rabia de haver comprado; si suben, rabia de que no compró mas; si compra, suben, vende, gana y buelan aun á mas alto precio del que ha vendido; rabia de que vendió por menor precio: si no compra ni vende y ván subiendo, rabia de que haviendo tenido impulsos de comprar, no llegó á lograr los impulsos; si van baxando, rabia de que, haviendo tenido amagos de vender, no se resolvió á gozar los amagos; si le dan algun consejo y acierta, rabia de que no se lo dieron antes; si yerra, rabia de que se lo dieron; con que todo son inquietudes, todo arrepentimientos, tododelirios, luchando siempre lo insufrible con lo feliz, lo indomito con lo tranquilo y lo rabioso con lo deleytable.
deepceleron
Legendary
*
Offline Offline

Activity: 1512
Merit: 1025



View Profile WWW
April 28, 2013, 04:02:43 AM
Last edit: July 17, 2013, 01:23:47 AM by deepceleron
 #419

bitcoincharts is alive again.

Every attempt I make to download the whole history, it often gets through a month or two but always ends up crapping out with a 502 error.

These errors persist even with the rebirth of bitcoincharts. They seem completely random, with no relation to the amount of data or the amount of time since the last request. I can wait five seconds between requests that are 1/20 the size of maximum and they still randomly happen. Of course the bitcoincharts API overview does say "Don't query more often than once every 15 minutes!", but that would make getting any amount of history beyond a single request take forever.

Therefore I have tried to make code that figures out what bitcoincharts wants and only gives up if it has way too many 502 errors. It slows down and requests less data upon errors to reduce hammering the server:

Link to post with download: https://bitcointalk.org/index.php?topic=6019.msg1929339#msg1929339
zhalox
Full Member
***
Offline Offline

Activity: 176
Merit: 106


XMR = BTC in 2010. Rise chikun.


View Profile
April 29, 2013, 12:52:10 AM
Last edit: April 29, 2013, 01:16:30 AM by zhalox
 #420

For some reason, this "fix" fails to download anything past the last week or so of trade data.  Is there any way to download data farther back?  My mtgoxUSD.scid file is missing historical trade data from April 19, 20, 21, and 22 Sad

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 [21] 22 23 24 25 26 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!