statistics

[matt313]

Hi,
I would like to do some statistics on cryptocurrencies' market curves (like mathematical fonctional analysis).

I would like to know where I can find raw datas, like a csv file containing BTC/ETH rate (per x minutes) between 1st of January and today.

Which python script do you know which can be installed at home and catch every x minutes a specifical rate (like on https://api.coinmarketcap.com/v1/ticker/bitcoin/?convert=EUR) and save it locally into a csv file?

Do you know what is the limit of connexions on coinmarketcap.com for a single IP adress? (performing statistics is different that doing a DOS!)

Thanks in advance!

[1715147466]

1715147466

[1715147466]

1715147466

[1715147466]

1715147466

[1715147466]

1715147466

[1715147466]

1715147466

[xdrpx]

I've been utilize price chart data in the form of JSON and fetching them from the following Blockchain.info URL's, coindesk and bitcoinchart's data:

1) Blockchain.info: https://api.blockchain.info/charts/market-price?format=json and https://api.blockchain.info/charts/market-price?format=csv and the API documentation here explains how you can view transaction information, fee information and addresses: https://blockchain.info/api/blockchain_api

2) Coindesk: https://api.coindesk.com/v1/bpi/historical/close.json?start=2013-09-01&end=2013-09-05 - So what I was able to do with this is allow the user input a date range (from and thru) and update charts which I built using VueJS charts. This would display the price of bitcoins between the date range. Here's the API: https://www.coindesk.com/api/. So far the prices here are almost live and I've not faced any issues with requesting it every 5 minutes.

3) Bitcoincharts has a limit of querying every 15 mins: https://bitcoincharts.com/about/markets-api/, and example data from KrakenUSD (https://api.bitcoincharts.com/v1/trades.csv?symbol=krakenEUR&start=1303100000).

For coinmarket cap and coincap's API's I have tried making JSON POST update requests every every 5 minutes and I don't see any limits imposed on it yet. So it should be good to go.

[Anti-Cen]

I've been utilize price chart data in the form of JSON and fetching them from the following Blockchain.info URL's, coindesk and bitcoinchart's data:

Agree, quick fast and easy and i have picked many of the useful Url's already so if you work in C# and VS then maybe we can help each other.

Have you tried talking to BTC nodes on port 8333 yet use from you own socket connection without using a library as an API
that is a million lines long ?

I am good at this type of stuff and to me it looks like giving birth to an elephant would be easier
and i could not even get a node to return a list of other nodes on network so never mind doing
a transaction.

Blockchain.info could go down, charge a fee or change the API and I bet they limit the request quota
so i want to avoid it but if you have trouble with the quota then I have a little tool that soon fixes it.

Best Regards

[matt313]

I'm begining to make some tries on python to collect datas.

This kind of script, run into Anaconda environment works :

http://blockxchain.org/2017/05/29/importing-altcoin-data-with-python-and-the-coinmarketcap-api/
http://blockxchain.org/2017/09/05/getting-started-with-the-cryptocompare-api-in-python/

Code:

top-10-every-5-mins.py

import coinmarketcap
import json
import pandas as pd
import time

market = coinmarketcap.Market()

# coins = json.loads(market.ticker()) doesn't work

coins = market.ticker()

for i in range(10):
    #this creates a dataframe with the top 10 coins
    coinArray = pd.DataFrame([pd.Series(coins[i]) for i in range(10)]).set_index('id')-

    #timestamps and stores the csv file
    location = 'Data/'+str(time.time())+'.csv'
    coinArray.to_csv(location)

    #waits 5 minutes until collecting data again
    time.sleep(300)

id,24h_volume_usd,available_supply,cached,last_updated,market_cap_usd,max_supply,name,percent_change_1h,percent_change_24h,percent_change_7d,price_btc,price_usd,rank,symbol,total_supply
bitcoin,10775900000.0,16762962.0,False,1514247862,237730650788,21000000.0,Bitcoin,0.71,2.02,-25.18,1.0,14181.9,1,BTC,16762962.0
ethereum,2474570000.0,96569089.0,False,1514247849,74292434667.0,,Ethereum,0.2,9.52,-3.64,0.054548,769.319,2,ETH,96569089.0
bitcoin-cash,1338570000.0,16875925.0,False,1514247852,50025641996.0,21000000.0,Bitcoin Cash,0.08,2.5,35.3,0.210183,2964.32,3,BCH,16875925.0
ripple,684181000.0,38739144847.0,False,1514247840,43424644416.0,100000000000,Ripple,-0.25,8.34,41.72,0.00007948,1.12095,4,XRP,99993093880.0
litecoin,805232000.0,54471008.0,False,1514247841,14957575426.0,84000000.0,Litecoin,0.41,-0.41,-21.81,0.0194701,274.597,5,LTC,54471008.0
cardano,121627000.0,25927070538.0,False,1514247855,10664063383.0,45000000000.0,Cardano,-2.31,6.18,-20.73,0.00002916,0.41131,6,ADA,31112483745.0
iota,172224000.0,2779530283.0,False,1514247852,9813103869.0,2779530283.0,IOTA,1.1,2.33,-16.87,0.00025033,3.53049,7,MIOTA,2779530283.0
dash,192648000.0,7775162.0,False,1514247842,9204081078.0,18900000.0,Dash,0.81,-0.27,0.79,0.0839347,1183.78,8,DASH,7775162.0
nem,118117000.0,8999999999.0,False,1514247845,8824724999.0,,NEM,-3.64,-0.57,11.58,0.00006952,0.980525,9,XEM,8999999999.0
monero,119962000.0,15523574.0,False,1514247842,5622297054.0,,Monero,0.95,7.98,-3.48,0.0256799,362.178,10,XMR,15523574.0


This is raw datas of all the coins. I should now program something to study a specifical currency.

How do you transform the raw datas into charts? vue.js?

Thanks for the links above! Do you know links like yours : https://api.bitcoincharts.com/v1/trades.csv?symbol=krakenEUR&start=1303100000 but for altcoins? Could be quicker as runing a script to catch historical datas...

[jmiro1]

Appreciate your work. May I ask what are you planning to do? I mean what kind of statistical analysis you are going to run? Are you trying find out some correlations? If that so, care to share us your work? I hope the answer is yes. All the best with your research.

[matt313]

Hi jmiro1!
I'm relatively in cryptocurrencies. I have a mathematician degree.
I will try to do some analysis on historical evolutions of the currencies, like : how does a pump&dump work? how "fractal" are the charts. Which differences/similarities can you find between the evolutions? For example BTC/ETH rates and BTC/Dash rates are not the same due to the differences of technology and target market between ETH and Dash.

And so on.

I have no idea where such personal research can lead. Some people are programing bots. I would like to understand deeper the markets.
First, I have to collect datas.

If you are interessed, do not hesitate to mp. I will have more time for that in January.