[CONTEST] $50 to whoever can implement a historical Bitcoin price API

[NotATether]

In case you haven't heard of this tool, bitcoin-chart-cli is a nifty little Bitcoin price tracker written in NodeJS that also spits out some technical indicators as well as support for some altcoins (though I only ever use the BTC tracker).

It's only downside is that it uses an aggressive API service CryptoCompare that will ban you if you accidentally leave the program repeating every few seconds. To get back, you need to pay them an expensive $75/month (not year) for an API key.

Now, I've been thinking, why don't I just rip that API out and replace it with another, unlimited API just for fetching the historical Bitcoin price?

Since I'm not in the business for writing APIs, I figured it would be best to outsource this to someone else. Now that there are a bunch of Preev clones out there, I'm sure at least someone is willing to take on the task.

Rules:

- There is no application, you are supposed to develop an API and host it, and notify me about that in this thread.
- You must make two endpoints:
--One for fetching historical price data for each day, for most common fiat currencies (the bare minimum is USD) as an array sorted in chronological (oldest data first) order.
--The other endpoint must return today's price for a specified fiat currency.
- There must be no periodic limiting requests. Basic rate limiting (say X requests per second) to protect yourself from DDoS attacks or using Cloudflare is fine, but the whole idea of this contest is to make an API that you can hook up price charts to without paying extraordinary prices, so no silly stuff like limit 100K requests/month.
- You need to include all daily prices starting from 2013 to be eligible for the reward - the aim is to make as complete of a price tracker as possible.
- The API must be usable without requiring an API key or registering for an account.
- This contest is open for a maximum of three months (90 days), and is subject to close at any time, such as if I select the best API;
- I reserve the right to pay out extra if you make a particularly good API (such as adding more fiat/cryptocurrencies or endpoints). I also reserve the right to not pay out the reward at all if every submission is (in my opinion) not good enough.
- If you win the prize, you are not allowed to take down the API, except for maintenance or if you have financial problems that prohibit you from paying your hosting, or you will be red-tagged.
- The prize will be paid out in BTC.
- I reserve the right to modify the rules.

You may use this thread for submissions or for discussing APIs, bitcoin price trackers, or the contest.

[1714556074]

1714556074

[1714556074]

1714556074

[1714556074]

1714556074

[NotATether]

Reserved for results

[bitmover]

Unless someone already have the server, the data and the whole backend enviro,  50 usd is a very low payment. People will even have to host the server, which is impossible to host for 50 usd forever without any limit.

Well, I know a free api that will do everything you requested. Binance api

I have used it another project with one request every 15 seconds, never had any problems.

--The other endpoint must return today's price for a specified fiat currency.

All biannce fiat work in this one.
https://api.binance.com/api/v3/ticker/24hr?symbol=BTCEUR
https://api.binance.com/api/v3/ticker/24hr?symbol=BTCBUSD
https://api.binance.com/api/v3/ticker/24hr?symbol=BTCBRL

Code:

{"symbol":"BTCEUR","priceChange":"-41.05000000","priceChangePercent":"-0.192","weightedAvgPrice":"21401.89187687","prevClosePrice":"21420.29000000","lastPrice":"21379.16000000","lastQty":"0.00039000","bidPrice":"21374.24000000","bidQty":"0.00239000","askPrice":"21377.62000000","askQty":"0.00424000","openPrice":"21420.21000000","highPrice":"21538.59000000","lowPrice":"21264.53000000","volume":"1633.59668000","quoteVolume":"34962059.51578010","openTime":1667645091825,"closeTime":1667731491825,"firstId":98744485,"lastId":98846205,"count":101721}

Or cleaner
https://api.binance.com/api/v3/ticker/price?symbol=BTCEUR

Code:

{"symbol":"BTCEUR","price":"21379.37000000"}

- You must make two endpoints:
--One for fetching historical price data for each day, for most common fiat currencies (the bare minimum is USD) as an array sorted in chronological (oldest data first) order.

Thid end-point will do the job

You need to configure the parameters.

symbol = BTCBUSD, BTCEUR, BTCBRL, etc..,.
interval = 1h, 6h, 12h, 1d, etc
startTime, endTime in timestamp
limit = max is 1000 (i.e. 1000 data following those parameters.

This is the endpoint for bitcoin in Euro data since october 2020

https://api.binance.com/api/v3/uiKlines?symbol=BTCBRL&interval=1d&startTime=1602547200000&limit=1000


I believe solutions like this, using free third party api from big companies (and their limitations) is the only thing you can get for 50 usd one time payment.

Binance limitations are very loose  , you can request basicly forever and you won't get blocked.

Unless someone is willing to donate their infrastructure to you to host your api.

Tell me if you have problems maybe I can help, I have been studying this api for some time .

You can also try coingecko and similar ones, which will probably work as well


For older data, you can merge with this one which is not being updated anymore (work with all currencies)
https://api.coindesk.com/v1/bpi/historical/close.json?currency=EUR&start=2010-07-17&end=2020-10-13

And coingecko, which is an amazing api as well
https://api.coingecko.com/api/v3/coins/bitcoin/market_chart?vs_currency=usd&days=max&interval=daily

[NotATether]

Unless someone already have the server, the data and the whole backend enviro,  50 usd is a very low payment. People will even have to host the server, which is impossible to host for 50 usd forever without any limit.

I might increase it to $200 or $300 (or possibly even $500 - that particular amount won't be done now though, as I've already splashed 400EUR on ProtonMail's "Visionary" plan that should cover my email, cloud storage and VPN needs for 2 years. So a second payout this size would put me over my monthly budget.

For now I can't justify any prize amounts higher than $500 for a simple price history API like this. After all, the odds of the service going "viral" are very low, and even if that does happen, a simple donate button on the site's webpage should cover maintenance costs - I mean I've done just that for BitcoinCleanup and raised about $100 for it [its funds are in a completely separate address that won't be mingled with anything else].


Yes, I have used the Binance API before and it's pretty good. But first let's see if we can make one that is more simple for devs.

[seoincorporation]

...
Yes, I have used the Binance API before and it's pretty good. But first let's see if we can make one that is more simple for devs.

It's impossible to make an API like this without a third party, the bitcoin price comes from the value on the exchanges and we have to request their information if we want that data.

To make your own API with unlimited calls then you have to request all the past data and start filling a database, then each API call will be a request to your data base and not to the data provider, and then you need to keep updating the data base information each X seconds.

Mount this kind of servers aren't cheap because you need a VPS with a nice amount of ram and cpu for all the calls, a huge hard disk for the DB and a big bandwidth.

Just think about the data base with a line for the bitcoin prize each 2 second since bitcoin gets on the markets, that's a huge amount of data.

I hope this information makes you see the full context of your request. A project like this could take months on API calls to create your own DB. The api calls isn't a big deal, is just make request to the db and print the data in json.

[NotATether]

To make your own API with unlimited calls then you have to request all the past data and start filling a database, then each API call will be a request to your data base and not to the data provider, and then you need to keep updating the data base information each X seconds.

Speaking about that...

@LoyceV do you have a data dump of Bitcoin prices lying around somewhere?

Maybe I'll send him a PM.

[bitmover]

You are 100% correct. The only way someone would provide what NotATether requested is if someone already has all that infrastructure and he wants to share it for 50 one time payment (basically a donation).

I think the best solution would be to use the free apis, which have all the data he requested, or to pay for the api if he need more calls (I don't think he needs, binance is basically unlimited)

It looks  simple just to have an api like that, but it is expensive because you need to make requested every few seconds/ minute, otherwise you will have a delayed data.  A database,  etc like you said.

I think that using those free api are enough. The historical data is the most problematic because you won't find since 2013 for free... but you can just merge coindesk api Data with the binance api. I can help with that if needed.

[seoincorporation]

Speaking about that...

@LoyceV do you have a data dump of Bitcoin prices lying around somewhere?

Maybe I'll send him a PM.

If you want to have the data of the bitcoin price by day you can get it here:

https://www.nasdaq.com/market-activity/cryptocurrency/btc/historical

They even give you the option to download the data as CVS and their data rage is from 3/08/2019 to Today. I like they give the highest price and the lowest one of the day. That way you can verify when we had some crazy bump and dump moves on the same day.

Another options for this data are:

https://coinmarketcap.com/currencies/bitcoin/historical-data/
https://finance.yahoo.com/quote/BTC-USD/history/

[LoyceV]

@LoyceV do you have a data dump of Bitcoin prices lying around somewhere?

Nope. I use CoinMarketCap.com when I need price data.

I know nothing about APIs, so I'll ask: why an API? All data fits in a very small TXT or CSV file.

[bitmover]

@LoyceV do you have a data dump of Bitcoin prices lying around somewhere?

Nope. I use CoinMarketCap.com when I need price data.

I know nothing about APIs, so I'll ask: why an API? All data fits in a very small TXT or CSV file.

Let's say he wants to make the chart of the last 7 days    or the last day (24h)

This data gets updated every minute or so. I believe api is the best way to deliver this kind of data, or a web socket.

The api what will make to call to the server/database and request the data.
I have never seen anyone delivering this kind of data in csv formats, but it is possible.

Csv is good for static data, not dynamic like this. Afaik

[NotATether]

@LoyceV do you have a data dump of Bitcoin prices lying around somewhere?

Nope. I use CoinMarketCap.com when I need price data.

I know nothing about APIs, so I'll ask: why an API? All data fits in a very small TXT or CSV file.

It's because bitcoin-chart-cli is a very small utility that doesn't justify making a download of several gigabytes of price data, just to display a chart. I checked the code, and it seems that only the last 50 days are fetched, in order to calculate moving averages.

[LoyceV]

It's because bitcoin-chart-cli is a very small utility that doesn't justify making a download of several gigabytes of price data, just to display a chart. I checked the code, and it seems that only the last 50 days are fetched, in order to calculate moving averages.

Are you looking for daily data, or much more frequent data? If it's daily, it's about 5000 days times a maximum of 7 digits (price in dollar cents). That's less than 35 kB.
Even if it's data for every minute (which I haven't found anywhere going back that long), it will be less than 50 MB.

[seoincorporation]

Nope. I use CoinMarketCap.com when I need price data.

I know nothing about APIs, so I'll ask: why an API? All data fits in a very small TXT or CSV file.

When you want to get data from a TXT file the pc has to read the file, then cut all the data that doesn't belongs to your query. But when you call data on a data base the computer doesn't read all the data base, the process jumps to the line you are looking to.

The API calls are a combo of database calls and JSON transformation to print the data.

Difference:
SQL

Code:

SELECT * FROM table-name WHERE your date-column >= '2013-12-12'  

Bash

Code:

cat db | grep '2013-12-12'  

It's because bitcoin-chart-cli is a very small utility that doesn't justify making a download of several gigabytes of price data, just to display a chart. I checked the code, and it seems that only the last 50 days are fetched, in order to calculate moving averages.

Maybe you could change the focus and offer a budget for the one who can change the api bitcoin-chart-cli, or even better, add multiple apis for the scenario where one gets locked.

Here the API path, it doesn't looks as a complex code, maybe you can implement another api by yourself.

https://github.com/madnight/bitcoin-chart-cli/blob/master/src/CryptoCompareAPI.js

[NotATether]

Maybe you could change the focus and offer a budget for the one who can change the api bitcoin-chart-cli, or even better, add multiple apis for the scenario where one gets locked.

Here the API path, it doesn't looks as a complex code, maybe you can implement another api by yourself.

https://github.com/madnight/bitcoin-chart-cli/blob/master/src/CryptoCompareAPI.js

Yes I saw that file. First I tried to hardcode a simple workaround where it connects to a Tor SOCKS5 proxy, but this API doesn't like that.

I have gigantic 3TB of Proton Drive storage that is just sitting there and collecting dust, so perhaps I could scrape price data and package it up there for safe keeping. The only problem is that that doesn't have an API, so it will be slow periodic manual uploads, while I query the Binance endpoint, which I have experience in doing.

[PawGo]

Did you consider https://www.coingecko.com/en/api/pricing?

It might be easy task, depending how big granularity you need on API side*, but $50 one-time payment is a quite low price for a "lifetime guarantee". It may cover a very simple/cheap web hosting for 1-2 years where you may run your script to download the data and re-pack it to desired API struct.

Do you want to have a response for a request for an average price for a given day, hour, minute? Or do you want API to return all prices for a given period and you do the calculations yourself?

[bitmover]

Do you want the data in a web project (js) or in python?

For $50 I can solve your problem using binance api (maybe some other). I will make a web page or a jsfiddle (or a python notebook) so you can see the solution.

About historical data, binance is limited to 2020.
I will host old data (static, 2010-2020, small file) in  the front end in a json file forever for free in my website ( or in yours )

You will probably never get blocked (unless you have like millions of visitors).

 Edit: I can use coingecko (or any other  ) as back up in the code as well, in case binance blocks your requests

[bitmover]

Hello NotATether

I had a free time and i made this jsfiddle for you. Using coingecko and binance api. I could use only coingecko as well.
Take a look and feel free to use in your projects.
http://jsfiddle.net/yd7q495u

Coingecko api free plan has a nice limit and data from 2013

https://support.coingecko.com/hc/en-us/articles/4538771776153-With-the-free-plan-what-are-the-rate-limits-
With the free plan, what are the rate limits?

10 - 50 calls per minute rate limits on shared infrastructure.

As a free user, you will be able to make up to approximately 2 million calls per month

Binance api also have nice limits. This fiddle is updating price from binance every 15 seconds.