Crypto prices, trades & volume in realtime excluding fake volume

[Dwynr]

Hello!

I've been working on a website for a while now that shows trading volume and prices in realtime excluding wash trading.

My main incentive for creating this site was me getting sick of all the fake volume exchanges listed on CoinMarketCap.
I'm using a list of exchanges created by BTI (http://www.bti.live/exchanges/) and presented to the US SEC when they applied for the BTC ETF.
Some of the approved exchanges still include (+-5%) wash trading on a few pairs and thats why I also wrote an algorithm that mostly detects strange behaviour and filters it out.

The main difference between this site and all other price trackers is that it saves all trades made on the exchange in a Database and then calculates price and volume rather than just trusting the exchanges API.
The site also comes with a REST and WebSocket API for free which you can use to build your own tools (mostly trading bots, I guess).

Feel free to join the Discord (https://discord.gg/eEQq5PF) if you find a bug, have a question or want to submit a feature request!

The website is in active development so expect things to change.

Link: https://marketcap.cc


__________________________________________________________________________

Semi-huge update: You can now see liquidity of every asset (collected in realtime from all exchanges' orderbooks), sort tables by clicking their headers and see high volume transactions on selected blockchains (whale alert).
Order book viewing will be enabled soon (for every market).
__________________________________________________________________________

Update: The RESTful & Socket.io APIs are now fully documented (https://marketcap.cc/api). Both are completely for free and have no limits, because data should be free Smiley If you need a specific RESTful endpoint for your project don't hesitate to contact me, I'll try my best to include it into the API.
__________________________________________________________________________

Update: You can now see Liqudity of every market, exchange etc. instead of asset-based liqudity only. Ranking for Liquidity also added.
__________________________________________________________________________

"Update teaser": I am currently working on a full port of the website to an App. The App will be available for Android & iOS, starting with an Android Beta release first. Here is a screenshot of the work in progress:


__________________________________________________________________________

Great news! Google Play just approved the official App for MarketCap.cc. You can find it here: https://play.google.com/store/apps/details?id=cc.marketcap.app
__________________________________________________________________________

[1714169380]

1714169380

[1714169380]

1714169380

[1714169380]

1714169380

[rijaljun]

I have a question, it's excluding fake volume? So, what will happen if exchanges use a fake volume? are they will have different volume between your site and the exchange site or... please give me some explanation about this.

[OmegaStarScream]

If you are retrieving the volume/orders using the exchange's API, how can you 100% distinguish the real orders from the fake ones?

[Dwynr]

I have a question, it's excluding fake volume? So, what will happen if exchanges use a fake volume? are they will have different volume between your site and the exchange site or... please give me some explanation about this.

Simply: yes.

If you are retrieving the volume/orders using the exchange's API, how can you 100% distinguish the real orders from the fake ones?

It is pretty easy to see if a market pair is getting wash traded by looking at the frequency and amount and quantitiy of each trade.
Please refer to this article I wrote: https://medium.com/@Dwynr/building-a-website-that-shows-real-prices-and-trading-volumes-of-cryptocurrencies-aa4d9a2f4772 if you want to learn more about it.

[elda34b]

Seems like you'll require a lot of storage to maintain the wash trading activity.

So, at what point your algorithm considers a wash trading occurs? Suppose there is an X market trade that was done by a user, will this consider as wash trading per se because the quantity exactly matches the sell/buy order?

[DiamondCardz]

So you're storing every single trade that happens on every exchange for every cryptocurrency pair? That sounds like it's going to take a fairly rapidly increasing amount of storage. Perhaps a better idea would be to store the prices and volumes for each day, and then store only the trades which are flagged as a wash trade - that allows you to calculate the real volume values without having to store every single transaction.

[Dwynr]

So you're storing every single trade that happens on every exchange for every cryptocurrency pair? That sounds like it's going to take a fairly rapidly increasing amount of storage. Perhaps a better idea would be to store the prices and volumes for each day, and then store only the trades which are flagged as a wash trade - that allows you to calculate the real volume values without having to store every single transaction.

The main reason for storing all trades is that I will open an API endpoint to fetch historical trades for the public. This can be REALLY useful for people doing TA, coding trading bots etc.
Data is stored in Redis (KeyValue in Memory Database) for 48h and then offloaded to a BigData database after that, where storage capacity isnt really a problem. Data growth per day is about 10GB.

[rijaljun]

The main reason for storing all trades is that I will open an API endpoint to fetch historical trades for the public. This can be REALLY useful for people doing TA, coding trading bots etc.
Data is stored in Redis (KeyValue in Memory Database) for 48h and then offloaded to a BigData database after that, where storage capacity isnt really a problem. Data growth per day is about 10GB.

So, data storage must be no problem for you. How about the whole data fetching process? Doesn't it will take longer time than taking from exchange API?

[Dwynr]

The main reason for storing all trades is that I will open an API endpoint to fetch historical trades for the public. This can be REALLY useful for people doing TA, coding trading bots etc.
Data is stored in Redis (KeyValue in Memory Database) for 48h and then offloaded to a BigData database after that, where storage capacity isnt really a problem. Data growth per day is about 10GB.

So, data storage must be no problem for you. How about the whole data fetching process? Doesn't it will take longer time than taking from exchange API?

No, fetching data for the statistics (24h change, volume, price etc.) is pretty fast since the data is in memory storage (Redis). After it has been processed and calculated its offloaded to the other Database.

[BrewMaster]

so you are excluding the "obvious" fake volume not fake volume in general. the exchanges can still continue producing fake volume and if someday your website (or the method in general) became popular they can easily mask it.

in addition to that you are putting too much efford and focus in something that doesn't really matter for sites such as yours that list coins. you are still making the same mistake as coinmarketcap.com and rank coins based on their market capitalization which is a completely meaningless and wrong approach.
not to mention that your site lacks the option to list prices against bitcoin whcih in my opinion is the only thing that matters, you only have USD option.

[Dwynr]

so you are excluding the "obvious" fake volume not fake volume in general. the exchanges can still continue producing fake volume and if someday your website (or the method in general) became popular they can easily mask it.

in addition to that you are putting too much efford and focus in something that doesn't really matter for sites such as yours that list coins. you are still making the same mistake as coinmarketcap.com and rank coins based on their market capitalization which is a completely meaningless and wrong approach.
not to mention that your site lacks the option to list prices against bitcoin whcih in my opinion is the only thing that matters, you only have USD option.

Ranking based on Liquid Market Cap will come in the next update. I will also include pricing against Bitcoin. I am taking any suggestions and will try to make the site better everyday. Not worth talking the site down since it's only been online for roughly a week now.

[Theb]

I've seen an article about you which I saw in the press section of the forum and I'm still a bit skeptical on how accurate you will calculate and segregate the fake volume from crypto exchanges, even removing the -/+ 5% won't be enough for you to have the true volume on each exchange. So can you explain a little bit more on how or what you will do to remove those "strange volume" in their data and what you will do to identify them? How do you even consider one to be a strange behavior in the first place?

[johndoughsmith2019]

The idea of being able to "see" true volume of different coins is very valuable. Looking at the website i real like the UI, it's clean, easy to navigate, and responsive. Haven't tried out the API so cannot speak to that.

The fundamental core of this project is the accuracy of the algorithm used for filtering out wash trade, of which I still have no clue on how it is done after reading you article (https://medium.com/@Dwynr/building-a-website-that-shows-real-prices-and-trading-volumes-of-cryptocurrencies-aa4d9a2f4772). If the algorithm are just detecting certain trading patterns, then I don't see how it can be used to detect wash trade with sufficient accuracy since exchange might have or come up with sophisticated wash trading patterns.

That being said, I do believe idea is still valuable even if the algorithm only filters out a majority of the fake volumes rather than 100%.

Great job OP.

[Dwynr]

The idea of being able to "see" true volume of different coins is very valuable. Looking at the website i real like the UI, it's clean, easy to navigate, and responsive. Haven't tried out the API so cannot speak to that.

The fundamental core of this project is the accuracy of the algorithm used for filtering out wash trade, of which I still have no clue on how it is done after reading you article (https://medium.com/@Dwynr/building-a-website-that-shows-real-prices-and-trading-volumes-of-cryptocurrencies-aa4d9a2f4772). If the algorithm are just detecting certain trading patterns, then I don't see how it can be used to detect wash trade with sufficient accuracy since exchange might have or come up with sophisticated wash trading patterns.

That being said, I do believe idea is still valuable even if the algorithm only filters out a majority of the fake volumes rather than 100%.

Great job OP.

The RESTful API will be documented in the coming days, the WebSocket API already is. The algorithm cannot truly filter out 100% of all fake volume, but I'll try to improve it over time. As you mentioned, it currently only looks for pre-determined patterns and filters them out. The next version of the algo will be self learning, which will cover almost all patterns an exchange might use (I hope, it's just a theory for now ).
Thanks for your feedback!

[rijaljun]

so you are excluding the "obvious" fake volume not fake volume in general. the exchanges can still continue producing fake volume and if someday your website (or the method in general) became popular they can easily mask it.

We can't really make exchanges to stop faking volumes anyway, it will be always happening and I don't think this project is aiming to stop it directly. All I get from OP is to remove those fake volumes from price tracker site and creating a place where people can find a more genuine prices.

Although it might only obvious fake volumem at start and volume fakers won't stop, at least OP has a big efforts to avoid tracking fake volumes and of course the site and algo will improve if OP really commit to develop this.

[Slow death]

https://coinmarketcap.com



https://marketcap.cc

The difference is very big, your site is excellent. I have a suggestion:

can you add the ann threads of the coins? something like that:

[Dwynr]

https://coinmarketcap.com



https://marketcap.cc

The difference is very big, your site is excellent. I have a suggestion:

can you add the ann threads of the coins? something like that:

Thanks for your feedback and yes, more information like website, block explorer, ANN thread etc. for coins will be added in the next update.

[BrewMaster]

Not worth talking the site down since it's only been online for roughly a week now.

sorry, i didn't mean to belittle your efforts, i was just criticising the site. as long as you work on the "ranking" mechanism that you use and don't rank coins based on their market capitalization, it will be a lot better than the alternative.

[Dwynr]

so you are excluding the "obvious" fake volume not fake volume in general. the exchanges can still continue producing fake volume and if someday your website (or the method in general) became popular they can easily mask it.

We can't really make exchanges to stop faking volumes anyway, it will be always happening and I don't think this project is aiming to stop it directly. All I get from OP is to remove those fake volumes from price tracker site and creating a place where people can find a more genuine prices.

Although it might only obvious fake volumem at start and volume fakers won't stop, at least OP has a big efforts to avoid tracking fake volumes and of course the site and algo will improve if OP really commit to develop this.

Thanks for your feedback and yes, I am comitted to continue the development. I am adding new features everyday, just check it out once in a while (or maybe use it full time).

[hello_good_sir]

It's a good attempt by a smaller company/single person to combat washed trading/market making/market manipulation, whatever new word they are using to call fake trade volumes - It seems like you are making a valid attempt to not counted the millions in faked volume, and even though I'm sure there will be some exchanges that can slip through, I congratulate you on your effort!

I do enjoy the design aspects of your listing site as well. Although I do get the feeling that text is hard to read, especially in dark mode and when you click a coin - the 4 boxes that are all black (black text, black background, etc) is really distracting and I don't enjoy it at all, maybe try it with white text?

Good luck mate, as long as this keeps being updated you'll have a banger on your side!