Blockchain Data Service- BDSBlockchain data service (BDS) is launched by JD Cloud's Blockchain Department. It stores the linked and unstructured data of blockchain in a structured way through technical means, and synchronizes it into a data warehouse of high performace in real time.
Users can use the blockchain data query tool to accomplish simple condition query and complex SQL query, so as to help innovative enterprises who need blockchain data to run business.
Blockchian Data Service has the following advantages:
Comprehensive: provide data for a lot of blockchain programs including BTC,ETH,EOS,LTC,XRP and BCH. Also define more than 100 data indexs involving users,tansactions,blocks and so on. All data could be queired with internet available wichi is accurate and efficient.
Convenience: provide a strongly interactive BI tool for data visualization which offers real time generation of various data reports plus graphical interfaces and also support conditional query by clicking and draging.
Speciality:support using SQL language to conduct many complicated querys.
Blockchian Data Service of JD cloud is the first product in China to provide online analysis service of blockchain data.
Product BackgroudIn recent two years, with the rise of blockchain technology, the whole industry is full of contention. There are famous public chain projects such as BTC, ETH, XRP, and well-known alliance chain projects such as Fabric, Enterprise Ethereum Alliance, Corda R3, etc.
Whether public or alliance chains, in fact, each is an independent island of information, data and information can not circulate between each other. In the data-oriented Internet era today , how to integrate the data and information of these block chains together and make use of the relatively mature data analysis tools to analyze and model the block chain data has become a question that JD Cloud Blockchain Department has been thinking about.
Finally, JD Cloud Blockchian Department upholds the original intention of product design and launches the Blockchain Data Service.
Product Framework:https://zhulang.s3.cn-north-1.jdcloud-oss.com/%E5%9B%BE%E7%89%871.pngFor blockchain data, because most of the public-chain nodes are deployed overseas, the deployment of full-node synchronization in the domestic environment often results in a synchronization failure. Relying on the strong fundamental capability of JD Cloud, we deploy lots of node overseas to ensure real-time synchronization of public chain data for each independent public chain.
We observed that the top three cryptocurrencies are BTC, ETH and XRP, among which the total storage space of BTC node is the least. At present, the total storage space of BTC node is 200 GB, while of ETH node it is close to 3 TB. The largest storage space of ETH node is XRP, which is close to 10 TB, and the normal physical server is no longer available for such considerable amount of data.
In the process of analyzing unstructured data of public chain, we initially used the exposed interface of the full node of public chain to invoke the original data of the query chain through RPC. This method is convenient, but has following problems :
Data timeliness: Block data synchronized by all nodes need to be persisted to disk before it can be obtained through RPC interface.
Performance loss: In order to responds to RPC requests, full nodes need to read data from disk which will have IO loss and CPU comsuption for processing all external requests.
After careful consideration, we modify the source code of full node in public chain directly and when full nodes synchronize the block data, we directly push the data to large data analysis platform, thus ensure the timeliness of data.
Custom dataFor customized data, we has innovated our unique custom data through combining AI, web crawler and other technical means.
Because of the anonymity of the blockchain itself, it is impossible to know the true ownership of each address in the block chain. JD Cloud Blockchain Department obtains all information related to blockchains on the Internet through web crawlers, combines it to the transaction behavior on chains, finally locates the attribution of address using complicated AI technology.
Big Data Analysis PlatformIn the aspect of big data analysis, based on block chain data and custom data like transaction amount, transaction times, active address and new address, JD Cloud Blockchian department has created many meaningful and exclusive data indexs after secondary calculation. These data indexs include address classification, market value ratio, user retention and so on.
Data analysis includes not only real-time data analysis, but also offline data analysis. For example, when classifying new addresses every day, it is necessary to combine the clustering judgment of full historical address data. This data index is currently used offline analysis, running data aggregation tasks regularly every day. At present, it takes about 3 hours to classify new addresses every day and we can predict that with the increase of amount of data. , the time consumed of data aggregation will increase accordingly.
Therefore, relying on the basic computing capability of JD Cloud, we have calculated more than 100 basic and complex data indexs in advance through data analysis cluster services.
For example, let's take a look at the table structure information of BTC statistics day transaction.
btc_statistics_day_transaction
https://zhulang.s3.cn-north-1.jdcloud-oss.com/%E5%9B%BE%E7%89%875.pngThere are many interesting data indexs, such as:
Resurrection Address: It means that this address had no trade action for a long time, and then suddenly traded today. If there is a large number of resurrection addresses on that day, in fact, it can indirectly indicate that users are beginning to be active.
Address Activity Rate: The proportion of addresses that generate transactions to the total number of addresses. Through this index, we can see the overall market activity. The higher the value, the more active the market.
Storage Rate: Represents the proportion of untransacted cryptocurrency to the total currency supply. This index reflects the circulation of cryptocurrency in the market. The lower the value, the higher the circulation rate of cyrptocurrency.
Block Link Data BI ToolWe also prived an interactive BI tool of data visualization which can generate the data report you want by simple click and drag. It also supports changing the data visualization results to various graphical interfaces such as bar chart, line chart, pie chart and so on.
Data report example:
BTC address tablehttps://zhulang.s3.cn-north-1.jdcloud-oss.com/%E5%9B%BE%E7%89%872.pngIf you can write SQL sentence, you can also write some complex SQL language to achieve the associated query between tables, so as to make more intuitive data analysis and comparison.
Table associated query example: BTC transaction data table and BTC transaction input data tablehttps://zhulang.s3.cn-north-1.jdcloud-oss.com/%E5%9B%BE%E7%89%873.pngDash board example: https://zhulang.s3.cn-north-1.jdcloud-oss.com/%E5%9B%BE%E7%89%874.pngApplication scenarioFor application scenarios, we can do a variety of application services based on these data. For example, several simple scenarios:
1.We could parse the operation status of a specific chain and provide rating reports according to the data we have.
2.Trading atlas analysis can be done by analyzing the transaction actions on chian, which also can be used to analyze the existence of illegal activities such as money laundering through cryptocurrency, illegal fund-raising and so on.
3.Through the analysis and comparison of various data indexs, we can make some business insights in blockchian industry. Also we could predict the future development trend of cryptocurrency and judge its potential value.
Product summaryJust as JD Cloud provides public cloud services including basic computing, storage and network services to all users, JD Blockchain department has been committed to the related research of the fundamental technology of blockchain, and has carried out the most basic data extraction and analysis services for the public chains. We are devoted to serve all groups or individuals that have entrepreneurship based on blockchain data, thus greatly reduce the entry threshold.
In the future, we plan to provide open source the blockchain data service, so that more developers can participate in. We can extract and analyze the basic data of not only public chain but also alliance chain and private chain. We are devoted to create a Google-like data query enginee for blockchian, where all Blockchain related contents could be easily queried here.
The blockchain department of JD Cloud has always followed the “GOOD” culture of Jd Cloud:
Growth mindset
Obsessed wth customers
Open source & Open minded
Data drive Product link:
https://bds.jdcloud.com/Twitter:
jdcloudbdsEmail:
jdcloud-bds@jd.com
