New much improved version of
assetparser.pychangelog:* asset holders, accounts:
** before only the workarounds for HZ were implemented, now all also works on NXT again
** funfacts accounts: number of asset positions, asset holding accounts
** HZ: pickle/unpickle all accounts (for faster debugging)
* assets:
** NXT: genesis account block time --> time calculations
** funfacts assets: simple asset shareholder analysis (mean median max, >10, >100, >1000)
* trades:
** Nxt version: sellers & buyers are given by API (faster & easier than HZ workaround of getting all order txs)
** HZ: extracted order ids can now also contain bidOrders, or freely choosable (good for 'shareholders.py')
** progress: show a "*" per downloaded asset
** funfacts: simple number-of-trades analysis (mean median max, >10, >100, >1000)
** analyzeAllTrades can easily handle 100k trades
* architecture:
** all asset-specific analysis is already or will go into shareholders.py
** most api calls now in an own module apiclient.py
* observables - what cannot be calculated for HZ now also include as empty columns, to allow easier platform mixing
* altprices.py - usd as a base, due to missing data
* logging - better and more consistent logging output
Future:Open for suggestions.
Sample output:NXT assetparser.py[03:51:30.118] nxt:
[03:51:30.118] API-downloading all assets ...
[03:51:31.365] API-downloading all assets: 572 assets. DONE.
[03:51:31.365] pickling 'getAllAssets' API results to file, for later reuse ...
[03:51:31.500] pickling 'getAllAssets' API results to file, for later reuse: DONE.
[03:51:31.500] API-Downloading all accountIds which are holding the 572 assets (async, MAXTHREADS=1000) ...
[03:51:34.831] API-Downloading all accountIds which are holding the 572 assets (async, MAXTHREADS=1000): 572 threads finished. DONE.
[03:51:34.849] Funfacts: Mentioned accountIds: 37762. Unique accounts holding assets: 11872
[03:51:34.851] Among the 412 assets with >1 shareholder (at least 1 more than the issuer) ...
[03:51:34.851] ... number of shareholders: min=2 mean=91.3 median=11.5 max=4157
[03:51:34.851] ... >10 shareholders: 213 >100 shareholders: 57 >1000 shareholders: 11
[03:51:34.869] API-Downloading trades (async) of 572 assets
[03:51:34.934] Progress: Each '*' represents one getTrades API-call:
* * * * ** *** ** ** ** *** * * ** * * * ** * * *** * * * *
* * * * * *** * ** *** ** ** *** ** ** * * ** ** * *** ** ***
* ** *** ** ** ** ****** * *** ** * **** ***** ** ** ** ******* *
** * ** * * ** * ** ******* * * * *** *** **** ***** *** * * * **** ** ** **** *** * * ** * **
** *** ** * ** * ****** **** ** * ** ** ** ******
*** * **** * * *** ** *
** ** * * ** ** *** * ** ** * ***
** * ** * * **** * *** ** * * * ***
* * ** ** * * * ** * * *** ** ** * *** ** ** ***
** * ******* *** ** *** **** *** * ** ** * **
** * ** * * * ** ***** * ** ** * * * * * * ** * ** ***
* ** * * * * * * * * * *
[03:51:41.230] API-Downloading trades (async) of 572 assets: 397 threads finished. DONE.
[03:51:41.232] From 397 assets with trades, a total of 109325 trades downloaded.
[03:51:41.233] Funfacts: Among the 397 assets with trades ...
[03:51:41.233] ... number of trades: min=1 mean=275.4 median=27.0 max=9046
[03:51:41.233] ... >10 trades: 244 >100 trades: 116 >1000 trades: 30
[03:51:41.233] API downloading accounts of all asset issuers ...
[03:51:43.644] API downloading accounts of all asset issuers: 572 threads. DONE.
[03:51:43.707] Loaded price data. Most recent: NXT=0.00005647BTC BTC=266.93USD (nearest btc/usd from: 07 Jul 2015 18:59:20 +0000)
[03:51:51.434] Analyzing 572 assets with 109325 trades. DONE.
[03:51:51.434] printing to file (nxt-assets_20150714-155151.csv) by redirecting stdout. ...
[03:51:51.477] printing to file (nxt-assets_20150714-155151.csv) by redirecting stdout. Stdout redirected back. DONE.
[03:51:51.477] analyzing output (nxt-assets_20150714-155151.csv):
[03:51:51.493] 572 rows(assets), 42 columns(observables) with 18249 non-empty cells (table 76% filled)
[03:51:51.588] ready. DONE.
Sample outputHZ assetparser.pyThis is more complicated, because the old API of HZ forced me to invent some clever work-arounds. But then it results in almost the same information:
[04:41:26.289] nhz:
[04:41:26.289] API-downloading all assets ...
[04:41:27.345] API-downloading all assets: 147 assets. DONE.
[04:41:27.345] pickling 'getAllAssets' API results to file, for later reuse ...
[04:41:27.384] pickling 'getAllAssets' API results to file, for later reuse: DONE.
[04:41:27.385] B.E.-API-downloading all account ids (from NHZ block explorer) ...
[04:41:29.595] B.E.-API-downloading all account ids (from NHZ block explorer): 4902 accounts. DONE.
[04:41:29.595] API-Downloading 4902 accounts (async, MAXTHREADS=1000) ...
[04:41:44.983] API-Downloading 4902 accounts (async, MAXTHREADS=1000): 4902 threads finished. DONE.
[04:41:44.993] pickling 'allAccounts' API results to file, for later reuse ...
[04:41:46.174] pickling 'allAccounts' API results to file, for later reuse: DONE.
[04:41:46.190] Funcfacts HZ: 2395 accounts are holding (median 230.0 mean 417535.7) HZ, and 752 are holding assets.
[04:41:46.190] API-Downloading trades (async) of 147 assets
[04:41:46.265] Progress: Each '*' represents one getTrades API-call:
* * * ** **** ** ** * * * * ** *** *** * * * ** ** *** ***** **** * *
* *** ** ** * * * * *** ** *** * ***** * ** *** ** * * ** * * * * * *
[04:41:47.607] API-Downloading trades (async) of 147 assets: 90 threads finished. DONE.
[04:41:47.607] From 90 assets with trades, a total of 2923 trades downloaded.
[04:41:47.607] Funfacts Trades: Among the 90 assets with trades ...
[04:41:47.607] ... number of trades: min=1 mean=32.5 median=5.0 max=611
[04:41:47.607] ... >10 trades: 29 >100 trades: 7
[04:41:47.607] extracting all txIds from trades ...
[04:41:47.607] extracting all txIds from trades: 461 trades with 2923 orders of type ('askOrder',): 1448 unique orderTxs. DONE.
[04:41:47.607] API-Downloading 1448 Order transactions (async) ...
[04:41:52.288] API-Downloading 1448 Order transactions (async): 1448 threads finished. DONE.
[04:41:52.289] extracting sellers of trades from askOrder -transactions ...
[04:41:52.289] extracting sellers of trades from askOrder -transactions: 2923 trades. DONE.
[04:41:52.289] API downloading accounts of all asset issuers ...
[04:41:53.654] API downloading accounts of all asset issuers: 147 threads. DONE.
[04:41:53.654] Workaround for OLDAPI - go through all accounts, to collect asset holders ...
[04:41:53.763] Analyzed 752 shareholders, with 1409 asset balances.
[04:41:53.763] Workaround for OLDAPI - go through all accounts, to collect asset holders: DONE.
[04:41:53.763] Funfacts Assets: Among the 87 assets with >1 shareholder (at least 1 more than the issuer) ...
[04:41:53.763] ... number of shareholders: min=2 mean=15.5 median=4.0 max=310
[04:41:53.763] ... >10 shareholders: 18 >100 shareholders: 3 >1000 shareholders: 0
[04:41:53.852] Loaded price data. Most recent: HZ=0.00000092BTC BTC=266.93USD (nearest btc/usd from: 07 Jul 2015 18:59:20 +0000)
[04:41:54.029] Analyzing 147 assets with 2923 trades. DONE.
[04:41:54.029] printing to file (nhz-assets_20150714-164154.csv) by redirecting stdout. ...
[04:41:54.046] printing to file (nhz-assets_20150714-164154.csv) by redirecting stdout. Stdout redirected back. DONE.
[04:41:54.046] analyzing output (nhz-assets_20150714-164154.csv):
[04:41:54.063] 147 rows(assets), 42 columns(observables) with 4146 non-empty cells (table 67% filled)
[04:41:54.165] ready. DONE.
Open for suggestions.
Availability For now:
... an earlybird approach - you could have this TODAY!
* How much would you pay for (private) access to the current static results table? With an option for 3 data updates later.
* How much would you pay for providing all static results (publicly) on your website?
For your own (
private!) detailed analysis ...
Buy snapshot spreadsheet tables of the 2 whole asset markets analysis, of these sizes:
* HZ: 147 rows(assets), 42 columns(observables) with 4146 non-empty cells (table 67% filled)
* NXT: 572 rows(assets), 42 columns(observables) with 18249 non-empty cells (table 76% filled)
Buy snapshot rows of selected (e.g. of your own) assets:
* 1 row(asset) with 42 columns(observables)
Tell me what you are interested in, and make an offer how much this is worth to you.Money talksInviting all ideas how to monetize this in the future.If you already found this interesting, informative, inspiring ... consider to
GIVEBACK.
Thanks!