Pruning

[21E14]

--

[franky1]

Has any progress been made with respect to pruning? The exp. regression trendline puts the blockchain size at ca. 150 GB this time next year, with the 130MB blkXXXXX.dat files being filled out once every minute and a half ( http://21e14.com/Blockchain/Blockchain.aspx )...

blocks have a 1MB limit each.. at the moment each block is only 20% utilised. meaning 0.2mb
6 blocks an hour over 24 hours, over 365 days

utility growth estimate = 10gb a year
full limit utilised estimate = 52gb a year

i see in no way that next year the block chain would be anywhere near 150GB larger.. the maths just doesnt work that way.

the only reason your seeing new 130mb files appearing on your system every few minutes, is because your system is updating historic data.. not live current data.. (your wallet is not synced yet)

at the very most live data is only 144mb a day
(1mb block limit x 144 blocks a day)

[fernando]

Has any progress been made with respect to pruning? The exp. regression trendline puts the blockchain size at ca. 150 GB this time next year, with the 130MB blkXXXXX.dat files being filled out once every minute and a half ( http://21e14.com/Blockchain/Blockchain.aspx )...

blocks have a 1MB limit each.. at the moment each block is only 20% utilised. meaning 0.2mb
6 blocks an hour over 24 hours, over 365 days

utility growth estimate = 10gb a year
full limit utilised estimate = 52gb a year

i see in no way that next year the block chain would be anywhere near 150GB larger.. the maths just doesnt work that way.

the only reason your seeing new 130mb files appearing on your system every few minutes, is because your system is updating historic data.. not live current data.. (your wallet is not synced yet)

at the very most live data is only 144mb a day
(1mb block limit x 144 blocks a day)

Well, that's a lot of data anyway. Not everybody needs the full blockchain, but it would be nice if something gets achieved about pruning it.

[zetaray]

I have read some discussions on a DHT blockchain, or a superblock, but nothing concrete has come out from them. I have switched to a light client for some time now, 20G is already too big for my computers to handle.

[franky1]

Well, that's a lot of data anyway. Not everybody needs the full blockchain, but it would be nice if something gets achieved about pruning it.

at the moment the blockchain can fit onto a 32gb SD card which costs only a few dollars/pounds so its not like people need to buy multiple servers, costing thousands..

secondly even as far back as the genesis block, there are still unspent coins. so trying to prune away blocks that have 100% spent values which also lead to new blocks that have 100% spent values to ensure the blockchain doesnt break due to having no previous block to reference, is at the moment a big task.

after all new blocks have references of old blocks, so there would need some sort of buffer of a block or two from an unspent block before getting to the pruned(spent) block.. thus out of all blocks made, there are very few that are prunable..

which brings me back to the first point. data is only a sub $20 data store. so the importance to prune is still not a big deal, and the solution wont show much of a change in data size (EG current 20gb would maybe get 1GB pruned off, max)

[franky1]

I have read some discussions on a DHT blockchain, or a superblock, but nothing concrete has come out from them. I have switched to a light client for some time now

there have ben many 'theories' of having what we have now (full nodes) to be called th future supernodes (containing all blockchain) and clients that just download headers, and only download a block that is relevant to their transactions/addresses from the supernodes

but this then dilutes the amount of copies of the ledger if this became the norm as only maybe 1000 people would want to be full nodes, making it slightly more riskier for the community.

20G is already too big for my computers to handle.

games like call of duty, world of war craft, titan falls.. all have datasizes that surpass 20gb. so most people truly into computing and gaming, dont have an issue with 20gb. just a shame you cant download these great games

[fernando]

Well, that's a lot of data anyway. Not everybody needs the full blockchain, but it would be nice if something gets achieved about pruning it.

at the moment the blockchain can fit onto a 32gb SD card which costs only a few dollars/pounds so its not like people need to buy multiple servers, costing thousands..

secondly even as far back as the genesis block, there are still unspent coins. so trying to prune away blocks that have 100% spent values which also lead to new blocks that have 100% spent values to ensure the blockchain doesnt break due to having no previous block to reference, is at the moment a big task.

after all new blocks have references of old blocks, so there would need some sort of buffer of a block or two from an unspent block before getting to the pruned(spent) block.. thus out of all blocks made, there are very few that are prunable..

which brings me back to the first point. data is only a sub $20 data store. so the importance to prune is still not a big deal, and the solution wont show much of a change in data size (EG current 20gb would maybe get 1GB pruned off, max)

Thanks for the info! If the pruning would only reduce size in 5%, then it is definitely not worth the trouble. I know that storage is dirt cheap now, it's just that it doesn't feel efficient to have that small cost in so many places 

[franky1]

Thanks for the info! If the pruning would only reduce size in 5%, then it is definitely not worth the trouble. I know that storage is dirt cheap now, it's just that it doesn't feel efficient to have that small cost in so many places  

not efficient?
the whole point of distributed ledger is the security of bitcoins. if there was only 1 database that everyone views.. then bitcoin would break. having copies of it on thousands of computers is the beauty of bitcoin. its the main point of how bitcoin works

[21E14]

Has any progress been made with respect to pruning? The exp. regression trendline puts the blockchain size at ca. 150 GB this time next year, with the 130MB blkXXXXX.dat files being filled out once every minute and a half ( http://21e14.com/Blockchain/Blockchain.aspx )...

blocks have a 1MB limit each.. at the moment each block is only 20% utilised. meaning 0.2mb
6 blocks an hour over 24 hours, over 365 days

utility growth estimate = 10gb a year
full limit utilised estimate = 52gb a year

i see in no way that next year the block chain would be anywhere near 150GB larger.. the maths just doesnt work that way.

the only reason your seeing new 130mb files appearing on your system every few minutes, is because your system is updating historic data.. not live current data.. (your wallet is not synced yet)

at the very most live data is only 144mb a day
(1mb block limit x 144 blocks a day)

The 1MB block limit itself will need to be upped, as the block size exp. regression trendlines themselves are at the current limit sometime between 8/'14 and 8/'15 ( http://21e14.com/blocksize/blocksize.html )...

Moreover, if the exponential transactional traffic growth patterns hold, 130MB worth of raw block data will be generated every 84 seconds ( y = 9.547e^-0.008x ) this time next year.

[franky1]

The 1MB block limit itself will need to be upped, as the block size exp. regression trendlines themselves are at the current limit sometime between 8/'14 and 8/'15 ( http://21e14.com/blocksize/blocksize.html )...

Moreover, if the exponential transactional traffic growth patterns hold, 130MB worth of raw block data will be generated every 84 seconds ( y = 9.547e^-0.008x ) this time next year.

where are your numbers even coming from??

84 seconds??

you do realise that a block is generated once every 10 minutes.. not 1 minute 24 seconds right?

and that lets say they raised it to 4mb per block. thats 4mb per 10 minutes..
or 0.56mb per 84 seconds
or simply 0.4mb per minute

bit 130mb every 84 seconds??? again where does this 84 second number or the 130mb number come from..

i think your talking about historic data download of the users bitcoin client while re-syncing and not actual blockchain block sizes of mining blocks..

[21E14]

The 1MB block limit itself will need to be upped, as the block size exp. regression trendlines themselves are at the current limit sometime between 8/'14 and 8/'15 ( http://21e14.com/blocksize/blocksize.html )...

Moreover, if the exponential transactional traffic growth patterns hold, 130MB worth of raw block data will be generated every 84 seconds ( y = 9.547e^-0.008x ) this time next year.

where are your numbers even coming from??

84 seconds??

you do realise that a block is generated once every 10 minutes.. not 1 minute 24 seconds right?

and that lets say they raised it to 4mb per block. thats 4mb per 10 minutes..
or 0.56mb per 84 seconds
or simply 0.4mb per minute

bit 130mb every 84 seconds??? again where does this 84 second number or the 130mb number come from..

i think your talking about historic data download of the users bitcoin client while re-syncing and not actual blockchain block sizes of mining blocks..

But of course, blocks would continue to be generated at the current pace. The blkXXXXX.dat files are used to illustrate the growth in Bitcoin's transactional volume, as reflected by the ever decreasing 'life span' of a blkXXXXX.dat file. Mind you, it tok 2.3 years to generate the very fist 130MB of raw block data. The 150th blk was generated in 3.4 days

Hence, the cited numbers are rooted in the aforementioned  exp. trendline described by [y = 9.547e^-0.008x], where y is the time (in days) it takes to fill out one 130MB blkXXXXX.dat file, and x is the variable [XXXXX] in blkXXXXX.dat.

In this model, the time will collapse to 0.000972371105114262 days (ca. 84 sec.) in the next 1000 blkXXXXX.dat files that would only take another year. By this time, the blockchain size will have expanded to ca. 150GB and would explode from there on.

Of course, we cannot say with certainty just how long this exponential growth phase in Bitcoin's presumed logistic curve [often used to illustrate diffusion of innovations] will last, but its impact is undoubtedly profound. Just look at the current 124,366,605 GH/s hash rate. It was 181,687 this time last year - 684 times less.

[Kprawn]

Sorry, if this is a stupid question. Would "Pruning" improve overall performance of the blockchain? {Transaction speed / Miner processing}? I know verification of tx are based on the hash power of the miners etc, but they still refferencing a ledger {database} and the less data there are, should in theory increase the processing?

The 144mb live data per day, is also still a lot, for people in 3rd world countries, who still use data caps on their internet. {In many countries 5GB & 10GB is the normal cap per month for internet}Remember they use the cap for other things too.
 

[dserrano5]

The blkXXXXX.dat files are used to illustrate the growth in Bitcoin's transactional volume, as reflected by the ever decreasing 'life span' of a blkXXXXX.dat file. Mind you, it tok 2.3 years to generate the very fist 130MB of raw block data. The 150th blk was generated in 3.4 days

But that "decreasing life span" has a limit. As of now, no more than 144 Mb of raw block data (144 blocks, 1 Mb each)  would be created in a day (in average) so the "life span" of a blk* file would be around 21 hours.