LoyceV
Legendary
Online
Activity: 3318
Merit: 16673
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 14, 2023, 10:52:17 AM |
|
I didn't know there's a procedure for it, I just compressed blocks and chainstate. 7z with LZMA2 algorithm could do better job, but i expect it'll be extremely slow. Even if it saves a few GB, that's only a few percent anyway.
|
|
|
|
LoyceV
Legendary
Online
Activity: 3318
Merit: 16673
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 14, 2023, 12:05:40 PM |
|
That's true. Last time i tried that with Bitcoin Signet, i got compression ratio 0.63 (505MB / 798MB). For comparison, compression you choose have ratio about 0.81 (397GB / ~486GB). This took over 2 hours on all CPU cores already.
|
|
|
|
Coin Gorilla
Newbie
Offline
Activity: 56
Merit: 0
|
|
February 14, 2023, 12:21:00 PM |
|
That's true. Last time i tried that with Bitcoin Signet, i got compression ratio 0.63 (505MB / 798MB). For comparison, compression you choose have ratio about 0.81 (397GB / ~486GB). This took over 2 hours on all CPU cores already. Does faster drive (something like PCI-e 4.0 NVMe or regular even regular SATA SSD) mean anything in this case or is it only CPU bound? I assume that in this case the CPU and core clock (IPC performance too) is the most important aspect?
|
|
|
|
LoyceV
Legendary
Online
Activity: 3318
Merit: 16673
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 14, 2023, 05:32:25 PM |
|
Does faster drive (something like PCI-e 4.0 NVMe or regular even regular SATA SSD) mean anything in this case No. The CPU is already the limiting factor, stronger compression means there's even less to write (per second).
Back to the growth of the blockchain: the daily amount of data increased a lot in the past week: [ ] blockchair_bitcoin_inputs_20230128.tsv.gz 2023-01-29 00:59 205M [ ] blockchair_bitcoin_inputs_20230129.tsv.gz 2023-01-30 00:59 200M [ ] blockchair_bitcoin_inputs_20230130.tsv.gz 2023-01-31 00:59 235M [ ] blockchair_bitcoin_inputs_20230131.tsv.gz 2023-02-01 00:59 221M [ ] blockchair_bitcoin_inputs_20230201.tsv.gz 2023-02-02 00:59 274M [ ] blockchair_bitcoin_inputs_20230202.tsv.gz 2023-02-03 00:59 257M [ ] blockchair_bitcoin_inputs_20230203.tsv.gz 2023-02-04 00:59 238M [ ] blockchair_bitcoin_inputs_20230204.tsv.gz 2023-02-05 00:59 289M [ ] blockchair_bitcoin_inputs_20230205.tsv.gz 2023-02-06 00:59 282M [ ] blockchair_bitcoin_inputs_20230206.tsv.gz 2023-02-07 00:59 363M [ ] blockchair_bitcoin_inputs_20230207.tsv.gz 2023-02-08 00:59 330M [ ] blockchair_bitcoin_inputs_20230208.tsv.gz 2023-02-09 00:59 319M [ ] blockchair_bitcoin_inputs_20230209.tsv.gz 2023-02-10 00:59 366M [ ] blockchair_bitcoin_inputs_20230210.tsv.gz 2023-02-11 00:59 328M [ ] blockchair_bitcoin_inputs_20230211.tsv.gz 2023-02-12 00:59 377M [ ] blockchair_bitcoin_inputs_20230212.tsv.gz 2023-02-13 00:59 447M [ ] blockchair_bitcoin_inputs_20230213.tsv.gz 2023-02-14 00:59 331M
|
|
|
|
Wronsk
Newbie
Offline
Activity: 12
Merit: 29
|
|
February 14, 2023, 06:37:04 PM |
|
I think that when the technology of data storage has evolved by changing scale, i.e. for example when the price of 10TB is at the current price of 1TB, it may be possible to increase the size of the blocks as this will not harm decentralisation, however we may be close to the theoretical limit in data storage
|
|
|
|
franky1
Legendary
Offline
Activity: 4228
Merit: 4490
|
|
February 14, 2023, 08:55:35 PM |
|
I think that when the technology of data storage has evolved by changing scale, i.e. for example when the price of 10TB is at the current price of 1TB, it may be possible to increase the size of the blocks as this will not harm decentralisation, however we may be close to the theoretical limit in data storage
(facepalm) 1990's hit the theoretical limit of portable data storage of 1.4mb(floppies) oh wait CD-rw's 1990's hit the theoretical limit of portable data storage of 700mb(CD) oh wait DVD-rw's DVD hit the limit.. oh wait blue ray an along side all of that flash drives portable hard drives so theoretical limit.. is what, and when exactly? and with a current 210gb bitcoin growth limit. lets call it 4 years 6 months per terrabyte how long is this deadline before if you wish to call 10tb a hard technical limit of storage thats 46years for geeks to invent new storage options
|
I DO NOT TRADE OR ACT AS ESCROW ON THIS FORUM EVER. Please do your own research & respect what is written here as both opinion & information gleaned from experience. many people replying with insults but no on-topic content substance, automatically are 'facepalmed' and yawned at
|
|
|
AverageGlabella
Legendary
Offline
Activity: 1232
Merit: 1080
|
|
February 14, 2023, 09:12:32 PM |
|
I think that when the technology of data storage has evolved by changing scale, i.e. for example when the price of 10TB is at the current price of 1TB, it may be possible to increase the size of the blocks as this will not harm decentralisation, however we may be close to the theoretical limit in data storage
We have not shown any signs of being close to the theoretical limit but we are slowing down. Franky is right that we have evolved very quickly from things that we thought would be with us for a long time. CDs were very popular and people thought that it would be a method of storage for many years but they are now redundant so redundant that people do not have dvd players and do not have a cd tray in their computers. I think the motivation for consumer storage might be the theoretical limit instead of the actual limit. hard drives wil continue to expand by a lot but they will be aimed at companies that need a lot of storage and cloud storage provides. The average consumer does not use more then 4TB atm and with other technologies slowing down and hitting their limit like games where the hardware is stronger then the things it relies on I think that could be what happens in the next 5-10 years. Consumer level hard drives might continue to be cheap but probably will not be made in smaller sizes corporate ones will probably continue to expand at a fast rate but will be more expensive to purchase.
|
|
|
|
nullama
|
|
February 14, 2023, 11:46:23 PM |
|
~snip~ We have not shown any signs of being close to the theoretical limit but we are slowing down. Franky is right that we have evolved very quickly from things that we thought would be with us for a long time. CDs were very popular and people thought that it would be a method of storage for many years but they are now redundant so redundant that people do not have dvd players and do not have a cd tray in their computers.
I think the motivation for consumer storage might be the theoretical limit instead of the actual limit. hard drives wil continue to expand by a lot but they will be aimed at companies that need a lot of storage and cloud storage provides. The average consumer does not use more then 4TB atm and with other technologies slowing down and hitting their limit like games where the hardware is stronger then the things it relies on I think that could be what happens in the next 5-10 years. Consumer level hard drives might continue to be cheap but probably will not be made in smaller sizes corporate ones will probably continue to expand at a fast rate but will be more expensive to purchase.
Yeah, I would agree that whatever is being sold mainstream gives you a good indication of what the average user is going to be expecting to be the maximum storage. For example, the new Mac mini with the M2 Pro comes with configuration options ranging from 512GB to 8TB. So that gives you an idea of what's mainstream these days. 512GB being the bare minimum to run a system, and if you need some extra space, you can get up to 8TB, without doing any installation yourself. So, clearly 500GB of Bitcoin data is not too much these days.
|
|
|
|
Sarah Azhari (OP)
|
|
February 15, 2023, 12:07:07 AM |
|
And another question that came to my mind while reading the OP was , why do i need to download the entire bitcoin blockchain data when I am not a developer? I mean what is the essence actually,
for me is just a sense of pride, I don't know next. maybe when having the time I want to learn how to make a Lighting network node, and earning satoshi from that, or create a wallet and be a developer like you said. and there is no harm if we have unused internet at home and pc because is not disturb also when running my pc with another program. actually, I step to download a bit by bit, certainly not so felt long if in the future I make the blockchain program or etc.
|
|
|
|
AmoreJaz
Legendary
Offline
Activity: 3108
Merit: 1102
Leading Crypto Sports Betting & Casino Platform
|
|
February 15, 2023, 01:50:00 AM |
|
And another question that came to my mind while reading the OP was , why do i need to download the entire bitcoin blockchain data when I am not a developer? I mean what is the essence actually,
for me is just a sense of pride, I don't know next. maybe when having the time I want to learn how to make a Lighting network node, and earning satoshi from that, or create a wallet and be a developer like you said. and there is no harm if we have unused internet at home and pc because is not disturb also when running my pc with another program. actually, I step to download a bit by bit, certainly not so felt long if in the future I make the blockchain program or etc. well, set aside the pride and all. if you are not a developer, there's no reason for you to download such heavy data. am using electrum right now for my btc holdings. light and txs are fast as well. i was using the btc core before but yeah, it came to a point i ask myself why i was using it. if you have no valid or worthwhile reasons, just go for wallets that are not heavy especially for beginners and just regular users.
|
..Stake.com.. | | | ▄████████████████████████████████████▄ ██ ▄▄▄▄▄▄▄▄▄▄ ▄▄▄▄▄▄▄▄▄▄ ██ ▄████▄ ██ ▀▀▀▀▀▀▀▀▀▀ ██████████ ▀▀▀▀▀▀▀▀▀▀ ██ ██████ ██ ██████████ ██ ██ ██████████ ██ ▀██▀ ██ ██ ██ ██████ ██ ██ ██ ██ ██ ██ ██████ ██ █████ ███ ██████ ██ ████▄ ██ ██ █████ ███ ████ ████ █████ ███ ████████ ██ ████ ████ ██████████ ████ ████ ████▀ ██ ██████████ ▄▄▄▄▄▄▄▄▄▄ ██████████ ██ ██ ▀▀▀▀▀▀▀▀▀▀ ██ ▀█████████▀ ▄████████████▄ ▀█████████▀ ▄▄▄▄▄▄▄▄▄▄▄▄███ ██ ██ ███▄▄▄▄▄▄▄▄▄▄▄▄ ██████████████████████████████████████████ | | | | | | ▄▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▄ █ ▄▀▄ █▀▀█▀▄▄ █ █▀█ █ ▐ ▐▌ █ ▄██▄ █ ▌ █ █ ▄██████▄ █ ▌ ▐▌ █ ██████████ █ ▐ █ █ ▐██████████▌ █ ▐ ▐▌ █ ▀▀██████▀▀ █ ▌ █ █ ▄▄▄██▄▄▄ █ ▌▐▌ █ █▐ █ █ █▐▐▌ █ █▐█ ▀▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▀█ | | | | | | ▄▄█████████▄▄ ▄██▀▀▀▀█████▀▀▀▀██▄ ▄█▀ ▐█▌ ▀█▄ ██ ▐█▌ ██ ████▄ ▄█████▄ ▄████ ████████▄███████████▄████████ ███▀ █████████████ ▀███ ██ ███████████ ██ ▀█▄ █████████ ▄█▀ ▀█▄ ▄██▀▀▀▀▀▀▀██▄ ▄▄▄█▀ ▀███████ ███████▀ ▀█████▄ ▄█████▀ ▀▀▀███▄▄▄███▀▀▀ | | | ..PLAY NOW.. |
|
|
|
LoyceV
Legendary
Online
Activity: 3318
Merit: 16673
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 15, 2023, 08:42:28 AM |
|
so theoretical limit.. is what, and when exactly? When talking about theoretical limits for data storage, I'm thinking about storage on a molecular level. You can't store much more than 1 bit per molecule. The closest thing I could find was 215 PB in 1 gram of DNA, which is still several orders of magnitude away from current storage systems. We have not shown any signs of being close to the theoretical limit but we are slowing down. Is that because of technical limits, or because newer storage types focus more on access time than size?
|
|
|
|
franky1
Legendary
Offline
Activity: 4228
Merit: 4490
|
|
February 15, 2023, 11:34:55 AM |
|
theoretical limit in data storage (in general) or just theoretical limit of current technology used on HDD? Even if you're talking specifically about HDD, theoretically we can see at least 80TB capacity[1]. --snip--
well, set aside the pride and all. if you are not a developer, there's no reason for you to download such heavy data. am using electrum right now for my btc holdings. light and txs are fast as well. I'll have to remind privacy concern is also valid reason to run full node. For example, you don't want random electrum server know list of address belong to same person. [1] https://www.pcmag.com/news/next-gen-hamr-platters-promise-80tb-hard-drivesits also not relying on one source supply of balance accounting imagine if a litewallet didnt tell you a payment arrived at your address. reliance on a litewallet means you only know what they choose to tell you. they can send you a tx, say its confirmed. but the utxo does not taint/track back to any coin mined nor any real coin existing on a blockchain. should they go down a deceitful route running a full node means you get to see it all and know all coins have a track/taint back to their origins of creation(coinbase reward) that validates that all coins are real coins and not ghost coins made up yes not everyone needs to know everything. but if your receiving regular payment from alot of random people.. just seeing a tx designated for you is not enough, you will want to ensure it is confirmed and has valid funding before releasing your part of the deal, because the more payments you receive the more risk of abuse if you did operate without a non independent checking method
|
I DO NOT TRADE OR ACT AS ESCROW ON THIS FORUM EVER. Please do your own research & respect what is written here as both opinion & information gleaned from experience. many people replying with insults but no on-topic content substance, automatically are 'facepalmed' and yawned at
|
|
|
Sarah Azhari (OP)
|
|
February 19, 2023, 12:01:08 AM |
|
--snip--
well, set aside the pride and all. if you are not a developer, there's no reason for you to download such heavy data. am using electrum right now for my btc holdings. light and txs are fast as well. I'll have to remind privacy concern is also valid reason to run full node. For example, you don't want random electrum server know list of address belong to same person. Maybe AmoreJaz means if don't run a full node, and just use the core wallet just store the fund. Sure, I was thinking before making the decision to download bitcoin core. because the core wallet is complicated, can not be used for transactions before the blockchain is completed, But cmon, core wallet is from satoshi nakamoto legacy, same as this forum we live. there is a sense of pride if we talk it on seminar or other forum if we used what satoshi create, bitcoin, core and bitcointalk forum.
|
|
|
|
LoyceV
Legendary
Online
Activity: 3318
Merit: 16673
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 19, 2023, 06:53:06 AM Merited by vapourminer (1) |
|
the core wallet is complicated, can not be used for transactions before the blockchain is completed For the record: you can send a transaction before Bitcoin Core is fully synced. The moment an input shows up in Bitcoin Core, you can send it. The tricky part is of course that it might have been spent already, and in that case your transaction will indeed be rejected by other nodes.
|
|
|
|
franky1
Legendary
Offline
Activity: 4228
Merit: 4490
|
|
February 19, 2023, 08:19:28 AM |
|
the core wallet is complicated, can not be used for transactions before the blockchain is completed For the record: you can send a transaction before Bitcoin Core is fully synced. The moment an input shows up in Bitcoin Core, you can send it. The tricky part is of course that it might have been spent already, and in that case your transaction will indeed be rejected by other nodes. if your copying over a whole wallet file that also contains UTXO details of your spends. then yes. it can pop up instantly when initiating core, and will appear as pending confirmation until synced but if just importing a private key you will have to wait for the section of a sync where by it recognises the association between the key pair to then display it mid sync while its building the blockchain those that only got involved in say 2020 and just imported a private key/seed key, will still have to wait until 350gb+ is download to get to blockheight sync of 2020+ for their input to show up.. luckily my 2012 stash pops up on GUI display in the first ~100gb* luckily my 2014 stash pops up on GUI display in the first ~150gb* luckily my 201x stash's pop up on GUI display in the first ~300gb* unluckily my 2022 play amount pops up after 400gb * (im hoarding thus dont want to spend these)
|
I DO NOT TRADE OR ACT AS ESCROW ON THIS FORUM EVER. Please do your own research & respect what is written here as both opinion & information gleaned from experience. many people replying with insults but no on-topic content substance, automatically are 'facepalmed' and yawned at
|
|
|
LoyceV
Legendary
Online
Activity: 3318
Merit: 16673
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
February 19, 2023, 08:23:53 AM |
|
luckily my 2012 stash pops up on GUI display in the first ~100gb* ~ * (im hoarding thus dont want to spend these) Are you using a hot wallet for long-term holding?
|
|
|
|
franky1
Legendary
Offline
Activity: 4228
Merit: 4490
|
|
February 19, 2023, 08:32:44 AM Last edit: February 19, 2023, 08:50:30 AM by franky1 Merited by LoyceV (4), vapourminer (1) |
|
luckily my 2012 stash pops up on GUI display in the first ~100gb* ~ * (im hoarding thus dont want to spend these) Are you using a hot wallet for long-term holding? my old stash is on legacy .. am i doing a LukeJR.. nope i have a few other features on my node i added myself so i just do "watch only" for the old stash thus not importing private, but importing public address just as a balance display but if it was time to spend them. then yes i treat it as a hot wallet meaning get a (offline) new set of keypair. to use only public address as change address. and then spend old private key to destination address+ to the new public address(change) i import. and then wipe drive. and redo it all again. its a hassle to spend old stash but that hassle is avoided by making it another reason to not wanna spend old stash.. making it complicated avoids easy temptation to spend i have not put my private key of old stash in my node..ever my 1frank funds are not my main stash. the 1frank funds was/are just a side 'dont give a crap' play money small fund. and i havnt bothered with that private key for 5 ish years which was the last time i had to do the hassle method of wiping hard drive and starting again due to treating it as a hot wallet in relation to those funds. newer play money amounts i have(different keys) as display of balance using public keys on the node. but spend on a different device thats easier to wipe and start again.(lite wallet)
|
I DO NOT TRADE OR ACT AS ESCROW ON THIS FORUM EVER. Please do your own research & respect what is written here as both opinion & information gleaned from experience. many people replying with insults but no on-topic content substance, automatically are 'facepalmed' and yawned at
|
|
|
larry_vw_1955
|
|
February 20, 2023, 04:50:45 AM |
|
luckily my 2012 stash pops up on GUI display in the first ~100gb* luckily my 2014 stash pops up on GUI display in the first ~150gb* luckily my 201x stash's pop up on GUI display in the first ~300gb* unluckily my 2022 play amount pops up after 400gb
wow franky that's alot of stashes! you were smart to get involve in bitcoin so early
|
|
|
|
franky1
Legendary
Offline
Activity: 4228
Merit: 4490
|
|
February 20, 2023, 05:04:09 AM Last edit: February 20, 2023, 05:16:22 AM by franky1 |
|
luckily my 2012 stash pops up on GUI display in the first ~100gb* luckily my 2014 stash pops up on GUI display in the first ~150gb* luckily my 201x stash's pop up on GUI display in the first ~300gb* unluckily my 2022 play amount pops up after 400gb
wow franky that's alot of stashes! you were smart to get involve in bitcoin so early luck.. not smarts its a learning process. dont stop learning i dont need to keep buying im self sustainable and live a good life thanks to bitcoin i choose to buy more when there are undeniably cheap dips worthy of grabbing more im no where near the desire to just sell it all. so i dont its not "smart". its just not needing to. yea i use small play money amounts as a general living/short term income accumulator but its like 0.0X% 0.0xxx of stashes so risk is low but my main goal.. i want to ensure bitcoin functions as a financial payment network to protect my value.. and is not turned into a meme-stock storage network from a technical side of data storage growth of 210gb a year is not disastrous growth. but how that data is used, is a disaster if majority of it is just memes. memes is very inefficient use of the available data growth and it limits/decreases of the financial utility of bitcoin
|
I DO NOT TRADE OR ACT AS ESCROW ON THIS FORUM EVER. Please do your own research & respect what is written here as both opinion & information gleaned from experience. many people replying with insults but no on-topic content substance, automatically are 'facepalmed' and yawned at
|
|
|
larry_vw_1955
|
|
February 21, 2023, 03:31:04 AM |
|
but my main goal.. i want to ensure bitcoin functions as a financial payment network to protect my value.. and is not turned into a meme-stock storage network
these major nft players are just spamming bitcoin with their 10,000 and 5,000 garbage nft collections non-stop i would imagine at some point. some of them seem to already be doing it. i say let the garbage stay on ethereum and ipfs/aws we don't need it on bitcoin. from a technical side of data storage growth of 210gb a year is not disastrous growth. but how that data is used, is a disaster if majority of it is just memes. memes is very inefficient use of the available data growth and it limits/decreases of the financial utility of bitcoin
it's all the devs fault for letting this loophole exist in the firstplace and then it's their fault again for not doing something about it. would a fix require a hard fork? if it's just a soft fork then they really have no excuses. but it seems like they don't even care and aren't even looking into how to put a plug on this hole.
|
|
|
|
|