Bitcoin Forum

If you know at least a little bit about how Bitcoin works and its scaling problem, you know that it all comes down to the storage capacities of average users who run full node - we can't allow blocks that are big enough to knock those users out of the network.

Now, technology keeps improving, and sometimes people argue that if we went from 700 MB CD's to terabytes of modern storage, it won't be a big problem in the future. Enter SSD - they are quickly overtaking HDD's because they offer much higher speed, but they come with a drawback - they come with much lower capacity than HDD's. This can mean that people of the nearest future might have less storage then they have now, or it won't improve much, and meanwhile Bitcoin's blockchain will keep growing.

I think that any ideas of potential hard forks for increasing blocksize (they resurface every time fees get high) should be forgotten for the next 10 years, until we figure the amount of storage an average user can dedicate to running a full node.

The storage for blockchain isn't a problem nowadays, but it will be a problem in the future, i mean, if now the bitcoin blockchain is bigger than 200Gb, then it will become huge in the next years, and only servers will be allowed to handle them.

And about SSD vs HDD, some coins like monero recommend to their users to run the node with an SSD because this way the chain could work better, or at least you will sync it fastest, but we know ssd isn't cheap at all and use them to run a node feels like a waste.

SSD storage is getting cheaper and cheaper. Between that and the use of pruning, I don't think there's much concern here. Efficiently accessing the UTXO set is a much bigger concern. So, transitioning to SSDs should actually be a good thing for scalability. From the Bitcoin wiki (https://en.bitcoin.it/wiki/Scalability#Storage):

I don't think HDDs will be phased out for as long as they have significant advantages over SSDs. They're going to have a solid niche until SSDs are priced competitively, so any person who wants to run a node should be able to easily get their hands on one. They're only going to get cheaper from here on out, so I don't think it's going to be a big issue.

We should definitely figure out the sweet spot, but I feel like compromise is necessary at some point. People should at least talk about it once we start getting full blocks from real adoption. I don't think it's a concern now because fees only really skyrocket when Bitcoin surges.

size of the blockchain is a problem but it is not the biggest problem we have specially since we have cheap storage nowadays with high capacity and we also have option to prune the blockchain. the bigger problem is the time it takes to verify these blocks, as they get bigger they will contain a lot more sigops which need to be run and verified. we have had improvements in speed with SegWit and will have a huge improvement with Schnorr that can help so i don't think we need to wait 10 years to be able to handle bigger sizes. we already can handle 4 MB blocks! and if we want bitcoin adoption to grow and not become obsolete we need to scale.


by the way the problem with UTXO set has nothing to do with block size that OP is talking about, UTXO set can grow even if block size was 0.5 MB since it is the "Unspent Transaction Outputs".

This is the most debatable topic for a very long time, just imagine the amount of money we have to spent for the mining hardware if you are running a mining farm and you think it will be a big hurdle to spend $70 for a 2 TB HDD and $130 for a 1 TB SSD  ::), what ever gives the best solution must be implemented for the general welfare of  the network. The hard disk price is not the big issue here which is stopping us from scaling, there are other issues which have to be looked at.

Who said it had anything to do with block size? :)

Since the OP was asking about SSD adoption's effect on scalability, I suggested that SSD adoption is actually good for scalability because of their efficiency in accessing the UTXO set. It seemed like a relevant observation...

sorry, i made the connection because this discussion is also about blockchain size which is then affected by the block sizes. :D

Storage is just part of the problem. Some would say that it's "not a problem anymore", but the truth is, it is for some users.

The other, bigger part of the problem is bandwidth, not all users have access or can afford the fastest internet connections.

You want real scaling? This is real scaling, https://bitcointalk.org/index.php?topic=5148054.0

It is likely that in the near future, SSD drives will become more accessible to people and then the problem will go away by itself. Well, or rather, the problem will remain as for desktop wallet storage volume will constantly increase and it is really a problem. It is likely that many will eventually switch to cold wallets like Ledger and this will partially solve the problem. Well, or the problem will be solved in a completely different way and on computers it will not be necessary to store a wallet that will take up a lot of space.

That's interesting, when I was running a full node, it was putting a ton of pressure on my old HDD's read/write resource, so I can see how SSD can improve it with their superb speed.

Sure, SSD's are getting cheaper, but for now it seems like we've faced a setback - I was recently choosing between SSD and HDD with the same price, and I picked 500 GB SSD over 2 TB HDD, because speed is important for my applications, and I no longer would be able to run full node, because I just can't put the blockchain on SSD, I need that space for other things.

You can prune it if you need the space. Open the bitcoin.conf file and type "prune =" without the quotes, and set it to 5,000 for 5 GB. It will only store 5 GB of the blockchain, prune the rest. But you will still need to sync, and validate the whole 200 GB of data.

Storage ? No. Not a problem at all.

There are a lot reasons why increasing the blocksize will harm the state of decentralization of BTC. But storage is not one of them.

You can buy a 1 TB SSD for ~150$ currently. That's enough storage for the next 15 years at least.
Even with a blocksize increase, do you think 1TB SSD's are the maximum for an average buyer ? That the technology will not improve in the next 10-20 years ?

Or one could simply buy a 4TB HDD for ~120$. You don't need the blockchain to be stored on a SSD. A HDD is fast enough for the blocks itself.
What you should store on a SDD is the chainstate folder, which has quite a lot read/write operations happening.

But is there really anyone who believes that the size of the blockchain is a problem?
Are spending thousands of dollars/euros for the purchase of any type of hardware and there is no money for an HD with the necessary capabilities? say 70/80 €?

The real problem is maintaining decentralization, encouraging, instructing users to maintain a knot, making them understand the importance.
And in an era where smartphones are supplanting PCs, it's important to really understand how essential and essential it is to keep the network neutral.

The size of the BTC blockchain is in my opinion a false problem.

Thanks for the suggestion, I totally forgot about it, and I might actually do it in the future. But the problem is that everyone started pruning, it would increase centralization, because few nodes will have the full blockchain and all new nodes will have to rely on them for the initial sync.


I don't think a lot of people are going to buy hardware specifically for running Bitcoin, why spend $150 if you can just run a light client even on your phone instead? And I know there are always Bitcoin enthusiasts who own multiple full node machines, but they are a minority, most people run nodes if they can spare resources on the machines that were bought for other purposes. It's true that the storage is increasing, but the demand for it also increases - programs weight more and more, Bitcoin blockchain is growing.

Either they do, or they don't.
But the same applies to the current state too.

Why are people running a full node nowadays if they can run a light client on their phone ?

This has nothing to do with storage.




A lot of people already own 1 TB HDDs.
With the increasing popularity of SSDs and their decreasing price.. what will the 1TB HDD's be used of in 5 years ? They won't be worth anything. Might as well use them to store blockchain data if you don't want to throw it away or store them on one of your big future-SSDs.




The 'demand' does not increase at that pace. By far not.

20 years ago you could barely store the operating system on an optical drive.
Now you get 8TB storage for a fraction(!) of the price. And those 8TB are way more than needed for 'programs' nowadays.


Storage really isn't any issue at all.
People who are running a full node today, won't have a problem running one in 20 years. At least not storage-wise.

You are grossly underestimating how serious of an issue storage really is in this case. In my opinion, this is easily one of the biggest issues with the Bitcoin blockchain. The number of people who are running a full node today are a small minority who are primarily trying to support the Bitcoin network in this case.

Better solutions need to be available in order to help their numbers grow. For example, the Neutrino protocol is helping people support the LN without having to download the entire blockchain.

In my opinion, 1TB SSDs will help in the future once their prices continue to drop with time.

How ?
If a 1 TB HDD costs ~40$ and has enough space to store the blockchain for the coming ~15 years.. how can you claim that storage is a problem at all  ???




And they can continue to do so.
The extremely low price for storage nowadays (which is drastically decreasing btw) doesn't prevent them from doing so in the future.

SSDs are becoming cheaper as years go by, and the storage capacities are also increasing too, so I don't think it'll be much of a problem in the future. But yeah, if the blockchain continues to expand at a rapid pace in terms of size, the miners have to increase their storage capacities as well, and so the cost of mining will effectively increase too. Then again, technological developments on that area isn't stagnating, so I think we're still okay. Just 40 or so years ago, we managed to get by with very little storage spaces compared to what we have today for very important events for Mankind, so we're good, I think.

Storage size is, indeed, a non-problem, at least not directly.
Bandwidth is more a bottleneck in many cases (great link, @Wind_FURY (https://bitcointalk.org/index.php?topic=5148359.msg51262009#msg51262009)) and also the speed of accessing the data is important (and we are back to the data size and HDDs too, somehow). SSD can help, but it doesn't have the lifespan of the HDD and SSD is well, let's say not cheap.

I was certain that the devs already know that scaling has to be improved, still got a nice surprise from reading ^^ that link.
And there's SegWit, the work on LN and I expect that more will be done in the near future, since unlike "the competition", Bitcoin really processes a huge number of transactions already and the numbers are expected to rise exponentially.

SSDs will indeed be cheaper with time and also the amount of capacity will be even greater and may even be replaced by new storage technologies such as intel optane or whatever. And with time, bitcoin will continue to grow where the capacity of nodes will be bigger and miners need more storage, but of course miners will not buy storage every day because this enters the depreciation costs of their mining results.

What is clear is that there is no need to worry because in the future technology experts will certainly adjust to existing industrial needs, and of course all will be synchronous as 40 years ago.

You not quite right. A SSD is a HDD all SSD are HDD but not all HDD are SSD got it?? You first got ata HDD then sata HDD.
SSD means solid state and doesn't move that is why it is so fast. It is like a memory card they are solid state too and have no moving parts and that is why they are fast. Basicall an SSD is a very big much better made and much larger in size version of a memory card like the one you have in your phone.

Gameboy games used to also use solid state and so did n64 and sega and the like.

Running the blockchain on SSD is not very expensive anymore. They use to be very costly but not so much anymore.

Yes and no, and you assume that "everyone" will start pruning. I will tell you that not everyone will start pruning, although archive nodes will decrease as costs of storing the whole blockchain increases.

It centralizes the source of the blockchain for the initial sync, but it's debatable if it centralizes the network, because all full nodes validate.

a full node archives and validates. thats why its called FULL
nodes that prune, filter, strip are not full nodes. they might aswell just be spv nodes

using torrent analogy. a prunned node is a LEACHER not a SEEDER

we should not classify prunned nodes as part of the main full relay network as they dont help the relay network with other nodes needing syncing. same goes for the stripped, filtered, bridged nodes that devs buzzword. (compatible nodes)
the prunned feature should be something only coded into lite nodes as it deceives people into thinking they are still a full node when its included in core. and other nodes that want to sync with other full nodes dont like wasting bandwidth on leachers that then are not going to help the network

as for things about blocksize vs latency/propagation. comparing things to a 2015 1mb block. things have already changed. blockheaders only and other things have made things (as gmax quotes) 88% more efficient on bandwidth. so with 1mb not being an issue in 2015 8mb is not an issue in 2019.
limiting maxsigops down to say 1k would also improve the risk of latency issues for the whole legacy tx sigop debate of 2015

as for SSD vs HDD
people that want $20/tx fee can afford the extra expense of a SSD. so dont play that argument
if someone cant afford $200* every ~4 years for a hard drive. then they should not be part of the same crowd praising for high tx fee's to be a persuasion tactic to deburden bitcoin of utility in favour of other networks.
after all if you think $200 every 4 years is too much then math says making 1-2 transactions a year is too much..
think about how many users will want/need to be a full node if all they could 'afford' was under 2 transactions a year


*8mb legacy block=400gb max buffer per year (2TB SSD =$200)

I can accept whatever "names" you want to call them. 8)


But how can they be SPVs if they fully validate? Validating pruned nodes accept transactions and blocks, validates them, and relays them if they are valid, they are part of the network. SPVs are not, and don't do anything for the network.

I mean, I can imagine a dystopian scenario where there's like a dozen of archival nodes left, they are owned by big companies and they charge a fee for downloading the blockchain, and they can even refuse for political reasons. But this can only happen in big block scenario, if Bitcoin will keep the same blocksize and will only make small increases in the far future, enthusiasts will still be able to run archival nodes.

The downloading part is what I would be more concerned about. It's really cheap to get enough storage space to hold your own copy, but it can take weeks for your core client to download it. It can take days with a high speed connection, but the majority of the world doesn't have access to it. For many of us 50 or 100 megabit connection is nothing to brag about, but in many countries 2 is all that you will get. Getting a node to run in such conditions, especially with the client messing up the database from time to time, requires a huge effort.

Do you know what is the easiest and most effective solution for that problem?

Waiting...And those with talents related to the industry could help working in places that develop the future of data storage, just like humanity didn't stay with those floppy disks or regular cds we shouldn't stay forever using hdds.

Instead of trying to change a technology with a lot of possibilities for the future it's a lot better to work in replacing something that is becoming obsolete.

I have 50 Mbps and syncing takes forever, it feels like my connection is not being fully utilized and also peer discovery is slow. Sadly I was too lazy too investigate this issue to find a solution, and eventually I gave up on running a full node altogether. So, even now running a full node is kinda problematic.

Not everyone needs to run a full node; my stand is that if you aren't capable of running a full node, just use a SPV client.

For those who are able to run a node, the internet connection is probably not the main issue. With the current state of Bitcoin, the downloading won't take that long with the simultaneous downloading from the connected nodes. The main limiting factor is probably with the disk speed and the CPU verification speed. In the long term, the disk space is probably the main constraints that we will face. Memory density isn't growing all that fast and the memory space is still fairly expensive.

but being prunned also dos not offer a full service for the network. so they dont deserve the title full node. they just leach data without being great at seeding, so there is not much point in starting as a full node if your just gonna prune.

my point is.
from the buzzword games of devs, spv means simple payment verification. there is no set node type. some are webwallets that rely on custodians that then let users re-verify. some leach from the relay network itself, and some are full nodes that then dont archive. but if you want to get more anal about the grey area's.. lets give it 3 categories instead of 2
lite=spv/web/phone app
compatible=is nearly full but misses something(verify(some dont fully verify). archive(prunned, stripped, filtered))
full=verify and archive

from the user function.
if you dont care about seeding other nodes you dont want/need to be a full node, if you only care about your own transactions and not the community collective. then you might aswell just run lite/compatible

I agree, pruned validating nodes might be a better way to call them.


I disagree, if it fully validates, then it's part if the network, and does its job for the network, pruned or not pruned.


::) Stop the social drama, and the "Core is evil" propaganda.


Stop. SPVs are not nodes, they don't fully validate, and they should not be in the same debate as full nodes, pruned or an pruned.

'fully validating' is a service purely for the user using the node to trust the data it got. its not a 'network' service
'fully archive' is a network service for other nodes to get data for their syncs

a prunned node cannot offer old blocks of say the first 5-9 years of bitcoin, thus useless for network services.
its stuff like prunned being treated like full that gives issues like what 'hatshepsut93' experienced. because even without any hardware or bandwidth withstraints i presume the nodes he was connecting to were not full network service nodes

imagine it like torrents. you see torrents list for a movie that has many users, but. 98% of users have only downloaded the packets for the final 10 minutes of the movie (the credits/casting list) it doesnt matter how fast the intrnet is or hard drive space is. by listing these 98% as seeders means theres only 2% chance of getting the full data.
and if your not keeping an eye on it to cancel download to particular user to then search out a better source. you nd up just waiting for the other user to disconnect themselves before your system rescans for a new source

...
anyway back to the topic of SSD vs HDD
when a system uses a hard drive as its 'virtual memory'/paging file(extended ram utility) then it makes a difference of th read/write speed. but just archiving the blockchain. thats more of a background activity after all the validation stuff is complete thus not really affecting the network propagation. after all a average old 7200rpm hdd is atleast 80mb/s.. blocks are not even 3mb so storing a block which occurs in ~10 minutes, takes less than a second.

so the real limitation is not really ssd vs hd for archiving. but actually ram utility for the validation/UTXO store

You call it whatever you want, but if the node validates and relays transactions and blocks, and make sure that they are valid, then it is part of the network.

Capable of participating in a UASF. 8)


But still not useless enough to make sure that everything follows the consensus rules. 8)

I had the same problem. With 100 Mb/s I needed about 3 or 4 days to download the blockchain with my qt client and syncing was taking at least 10 minutes if I remembered to start it up every day. When I went somewhere for the weekend and forgot, I had to wait like 30 min the next time I started the client. To top it off it was all happening more than a year back. Right now it must be much worse. I still have my outdated qt client but not running it anymore.


Yes, not everyone has to do it, but what will be the incentive in the near future? If every year it takes longer to download the blockchain and you need a better computer to run a node, why would you? Surely there will always be enthusiasts willing to do it but as years pass by there will be less and less of them. There should be either a way to make it easier and cheaper or a way for the node owners to profit from it.

IMO disk space is still very cheap compared to the cost of electricity and the other components you will need to run a node. You can get a 3tb drive for $50. Isn't this cheap? It will probably die a natural death before blockchain outgrows it.

i know you love to derail. but ill bite
1. the UASF was just in name, not in action. it did not require user opt-in
2. 'compatible nodes' including legacy nodes that dont validate segwit still would have functioned under uasf
3. now to finish off the meander.. research 1037.. its an important number in relation to full node

now getting back to the topic.
ssd vs last gen HDD is not expensive. especially when comparing to to cost of tx's those in favour of fee wars, when you add up utility over a 4 year period

if people are ok to spend hundreds over the years for transactions. then spending a bit on a hard drive every 3-4 years is a non issue.
but if you wanna prune, and just be a leacher and not a sync seeder. then just start as mediocre node and be happy, stop pretending your still full node

a bit confusing here. some devices are hybrid,  HDD + SSD. But no, SSD is not a HDD.

HDD means hard DISK drive, a magnetic disk, SSD is not a disk.

SATA is another different thing, the type of connection cable / data transfer of the device.

I do not "love" to derail. You simply post a lot of misinformation in the forum.


I do not know what dimension you're living in, but nodes give demand to what miners produce. Blocks. The UASF forced the miners' hands to activate Segwit because there was a threat from users that it would reject non-Segwit blocks if they don't activate. In truth, I believe soft forks should be activated as a UASF. Next time. 8)

You just noticed that now ?

He is a known troll on this forum, trying to spread misinformation with his sciolism.

The best you can do, is to put him onto you ignore list and enjoy a less retarded forum from this moment on.

The best we can do is to be informed ourselves, and counter-post the correct information. We don't want newbies to be misinformed just because all informed users are too afraid to debate him.

Ignoring him only encourages him.