Congratulation, Bitcoin has reached 500 GB size hard disk data

[LoyceV]

People also could try built-in compression feature on the disk format. IIRC NTFS and BTRFS have such feature.

I would strongly recommend not to do that. Disk compression is a thing from the past, when disk space was scarce and expensive.
Even if you manage to reduce Bitcoin Core's data size by 20%, you'll go from 500 to 400 GB. That's 100 GB less, which is worth like $2. If you're that low on disk space, just prune your node and save 490 GB. Just imagine you need to use recovery software on a compressed disk.

[1714474100]

1714474100

[1714474100]

1714474100

[1714474100]

1714474100

[1714474100]

1714474100

[1714474100]

1714474100

[1714474100]

1714474100

[LoyceV]

Few days ago my node (Bitcoin Core on mainnet) got corrupted and i had to perform reindex. Reindex process took almost 2 full days even though i allocate 10GB RAM

Do you know what your current bottleneck is? If you gave dbcache 10 GB (which is excessive), did you have enough left for the rest of your OS (without swapping)? Still, 2 days (10 GB per hour on average) still isn't bad.

my hard drive is 3.5" (which is faster than 2.5" which used on laptop or many external HDD).

I think a 2.5" disk can have a lower access time, because the arm has to move less far.

i'm not sure if it's a thing from the past since Windows 11 still show this feature and BTRFS is relative new disk format.

The fact that Windows shows something doesn't mean there's a technical reason for it. It means people are willing to pay for it

[LoyceV]

Definitely HDD. I have 32GB RAM and at that time i only run few RAM-intensive application.

I've synced much faster on a server with HDD and 16 GB RAM, that's why I asked.

And if people willing to pay for it, it means some people still use that feature or can't move on from that feature without high cost/huge downtime.

Now you assume all people are rational. My guess is many people just like to click "compress" to make them feel like they're in control. Kinda like running defrag.

[franky1]

But isn't corporate the main reason Microsoft goes so far to preserve old feature/backward compatibility?

incase your subtle trying to infer that microsoft still technically supports windows 98 and xp and even vista.. well lets have a shot at debunking that

many business by default upgrade every 4 years as there is a 4 year tax relief on business tools(they can write off a % of PC equipment costs against tax for upto 4 years)

many software providers end their technical support after X years because most of
their customers have upgraded to a new PC with new operating system by then

this makes microsoft then not need to provide tech support(OS patches) for 10yo systems because most corporate customer have upgraded in 4 years and consumers within 4-8

old features like opening an old *.doc file in latest MS word is different than no longer providing software updates/tech support/bug fixes for word 2012

so even microsoft move forward with the times and gives up trying to support systems over 10 years old

heck even core no longer support XP or Vista era PC's

[franky1]

i only said that as i have seen in many topics about chaindata size and hard drive capabilities(like this topic), where people talk about bitcoin hardware age suggesting that bitcoin should support hardware from back in 2005 as the reason they cry that 500gb is "to big" "to slow"

they usually side step first by giving other examples of big tech still supporting XX year old stuff.. then reign it in to use as an example that bitcoin should emulate that and stick to floppy disk data and all that other BS 'cox corporation X does it'

im just saying 500gb is not big or slow.. after all we are in 2023
not 2003 nor 2008 nor 2013 nor 2018

funny part is some people that pretended to be "data saving conservatives" are now saying, let bitcoin bloat up the 3mb weight space with useless ordinal meta data unrelated to making a payment transaction peer to peer

personally 500gb of 14 years of international currency 'be your own bank' statements of millions of users is pretty small lump

[LoyceV]

Definitely HDD. I have 32GB RAM and at that time i only run few RAM-intensive application.

I've synced much faster on a server with HDD and 16 GB RAM, that's why I asked.

It's possible your server has newer/faster HDD. My HDD is about 4-6 years old.

I don't really expect a large difference, it's still HDD. I can't find the age, but it has 7768 power on hours (less than a year).

Code:

Model Family:     Toshiba 3.5" MG03ACAxxx(Y) Enterprise HDD
Device Model:     TOSHIBA MG03ACA100
~
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
~
SATA Version is:  SATA 3.0, 3.0 Gb/s (current: 3.0 Gb/s)

Given enough RAM, I don't think the disk speed matters.

I'm running a new performance test on a cheap hourly paid VPS, this time with 4 CPU cores and 16 GB RAM. In less than 3.5 hours, it's progress it at 24% of the blockchain (and downloaded 105 GB).

[Artemis3]

personally 500gb of 14 years of international currency 'be your own bank' statements of millions of users is pretty small lump

Plus porn spam...

I never had issues with ram for node syncing but my internet connection has always been so slow (currently under 4mbps) that it didn't matter anyway. An acquaintance with like 100mbps synced in like a day or two, without changing anything from defaults. At least in Linux, any ram you have unused gets assigned for "buffers" which helps for disk i/o. I guess those suggested optimizations can help a bit, like putting chainstate in a ram disk, on mine (pruned) that folder is currently at 3.6g (at 2019-03-28, still syncing).

[LoyceV]

my internet connection has always been so slow (currently under 4mbps)

At that speed, it takes you at least 12 days to sync. Still not bad for a one-time event.

on mine (pruned) that folder is currently at 3.6g (at 2019-03-28, still syncing).

Even though you shouldn't do it, I can just get you a pruned and up-to-date download and save you a few days. Unless you're trying to update an old wallet, in that case you really have to go through all the blocks.

[Sarah Azhari]

I really don't know what is bottleneck means which I find in the post above, Not only here, I always find it somewhere, youtube, forum and etc.

So how to make it not happen (bottleneck) if I want to build the PC just to run the bitcoin blockchain with low spek PC (like pentium III) ?. where I can find the tool to calculate to not happen?. it will be redundant if I bought the high speech ram or hd but bottleneck.

[nullama]

I really don't know what is bottleneck means which I find in the post above, Not only here, I always find it somewhere, youtube, forum and etc.

So how to make it not happen (bottleneck) if I want to build the PC just to run the bitcoin blockchain with low spek PC (like pentium III) ?. where I can find the tool to calculate to not happen?. it will be redundant if I bought the high speech ram or hd but bottleneck.

Basically it means that things slow down because of that one component. That's the bottleneck.

If you change that part of the system with something better, the whole system would perform faster.

[larry_vw_1955]

10 years from now, we will laugh about how small the blockchain was in 2022...

you might even be laughing about that sooner. if nfts get a foothold into bitcoin.  

[seoincorporation]

10 years from now, we will laugh about how small the blockchain was in 2022...

you might even be laughing about that sooner. if nfts get a foothold into bitcoin.  

True, the blockchain size can grow up too fast from now with those NFTs, some days ago I even see a zero fees transaction for close to 4mb, and the crazy thing is it gets confirmed, the miner should be crazy to include one a transaction like that in the block, but well, it happens. And it was the biggest block in bitcoin history. For me, that looks like a huge vulnerability in the blockchain. But is just my point of view.

[franky1]

10 years from now, we will laugh about how small the blockchain was in 2022...

you might even be laughing about that sooner. if nfts get a foothold into bitcoin.  

True, the blockchain size can grow up too fast from now with those NFTs, some days ago I even see a zero fees transaction for close to 4mb, and the crazy thing is it gets confirmed, the miner should be crazy to include one a transaction like that in the block, but well, it happens. And it was the biggest block in bitcoin history. For me, that looks like a huge vulnerability in the blockchain. But is just my point of view.

today alone i seen
https://www.blockchain.com/explorer/blocks/btc/775085
3.8mb 149tx

https://www.blockchain.com/explorer/blocks/btc/775091
3.16mb with only 789 transactions

the usual average was about 1.3mb with 2000 tx
which if all that cludgy code of miscounting bytes and favouring 75% of block for (witness/metadata) aka non tx payment data was removed, it COULD have allowed
~4750tx for the 3.1mb block    ~5850tx for the 3.8mb

but nah the "powers that be" dont want bitcoins to increase tx count with lean tx data.

so yep. expect the blockchain to fill up with more data(up to210gb per year at the expected rate of 52500blocks per year of 4mb) at a average higher than 1.3mb per block, but without increasing the tx count per block and instead decrease the tx per block (facepalm to those that let it happen)
(wait de-ja-vue,, i think i remember saying this many times for a while now)

i wouldnt mind the 4mb so much if the data inside the 4mb was a tx count increase of 3-4x the norm.. but not a friggen decrease of the norm tx count while a bloat of the data.

[LoyceV]

some days ago I even see a zero fees transaction for close to 4mb, and the crazy thing is it gets confirmed, the miner should be crazy to include one a transaction like that in the block, but well, it happens. And it was the biggest block in bitcoin history.

I searched a bit: it's block 774628. The (crazy) transaction you're talking about: 0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0ae. So there's a shitload of witness data, and apparently someone stored DOOM in the Bitcoin blockchain.
I guess the miner did this on purpose, and I also guess most miners would prefer to earn transaction fees instead of doing this, so blockchain spam shouldn't become a big problem.

[seoincorporation]

some days ago I even see a zero fees transaction for close to 4mb, and the crazy thing is it gets confirmed, the miner should be crazy to include one a transaction like that in the block, but well, it happens. And it was the biggest block in bitcoin history.

I searched a bit: it's block 774628. The (crazy) transaction you're talking about: 0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0ae. So there's a shitload of witness data, and apparently someone stored DOOM in the Bitcoin blockchain.
I guess the miner did this on purpose, and I also guess most miners would prefer to earn transaction fees instead of doing this, so blockchain spam shouldn't become a big problem.

This transaction wasn't the one with Doom's code, this one was an NFT and you can see the image here:

https://ordinals.com/inscription/0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0aei0

All that data for a terrible paint image is just fun. And The ones who create the transaction make a deal with the miner to pay him for the confirmation outside the block as we can read in this tweet:

https://twitter.com/guzmanpintos/status/1621123165949931520

So, I think this was just to demonstrate the limits of the ordinals, but the world and the bitcoin blockchain aren't ready for that tech, if they fuck up the blockchain we will need a hard fork to fix all this stuff.

[franky1]

i still laugh facepalm and nod in disagreement at how they promoted taproot as a "solution" to cut down on weight of meaningless data.. yet. it has produced the opposite result

this one was an NFT and you can see the image here:
https://ordinals.com/inscription/0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0aei0

a block of  3.955mb(facepalm) with only 63 transactions(facepalm) where 1 is taking up 3.915mb of space(facepalm)

yet another example of how core devs have really sh*t the bed on this one and failed the community in soo many ways

[Sarah Azhari]

True, the blockchain size can grow up too fast from now with those NFTs, some days ago I even see a zero fees transaction for close to 4mb, and the crazy thing is it gets confirmed, the miner should be crazy to include one a transaction like that in the block, but well, it happens. And it was the biggest block in bitcoin history. For me, that looks like a huge vulnerability in the blockchain. But is just my point of view.

is the lower fee can make more Megabyte of data?,
for example, 0 sat makes 4MB of data, so if ! use 10 sat fee, it could reduce 4 MB data to be 2 MB data?.

so blockchain spam shouldn't become a big problem.

I just hear blockchain spam, can this make full data Hard Disk unexpected?
today is about 480 GB of blockchain data, is that spam included?, and how much GB approximately?

[franky1]

the most the blockchain can grow per year is 210gb
which is 4mb per block for ~52500 blocks per year

however previous to this year blocks were averaging 1.2mb which was ~63gb a year
2016 was 1mb which was 52.5gb

as for fee's
for the spam data.
becasue its sat in the "weight area" it is not treated the same as normal bytes so it gets a discount, plus due to other cludgy stuff they can pay 1sat per virtual kb and be alot cheaper.

thus instead of having a system of 4mb blockspace for payment where fitting say 4tx per kb of payment means a lean 1mb being 4000tx meaning 4mb being 16,000tx

even in a lean system of such using legacy tx, would be trying to charge 1sat/byte minimum which would fill a block for 4000000sats (0.04btc($902))
yet the way that this spammy thing has been allowed they can fill a block for 1sat/1kb
minimum which would fill a block for 4000sats (0.00004btc($0.902))

[larry_vw_1955]

i wouldnt mind the 4mb so much if the data inside the 4mb was a tx count increase of 3-4x the norm.. but not a friggen decrease of the norm tx count while a bloat of the data.

sometimes you make really good sense franky. if you keep up this high quality level of post you might find yourself right back on the development subforum. but i guess that's only if greg hates nfts too.

nfts though, they really need a home. there's no where else you can store them "on chain". except for bitcoin that is.  

a block of  3.955mb(facepalm) with only 63 transactions(facepalm) where 1 is taking up 3.915mb of space(facepalm)

how many nfts was that 1 transaction? if it was just one then that really sucked up alot of blockspace. wonder if it was a picture of a monkey.

[franky1]

greg doesnt actually care much

its the usual certain group of drama queens that cried to him years ago and he snapped where i was the main critiquer and scrutiniser thus causing most controversy that triggered the trolls more than greg. thus he thought easiest way to nip it in the bud was to pander to the cry babies and shut them up by shutting me up
(and that win for them has energised and powered them since thinking they have power)

yea the group that keep quoting about the "greg banned you" were the ones that  pushed for it to happen in the first place. so they are quoting their own actions as confirmation bias. but pretending its third party independent source proof.

now its just over zealous achow that is mod heavy because he is a stiff fangirl of a certain clan..

funny thing is greg comes on the forum alot but doesnt delete my posts, but when achow is online he deletes posts.. not due to any insults in a post nor inaccuracies but if it mentions  certain topic of a subnetwork in any critiqued/scrutinised way (basically a non kiss assly Pro- positive manner of snake oily sales pitch). its gone

achowe is also going power crazy on github in recent weeks wanting to add more moderation hierarchy to clense out those opposing cores roadmap
(again not independent moderators.. but them moderating, to moderate out independent minds.)

so i dont expect things to change any time soon in that regard

as long as kiss-assy self promoting echo chambers of centralised group is tolerated more than independent minds of research, review, scrutiny and critique.. it wont change

notice the lack of critique/opposition speak about ordinals in the tech discussion. the category should be seeing lots of posts about how core messed up or atleast asking for fixes.. .. but instead silence and alot of "lets see what happens, lets not make a rash move to stop it" stuff

things are getting worse, not better