Bitcoin Forum

I heard the news that bitcoin is most close to receiving 500 GB size hard disk data, so I am just confused if a newbie or beginner tries to start download bitcoin, how much minimum hard disk he must buy and prepare?

how many blockchains are there until bitcoin is mined?, is possible 10 terabytes?,

Is the default hard disk still good to use, or move to SSD?

I just try to download, but when over 3 days, my hard disk is slow to receive blockchain data, seems low in header sync when opening and closing bitcoin core. (I use a default Hard disk of 10 terabyte)

Ideally, put at least chainstate on SSD, and use 4096 MB dbcache. With those settings, any modern computer should be able to sync the blockchain within a day. Assuming your internet connection can handle it.
1TB is enough for many years to come.

If your internet is 100 mbs speed.
your ram is 16gb
use a 2tb ssd.

for the os and the blockchain.

once you down load the blockchain clone the ssd with the os and the black chain on it.

put the clone in a safe place

you should be good for years and years and years.

plus if the drive dies just swap in your clone and it will synch up fairly quickly since you have the first 500gb on it.

I don't understand about this, where the setting? Maybe because of this i have problem in half of full bitcoin chain when use 2Tb HDD sync

Is ok when clone it without os?.
 I try to download full blockchainu on external hardisk and change the target to external HD when opening core. So if that's dangerous lost the data, maybe i have to reformat it and create the OS

Blocks will be mined and added to the blockchain for as long as Bitcoin exists. The whole point of the current block size limit is to allow people to have the full blockchain on their computers without making a significant investment into storage. If there's a low barrier for running a full node, then the network will have a huge number of nodes, which is good for decentralization and security.

There's no point in predicting blockchain size many years away from now, because it will depend on the state of storage technology. If there will be some sort of revolution in this field and average people would own drives with hundreds of terrabytes, you can expect the devs to raise the block size limit to slightly improve network capacity.

Settings > Options > Size of database cache > 4096 MiB.
But: only do this if you have enough RAM. In my experience, 8 GB works okay on Linux (without active swapping), it depends on the rest of your system how much you'll need. If you're low on RAM and need to swap on the same hdd, that will make your download much slower.

Is there actually a reason that you need or want the full blockchain on your computer? I mean if you think it is necessary then do it, but if you have to buy a new hard drive maybe it is not really worth it just to have the blockchain. You can just use a pruned wallet.

Far as I know, a pruned node is helpful if you really want to make disk space economy but does not help you much if you want to run your own Block Explorer or gather all sorts of information from the Bitcoin Blockchain.  While both are still verified and safe, a Full Node is like the complete version of the Blockchain.  You have it all right on your computer and you can make use of it if necessary.

I do not remember honestly but I think you can not use a pruned node to import a new wallet and sync it up.  I think you will need to go through the entire Blockchain again for that.

Moreover.  You are going to help the Bitcoin network by running a Full Node.  You will gather block information, but you will also provide others yours.  This is how the Decentralized Peer to Peer thingy works.

-
Regards,
PrivacyG

I think you mean an old wallet: new wallets don't need to check for old transactions. In fact, a pruned node can update an old wallet, as long as the data goes back to the last time the wallet was connected. So if you prune Bitcoin Core to 100 GB, and only update your different wallets once a year, it still works fine.

A pruned node can also propagate the latest blocks to other nodes (which means you'll upload a lot less).

(facepalm)

you are pretty much saying a bittorrent of a movie file where you have watched the entire movie. but then only keep  the last 5 minutes of a movie file will allow others to watch.............. (now lets be honest) only the movie credits of the guys involved in sound effects

your ignoring that blockchain. is a blockchain not a "relay current tx" or "propagate latest block" network

do you think a bittorrent system of mvie files would sustain and work well if majority of users only keep the credits final 5 minutes. would that really appease everyone wanting to watch whole movies where instead of 1000 seed sources 1 leacher. they only have 2 seed sources and 999 leachers

if you are not decentralising the blockchain by archiving the blockchain your not supporting the network

if majority of the network is pruned. there is only a minority of the network seeding the blockchain.
meaning more centralised, less distributed

imagine the bandwidth demand increase by new users who can only get initial block download from low minority. that low minority then has bandwidth bottlenecks which means everyone leaching from them get slower IBD time. meaning more delays and cries that "it takes too long"

if all you care about is relaying latest tx or latest height blocks.. you are not looking to be a full node. you can be happy with a light node

stop pretending that prune=full
learn the word full. its true meaning

full nodes is a term that existed before prunning did. because fullnodes done the full job of verification and archiving.

pruning is a step down from full.
..
with that said. individuals that dont care much for protecting the network. can and do have free choice to prune. but they need to be made aware that they are not fully supporting the network.

bittorrent analogy. dont pretend a leacher is a seeder if the leacher only wants to have the final 5 minutes of movie credits

I'm not ignoring anything, I'm just stating a fact. Here's another fact: my full node uploads 28 times more than it downloads. When I ran a pruned node that was allowed to upload, it still uploaded more than it downloaded. Many people run Bitcoin Core without uploading, and after their initial sync, those people can get all the blocks they need from a pruned node.

Bittorrent doesn't need updates every few minutes, this comparison makes no sense. But if there's a TV-series with 200 episodes, you'll find there are more seeders for the last episode than for the first episode. And that's fine, because there are also more downloaders for the last episode. So maybe your comparison makes sense after all, just not in the way you intended.

I'm supporting the network just fine:
By now I've uploaded much more than I've ever downloaded.

Each pruned node still verifies all the blocks, and from the network's perspective, a full node that isn't uploading still contributes less than a pruned node that uploads the latest blocks.

I have never experienced any bandwidth shortage. When I'm downloading, it downloads as fast as my system can handle. And when I'm uploading, only 0.028 of the allocated 0.5 TB per day gets used.

Most cries come from their own hardware limitations.

Or you can be happy with a pruned node. The choice is yours. Bitcoin gives you that freedom.

i agree with you that listening only mode is also not supporting the network and not having full features of bitcoin core activated. thus not also a full node status

but while you endlessly want to only discuss current block current tx relay stuff.. your forgetting the decentralised blockchain invention.. yep you dont want to compare the lack of facility of security of distributing the full blockchain

you trying to call pruned "full" is the ignorance on your part

pruned/listen only are "self verification" for personal use and personal security of accounting. but thats not "full node" for the network facility of all users

and by the way.
thanks for showing your data ups and downs. it helps show that you are well below the average limits of todays technology

unlike what a certain other person thinks where we are breaking limits today and bitcoin cant scale

but here is the thing

if there was say 10k nodes full noding in full feature activates modes
the pressure is off of you to not need to be seeding the 100k mix of less than full nodes

if that full node changed to 1k due to promoting pruning as "full"
the pressure/bottleneck then multiplies

my whole point is
its ok if you choose if you do or do not want to support the network. but own up to your choices.
dont switch features off and then pretend you are supporting the network because you read on a forum that someone said that switching off features was still full

instead people should be calling out the crap
make sure the people on the forum are not kissing your ass and telling you everything is ok to switch network services off. when infact switching off features then centralises a smaller pool of full network feature activated nodes

oh and then dont read silly forum posts that bitcoins full feature activated nodes are more centralised because technology of 2022 is actually a myth and its actually 2007
because thats just the opposite of truth

My personal recommendation is at least 1TB storage to ancipate blockchain size growth.


There's no upper size limit of blockchain size.


HDD is good enough based on my experience if you either,
1. Move chainstate directory to SSD and add symbolic link.
2. Set Bitcoin Core to use >=4 GB RAM so most file on chainstate directory is loaded into RAM.


To be precise, pruned node share latest 288 blocks no matter how much latest block is stored.

Is 500GB a lot? 5TB drives cost less than $100, so $20 for 1 TB. $20 gives you enough space to run a system, some basic software and have enough space to run full node at least until 2025. If you're a bitcoiner, you should be able to spend $20-50 every few years on your hobby.

An interesting fact: monero uses more space than bitcoin. Its current requirements are smaller but it's much younger than bitcoin and holds less transactions but it requires more space per transaction, so if it keeps growing it's going to require much more drive space than bitcoin in 5-10 years.

I did not do such a thing. When I say full node, I mean Bitcoin Core with the full blockchain.

Many budget laptops nowadays come with with 256 GB. It's very fast storage, but can't easily be increased. On desktops you're right, storage space shouldn't matter.

I was especially surprised by how slow it is to sync. It took about 10 times longer than Bitcoin, on the same server.

funny part is those not wanting to scale bitcoin. think spending $20-$50 every few years is bad.. and people should use altnets without being a full noder.. but think bitcoiners spending $1-$3 every day/week/month on network fee's is good,  

seems they fail basic math

average pc upgrade $100 hard drive every 4 years+
=$25 a year
=~$2 a month
not everyone has to pay because not everyone NEEDS to be a full node just to transact
some call this "unaffordable for the masses"
vs
$1 a tx (daily user)
=$30 a month
yet everyone would have to pay this just to transact
some call this "affordable and harmless for the masses"


$2 a month vs $30 a month. which is the number that hurts pockets more for daily users
answer: the tx fee. not the hard drive

A Seagate Ironwolf 12TB 7200RPM, which although being sold as a NAS drive does work very well for blockchain storage, is under $249
Use anything you want for your boot / OS drive and stick the blockchain on there and you are good till just about the end of time.

I use a pair of slightly smaller wolfs mirrored together and they hold the data for 2 BTC nodes, and an ETH node and I/O utilization even under the IBD I am doing now for a MyNodeBTC I am setting up for someone is under 15%.

Drive storage is cheap. Used 1TB drives are just about free.

-Dave

Let's wait for the 1TB size then we can say that we are going to hit 200k haha. Storage is not a big issue in my views for the year 2022 and hardware resources as well in this time or era we are living we own resources but their efficient utilization is the issue its still developing thats a good mode.

Downloading Bitcoin is not correct!

You actually meant about downloading a complete Bitcoin blockchain to your wallet and store it at your computer (hard disk), that is for a Bitcoin full node.

Pros and Cons of Bitcoin Node types (Full node and Prune node) (https://bitcointalk.org/index.php?topic=5213552.0)

On another hand, a newbie or a beginner does not have to start with a full node or Bitcoin Core. They can have a more affordable option, with SPV wallet like Electrum wallet. With SPV wallet, they don't have to prepare too much storage space like Bitcoin core (either full node or prune node).

Electrum: https://electrum.org/#download and verify it (https://bitcointalk.org/index.php?topic=5240594.0)

All of my Hard Disks have failed after about two to three years.  As in, they became extremely slow up to a point where they were practically unusable.  HDD's are cheap but they also have a short life span.  Is this not a problem you encounter?  For me this is why I rather choose an SSD instead.  Although more expensive, that one I can truly say could last you possibly forever.  I at least never had an issue with any of mine.

-
Regards,
PrivacyG

Normal hard disks don't get slower, they either work or they break (or start showing bad sectors).
It sounds like you too fell victim to SMR disks (https://en.wikipedia.org/wiki/Shingled_magnetic_recording). It's a nasty trick hdd manufacturers pulled (without informing the buyer), and creates to a disk that gets very, very slow after you've used it for a while.
TL;DR: don't buy SMR disks.

old (platter) hard drives do degrade and slow down. you can normally tell because you can hear the bearings in the spindle of the platters or the actuator arms of the 'head' (reader/writer) get more clunky/noisy

when hard drives get full you hear it the most. because they have to search data across many different  sectors(locations on the platters)


you are correct there is no physics NEED to be full node just to make a transaction and use bitcoin.
however it needs to made clear the different features that can be opted in or out of being used . and knowing the differences.. rather than some that just pretend pruning =full where they want everyone to prune to cause centralisation.

there is nothing wrong with choice.. but having informed choice is what should be done

trying to get core to have prune mode set by default. is not really a good thing. yes give the option to use prune. but not try to push to make it the default, where then majority of nodes would then  be pruning thus not having a distributed blockchain

i laugh when i see the same dozen people recite the same outdated, debunked scripts that try to push for:
high tx fee's as being ok/normal/to be expected "just pay more"
less utility on bitcoin via pushing users to use altnets.
trying to say bitcoin is unfit due to 1990-2007 tech.

and all doing so just to promote their altnet as the bitcoin 2.0

10 years from now, we will laugh about how small the blockchain was in 2022...

Just to avoid confusion: a pruned node is a full node.

Pruning is full with the exception that you dump blocks later on. Full doesn't stand for full blockchain, but for full features; so no centralization. There will always be nodes willing to store the entire chain (mainly merchants), so we won't ever be left without it.

Now I'm confused: my definition of a full node was Bitcoin Core including a fully downloaded, stored and verified blockchain. I checked the first 4 definitions on Google, and it's a 50-50 tie between whether or not pruned nodes are included.

From a quick search:

Which site says that full node requires store of the entire chain?

the term full node describes full verification and archive of full blockchain
why: because pruning did not exist in the days of the term being used. the only option was full verification and archival

pruning came later which involve changing the network service flags of what options the node is using. the network service flags showing
[NODE_NETWORK_LIMITED] but not[NODE_NETWORK] means its pruning

same as nodes pre segwit running now(pre 0.12.2).
they dont have [NODE_WITNESS]
 they do not relay segwit tx nor verify segwit transact witness. they have a stripped down version of the blockchain missing data so they do not even act as IBD seeds for the network. they have by core buzzwordatory been called "downstream" nodes in the same grouping as what lite wallets sit at the outer circle/edge of the network

they are not full node. even though many pretend that "backward compatibility means older nodes are still full nodes because they blind default "is valid" "(facepalm) but they accept blocks with out checking the witness is signing for the tx data thus they are not fully verifying or storing or relaying such segwit blocks/tx

Numbers 1 and 4 (for me) on Google:

So, after i read all posts and a bit understood, I'm prepared to buy NVME M.2 SATA (can be 1-10 GB/s write and read) and prepared that SSD as an external hard disk to run the bitcoin core outside my laptop, because I have an old laptop that hasn't slot for that SSD. Is it's oke for that?, (thinkpad. core i5 and I want to upgrade ram also to 16 GB).  I want buy Msata SSD before, but after research it's not best for (write and read - 500MB/s)

So it's oke 1 terabyte SSD for the next 10 years?. my budget still not enough to buy more byte.

current blocks are about 1.4mb and there are 52,500 blocks a year average. so ~74gb a year =740gb a decade
meaning yea a 1tb could do that job based on current stats..

however should devs finally remove the cludgy bad math miscount code
it could easily become 4mb blocks of proper legacy utility meaning ~210gb a year which is if you want a PC to last 4+years, fine. but 10 years presuming they will finally shift the goal post by removing the cludgy math code legacy limit within the next decade, its to be a 2.5TB+ hard drive

there is not much difference between a 1tb vs 4tb in price its not like the price is 4x. so better off on the long run to have a 4TB hard drive

Maybe for the next 2 or 3 years the price of 4tb NVME M.2 SATA could be low from current price, and I have to move to new 4 TB .
So, is okay If I just copy from the old to the new SSD?, is that affect when rewrite a new blockchain?, Because Ive experienced copied from old to new disk, I have much time to wait just sycn header wallet and chain.

And, I still doubt if 1 TB could be enough for the next 2 years because if there is an increase in the new miner and new bitcoin transactions, could be not as our expected in only 1 year we got an additional 500 GB from now.

at the moment there is a hard limit of 4mb.
there is a cludgy limit that keeps data throttled down at about a 1.4mb average, keeping tx count allowance low

there are also rules for ~105,000 blocks in 2 years

so simple math 105k blocks times 1.4 =~150gb for 2 year (650gb total)
or upto 420gb a year using the uncludgy max 4mb rule (920gb total)

chances of devs suddenly raising the 4mb block limit above 4mb is pfft. seems like something not on their roadmap this year or next. so presume we are stuck with the restraints of the 4mb limit. where they "may" remove the cludge to atleast get to 4mb of data per block instead of throttled down at a average of 1.4mb.

and most would prefer the cludgy limit to be taken away to then actually allow more utility of the 4mb for actual tx count increase anyway
and progress from there in a few years after that

so for 2 years guess around the 650gb total for full archival at current rate-920gb for full archival if cludgy code removed to full utilise 4mb block limit

there's a revolution going on right now actually. 22 TB hard drives. problem is, it is out of reach of "average people" unless they're willing to pay $600.
https://www.westerndigital.com/products/internal-drives/wd-gold-sata-hdd#WD221KRYZ

it will probably stay like that too. they'll stop making them at some point but they won't just start lowering the prices too much.

The NVMe is very fast, but won't be able to use that performance if it's connected through USB. NVMe is slightly more expensive than SSD, and you won't use that speed at all in the many years you're using it after the IBD (initial block download).
So here's what I would do: can you replace the storage inside your laptop for a bigger one? If your laptop uses an HDD now, it will become much more responsive if you replace it for a (sufficiently large) SSD. And fit your old HDD in a USB case for other uses. You'll need to clone or reinstall your OS if you do that.

So replace the HDD :) It's the best upgrade you can do for old computers (and adding RAM helps too of course). Any chance your laptop has a build-in DVD player? If so, you can fit more storage in it's place.

You won't need more than that for downloading the blockchain. Chances are your current hard drive reads much less than that, but more importantly, the access time of a HDD is much, much worse than SSD.

I wouldn't plan 10 years ahead for an old laptop ;) In my experience, I tend to fill up all storage no matter how much space I have.

More miners lead to a higher difficulty, not more blocks. I see no reason for the blockchain to grow more than ~100GB per year in the coming years.

The thing is...... technological advances in storage and bandwidth are growing faster than what the Blockchain are growing.... so it's not a big issue. If storage are a problem.... then run a pruned node, but hard drives are so cheap these days, you can simply buy a small 2tb external hard drive and be good for a couple of years to come.  ;)

This is only an issue... for the people who wants to make this an issue. (like the Bitcoin haters and Bitcoin's competition)  ::) ::)

This is all true, I just wanted to clarify to the OP that he does not really need the additional disk space if he can't afford it. To me it seemed like he was a little bit skeptical if it is worth it to invest into a new hard drive, and if you hesitate to buy a 20 USD hard drive than you should probably use your money elsewhere than to support the bitcoin network with a node.

It depends if you are looking for new or used and where you are located.
If you know people in tech and such they are (including myself) giving away used 1TB drives a lot of the time. They have just about 0 residual value on the secondary market here in the US. 2 TB are not far behind if they have high hours, low hour drives are usually 'pizza' money.


You really don't need a WD gold for this, as I posed above I have ironwolfs running 2 bitcoin nodes and a ETH node and they are not even noticing the load once the IBDs are done.


SSDs have a finite lifespan, the bits can only flip a certain number of times before they stop working.
I have had both SSDs and spinning drives die. A lot also depends on the environment, a spinning drive in a sever in a data center with proper cooling and power with very few on - off cycles is probably going to last far longer then an SSD in your kids laptop.

Either way important data should be mirrored anyway.

-Dave

theoretically, i read you can transfer up to 2GB/sec through usb via nvme. so it's not exactly slow. but yeah you have to have usb 3.2.


that statement makes no sense. unless by SSD you mean "SATA SSD" but even then you're completely wrong. You can get 2TB NVMe SSDs for the same price you get ones in the SATA form factor. external SSDs if anything seem to be at a price premium to every other form factor so what you're doing is paying for the novelty and convenience of portability but nothing more.


yeah but if you go by TBW rating, your computer will be obselete before any of that becomes an issue.

far longer? any data to back that up? :o

Yes, I research and found can be replace with SSD SATA 3, 2.5". I just don't know how to clone it, can I clone it outside? (use that SSD as external first than clone it)?

My laptop don't have DVD so I can't replace with Caddy HDD, about ram, how much memori bitcoin core need when running downloading blockchain?.

Maybe hard when have only 4gb ram, I have 2 tb HDD, then, I will upgrade RAM only when better than replace the HDD, because that many cost for that.

This is the simplest answer because you can save money and it guarantees you to use it for a long time because in the future hard disks will become cheap and when you need more space, you can buy them later. This is expected since there will be more users in the upcoming days and transactions are mostly multiplied and Bitcoin Core's data will also expand. This is also a good thing because the more people will come, the more we are guaranteed to last longer because there will be more liquidation, and bitcoins as assets will be more recognized in the future.

You're right, the price difference is much less than I expected. Comparing SATA-600 and M2, there's only a 10% price difference in the low-end range.
I've updated my post.

Assuming you're using Windows, I'm not the right guy to answer this question.

Are these your current specs (4 GB RAM and 2 TB HDD)? If so, upgrading RAM will help a lot. On a Xeon server with 16 GB and HDD, I've downloaded the full blockchain in much less than a day. With enough RAM, the HDD isn't a problem, but with the chainstate directory on HDD, starting up Bitcoin Core takes significantly longer.
In your case, both more RAM and replacing HDD for SSD will give you a significant performance improvement.

OP mention old laptop, so it's unlikely his laptop has USB 3.2. Additionally, both parts (laptop and ssd external case) need to support USB 3.2.


Additionally, Bitcoin Core have very low write amount compared with other full node software[1].


I don't know about the SSD lifespan. But at very least, proper cooling increase data retention on SSD[2].


On my experience, the startup only took 1-2 minutes until Bitcoin Core window (which show tab "Overview", "Send", etc.) appear.

---

[1] https://blog.lopp.net/2021-bitcoin-node-performance-tests-2/ (https://blog.lopp.net/2021-bitcoin-node-performance-tests-2/), section "Disk Resource Usage".
[2] https://www.jedec.org/sites/default/files/Alvin_Cox%20%5bCompatibility%20Mode%5d_0.pdf (https://www.jedec.org/sites/default/files/Alvin_Cox%20%5bCompatibility%20Mode%5d_0.pdf), page 27.

Interesting comparison: Some (shitty) node writes 150 times more to disk than Bitcoin Core. I was surprised by Bitcoin Core's results though: 393 GB disk writes and especially 140 MB disk reads was much less than I expected. I would have expected Bitcoin Core to be all over the disk all the time, to verify each new transaction with existing blocks.
I also assume this would be much, much worse if Bitcoin doesn't have 10GB RAM to consume. Without that, the performance would be much less and the disk would read and write a lot more.

This isn't really relevant for consumer use. Consumer SSDs are much better at data retention than enterprise SSDs, and most consumers don't keep their drives powered off at high temperature for prolonged periods of time.

yeah i guess i overlook that. probably he has usb 2.0 which is 480 mbps or about 60 MB/s but the thing is that is theoretical. In real life, he might see half that. whether 30 MBps is good enough for someone I don't know. but with alot of older computers that's what you're going to get.


Yeah I've heard that SSD is not as resistant to heat as normal HDD. But I've never heard of story of someone that said they had stored data on an SSD and then unplugged it and then in a year later they couldn't read it. Apparantly that's a thing though.

i thought enterprise meant "better" more durable etc. particularly TBW rating. like intel optane.

well, then clearly you must think most people only have one computer. what about people with a handfull of them?  :o

Me neither.

Like I said: data retention is a thing for enterprise SSDs. Here's some worst case clickbait reading (https://hardware.slashdot.org/story/15/05/10/0936213/enterprise-ssds-powered-off-potentially-lose-data-in-a-week).
So don't buy enterprise SSDs for your long-term backups :)

I can't say for certain if it helps clarify things or just muddies the waters, but I've often seen reference made to "archive" full nodes.  Perhaps that helps to distinguish between full-pruned and full with complete initial block download?

Nomenclature has to move with the times.  The technology is bound to evolve, so the language needs to keep pace with it.

seems you know the word "fool" but not learned the word "full"

a full node does all network features. thats why its called full
this includes being a archive seed of the FULL blockchain for other peers wanting to initial block download

turning off feature or limiting how many blocks you are willing to seed to other peers is less then full.

it also reduces a diverse subset of nodes able to service new users and irregular users that dont sync up often.
causing a more centralisation effect of lesser nodes having to take on that service.

a full node is (simple logic common sense) a node that does all features

maybe we should call pruned nodes a "fool" node to meet your style

Speaking of fools...


Who says it's necessary for every node to support the network?  There are other factors to consider when deciding whether or not to run a node.  Factors which you consistently appear to be oblivious to.  Sometimes people do have to run a node for selfish reasons and that's perfectly okay.



Said the ignorant, myopic, self-absorbed fascist.  No one declared you arbiter of who should or shouldn't settle for relying on SPV and accepting the drawbacks of doing so.  Not your call.  None of your business.  People can decide that for themselves.  Kindly shut up, Blockchain-Hitler.  You don't dictate the needs of others.

only a few days ago you quoted me saying before that , that not everyone NEEDS to be a full node just to use bitcoin..
so you know my stance
but those that WANT to be a full node should ensure they are being a full node. and not following fools idea's that a less than full node is a full node

YOU are the one telling people to use bitcoin less and use other networks instead
YOU are the one telling people to stop being a full node because thy can be a fool node . without even highlighting what users are not able to do or offer by being a fool node instead of a full node

I do.  And your stance is abhorrent.  It basically amounts to your belief that there's a "lower class" of network participant that only "deserve" SPV.  You believe privacy and security should only be privileges enjoyed by those who can afford to run a full node.  You also believe that cost should be higher than it is.



You are once again holding people to standards that only exist in your head.  If someone can obtain the privacy and security they require by running a pruned node, then that's their choice.  You don't get to declare that they somehow aren't "worthy contributors" to the network and therefore don't deserve to exist.  Or, at least, you can declare it all you like, but all it's going to achieve is that we're going to judge you to be a vile extremist who spews poison.



I'm telling people they have the option.  You don't want people to have that option.  You are Blockchain-Hitler.

you pretend to talk about choice. but then applaud that there is no choice or that someone else is threatening your idols.

so you call out that bitcoin is unfit for use for an underclass. and people should use other networks as their only choice or use a fool node pretending that its a full node by you trying to stroke their ego and say that its the same thing(facepalm)
or if they dont like your games. if they do not like the way a central dev team plan things.. you tell them to go fork off to another network

so YOU try to make it sound like bitcoin should only be used as a payment rail for reserve settlements. for the elite exchanges and subnetwork hub managers to settle on, where mere normal users should instead use other networks and be bunched into being managed by hub managers having discretion over if they will sign and agree with a payment you want to make. which they can refuse and hold hostage

and when that doesnt work to convince people to downgrade from wanting to be an actual full node.. YOU join the hymn sheet singers that technology is stuck in the 1990's-2005 era where being a full node is for only the rich and people should use the fool node setting pretending it means the same thing as full yet again

so YOU then try to sway those that want to be a full node not to be.. and instead be a fool node, even when they say in this topic they want to be a full node. by you pretending they are a full node when infact they become a fool node if they followed your advice. and you are not giving them an INFORMED choice of what the actual difference is

please grow up

Makes me wonder what's going to happen to altcoins with gigantic space requirements over the next several years. Maybe they will see less adoption, which would correspond to a price decline. The ones using the largest amounts of disk space are already centralized to the extreme and could fold if the node operators can't pay their storage bills.

Post when you see a Proof of Stake node operator go bankrupt.
Odds are that won't ever happen.
If cloud storage got too expensive, one could easily setup a RAID config at Home or Office and Node from there.
Or realize this is the 21st century ,
and there are 20 terabytes available in a single hard drive for less than $400 bucks.
ie:  Seagate Exos X20 20TB SATA HDD

---

PoW miners are already going Bankrupt, why don't you worry about them?
Because that is happening right now.

https://www.coindesk.com/tech/2022/09/16/chandler-guo-predicts-90-of-pow-miners-will-go-bankrupt/

https://wolfstreet.com/2022/11/23/lose-1-7-billion-on-519-million-in-revenue-bitcoin-miner-spac-core-scientific-shows-how-bankruptcy-a-year-after-going-public/

https://bitcoinist.com/crypto-mining-firm-goes-bankrupt/


Funny how btc cult members are so worried about Proof of Stake causing bankruptcy from energy costs, (Which we have yet to see one)
but care nothing for the fact that Proof of Waste is bankrupting PoW miners like it is going out of style.   :D

How did you know about slashdot?  ;D

But that link was a hoax. Look at the bottom of the page for the real explanation:

https://hardware.slashdot.org/story/15/05/25/1428246/no-your-ssd-wont-quickly-lose-data-while-powered-down?sdsrc=rel

enterprise ssds should be fine.
https://www.pcworld.com/article/427602/debunked-your-ssd-wont-lose-data-if-left-unplugged-after-all.html

give me the choice between an "enterprise ssd" and some consumer one and i'll take the former all day long. every day.

My guess: nothing. Many people keep their altcoins on an exchange to speculate, and the ones who use their own wallet, use something that supports many different shitcoins.

PoS coins are usually staked in pools/custodians. so no one cares to mine it or use the coin actively so no one cares to be a full node of blockchain security for them coins

when it comes to PoW (real asset coins)
people want to protect the network and support it.
(disclaimer to counter act a troll girls shinanigans. people are not forced or need to be a full node, but if they want to be, they should be a full node not a fool node)

bitcoins lack of scaling onchain is enon-politically hindered with silly excuses that bitcoiner full node users have a physics technical impossibility of being a full noder, with silly games of it being 1999-2005 right now

technology can and does grow over time. what seemed impossible in 1999 is seen as basic expectation now

things like the 4mb weight is for 5 yeasrs(along time in technology terms) been deemed safe. and yet cludgy code is impeding that 4mb utility with bad math to keep it hindered at a 1.4mb average block. reducing the utility while allowing congestion and fee premiums

since that 4mb safe announcement 5 years ago technology has moved on further still.

4mb block is a growth rate of 210gb a year. i have something the size of the nail of my thumb that can hold more than that.

a hard drive able to have a life cycle of average 5 years replacement(most upgrade their pc's in that timescale)
is 840Gb which is at a cost of not even $1 a month

yet while crying about $1 a month cost they also applaud that $1 a tx is reasonable

there are people that pay more money upfront on games consoles, where they then subscribe to services at $7 a month which use more bandwidth and hard drive space in that sale life cycle.

yet for some strange reason want to say bitcoin is broke/cant work at those same levels of activity for millions of users. and then they suggest people should stop using bitcoin and use another network

I just bump this thread because I was synch all blockchains yesterday

https://iili.io/HchxsF2.png (https://freeimage.host/)

The full size is 480 GB including blocks, chain state, and wallet folders. The Files such as debug, mempool.dat, and peers.dat is including also about 15 MB.

Blocks is mainly important, and have size 475 GB which contains 6,797 files
while Chainstate has size 4.73 GB

In my calculations, between Q2 and Q3 this year, the size of the block will reach 500 GB of data on my hard disk. what yours tought?.

Some people have more than that in just their Google Photos account.

It's really incredible to think that the entire history of all Bitcoin transactions from 2009 to today can fit in half a terabyte.

This is a really well done job by the Bitcoin devs. Really good job.

And the processing speed of the nodes has been constantly going up, it's way faster than what it used to be.

And new 1TB 1TB SSD NVMe PCIe are at $50 or less so there is not even an appreciable cost to putting your OS and your node on one.
Just bought a used 7th gen Dell laptop from Craigslist for $65, had a 4GB stick of RAM sitting around to bring it to 8GB and bought one of those drives.
So for under $125 another full node with lightning and a block explorer and a few other things is made.

-Dave

i heard that western digital is thinking of coming out with a hard drive that really is a tape drive inside.
https://www.techradar.com/news/i-cant-believe-this-huge-storage-company-wants-to-mix-tape-and-hard-drive-technology

would you guys use one instead of normal hard drive for running bitcoin core?  :o

alot of people are still scratching at their devices wishing to say how that the blockchain is "huge" and cant scale and how blockchains are not fit for function

you know the types of people. that still want to pretend technology is in the 1990's

heres a prospective for how huge 500gb is
500gb.. double it. and then you have

https://i.imgur.com/Dfn1bKK.png

its not a case of 500gb is "huge"
its that something that can secure hundreds of millions of peoples value, and 14 years history of transactions can all fit in the palm of your hand

I'm not sure what the developer did until increased my HD data?.

So can we small that blockchain data?, 
like WinRAR, we can compress the data from 1 GB to 200 MB, maybe we can compress it from 475 Gb to 95GB, and node still we can use for running and transactions on the bitcoin core.

Bitcoin devs make transactions and the data associated with it as compact as they can.

Also they make it so that processing the data is fast, so that the end result is that with updated software you can get faster and faster processing speeds of the whole blockchain.

Every new version of Core you download usually is faster than the previous one.

There is a limit of how much you can compress data in a lossless way. That means that if there's redundant data, sure, that can be compressed a lot, but if all the data or most of it, is vital, then you can't really compress it a lot.

For example, if you have one million zeros in a file, you can compress it by a rule that encodes "write a million zeros here". You'll have a huge compression rate.

On the other hand, if you have random numbers that follow no pattern, you have to just write them down, without any possible compression.

Bitcoin devs are making a good job of keeping the data structures as small and efficient as possible.

That's not targeting the consumer market. Tapes are for backups, not for data you use.

Consumer response: That's €159 for something Electrum does for free.
Micro SD cards aren't really meant for blockchain storage, and even though it can work, I don't think many people do it.
Years ago, budget laptops came with 512 GB HDD. Nowadays they come with less\: 128 or 256 GB SSD/NVMe storage. The storage is much faster, but the small size doesn't work for downloading the blockchain. Many are still sold with 4 or 8 GB RAM. The more expensive models come with 16 GB RAM (which often is the maximum) and 512 GB or 1 TB. But the average consumer won't buy them. They also won't upgrade their storage capacity.
It's actually one of the reasons I'm still using my old laptop: it fits 2 disks.

Blockchain data is more or less random data, which is hard to compress. I just tested it:
The compressed file got 15.58% smaller. That's really not worth making the data inaccessible.

I haven't tested it, but I expect that exchanging my HDD for a high-end micro SD card wouldn't make it any slower. It will depend on the setup though: I currenly only have my blocks directory on the HDD, not chainstate.

I can see it happening more and more on budget setups. There are a lot of laptops and PCs out there on the used market with smaller SSDs. Remember 6 years ago the 6th gen Intel processors were new and a lot of machines were coming with 256gb SSDs. Small by today's standards but faster then spinning drives and more then enough for an office PC. Around here they are all over the used market for under $200 with a warranty from a store and under $100 on the Craigslist / Facebook marketplace from people.

The SSD is not going to be big enough for the blockchain. BUT, just about all of them have SD slots. So any 1TB microSD with an adapter to regular size and you are done.
Yes it's close to $100 to get a 1TB card vs. $50 to replace the drive but not everyone is comfortable taking apart a PC. Sliding an SD card into a slot it easy....

Don't know if there are going to be a lot of people doing it. But once again for ~$250 a full node with room to grow. And it's a useful PC when you want to upgrade to something else.

-Dave

my point is that those people who shout "blockchain huge" and "bitcoin not fit for purpose"
are the ones that then reference some windows vista era hardware,
you know sata 1 or even IDE cable hard drives

where we all know solid state memory is faster, even on "average" priced SSD/ micro media cards

and no. its not a opportunity to show high spec hard drive(sata 3) vs low spec microsd

is it right if use micro SD express with PCIe technology inside? because when using that new technology, we can write data about 1 GB/s.
So if that micro SD is good to copy all data bitcoin, it will be very convenient and efficient to bring it on everywhere and anywhere if use it on different PC.

https://hexus.net/tech/news/storage/127829-microsd-express-integrates-pcie-nvme-985mbs/

An embedded tape would still be more expensive than a normal one (LTO-9 tapes retail for about $130 a pop) because of the extra electronics but you don’t need a tape drive to get started. As long as it sits somewhere between tapes ($4 per TB) and enterprise hard drives ($20 per TB), there will be a significant market for it.

i think price (ultimately) determines who ends up adopting it. but consumers pay $20 per TB for HDDs all day long don't they?

but if it was faster then that becomes less of a rule right?

What this alludes to is the intriguing possibility of getting the basic components of a tape drive merged with the actual tape media in a bid to reduce the inherent environmental and technological complexity of tape libraries as well as improving the access time by at least one order of magnitude.


how fast do you need to access your data?

but the blockchain is kind of like a linear structure in that blocks are ordered in a time manner one after the other. older blocks come first. seems like tape would be perfect for that. you don't need to access older blocks as often as you do newer ones probably. for a casual user of bitcoin core that only needs to do a transaction every so often, i don't see how access time would be a huge deterrent to using a tape drive.

How often do you want to use your own Bitcoin Core installation on someone else's computer? I'd never do that, just like I don't check my email on systems that aren't mine.

It's not about the price, it's about the use case:
Tape will never be fast when it comes to random reading. One order of magnitude faster is still very slow.
I see another problem with this: tape should be stored in a safe location, which means physically removing it from the system after the backup is done.

Use prune, it solves all those problems.

PCIe is just the port, the actual speed depends on microSD itself and most microSD isn't fast enough to utilize maximum speed on PCIe port.


On theoretical level, it's true blockchain has linear structure. Bun on implementation level, there are many random read/write such as read/update index file and read certain UTXO/block.

ok some people are saying about data rate of read/writes

so lets say we have 773377 blocks.. and over the 14 years average block is about 1mb (remember we had hundreds of thousands of blocks under 1mb in early days

now lets say we put a sync time of 24 hours
24 hours is
1440 minutes which is
86400 seconds

which is 8.95 blocks per second ..so about 9mb a second (rounded)

so when saying 'but ssd is more then platter', 'but both are less than tape'..
..it doesnt really matter much because ultimately if you take away the download time and the validation time..
 it takes the amount of data per second to read/write, less time... meaning there is alot of idle time between writes in that 9mb/sec
so because there are plenty of gaps between writes per block every ~0.1sec.. (due to the other tasks happening unrelated to writes happening between each block) there is no harm or over use of a data store even if said data store is cheap

But you know that Electrum cannot function without these data stores slotted into full nodes in the first place.

We're not asking people to run full nodes - we have enough of those for now. But more SPV and hardware wallets is absolutely a must.

Besides, I wouldn't gloss too much over storage capacity as ultimately it's the read/write speeds that matter the most.

i was actually being favourable with my rounding of numbers to the hard drive doomsayers.. whilst still debunking them, i

however if you really wanna go full on best case if you compute it more accurately
the read-write data per second even with rev, blk and all other files. the data is still not even close to the basic read-write rate of cheap data storage

for instance the 14 years of blockchain is actually closer to 0.65mb per block average
meaning even handing 10 blocks a second. in octuple the file saving(im going extreme now)
is still only 48mb a second.. far below standard storage media capability and still able to sync within 24 hours

yes lopp was showing how core can sync in 7 hours ..

my point is the other type of people saying bitcoin cant function, hard drives are bad, bitcoin bad, bitcoin takes weeks.. needed to be debunked. but fairly so they cant come back crying with new absent theories

and so, even on the most cheap boring hard drive a sync can be done with no storage device burnout in way under 24 hours, no special equipment or expense needed. (as lopp also shows)

but i didnt want to be stating the overly efficient numbers i was trying to be fair to all parties by not being the optimistic "7 hours" guy

I don't think many people here are saying it's not possible to sync Bitcoin Core.

I'm pretty sure it can be much faster too, given powerful enough hardware. Unfortunately, I no longer have access to the 32 core 256 GB RAM dedicated server I was given to play with a few months ago, otherwise I would test it right now.

Bitcoin functions, that's a fact. Some people indeed have hardware or configuration problems that makes Bitcoin Core take weeks or longer to sync.

A cheap boring hard drive isn't enough, you'll need RAM and CPU power. Take an old netbook with 1 GB RAM and 1 TB HDD, and you won't be able to sync Bitcoin Core in less than a month.

With a 2020 Mac Mini you have the storage needed and also it syncs the whole blockchain roughly overnight or so.

That's a pretty normal device, more than two years old already, and has zero issues to sync the blockchain.

I don't think there's an issue with the size of the blockchain and the speed of devices.

not too often, I ever try it on my external HD. and I just used it for downloaded blockchain data, I don't put the main wallet folder inside. Because I have an office with good Internet, and that can spend my money also if I used it on my home with not good internet.

You look good to calculate complicated mathematics, so after 3 days my bitcoin folder size increased from 480 to 481 GB, I am just curious what time the blockchain data reached max 850 GB?

@loyce
the MANY topics talking about blockchain vs hard drive size.. mention the same doomsday about how hard drives and blockchains cant cope.. its the norm expected evolution of the topic of such. so best bite the bullet and state the facts before the fud
 funny part is you start by saying not many say it, then.. YOU go and say it and re-join the club

@sarah
850-481 = 369
imagining a 1.3mb average data load now per block
~ another 283k blocks
so about the 1.05m blockheight mark.. about 5 years from now

Well lets say you could get a 10TB embedded tape drive for $50. people would find uses for that. even if the access time was on the order of seconds. i would expect tape to last longer than hard drives too.


Lets say they got it down to 10 seconds access time. I bet some old cdrom drives that people use for running Live CDs has seek times of 1 second no doubt. Yeah it's a bit slow but you learn to deal with it. Same idea here. For all the TB of storage, you would be willing to put up with it, maybe not you in particular but some group of people would no doubt.

You're thinking of tape only being used to backup data. Maybe think again if embedded tape drives ever come about and have sub 10-second access times. People would definitely use them to store videos and movies and stuff. maybe even the blockchain. Tape doesn't have to be just for "backing up". Why would it need to be?

I have a feeling too that tape would outlast hard drives. In terms of durability. As evidence of that, look at all the vhs tapes that are from 30 years ago still work. how many hard drives you know that old?  :o

I don't know why people want to specifically buy SSD drives for this?...... you can go much cheaper and just buy bigger SATA drives for the same price or less. SATA drives are also slower to boot up and slower in retrieving data than SSDs, but will a few milliseconds really make that much of a difference for average users?

You get some pretty good high performance SATA drives within the 1TB to 2TB range, with a lot of cache and good RPM that will be more than enough for the average user.... with an affordable price tag.  :D

As always: it depends :)
It's a fact that SSDs have much better access time than HDDs. Linear throughput isn't relevant.
I've synced Bitcoin Core recently on a HDD within a day, but it was a server with enough RAM, and high enough dbcache. There are multiple factors that can be the bottleneck if they're bad enough: RAM, disk, CPU or internet. Lack of RAM can be partially substituted by a faster disk. But you know all this.

Sure, and some people are still using the Commodore Datasette (https://en.wikipedia.org/wiki/Commodore_Datasette). And even that thing did the only thing tape can do: linear reading the entire data file.

Yes! Once used to an SSD for booting, you don't want to go back.

yea yea yea..
but the topic.. is hard drive. so to address the hard drive factor.
there is no need for some special hardware or expense of needing some large super fast device/server

boom. end of story

we could digress and say if you want to download the blockchain faster (as just a download process) by ensuing you have fibre and setting peers to be a higher number than 1-2 (to get more then ~9 blocks a second*)
*(where 36mbit/s is needed for the average pre 2017 data and 80mbit/s post 2017 data)
(then only ~1mbit/s once uptodate/synced)

we could digress and say the validation of blocks can be reduced in delay/bottleneck with high multithread CPU and high ram.. but it actually makes no difference if you have 256gb ram or 32gb ram
when a validation process only needs (purely for validation) 1gb. then you only need maybe 4gb to do all your normal windows desktop activity, plus active scan antivirus plus watching a movie/gaming while you wait for the sync while also having ram spare for the sync
having 256gb of ram on a server is wasted/overkill

there is no need to buy a whole server (usual cries).
bitcoin can sync fast enough on normal consumer bought equipment in their home
even the core devs have deemed upto 4mb a block data as being consumer safe and not cause a need for the doomsday cries of some people you know well shouting "but centralisation" "but servers"

---

i know you're subtly now admitting defeat that hard drives are not the issue without flat out saying so. by avoiding continuation of hard drives to then digress to talk about ram, internet..
for instance wanting to digress the discussion to now talk about Pc's with 1gb ram(loyce:'Take an old netbook with 1 GB RAM').. but come on.. even a decade ago normal retail bought pc's had more than 1gb ram. so dont do that silly narrative (subtle)"we are stuck with windows-XP era technology so we are doomed and bitcoin is slow" please dont follow your chum group narrative of excuses of decades old tech as sync delay reasons
as you sound like the fools of "but HD snapshot cameras wont be a thing because floppy disks only hold 1.4mb"
that mindset was soooo 1990's.. so dont be like that

and stop the subtle narrative that you only experienced a 24 sync using a super spec server that you wish you had access to, i know all the scripts your chums repeat, no matter how subtly you imply them here.. trying not to be so verbatim, but still trying to hint/push them

and no dont even hit the reply button with the script of "but i never said what you said i said, strawman". i know the silly narratives of your chums, in many topics. i know the subtlety you are trying to imply, but making it not sound like their narrative, whilst you still hint at the same script propaganda points they always do.
(again highlighting the whataboutism of a '1gb ram notebook', hinting of your server experiences)

Bandwidth-wise, sure, the network functions brilliantly. We have tens of thousands of nodes, some of which have the best Internet speed universally. There isn't going to be a difference if people launch more nodes. But running a full node is without doubt an important part of the system for those who value privacy and security. I'm of the opinion that running a node is rather of an individual's benefit, than a network's benefit.

And that's before we even mention that owning a lightning node requires running a full node.

oh look the usual crowd, and then the usual side-ssles pitch to a silly subnetwork that has no blockchain
(facepalm)

though this topic is about bitcoins blockchain size. seems a certain group love to  do all they can to subtly and implicitly pretend bitcoin is only for the elite or pretend everyone they know things it is and all that crap. while then adding in a sale pitch at some part in their conversation about another network.. boring not original. shameful

funny thing is majority of promised use cases of lightning wouldnt work.. let alone if you needed a laptop when you go to just purchase a small pack of chewing gum or a small coffee in those small use cases

reality:
lightning is a thing where idiots load up funds from multiple blockchain networks. and bridge to lightning(much the same as many other subnetworks that do the same thing(yep lightning is nothing special)) locking up value with a hub/manager and use their lite-app on their phone
lightning is not even a network thats made to work only with a bitcoin blockchain. infact people can create balance without any funds locked at all.. thats how bad it is, it can fractional reserve

but yea doesnt matter much anyway because there are many other sub networks that have more liquidity locked up from bitcoin locks. to then play around with silly micro amount units, becasue many realise lightning is the unfit network people need to know about.

even other similar buggy networks like avalanche which is only 1 year old has more liquidity locked than lightning. so i keep wondering why are these same fools  trying to say bad stuff about bitcoin, trying to divert people away from using/securing bitcoin.. just to then promote some other network called lightning which is not even going to meet expectations or fulfil its promises

but hey. if they dont like me calling out Lightning flaws. maybe they need to stop snake oil selling lightning while trying to pretend they are bitcoiners

maybe they should/could try to back things up with stats data and math. not their subtle crappy attempts to try to say bitcoin doesnt need more nodes, bitcoin can function fine with les people whilst then promoting another network
idiots

Funniest part is that it's 2023, and you still don't know what's lightning.

People use these cool devices a few years now. Check them out: https://www.amazon.com/s?k=smartphones

How? Please tell me, and I'll finally stop locking funds like an idiot. Creating money out of thin air instead of having a job never crossed my mind to be honest.

very first lightning payment was done on using value pegged from........ LITECOIN
people can do that.. its not just locked to bitcoin

users do not need to lock funds just to access lightning funds
a. get inbound balance from hub managers (giveaways, faucets)
b. thor turbo gave msat balance with no confirmed funding utxo
c, users dont even need to be a 2-of-2 co-signer of funds to ensure no cheating
    = cheating can happen
    = ownership of payment promises are not guaranteed

and yea there are many other subnetworks that act just like lightning (bridging between many networks)
lightning is not special or unique. its 7 years old and has less liquidity than other subnetworks

maybe if you tried to research you will see more then the small pamphlet advert you have been reading and narrowing your views around

---

back to the topic. unlike some that want to
-pretend bitcoin only works running a server (facepalm)
-pretend bitcoin only works running on a super high spec hardware designed mainly for server architecture (facepalm)
-pretends it wont work on normal consumer class hardware available today (facepalm)
-pretends its only for the elite.. (facepalm)
-pretends bitcoin cant work by using [insert hardware spec pc of 18 years ago]  (facepalm)

bitcoin can run on normal consumer hardware and even core devs admit it
so all these subnetwork salesmen.. do YOUR RESEARCH
and realise your subnetworks have the flaws!

Interesting ! good information. I am thinking to do some fractional reserve banking with BTC. I need further information.

---

don't worry about blockchain data size, there we will find new solution to it by segmenting the data size.

So, we do have not to worry much if having 1 terabyte HD right now, because 5 years is enough to start saving money if the user wants to upgrade to 5 or 10 TB. The problem is, If we change to a new HD (from 1 TB to 5 TB), will it Synchronization speed will be the same as before? Because I tried it and changed the target to external HD, it need more time to verify the block and syncing hardware

changing from lets say a 2016 era pc to a 2026 era pc(second hand in 2028 thus cheaper) you will notice benefits of the initial sync due to hardware efficiencies  (maybe 7hours or less)
however you wont see the benefits now if you just swapped CPU ram now(keepong blockchain backup) because you are already in the 'after initial blockchain download' stage.. so the normal daily activity of 'live' data flow to stay at blockheight is meaningless to a computer

but the initial block download being done on a 2016pc is still not bad (14 years of data in 24 hours) and with a 2026 PC will be better

all in all. most people upgrade their computers every 4-8 years normally. and for reasons outside of bitcoin. so thinking you need to specifically invest to stay up to date just because of bitcoin is a not even a thing people need to think or worry about

you only hear the FUD from idiots that cant math, and who just want to call out that bitcoin is dead, not fit, broke so they can advertise their silly subnetwork as the solution. even when their subnetworks have a heck of alot more flaws

Yes after Upgrading my RAM yesterday, seem like don't meaningful changes, loading block index is the same as before, but I receive meaningful change now, I can open many applications at the same time besides bitcoin core wallet.

I always open my bitcoin core wallet in the morning and close it after the blockchain is completed. I am just curious because always received a different number of blocks left. Today is left 115 block, and yesterday is 210, I ever remember have 110 left, so why is different at the same time?.

https://iili.io/H0bwNRI.md.png (https://freeimage.host/i/H0bwNRI)

its not a hard rule of 1 block per 10 minutes
nor a hard rule of 144 blocks a day..
those hare humanised simplified implicit/implified expectations.

explicitly
the rule TRIES to average out that there should be 2016 blocks every fortnight by adjusting the difficulty. but blocks can appear at anytime.. after 1 minute or after a couple hours.. meaning some days can have 100 some can have 200
on an average day there should be about 144block

What's your current system setup? How much RAM, what kind of hard drive (HDD/SSD/internal/external), which CPU, which operating system and what internet connection do you have?

I remember the days I switched from 4 GB to 8 GB and later to 16 GB. More RAM makes everything run faster. If only my laptop could handle more than that.

If I may ask: why? If you have enough RAM now, you could just keep it running.

This picture is the reason I asked for your system setup. My laptop is ancient, and catching up on 15 hours of blocks takes less than 2 minutes. It could be less than 1 minute too, I barely notice it.

bitcoin core 24.0
OS Windows 10 64 bit,
RAM 16 GB
Hard drive SSD NVME but, core wallet target on HDD internal 1 GB
CPU AMD A9
Internet connection: Wifi hotspot from phone cellular.

Sometimes when the core wallet opened, my internet is a bit slow to open the forum.

is that time left 77 minutes make you curious?. That screen I take 2-3 minutes after opening the wallet, but I remember less than15 minute  download completed.

FTFY (I guess).
Any chance you can put your chainstate directory on the SSD? I don't know if Windows can do this, on Linux I have .bitcoin in my home directory (on SSD), and the blocks directory in there is a symlink to my HDD.

It could very well be this is your bottleneck now.

That's all the more reason to keep it running: once it's synced, it will only download ~10 MB per hour and reduce the load the next time you turn it on.

Yes, the 77 minutes looks terrible, and 15 minutes is still long, but if your data is limited to 5 Mbit/s, it makes sense.

Starting with Bitcoin can require a fair amount of storage space. Currently, the size of the Bitcoin blockchain is around 500 GB and growing. As a beginner, you'll need to allocate at least this much storage space to fully download and validate the blockchain. It's recommended to use an SSD (solid state drive) as they are faster than traditional HDDs (hard disk drives) and can make the process of downloading and synchronizing with the blockchain much quicker. If you're using a 10 TB hard drive, it should be enough for the current size of the blockchain, but keep in mind that it will continue to grow over time.

You can always run a prunned node, that means that you do download the full 500GB and validate it, but you only keep in storage a fixed amount, say 2GB or whatever, of the latest data.

So, in reality you don't need a large disk if you simply want to run a node.

Running a pruned node has downsides for users. It can limit your ability to validate older transactions and blocks. In the event of a dispute, it can be more difficult to prove the validity of your transactions. Additionally, you won't be able to fully participate in the decentralized network as a fully validating node, making the network less secure.

I have a good internet connection with a speed of 100 Mbps. I have a Western Digital 1.5 TB HDD which has around 40 MBps write speed.
Yet it took me a good amount of time to download the whole blockchain. I used to keep it running in the day time and shutdown by night.
It took me around 2.5 days to sync the data completely. I was running it on a raspberry pi 400 due to which I think it took time.
May be a better configuration PC would have downloaded the data faster.

Talking about SSD, what I have heard is that it won't matter much if you have HDD or SSD because the internet download speed and system configuration will restrict the speed of downloading the blockchain data.
So I thought it would be better to use a HDD instead and save cost on it.

With a Pi, I expect the limited RAM and CPU to be the bottleneck.

RAM and SSD can (more or less) replace each other's function when syncing Bitcoin Core. With enough RAM, you can increase dbcache and don't need to read/write large amounts of data for chainstate. If you don't have enough RAM, an SSD significantly improves syncing time. If you have both, it's still the best. For what it's worth: my blocks directory is still on HDD, but if I ever buy a 2 TB SSD, I'll use that. It also improves performance when importing for instance an old private key.

You're talking to a chat bot spammer.

This was what we are discussion before, SSD or HDD is just storage where that function is only effective at the beginning. After download and initial complete, SSD is just storage whose function is not far away the same as HDD. So, SSD just spends your money if the core wallet function is just downloading the blockchain and running it. And, if you were bought SSD and want to upgrade, better to make that SSD as bootable OS and HDD inside and change it as bitcoin core wallet storage, this is what I do now

You always validate all the transactions in the blockchain, prunned or not.

When you're running a prunned node you simply remove older data that has been already validated.

You're usually checking transactions that are relatively current, so there are no major issues in running a prunned node, you can also set the size to whatever you like so you can have say the last 5 years of Bitcoin transactions. That is plenty enough for most people.

when new tx formats come out(rule/format change to consensus). unless you have upgraded to have new format recognition (they done away with/removes requirement of the 2009-2016 upgrade requirement pre consensus activation)

you wont be validating everything... you will be treating new formats (in an unupgraded node which does not recognise) as 'default: isvalid' without the thorough checks
thus the dev politics is making people more reliant on one brand of node to be the most uptodate that people should upgrade to the moment new versions are announced(the extra flaw of the centralisation debate in other topics)

Just saw a advertisement news article(note I didn't call it news or advertisement )

Seagate is coming out with 18 and 24 terabyte HD's this year.

no longer worried about 500 gigs. 

Although id like to see in the future is nodes that can download in reverse so full nodes can function.  but have a minimum of the last year of data. 

That's not how a blockchain works. You need to start from the genesis block to be able to verify everything belongs to the same chain.

to mitigate initial sync /initial block download time there are some options

one is a 'ball&chain'
where say for instance its assumed that block 0-630,000 are assumed as immutible. and mile stoned where you just download a ball of UTXOset(lump of data of utxo with its own string ID) of which you then build a chain from block point 630,001 assuming and calling the block 630,000 as the 'genesis' (first block of full chain data)

thus not needing to keep any "spents" of blocks 0-629,999 in the form of a chain

ofcourse it would need some coding and activation for nodes to understand this concept, but thats a discussion for the future

How would you verify transactions then? Having a incomplete blockchain would not be able to verify that you are on the right chain. The only exception to this are pruned nodes but then they do not function like a complete node. There is no way around the increasing blockchain size except for better hard drives. Most hard drives up to 4TB are cheap today. I think by the time we reach higher then that higher capacities will be cheaper to and you could split the blockchain onto two hard drives and still keep the integrity of the blockchain.

its a middle of between pruning vs full chain.
not saying its a proposal. just saying there are ways.

same principles apply. but if whole network deems blocks 0-630 are immutable you no longer need all chain data of such and just the state of position from 630k, everyone agrees state of utxoset of 0-630k can be milestoned and given a strong id ..where previous to 630k is just utxo of position upto block 629,999

but as said that would require whole network activation of new data set recognition and rule defining the 'ball' as the dataset of blocks 0-630k state

the problem with these higher capacity hard drives is #1) they cost alot of money and #2) they probably won't last as long as older lower capacity hard drives until they die for one reason or another.

if you have an 18TB hard drive then you got alot to be worried about as it is including what you're going to do if it dies.  :o because that's alot of data to have to backup....

So if HD is not enough space to save the blockchain, if I am a student college that has a student email with 1 terabyte of free google drive. Is it safe to save bitcoin folder to google drive?

I have downloaded google drive.exe and installed it as a local (E) google drive, which I think must take advantage of the moment.

You'll have to trust whoever you download the data on the first 630,000 blocks from. One of the main reasons to run your own Bitcoin Core node is so you don't have to trust anyone, and the only way to do that is by starting from the genesis block.

I wouldn't do it.
It's probably safe in the sense that Google isn't going to change your blocks to make you believe some addresses have a different balance, and it can probably work if it's mounted as a network drive. But what if it gets disconnected? Bitcoin Core won't be able to save it's data, which will lead to data corruption.
And there's really no point: if you want to reindex your wallet, you'll have to read all block data through your internet connection again. If you want your node to upload data, it will have to first download every byte it uploads, so effectively it passes through your internet connection twice.
If you're low on disk space, prune is much easier.

Downloading software from Google sounds terrible for privacy.

by the way its not my idea, nor is it even a proposal. its just demonstrating that there are ways

and like i said it would have to be a future discussion thing where everynode has a known milestone ID for block 630k utxo state.. this would be given a strong Id of the utxoset where the network nodes identify and share this ID for 100k blocks+ before being settled into data law of consensus.. thus plenty of time to notify nodes before even then taking a ball id, which obviously matches the data because they are all milestoning the same data

after all its like having al blocks and knowing the ID of block 774435 today compares to everyone around, to know we are all working on the same chain if exact data matches the ID and the id matches id of block heigh and everyone is at same chain height with same id means they all have the same data

but the ball idea.. is not full blocks, but a mile stone  blokckheight in the past, back long enough to be considered immutable where the spents are 'pruned' out already
and the hash of the ball is easily verifiable to a network known milestone much the same as the genesis hash is known throughout the network to keep nodes united and building on the same milestone

It's not matter of time, but rather whether someone would bother modify existing full node software to add such feature. It can be done by downloading all block headers then download blocks in reverse which skip many verification.


UTXO commitment also has similar idea.


Is this statement based on your own experience or certain article/research?

I do not mean to attack your idea but I think the integrity of the Blockchain is more important then saving a few bucks off already cheap technology. Those that are interested in running a full node will want that integrity so I do not think that this discussion in the future will lead to anywhere but recommending multiple storage locations. I think our full node and pruned nodes already cater to their purpose and we do not need something in the middle which will only confuse things.

You can't store Bitcoin Core blocks directly to the cloud. First you have to mount the file system using something like FUSE and rclone, but that is prone to unexpected unmounting errors so it is not something that I particularly recommend.

Besides, irrespective of your storage quota, there is a 750GB monthly transmission limit between your computer and Google drive, and Bitcoin Core already jumps half of that, meaning this method will become impractical very soon. It is already impossible to accomplish on some other clouds.

Cloud is probably more expensive then just getting 4 hard drives of 2-4TB and connecting them up to the node. 8TB-16TB probably means you will not run out of space for the node in decades and it would come at a cheap cost. You could use 2nd hand hard drives to keep the cost down if you wanted. Probably spend around $50-100 to get those 4 hard drives or you could just buy more as you are needed and put them in a caddy together.

Bare in mind, that SD cards are more prone to failure than traditional storage methods like hard drives, and SSDs. SSDs, are generally more expensive, along with SD cards. Hard Drives do the job for storing the blockchain, have stood the test of time, and are generally quite reliable, and cheap. It doesn't matter what brand you have of hard drive, we've got to the point that most have decent longevity, and are relatively cheap to replace when they finally give in.

When you're looking at storage methods for the blockchain, at the moment it has to be a hard drive. They're the most cost effective, while your second option would likely be a SSD card, but they have a limited amount of lifespan, that's determined by the cell usage, so if you're writing a lot of data to the SSD you could potentially hit that relatively soon compared to that of a hard drive. Although, the amount you're writing to the disk with the Blockchain isn't massive when you take into consideration 24/7 RAID servers that are basically operating constantly.

That being said, hard drives are generally bigger so won't fit in the palm of your hand as much as a SDcard, although that entirely depends on how big your hands are doesn't it ;). Most SSD cards today are tiny, and aren't far from a SD card, especially m2 SSDs. Although, I don't think many users here would have a problem accommodating a hard drive that can store massive amounts of data at a small cost compared to the alternatives. 

Either the RAM, CPU or the storage method. Some users might cheap out on the SD card, and get one that's got relatively low write speeds, which could be the bottleneck.

If anyone really cares the new price everywhere I can see here in the US for a 2TB Samsung 870EVO SSD is $149.99
Amazon, newegg, Microcenter, wherever it's $150.
These are all brand new sealed in a box drives.

Just about EVERY 4TB spinning drive is under $100 until you get into the higher end ones which you DO NOT NEED for this. Yes, your IBD might take a bit longer if the rest of your system is good enough. But if there is ANY other bottleneck be it RAM, CPU, download speed, then the drive speed does not matter.

Back to what I said back in November higher hour used 1TB and 2TB drives are being given away by a lot of people since they have NO real value anymore.

-Dave

I was thinking it also if the internet is not stable.
Right now, I still have enough space, I just want to make it mobile.

Student got 1 terabyte gdrive when login in using college email. and maybe more than 5 terabytes when you have enough shrewd to modify your friend email using Cbackups software.

My solution: I use different wallets for different purposes. Bitcoin Core stays at home, Mycelium/Coinomi/Blue Wallet/Phoenix Wallet are for the road (and funded with no more than I can afford to lose).

the way the visualised network topology plays out recently

majority of "always on" fullnodes* belong to the economic nodes crowd(merchants and services), compared to where many usernodes just drop in-out once a week-month to re-sync and then move some main stash funds to daily-spend wallets(mobile/lite wallets)

this is why some people think usernodes are unimportant to the network security and IBD seeding, thus they make features of "backward compatible" and pruning. where user nodes dont have to upgrade node often nor need to store all data 24/7
or thats how they want to promote it

not saying its right or wrong or making any social drama. its just how the network and dev/peer politics plays out

* yes fullnodes = full validation of latest ruleset, full archiving and full broadcasting
not "fool nodes" =pruned enabled, listen only enabled and out of date to ruleset

---

if your not in need of moving large sums or not receiving multiple payments where you just instead want to be able to play around with small amounts daily. then using a cell phone wallet of small daily spend amounts whilst then having a full node* separate for weekly-updates sync session(when stable electric is available) for your main bigger stash of coins would be your best case scenario of utility

but each person is different, you might(unknown to us) be a business getting alot of payments and wanting to pay alot off suppliers so may need a full node* more. where as after work just spending personal smaller amounts you use a cell phone app

man, you're talking about HHD right? it is slow as fuck.

Compressing data into a format like .zip or .tar can roughly decrease file sizes by a factor of 1/3rd.

500 / 3 = 166 gigabytes.

While I do not support most updates or amendments to bitcoin core, I do wonder if compression of file sizes may become necessary at some point to maintain node support.

You're wrong on several points: First, .tar doesn't compress on it's own. Second, dividing by 3 means a 2/3rd decrease in size. But most important: compression of (more or less) random data isn't that good. I just tried gzip, and block the file got only 14% smaller.

You can keep the block data on the "slow as fuck" HDDs and only the chainstate (and indexes if you have them) on a small SSD. And it's already very fast.
Even more, you no longer need ultra-speed after the IBD, so you can move then everything onto HDD. That's what I did not long ago I had to do an IBD (external HDD is not so great when one doesn't have UPS).


Did you actually try this out? I would be overly surprised if you get for real these numbers. Plus, as LoyceV said, your math is wrong.

You might not need a 4tb hard drive now but would it be better to get one if you plan on running a full copy of the blockchain to future proof? It would be cheaper to buy a bigger hard drive now then having to upgrade your 1TB in the next years. SSD would be a waste of money they are only needed for running operating systems to increase the speed but for storing the blockchain a HDD is the best option.

This philosophy is similarly applied when setting transaction fees manually. You're essentially tipping the miners for doing their job and providing the work/effort that keeps the network going and stuff.

Saving a few bucks here and there is found everywhere throughout crypto. Buisnesses and platforms will convince you that paying attention to savings in the present will help you in the future. You're still giving them value at the end of the day, and your value is being taken. Who tf cares how much?

So you apply that logic here, and it's like why worry about the PRICE of the bitcoin instead of weighing how many satoshis you currently have your hands on in the game.

The blockchain, in gigabytes, is less than 600. I know that it's growing more abruptly than a few years ago, but 2 TB is enough for another 30 years according to NotATether (https://bitcointalk.org/index.php?topic=5301075.msg55857178#msg55857178) (who's proved a little bit inaccurate, but even if you take 10 years, it's still sufficient). 

What does transaction fee have anything to do with running a full, non-pruned node in an HDD? What philosophy?

lets assume that (unlike a certain groups desires) that we do not grow the blockchain by just 60gb a year
nor wait 30 years before chaing parameters of the dev politic code cludge preventing scaling

lets assume we finally convince devs to remove some code cludge and have full utility of the 4mb space per block, (they have said for YEARS is network safe way way way before certain peoples desires of a "wait patiently" for 30 years(facepalm))
(full utility of 4m safe space is not a big ask)

1mb=52.5gb so 4mb is 210gb

lets say we round of the year being patient and get to about 500gb by year end 2023
and start afresh uncludgy in 2024 with better transaction count capability in 2024

december 2024 =710gb
december 2025 =920gb
december 2026 =1.13tb
december 2027 =1.34tb
december 2028 =1.55tb
december 2029 =1.76tb
december 2030 =1.97tb

by 2030 people will be wanting to upgrade their computer again anyway.. obviously owning a 2020 computer that has 2tb(average expectation) and having it already for a couple years and still good until 2030 it would be about time to upgrade anyway

people still sell sub terabyte hdds on ebay and no they're not giving them away for free. depending on the hdd model, people might really cough up some big bucks for 1 or 2TB USED hdd. i've seen it.

if your paying premium for a used hard drive(sub 1tb).. at a higher price than a new hard drive with more capacity(retail 2tb) and where if on the same site you admire... then find a 1-2tb USED which is also bigbucks..  your searching the wrong listings

maybe its time you stop using ebay,  searching for particular models, as the models you see on ebay are premium

its much the same as a games consoles
people can buy new games consoles at retail stores cheaper then some ebay listings..
the fault lays in people search ebay for games consoles

Hosting anything that is not student related on that cloud account is against their terms of service and could mean that the cloud storage gets terminated. It is only limited to the time that the person is a student to so you would have to think about long term solutions even if you could host the blockchain in the student cloud that a couple of years. It is not a permanent solution but a cheap 1tb, 2tb, 4tb hard drive is a permanent solution.

Interesting. didn't know that before. Thanks! For the 'chainstate' (and indexes if you have them), where can i find them and what are these folders or files exactly? 

So, basically are you meaning that after the IBD, it is okay to move the blockchain data to a HDD (keping the indexes in SSD)?
Thanks!

I don't know what they contain. Files  :D
chainstate and indexes are folders in the data folder, usually at the same level with blocks.
But it depends if you use standard settings/config or you've already set up something, since there are config settings for them.
You may want to read:
https://en.bitcoin.it/wiki/Data_directory
I don't know the settings for those folders, I've always used symlinks.


What I meant was that the blocks can always stay on HDD (including during IBD!). And after the IBD even having everything on HDD no longer hurts that much.

ok. a bit of confusion here. The purpose is to prove that using HDD is not losing too much comparing using SSD.

So, you are saying that "blocks" directory can be stored on a hdd, but "chainstate" directory should be stored on ssd to not to lose any speed.

Some time ago i did that, although using Bitcoin Signet (similar with testnet, except SegWit is always enabled).


The result isn't bad, but we know strong compression/decompression process require more CPU/RAM.

---

Unless you claim your study/research is related with Bitcoin :P. Although AFAIK storing file which violate copyright is more risky.

It's much more impressive than I expected. 7z is probably also more powerful than what was previously proposed, depending on the compression level of choice, still, I'm impressed.
Does the node need the entire blockchain "at hand" in order to validate transactions, or it has the UTXOs somewhere else? Because if it needs to continuously unpack old blocks then indeed the computing power spent probably not worth it. And if one wants to save space he can just use pruned node.

To be specific, that 7z use LZMA2 algorithm which is known to be slow.


No, Bitcoin Core has directory called chainstate which contain all UTXO. That's why people say pruned node still perform full verification.


It's also not worth if you need to look up for specific transaction or block frequently, which is needed when you run self-hosted block explorer or Electrum server.


People also could try built-in compression feature on the disk format. IIRC NTFS and BTRFS have such feature.

Maybe but I think you have to be registered with a college to qualify for the student cloud? You have to show proof of enrollment too. AFAIK that is how it works.

a lil bit of research reveals
students get a college-uni email address
then
when signing up to google they have to give them their college-uni address and an email confirmation is sent to students uni-college email

..

anyways this topic had started with the OP saying he had a 10 terrabyte hard drive and wondering why it was taking 3 days to sync

its been noted several times its not due to the hard drive. with multiple examples why..
. and that includes that a ~774,000 blockheight of ~450gb would be even at 10 blocks per second only be 24 hours and thats on a slow day. but where even generic hard drives can handle way way more then that.

thus debunking the hard drive being bottleneck theory
thus not a storage amount problem or a write-read speed problem

even 6-13mb a second is way less than the limits of even old hard drives
(plus all the other blk rev file stuff ontop still doesnt exceed hard drive tolerances)

.. all that said i do find it funny how people went into a whole tangent to try to get someone to buy some majorly expensive new server-grade hard drive and pretend bitcoin wont be good unless large server cost investment went into it (facepalm)

however its more likely a bandwidth (intermittent internet) issue

ram, also deals in GB amounts not megabyte amounts
also even if ram is over utilised the pc can use hard drive as virtual memory (page file swap files) but again they are dealing with GB amount of data store/processing but only need a few mb per second utility.

the silly delays in syncing i see today happen more often by people using slow network protocols like Tor and VPN that have the worse ping rates and bandwidth speeds even if your device is linking to an ISP at highspeed
its the propagation/relay between nodes that slow the momentum down

i say this because
if hard drives can tolerate terrabytes of data
hundreds of mb of read-write speeds

ram deals in gbs of data and even using virtual memory hundreds of mb of read-write speeds per second

the only low baring fruit that is probably going to be only a few mb/s would be the internet. especially if using tor-vpn bad ping rate utilities

and searching OP post history, they like to talk about utilising tor alot. thus i feel that is their actual issue

but i still find it funny how people are trying to make it look like bitcoin cant function on hard drives bought circa 2018-2023 and pretending everyone needs to by some high spec hard drive that only servers use

I would strongly recommend not to do that. Disk compression is a thing from the past, when disk space was scarce and expensive.
Even if you manage to reduce Bitcoin Core's data size by 20%, you'll go from 500 to 400 GB. That's 100 GB less, which is worth like $2. If you're that low on disk space, just prune your node and save 490 GB. Just imagine you need to use recovery software on a compressed disk.

Do you know what your current bottleneck is? If you gave dbcache 10 GB (which is excessive), did you have enough left for the rest of your OS (without swapping)? Still, 2 days (10 GB per hour on average) still isn't bad.

I think a 2.5" disk can have a lower access time, because the arm has to move less far.

The fact that Windows shows something doesn't mean there's a technical reason for it. It means people are willing to pay for it ;)

I've synced much faster on a server with HDD and 16 GB RAM, that's why I asked.

Now you assume all people are rational. My guess is many people just like to click "compress" to make them feel like they're in control. Kinda like running defrag.

incase your subtle trying to infer that microsoft still technically supports windows 98 and xp and even vista.. well lets have a shot at debunking that

many business by default upgrade every 4 years as there is a 4 year tax relief on business tools(they can write off a % of PC equipment costs against tax for upto 4 years)

many software providers end their technical support after X years because most of
their customers have upgraded to a new PC with new operating system by then

this makes microsoft then not need to provide tech support(OS patches) for 10yo systems because most corporate customer have upgraded in 4 years and consumers within 4-8

old features like opening an old *.doc file in latest MS word is different than no longer providing software updates/tech support/bug fixes for word 2012

so even microsoft move forward with the times and gives up trying to support systems over 10 years old

heck even core no longer support XP or Vista era PC's

i only said that as i have seen in many topics about chaindata size and hard drive capabilities(like this topic), where people talk about bitcoin hardware age suggesting that bitcoin should support hardware from back in 2005 as the reason they cry that 500gb is "to big" "to slow"

they usually side step first by giving other examples of big tech still supporting XX year old stuff.. then reign it in to use as an example that bitcoin should emulate that and stick to floppy disk data and all that other BS 'cox corporation X does it'

im just saying 500gb is not big or slow.. after all we are in 2023
not 2003 nor 2008 nor 2013 nor 2018

funny part is some people that pretended to be "data saving conservatives" are now saying, let bitcoin bloat up the 3mb weight space with useless ordinal meta data unrelated to making a payment transaction peer to peer

personally 500gb of 14 years of international currency 'be your own bank' statements of millions of users is pretty small lump

I don't really expect a large difference, it's still HDD. I can't find the age, but it has 7768 power on hours (less than a year).

Given enough RAM, I don't think the disk speed matters.

---

---

I'm running a new performance test on a cheap hourly paid VPS (https://bitcointalk.org/index.php?topic=5434679.0), this time with 4 CPU cores and 16 GB RAM. In less than 3.5 hours, it's progress it at 24% of the blockchain (and downloaded 105 GB).

Plus porn spam...

I never had issues with ram for node syncing but my internet connection has always been so slow (currently under 4mbps) that it didn't matter anyway. An acquaintance with like 100mbps synced in like a day or two, without changing anything from defaults. At least in Linux, any ram you have unused gets assigned for "buffers" which helps for disk i/o. I guess those suggested optimizations can help a bit, like putting chainstate in a ram disk, on mine (pruned) that folder is currently at 3.6g (at 2019-03-28, still syncing).

At that speed, it takes you at least 12 days to sync. Still not bad for a one-time event.

Even though you shouldn't do it (https://bitcointalk.org/index.php?topic=5177528.0), I can just get you a pruned and up-to-date download and save you a few days. Unless you're trying to update an old wallet, in that case you really have to go through all the blocks.

I really don't know what is bottleneck means which I find in the post above, Not only here, I always find it somewhere, youtube, forum and etc.

So how to make it not happen (bottleneck) if I want to build the PC just to run the bitcoin blockchain with low spek PC (like pentium III) ?. where I can find the tool to calculate to not happen?. it will be redundant if I bought the high speech ram or hd but bottleneck.

Basically it means that things slow down because of that one component. That's the bottleneck.

If you change that part of the system with something better, the whole system would perform faster.

you might even be laughing about that sooner. if nfts get a foothold into bitcoin.  :o

True, the blockchain size can grow up too fast from now with those NFTs, some days ago I even see a zero fees transaction for close to 4mb, and the crazy thing is it gets confirmed, the miner should be crazy to include one a transaction like that in the block, but well, it happens. And it was the biggest block in bitcoin history. For me, that looks like a huge vulnerability in the blockchain. But is just my point of view.

today alone i seen
https://www.blockchain.com/explorer/blocks/btc/775085
3.8mb 149tx

https://www.blockchain.com/explorer/blocks/btc/775091
3.16mb with only 789 transactions

the usual average was about 1.3mb with 2000 tx
which if all that cludgy code of miscounting bytes and favouring 75% of block for (witness/metadata) aka non tx payment data was removed, it COULD have allowed
~4750tx for the 3.1mb block    ~5850tx for the 3.8mb

but nah the "powers that be" dont want bitcoins to increase tx count with lean tx data.

so yep. expect the blockchain to fill up with more data(up to210gb per year at the expected rate of 52500blocks per year of 4mb) at a average higher than 1.3mb per block, but without increasing the tx count per block and instead decrease the tx per block (facepalm to those that let it happen)
(wait de-ja-vue,, i think i remember saying this many times for a while now)

i wouldnt mind the 4mb so much if the data inside the 4mb was a tx count increase of 3-4x the norm.. but not a friggen decrease of the norm tx count while a bloat of the data.

I searched a bit: it's block 774628 (https://blockchair.com/bitcoin/block/774628). The (crazy) transaction you're talking about: 0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0ae (https://blockchair.com/bitcoin/transaction/0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0ae). So there's a shitload of witness data (https://mempool.space/tx/0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0ae), and apparently someone stored DOOM (https://decrypt.co/120459/will-it-run-doom-bitcoin) in the Bitcoin blockchain.
I guess the miner did this on purpose, and I also guess most miners would prefer to earn transaction fees instead of doing this, so blockchain spam shouldn't become a big problem.

This transaction wasn't the one with Doom's code, this one was an NFT and you can see the image here:

https://ordinals.com/inscription/0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0aei0

All that data for a terrible paint image is just fun. And The ones who create the transaction make a deal with the miner to pay him for the confirmation outside the block as we can read in this tweet:

https://twitter.com/guzmanpintos/status/1621123165949931520

So, I think this was just to demonstrate the limits of the ordinals, but the world and the bitcoin blockchain aren't ready for that tech, if they fuck up the blockchain we will need a hard fork to fix all this stuff.

i still laugh facepalm and nod in disagreement at how they promoted taproot as a "solution" to cut down on weight of meaningless data.. yet. it has produced the opposite result


a block of  3.955mb(facepalm) with only 63 transactions(facepalm) where 1 is taking up 3.915mb of space(facepalm)

yet another example of how core devs have really sh*t the bed on this one and failed the community in soo many ways

is the lower fee can make more Megabyte of data?,
for example, 0 sat makes 4MB of data, so if ! use 10 sat fee, it could reduce 4 MB data to be 2 MB data?.

I just hear blockchain spam, can this make full data Hard Disk unexpected?
today is about 480 GB of blockchain data, is that spam included?, and how much GB approximately?

the most the blockchain can grow per year is 210gb
which is 4mb per block for ~52500 blocks per year

however previous to this year blocks were averaging 1.2mb which was ~63gb a year
2016 was 1mb which was 52.5gb

as for fee's
for the spam data.
becasue its sat in the "weight area" it is not treated the same as normal bytes so it gets a discount, plus due to other cludgy stuff they can pay 1sat per virtual kb and be alot cheaper.

thus instead of having a system of 4mb blockspace for payment where fitting say 4tx per kb of payment means a lean 1mb being 4000tx meaning 4mb being 16,000tx

even in a lean system of such using legacy tx, would be trying to charge 1sat/byte minimum which would fill a block for 4000000sats (0.04btc($902))
yet the way that this spammy thing has been allowed they can fill a block for 1sat/1kb
minimum which would fill a block for 4000sats (0.00004btc($0.902))

sometimes you make really good sense franky. if you keep up this high quality level of post you might find yourself right back on the development subforum. but i guess that's only if greg hates nfts too.

nfts though, they really need a home. there's no where else you can store them "on chain". except for bitcoin that is.   ;D

how many nfts was that 1 transaction? if it was just one then that really sucked up alot of blockspace. wonder if it was a picture of a monkey.

greg doesnt actually care much

its the usual certain group of drama queens that cried to him years ago and he snapped where i was the main critiquer and scrutiniser thus causing most controversy that triggered the trolls more than greg. thus he thought easiest way to nip it in the bud was to pander to the cry babies and shut them up by shutting me up
(and that win for them has energised and powered them since thinking they have power)

yea the group that keep quoting about the "greg banned you" were the ones that  pushed for it to happen in the first place. so they are quoting their own actions as confirmation bias. but pretending its third party independent source proof.

now its just over zealous achow that is mod heavy because he is a stiff fangirl of a certain clan..

funny thing is greg comes on the forum alot but doesnt delete my posts, but when achow is online he deletes posts.. not due to any insults in a post nor inaccuracies but if it mentions  certain topic of a subnetwork in any critiqued/scrutinised way (basically a non kiss assly Pro- positive manner of snake oily sales pitch). its gone

achowe is also going power crazy on github in recent weeks (https://github.com/bitcoin/bitcoin/pull/26890) wanting to add more moderation hierarchy to clense out those opposing cores roadmap
(again not independent moderators.. but them moderating, to moderate out independent minds.)

so i dont expect things to change any time soon in that regard

as long as kiss-assy self promoting echo chambers of centralised group is tolerated more than independent minds of research, review, scrutiny and critique.. it wont change

notice the lack of critique/opposition speak about ordinals in the tech discussion. the category should be seeing lots of posts about how core messed up or atleast asking for fixes.. .. but instead silence and alot of "lets see what happens, lets not make a rash move to stop it" stuff

things are getting worse, not better

well that's good to hear.

by banning you on the dev subforum they basically prove that they don't follow the philosophy of bitcoin which is decentralized and not centralized control. you been around a long time and for them to just ban you is kind of like almost banning someone like satoshi  :o


yeah expect up to 210GB per year from now on i guess. unless miners start censoring transactions or something. plus more expensive transaction fees. that's an obvious. but OT here.

I'm not sure what you want Bitcoin Core devs to do about this. Miners have always been able to fill blocks with as much spam as they can fit in, and the assumption is that most miners won't do it, because they earn more if they include paying transactions. And most miners indeed prefer making money over spamming the blockchain.

Your math is way off.

a. code is not AI(self coding/expanding) nor does an ASIC make the code.. nor do the hundreds of thousands(of full nodes). DEVS do
if you thing its miners fault.. its not . its the code DEBS RELEASE that allow thing to be soft thus not in a ruleset

devs can solve it by making hard rules like limiting bytes per witness..

dont be part of the madhatter crew song sheet echoing out that its the users or miners fault and devs are innocent. its the devs that write the code and bitcoin relies on code. and since 2017 core have had defacto soft access to add what they like, while treating any other brand that wants to as opposition/traitor devs
(but it was funny to see you do the usual dev defence game. (blaming everyone else apart from devs)

..
b. as for my math. yes i made a typing mistake on the valuation conversion.. awww. petty knitpick

seems a shame(more so shameful) that you are quick to point out my bug of insignificance to the community(my sat to $ quick demo).. but want to hide away from pointing out core devs very significant bug that does affect the community(code bug of code THEY created)

are you really that bias (by being be a core dev suck up) and not want to have them treated as non-gods?
they are human. they make mistakes. and they should be critiqued and scrutinised about bugs THEY CAUSE

Your retry-math is still off:

I don't get it: first, you argue hard disk space is not a problem and we can buy many TB disk space at low rate. And then you're complaining that someone wasted 4 MB once. Someone proved a point, good for them.

@loyce
if you want to be some grammar nitpick and 'oh look lets raise awareness that someone on a forum was typing too fast and didnt care about minor details of an  example demo...
.. but then ignore an ACTUAL friggen CODE BUG in the bitcoin network that actually affects many peoples utility of bitcoin.. well you play ignorant pedanticism

you try to down play real bugs but waste more time on being a pedantic grammar knitpick..
.. it shows more about your failings than mine..

as for the point of the context..
its not about the 4mb. its about how useful is it.. emphasis useful 4mb. EG is the space being used wisely.

those memes are DEAD weight
they have no value. they have instead caused other transactions to be delayed and now other people to pay more to try to outbid such dead weight

i find it funny how your clubhouse have for years have said you dont want to see "coffee amount" in the blockchain but happy to see 300k-4mb memes

to me that 4mb space should be used FOR LEAN PAYMENT TRANSACTIONS
because.. yep bitcoin is about payments not memes

with an average lean tx being ~250bytes
meaning 4000 for 1mb meaning 16000 for 4mb
that could have been 16,000 peoples wages or goods purchases. not a single persons MSpaint doodle
or at the average expectation of 2000tx. could have been 8000tx

and no its not "just one" or just "once" meme. there are hundreds of them. thus it needs to stop before it becomes more memes per day/week (causing congestions and fee wars)

as for your "just once"
before this saga.. blocks struggled to get to 1.5mb but average blocks had ~2000tx
so lets just give a few examples of blocks over 3mb lets just see if there was a healthy 2x tx count (4000tx)
https://www.blockchain.com/explorer/blocks/btc/775287 3.35mb 893tx
https://www.blockchain.com/explorer/blocks/btc/775286 3.88mb 230tx
https://www.blockchain.com/explorer/blocks/btc/775285 3.16mb 1083tx
https://www.blockchain.com/explorer/blocks/btc/775283 3.41mb 756tx

and thats just in the last 4 hours!!

.. and you say "just once" (pfft) as if bitcoin has only seen one meme ever

Don't get me wrong, I don't disagree. But I'm also more realistic: Bitcoin transaction fees are a free market, and if someone is willing to pay 1 sat/vbyte to get a miner to include his BS, he can. If that leeds to congestion and fees go up, at 25 sat/vbyte he's already looking at 1 Bitcoin for 4 Mb.

I noticed low-fee transactions piling up (https://jochen-hoenicke.de/queue/#BTC,24h,weight), and usually that means someone is consolidating many small inputs. I'm not sure how I can quickly check if there's a "BS-txid" in those blocks, do you have a link?

at first i thought.. google can help you in like 12 seconds..
then i thought if loyce hasnt found a good source in years then maybe google is not good for him

so i done a 12 second google for you
and again thinking maybe just throwing numbers at you would just get you reacting (usual social club way) of trying to find fault to ignore the context and not think.. so i thought how would social drama queens prefer to be educated about tx size in the mempool and blocks. and so i found the perfect representation even you might be able to see..

so
https://txstreet.com/v/btc

notice how big the southpark characters are waiting for the bus
the bigger they are the bigger the tx weight.. click on the character to get tx details


you can then put the txid into this to see if it has a ordinal
https://ordinals.com/inscription/<txid here>

have a nice day

It's hard to explain how bad is this for the blockchain... This time i'm with franky1, the blockchain is under attack, those memes are cancer, and the only ones who can fix this shit are the coin developers... Bitcoin was made to be a payment system and not a meme engine. And memes are not the main problem, I want to see how are we dealing with some nasty porn and pedo images in the ordinals.

Consequences:
*A blockchain full of nasty porn and illegal images.
*Blockchain size going too gib in a short period of time.
*Tons of transactions stuck.
*People leaving bitcoin and migrating to other blockchains.

If you remove anything from the blockchain you re compromising one of the fundamentals of btc. Not being able to alter the blockchain. It is not up to the developers it should be up to the miners whether or not they accept a block and there should probably be more scrutiny when they do but you cannot do that because all they care about is getting their cut and processing a block is hard enough. Rejecting it would cost them. You are never going to stop people from putting this one the blockchain and if you ever do you compromise the whole idea of the blockchain. 

having op codes that allow things through without any validation rules is the trojan back door..
those opcodes are treated as features to allow "backward compatibility" of not requiring nodes to upgrade to allow a new format in by node law..

these flaws need to be dealt with and harden consensus where nodes should validate everything. and new features only allowed if majority nodes are ready to validate such new features. as thats the true old purpose of consensus that made bitcoin great

the softening of the consensus rules by allowing these soft entry systems of new format is the flaw.

the solutions are simple. and doesnt need a re-org. doesnt need to get everyone to just pay more.(yep im saying it.. letting the flaw stay open and encouraging "pay more2 is not the solution)

the solution is
a. harden consensus again
as of block 7XX,XXX
each opcode has a rule. where each opcode has its own byte count math. to limit each opcode from being able to bloat KB per input

this achieves multiple things and benefits alot of things for the community, without rejecting existing blocks

The blockchain is under attack every time someone attempts to capitalise on recent events for their own misguided agenda.  And that's what is happening here.  Don't buy into the fear-mongering from those who would weaken Bitcoin.

Just imagine we increased the size of blocks, like the skeezy opportunist wants.  That would make it cheaper to spam the blockchain and more expensive to run a node.  That just opens the door to more novelty images and then the same skeezy opptunists will tell us we need to increase the size of blocks again, because it's all they can comprehend.

Bitcoin has coped with spam attacks before and will do so again.  Buying into the "Chicken-Little-sky-is-falling" nonsense is only serving the interests of those looking to undermine Bitcoin.

Of course this is for the future, whats done is done. But action should be taken, even if spamming has been possible in the past, its about time something is done about it. I particularly don't feel like running a node for the benefit of parasites of the blockchain, we do it for Bitcoin transactions not whatever else that are not Bitcoin transactions. Quite simple. Judging the spam itself is not what this is all about, i don't care if it is memes, NFTs, or whatever content the State hates to fuel the FUD against Bitcoin.

This reminds me of Jan 2018, the network was saturated, but improvements were made, and its been quite nice until this happened. And its only beginning, what Ordinals and accomplices did, others will soon follow; and left unchecked we will repeat the eth network collapse here as well.

at least you come up with possible solutions to the problem rather than just complaining about it. this was a rather high quality post. but it went over my head a bit but it does sound like the original intention was not to let someone abuse transactions to make them arbitrarily large in size as far as bytes go. big bug in bitcoin. no transaction processing system should be without a byte limiter  >:(

I wonder if bitcoin would be better suited if the entire network was restructured to work in a SQL context natively, and the RPC system was better designed. See the concerns here:

https://bitcointalk.org/index.php?topic=5159374

its not just about hardening the rules of each tx format by applying a byte limit to each possible tx format to prevent wasting space on bloat
(pre-empt knitpickers, no im not saying <80bytes for all, nor have i ever,  just rules that stop 'upto 3.9mb')


its also by not having a bypass to allow a new tx format to slide in without rules(which can then re enter a mechanism of allowing bloat, due to lack of rules)

tightening up consensus again. by needing nodes to be majority ready to verify each format including a proposed new tx format is the safe way to verify the network. as thats called network security

the silly softening of consensus years ago was a mistake devs made.

the point of a decentralised network of FULL NODES is that FULL NODES are verifying all data.

no new unruled tx formats should be able to slide in before majority of full nodes are ready to support the new format.
where said new format has rules of limited byte utility to avoid bloat

it feels like we're trying to patch up holes in a shooting barrel full of fish. once this one gets patched up where is the next hole going to appear with water gushing out of it? sad that bitcoin got to this state. if the public ever gets wind of this, we might expect a price correction.  :o

thats why i said patch the holes AND strengthen consensus to remove the soft implementation crap
thus in future new formats will only be acceptable if and when majority nodes have upgraded BEFORE activation to consent to activation (real hard consensus again)
so that full nodes can verify everything and not be a soft system of..
.. old nodes just not needing to upgrade and just letting things though as "isvalid"(current soft consensus system)

rather than just patch the taproot opcode of ordinal bloat

I haven't found the transactions, but if they actually pay 1 sat/vbyte, isn't it just fair game? I don't get why anyone would be dumb enough to pay hundreds of dollars to spam the blockchain with a stupid picture, but if it's anything like NFTs, they're building up for a ponzi and it will eventually die off again.
Franky1 claims they can pay 1000 times less, but hasn't shown an actual transaction other than the one where a miner made a special deal:

Bitcoin was also designed to store (hashes of) data. I don't really get why suddenly transactions this large are allowed, but even without it, many individual transactions could accomplish the same result: large full blocks.

According to jochen-hoenicke.de (https://jochen-hoenicke.de/queue/#BTC,24h,weight), fees are dropping to the minimum all the time. But my transaction with 1.05 sat/vbyte isn't confirming since yesterday. It looks like prioritizing fees doesn't work the way it should.

loyce.. you know more then you pretend
this is not a time to play dumb

you know that a transaction spending 0.00001000
can just set itself up where the ordinal of 3.9mb utxo and the outputs being 0.00000999 to its own change address. meaning losing just 1 sat as a fee

your groups whole "free market for fees" "no fee rule please" promoting of the fee market has been such for years

there is no fee structure of rule. made into consensus or in block templates..  its just where core devs have guided users nodes to follow some crap math that pushes for fee growth. but that crap math is not enforced by consensus rule of post confirm block validation. nor so by mining pool managers collating preferred transactions, which they can collate and can accept any tx they like even if it pays just 1 sat for the whole tx meaning 0.00000001 for 3.9mb

You know, you should really work on your attitude. Try it.

I know nothing about "ordinals", but I do know miners love money. If someone pays them 1 sat to fill a block, why are they even accepted in their mempool? Drop them, and include my transaction that pays over 200 sat for just a few bytes!

they do empty blocks. so they dont care for fees as much as you think
they actually like collating data with least sigops
as thats less time between block attempts wasted. and allowing more time to churn through nonce and extra nonce to win the actual large reward
6.25btc = $140k is alot more to worry about winning due to a speed race.. than a $1-$1k in deciding which tx to include/exclude

tx with many sigops takes more time in collating tx's in their mempools.
so they would prefer a mempool of least sigops. and blocks of least merkle tree hashing and sigop checking a data block of

and if they can fill a block that uses only 1 sigops.
but where added bonus for them pre-planning this, can cause a congestion where everyone else then increases their fee rate. then thats a game they can and do play..
also, later when there is not a ordinal to add in that block session. they can get everyone elses raised fee tx's thus they win that way..

its just the same business accounting decision as empty blocks..  where its done to cause congestion to force everyone to pay more, for later blocks

as for my attitude. seeing your silly game you play socially for years causes me to lose respect for you. this is not the first time we have talked and its not like your some newbie.

you spend soo much time analysing social data of forum posts i know your not dumb, so dont play dumb. dont act like you dont know how to research and analyse data from sources.
i know your agenda and what side you lean towards so dont play ignorant that your some unbiased unknown user that has no social club

this is genius stuff franky. you must be a miner or closely associated to them to know this kind of thing off the top of your head.  ;D

years of research and running scenarios and learning.. plus a bit of economics common sense and math

the other thing is
in short and on topic
pools can fill their own blocks with >3.99mb of data per block~210gb a year.. at no cost. because the fee on the tx became the fee reward in their coinbase(no real cost) and other average joes cant compete against that

waffly explainer
when its the pool making its own meme(nft/ordinal)
it doesnt matter what fee they attach. 1sat for 3.9mb or 6btc for 3.9mb

by them putting it into THEIR block attempt template (not broadcast it if not a block solved win)
the fee amount goes back to them. so its a cancel out out the cost

money out as tx fee money in as coin reward=0 money really moved

so the average joe user cant really fight off these with a fee war.

those promoting the crap to want a fee war just want to punish everyone and make bitcoin less appealing to use(dead weight bloat, payment congestions, premium fee and such) so they can promote people to use other network as a payment rail. and make bitcoin a non main system for payments(their end goal)

High-fee transactions in the ballpark that you mentioned are a rarity, so that once all of them are included in a block there is ample room for transactions paying normal fees in a block.

I highly doubt that miners would do empty blocks if the mempool tip balloons to 100 vMB or even 30 vMB, for example.

pools can put in a deadweigtht tx (nft/ordinal) of 3.9Xmb
put in any fee they want even 1sat or 2btc. knowing its in THEIR blocktemplate so they get that fee back. meaning it costs them nothing to bloat a block with dead data(not broadcast unconfirmed tx for other pools to steal that tx fee, but only put into a malicious pools blocktemplate where, should it win the block solve attempt. the tx is locked to that block thus only the malicious pool gets its coin back in its-own coinbase reward.)

this is where the pool only needs 1sig-op and 1 merkle tree hash to create a template full of dead weight data. which is free(no cost) and fast(2 operations) so they can speedily begin their blockhash asic mining of said blocktemplate

where they are then advantaged to be ahead to mine a block due to lack of op's processing delays. to actually mine and thus win the main reward of 6.25btc before competing pools.

pools dont care about the small amount of tx fee bonus as that requires adding in alot of transactions with alot of sig-ops and needing alot of merkle tree operations to fit in many transactions.. which would delay them from starting a asic mining hash process on that template thus risking them winning the 6.25 main reward

to incentivise a pool to take time adding in many transactions of small amounts of data but many sig-ops and merkle tree operations requires said transactions to pay alot to bribe pools to take that risk of delay

if people are forced to "pay more" then transactions become to expensive to be useful for a payment system so people stop using bitcoin as a payment system. thus .. less transactions willing to be made. thus needing transactions to pay more to get a total worthy of being added

snowball effect. bitcoin stops having users making payments on the bitcoin network due to bloats and expense.
thus bitcoin loses its utility

..
bitcoin should not allow this game to be played as it does not help the community.

---

now with all that said
a block of proper payments tx's of average joe users is about 2000tx of about 1.3mb(it was before ordinal junk)
meaning average tx is about 650bytes

how much do you think 2000 tx of 650bytes needs to pay to incentivise a block to ignore its easy bloat 2operation template. to instead have a min 2000sigop+ and 2000 merkle tree ops+ delay risk of losing their 6.25 goal per 10min reward chance

for knitpickers
https://api.blockchain.info/charts/preview/avg-block-size.png?timespan=1year&h=405&w=720
https://api.blockchain.info/charts/preview/n-transactions-per-block.png?timespan=1year&h=405&w=720
time of post is 2292tx for 2.11mb = 918.bytes per tx due to ordinal junk

so how much do you think people should pay to fight the ordinals

If/When miners start creating empty blocks we can create a minimum fee that would not stop people using btc but would stop this kind of attack but I do not see the motivation for a pool or miners to do this. What would they gain from congesting the blockchain full of empty blocks? Miners have paid 1000 of dollars for their mining equipment and you think instead of earning money they are going to sabotage something that earns them money? The margins for profitability is low atm so they need to be generating profit when they can.

Pools do not guarantee to get the fee back they are competing against other pools unless all pools banded together and attacked the Blockchain this is unrealistic because they will never band together because there is no motivation for them to sabotage something that earns them money.

not all pools want to cause congestion to pressure users to pay more
but it does happen

and for everyone that makes a tx. they are always, at every payment they make, thinking "will my tx get into the next block, i hope it does"
so they pay a certain amount to ensure they are more of a priority

meaning each payment they are paying is not 1sat but more..
just to not think.. just to not be on edge with "hope" wondering.. but instead at peace confident..  and more likely to be in very next block, thus not thinking

instead of in a situation that they are more likely going to be left waiting upto a couple days
.....
this confident not concerned base rate. is at a current average of more than $1 a payment to be deemed next block priority BEFORE ordinals was an extra concern

these things like empty/ordinal filed. now adds another layer of pressure to pay a lil extra higher to bribe a pool to not even attempt a ordinal fill, meaning a payment delay for normal tx.

---

lets use an average example of a payment filled block
average tx per block is 2000 for 1.3mb = 650bytes

average person using bitcoin for a payment of goods or service is paying
a.@1 sat per byte = 650sat ($0.15) each  total $300 for block winning pool
b.@2 sat per byte = 650sat ($0.30) each  total $600 for block winning pool
c,@10 sat per byte = 650sat ($1.50) each  total $3k for block winning pool
and so on
d.@100 sat per byte =650sat ($15) each  total $30k for block winning pool

vs time to waste doing sig-ops and merkle ops
1. 1sig-op and 1merkle tree hash(ordinal) = filled block 3.9Xmb bloat
2. 0sig-op and 1merkle tree hash(coinbase) = empty block 0.001mb bloat
3. 2000+ sig-op and 2000+ merkle hash = block with 2000 payments

so that 3rd option
how much time risk vs reward would you want
to risk doing (3) to put at risk for not getting (6.25)$140k block reward prize

would $3k cover it(c) or $30k(d) to ensure the blocks are always prefered filled with payments and a continual bribe to pools to never be empty/ordinal filled

and how much would users put up with to pay for their payments to facilitate this constant bribe to never do a empty or ordinal filled block attempt again

no one can know when a pool is currently making a block template of ordinal fill or empty. so users have to be on the constant bride to deter them from trying

so how much would this base fee bribe need to be to make it a consideration to fill blocks with 2000 real payments

and not have a "dang it they are ordinal filling" reactive thing of dang my tx didnt get in this last block i must RBF my tx with new higher fee to get the next one

how much base fee do people need as a constant min to be just a simple i feel this is enough for us to pay to always be in the next one and not worry about empties or ordinals delaying us

.. because thats the game certain people want users to play
to fight off the risk of having their payments delayed by a block+ people have to pay a min base of X as a constant bribe to ensure pools dont do empty/ordinal so they dont ever have to worry about a possible wait for block+1 delay

so how much do you think pools would want to get as an assurance that blocks always stay at a leanish 1.3mb for average 2000 payments (lean ish 68gb a year)
without constant worry of "will it be this block, did i pay enough to mitigate risk of a ordinal/empty block) constant dilemma thought every payment

ill give you a hint (c) 10sat per byte is not enough bribe for all users to permanently bribe pools away from ever doing a empty/ordinal filled block again

because at 10sat per byte average.. pools still done ordinals..
so to have the confidence of next block. people will end up paying more as a average fee to regain confidence of no worry next block confidence

i couldn't agree more with this. it's cool that a technology could exists to store files on chain but that was never what bitcoin was meant for. and it shouldn't be turned into that while at the same time punishing people that want to use it for its original intent and purpose.  ;D

they should make their own blockchain designed to store data on chain and leave bitcoin alone. i don't want bitcoin to end up like ethereum did with huge transaction fees. eth seems to have solved that issue to some degree with their upgrade but their fees are still high as in you could pay $5 to send eth. thats way too high still.  :o

bitcoin is a payment system and should remain so
its a payment system for the unbanked because a purpose for "banks" is to store value and save up accumilate value

so its also an investment in your future wealth
so being deflationary and a investment is also part of its function

..
as for things like scaling bitcoin to allow more people to do that. we have been discussing scaling solutions for years

its not about bloating up blocks with spam like a certain group think. and they pretend their opposition are the instigators of such false narrative(shifting the blame)
its about increasing the payment tx count per block for the unbanked to be more. (more transaction count of genuine people transfering value) and progressive expansion of the blocksize (not large leaps which certain group think scaling means)
EG the change from the 1mb to 4mb blockspace did not offer lean utility of a 4x tx count. it instead kept a cludgy code limit of payment data to stay below 1mb and offer 3mb for bloaty "witness" crap

removing the cludgy witness crap code that ihinders payment data from fuly utilising the 4mb block would allow more payment tx into a block, instead of only benefittig businesses that want bloaty complex or deadweight crap on a block

the solutions are in many area's, but are not what others call "bigger blocks"(of deadweight spam)

its things like
a fee formulae that charges spammers more(rather then everyone at same time)
lean transactions without excess wasted bytes
and a progressive block increase when demand needs it

fee formulae:
if someone is spamming by spending a utxo thats fresh(only a few confirms(less than an hour) they should be paying more then someone that only buys things once a day
this can be coded very easily to look at a UTXO blockheight vs current blockheight tip and charge accordingly
EG
144/age = sat per byte
so 1confirm is 144
144 confirms is 1sat per byte
and this can be enforced both at tx relay and at block verification to reject if rule is not followed
(some people pretend its impossible or shouldnt be allowed as they want spam, they want spam to be cheap.. they pretend its for "freedom" but whos freedom.. freedom of the spammer. well a spammers freedom ends up limiting genuine peoples freedoms due to the spam )

if someone is wanting to bloat up a tx with too many inputs/outputs. they are not a independent user, they are usually a business.(how often have you needed to pay 200+ people at once.. 144 times a day)
thus those types of large space taking up tx's should be disadvantages and should pay more

they already save money buy batching up outputs instead of doing 200+ '1in-2out'
so they dont need to also save more via other bad math trickery to get "discount"

instead those large tx already getting extra so called "discount" by certain things. need to be brought back inline where being an individual independent user is not paying excess for wanting to make a lean transaction
its not just a byte per sat thing.
its
a. counting bytes and sigops counts (signature operations) and not offering discount to signature operations as that only benefits the businesses that do complex batched payments. and not the individual who only wants to spend their funds from one address(1sigop)
b. counting outputs. where having more then XXX outputs adds a sat per byte multiplier

these things can then penalise those that want to make big transactions multiple blocks a day. while not penalising everyone to equally pay more to fight off the spammy bloaters

thus allow more space and freedom for individuals to put in more transactions of genuine spending/payments. compared to letting businesses take up all the space of a block with only a few of their transactions.

thus allowing more tx per block
other things like enforcing a bytes used per input/output. so that it stops an input having FREEability to load upto 3.99mb of bloat.

EG taproot promise was their 'witness' would look like a <80byte signature. even if they used multisig feature on taproot address

yet that rule has not been enforced.
each opcode can easily be seen as offering differing features and each opcode (tx format/feature) should have enforced rules which consensus checks and validates a tx meets its function

bitcoin needs to harden up its consensus rules again not soften the rules.
certain people think bitcoin has no true consensus, no rules and shouldnt have rules. but, code is rule and rule is code. consensus is where rules are set and then verified to meet the rules where by it keeps everyone happy and united in a common purpose and utility
consensus is not about letting everything through unchallenged/unverified and throw users off that disagree with a trojan being added. its about that new formats are not added until there is mass majority of nodes have the node ready with a certain ruleset so they can all verify the same rules, before a new format is activated

this then stops unreviewed bugs being added becasue it requires people to review new features and upgrade their fullnode to accept new formats. thus prevents bad crap just being added unchallenged

these things can be done but a certain group doesnt want them done

these things can be done and it will make it so there is less spammy deadweight tx so letting more normal user genuine payment tx per block thus adding utility without penalising each user. whereby more tx count without more bloat is thus efficient use of blockspace for payments

i would rather have the current 500gb and expanding by upto 210gb per year be of efficient payments of lots of people. rather than spam deadweight of a few people

500gb of 14 years of millions of peoples payments is a great achievement
we should not be thinking of letting the next 210gb be of just a dozen people throwing in a combined total of just 52k memes
but instead a year of an average tx of 650byte allowing for 323,076,923 payments per year for that same space, but instead of 52k memes

so why can't someone fork bitcoin and remove the ability to do this ordinals thing? i think that might be a successful fork. at least it would have a purpose. some people would move over to that chain, some maybe wouldn't. maybe a bit OT but still. interesting question.

majority of ful nodes are of the core brand
it requires convincing core devs to implement

hard to do this when certain core devs have moderation privileges and delete any posts speaking critically about ordinals or even offering a solution to it.
(they only want positive pandering)

just read all those that call ordinals a feature in the different technical discussion parts and the lack of discussion of fixes

even in places like IRC, the mailing list and github they are ramping up moderation to more higher levels to remove any core roadmap critiquers

the only option the core devs and their fans(of the dev-gods) want to offer is people to create a altcoin fork and see who follows the altcoin(facepalm) which is not a 'option to fix something on bitcoin' its a go away and let core do as they please unscrutinised, non-independently reviewed. un-critiqued

What's bad with BitTorrent?

Anybody can. There is just no point in doing it. First and foremost, for Bitcoin to be censorship resistant nobody can practice censorship; you do the moment you fork with that new rule. Second, you don't solve the problem with junk data, because there are nearly infinite manners to store junk on-chain, unless you give up forward compatibility (even that won't make it impossible though).

blackhat your using too many of doomads scripts
you are also sounding more of an idiot the more often you talk like him

try and break away from his scripts and think for yourself more often

consensus is about rules that nodes verify. where transactions that fit the rules have no obstruction..
EG rules like a transaction must spend value where there is a signature signing a message of tx data of the funding key
the value being spent must be the value that is associated to the funding key(utxo). these rules are important because it stops people just randomly spending other peoples funds and stops random coinis from just being made

consensus is the consent of mass survey(consent-census). meaning they meet the rules of the majority which the majority are consenting to uphold
thus its not a permissionless system in the way doomad thinks
its a permissionless system in regards to the ownership of the funds of the key you hold the key to that does not require someone elses signature be default (unlike your favoured subnetwork bridge)
in short only you can move the value you own.
and as long as you meet the rules you can spend your funds without needing anyone elses signature

have you maybe in your wild teenage years of exploring relationships. learned terms like consent. and how its different from permission

Damn isn’t that is one of the concerning issue over the period of time? I mean though I can read through some posts which states 2 tb is enough for next decade to come but it is still huge disk space and not everyone is able to buy SSD and maintain it that way over the time. Devices get corrupted, they need to be upgraded all the time. Also, no one is just going to use it for bitcoin core, there would be other stuff too! So how developers are overcoming this issue in the future? Do we have technical plans for this or it’s just gonna be same way?
Also, how does wallets like mycelium etc work on the smartphone with little space? Is the data always on the servers or something?

Watch your manners. This is an adults' discussion.

Every 2 years or so, we frequently notice serious drop in the prices of a gigabyte. According to backblaze (https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/), the cost of 1 GB dropped from around $0.033 in early 2017 to $0.015 in late 2022. That's about 50% within 6 years. I don't believe the cost of storing the blockchain will be less or equal with the current state, but I do think it will be cheap enough for a decade. In the end, for regular users, pruning if acquiring the disk is not possible, can provide the same benefit (but it will cost in time, due to future resyncs).

having a soft consensus where nodes are no longer needed to be responsible to verify everything .. BAD
(secure blockchain policy, made insecure)

having pruning where nodes are no longer having to archive full blockchain.. BAD
(decentralised and distributed blockchain les decentralised/distributed)

seem idiots have no clue what the bitcoin principles are.. or they do know but too busy paraphrasing scrips of their colleagues to think about what they are endorsing

idiots wanting to say that nodes that do not have strong hard rules to verify and dont archive are still deemed in their eyes "full nodes".
are just childish idiots pretending they are having adult conversation but still get upset when anyone dares say anything bad against them

if you like having a node that prunes and also doesnt have strong consensus rules to verify everything. then you are running a FOOL node not a FULL node

In adult conversations, people don't just repeat their empty wishlist over and over for years on end, like a gormless toddler at Christmas.  You keep asking for things that are not possible.  Tell us how you achieve the goal of having no softforks at any point in future.  You've had since 2017 to come up with something.  Surely you have an idea by now?  Is your best answer still "be a nazi piece of shit and dictate what devs are and aren't permitted to code"?  I'm guessing your primitive brain still hasn't progressed beyond authoritarianism.  You are the child.  A whiny, petulant one, at that.

https://i.ibb.co/417kTJ9/franky1-ordinals.png

if we leave out the "that prunes" part then it reads like this:


if you like having a node and also doesnt have strong consensus rules to verify everything. then you are running a FOOL node not a FULL node


i assume that statement is still true. thing is, it's not like an individual running a node has any control over the consensus rules. how does someone running an individual node make their node have "strong consensus rules"? i don't think they can. they have to just obey what the developers put into those rules. so i don't see why we can really find fault with someone like that. as far as pruned nodes, i thought they still can verify transactions and have the full list of consensus rules too. maybe i'm wrong.

but in this day and age, if a serious bitcoin user can't afford a big enough hard drive to store the blockchain then maybe it's time to sell some btc and get one. right? we're talking 2TB should be good for another few years!  :o 6 or maybe even 7 to be exact.

If Bitcoin was closed-source and the wallet software had automatic updates which can't be disabled, then you would have to "obey developers".  But that's clearly not the system we have here.

Devs can propose new rules and then users/miners make a conscious decision as to whether or not they choose to download and install an update containing any new rules.  New rules don't take effect until users and miners are running the code.  As an individual, it may be problematic to enforce your will.  But as a collective, we have the final say.  Not Developers.

im not dictating anything because as you admit. i have not released code to control the network.
so that proves you are wrong about me being the network dictator
but who's code is the solo code reference client everyone else follows.. oh yea core.
so what does that make core

how can i be (your words) "irrelevant" and "dictator" at the same time
im not a leader, i dont want ass kissers i dont want to recruit people nor have folowers i have not obtained control..

do you even understand the word dictator
or did no one buy you a dictionary at christmas like some may have suggested you needed

its funny how your narrative changes to shift blame or deflect who is doing what

those that say the truth only have one narrative that sticks.. those that cant decide what to say each week. are the ones that either dont know or are lying

devs can code what they like but what the network decides should be decided by majority, as was the case 2009-2016
you love the idea that the majority does not decide anymore 2017+ and instead allow core devs to be the ones that slide things in without a consensus (consent by majority)

your ok with core throwing things in without needing the majority ready to verify whatever core slide in

you are ignorant to history and facts and ability when it suits you. but then twist it to fit your defense league team motto when it suits you

consensus was strong until it went soft

again funny part is your narratives change depending on which one fits some core defense narrative
EG you forgetting that things went soft,, yet then pretend core are the trusted group because you then pretend the majority consented to core.. (you pretending consensus is hard majority)
but then say the majority didnt need to consent because the network is now permissionless and soft(backward compatible) to not need consent to allow core updates and you are happy core can throw things in via soft (abstinence)

you pretend the network never was hard and cant go hard again
even though it can
but you dont want that to happen
you are willing to REKT anyone that tries

---

the reason you get emotional about my opinions and suggestions is because they contain truths and idea's that can harm cores central power if the majority listened to it. so you want me to shut off, f^^K off get banned.
you fear the majority going against core control. and thats what you are afraid of

---

so answer one question
did core get to do taproot due to majority consent of the masses to cause a consensus activation of majority of nodes all ready, as a majority collective to fully verify taproot
or
did core get to do taproot due to soft abstinence let a (trojan)new format in.. where there was no majority of nodes ready to fully verify taproot when it was let in, thus causing issues the network cannot undo due to lack of ability to have ruled it out or object to it in the first place

And thus the tedious and banal merry-go-round continues to spin.

franky1 asserts softforks shouldn't be allowed.  I ask him how he would achieve that.  He avoids the question and claims censorship.

franky1 asserts core are in full control.  I point out that devs can't make changes if users don't run the code.  He claims users are "mindless sheep" and run the code anyway (This seems statistically unlikely, given that there are over 13000 publicly reachable Core nodes in operation.  They can't all be sheep).

franky1 asserts there should be more clients to choose from and users can then have options about what code to run.  I ask where franky1's code is and he acts like he somehow has the moral high-ground for having no code.  If you want an alternative, the onus is on you to present a viable alternative.

franky1 asserts he is an early adopter who is quite wealthy and financially independent.  I point out he could hire a developer or two to code something he would approve of.  He seems to think he should be able to order devs about for free.  He wants all of the influence, but none of the accompanying responsibility.

Rinse, repeat.  Time and again.


He has all the hallmarks of a successful cult leader, if it weren't for that fact that hardly anyone listens to him.  ::)

---

That's not a "how".  If I said "How do you bake a cake" and you replied "In the oven and then slap some icing on top" it's pretty clear to any onlookers that you've skipped a few important steps.

@doomad
i have answered the questions (hardening consensus) (adding rules that strengthen the byte count per opcode to prevent databloat abuse)(fee formuleas that penalise spammers more than everyone)
and yes they can all be implemented.
YOU play the amnesia card, to distract from the actual issues to spin it back on "franky needs to do something"

grow up you are no PR guy, nor even know what you are defending
try to read more code and less social chatter

if you forget. read your own post history to remember the topics you have spoken of rather then asking me about what you have asked for before. and then you will find the answers in replies to it

And another question that came to my mind while reading the OP was , why do i need to download the entire bitcoin blockchain data when I am not a developer? I mean what is the essence actually, I personally would prefer to keep it simple just as I have always kept it from the beginning, that is, buy bitcoin from exchanges, move them to my non custodial wallet for holding, and if for any reason I decide to sell, I move them back to an exchange and sell, its that simple .

I probably will need to become a developer to understand the benefits of downloading the whole bitcoin core network to my computer.

YEARS ago. people wanted to protect the network by lots of people having the blockchain distributed across lots of computers so that no single location had a master copy. thus reducing the risk of someone just changing a tx value and then being the master. everyone just complying that its the master record

next was the ruleset of what was acceptable to be allowed in a block. and thus what blocks were allowed to be added ontop previous blocks. to ensure the data and value within followed a set of rules

EG 2 people have a pile of lego
if each person has rules of 2x2 yellow followed by 2x2 red followed by 2x2 blue
they would all end up with the sale tower of sale colour combinations

where again due to network distribution YEARS ago majority had to upgrade their node to be ready to verify a new rule (majority ready meant rule was safe and accepted to be activated and used) and such then a new rule would activate and he network would verify the data follows the rules including the new rule(consensus)
.....
but since the consensus had been softened alot since 2017.
it no longer requires majority node upgrade pre new rule activation. and with features like pruning old data... alot more nodes no longer hold the entire blockchain.. but are social conned into being told they are stil full nodes defending the decentralised network(facepalm)meaning there is a smaller amount of nodes that do fully archive who are in less numbers so more pressured and treated as master nodes(real full nodes) and others are fool(leacher) nodes

its not just that less nodes fully archive. but now those that do have to seed it to more leecher nodes. meaning more leecher nodes are relying on a smaller source of seed nodes for blockchain data. leachers nodes just follow whatever he full nodes give them. and all the other risks that implies

and all that combined changes over the years has weakened the network to let in silly transaction formats that allow <4mb of bloat, untested, unverified and treated as (default:isvalid)
meaning allowing alot of deadweight data of no financial payment function to fill up a block.. lowering the transaction count per block. wasting the space

which will just make more people not want to be full nodes if all they are storing is a x% of unwanted memes

..
if you dont want to be a full node user you are not forced to. you can just use a lite wallet. that just makes transactions and moves your value
but if you want to be a full node.. be a full node and not a fool node

I don't feel like running my node if this situation doesn't improve. Using my limited resources to benefit a bunch of spammers? No thanks.

If I want to start my own full node, is it better to simply let the client update on its own? Or is there somewhere I could download a trustworthy bootstrap? Would the bootstrap save any time in downloading?

8)

Do not download the blockchain from a 3rd party that goes against the principle of having a full node. If you want to run a node you want to make sure you download the blockchain from the source and it verifies correctly because if you download it from a 3rd part they could have altered the blockchain without you knowing. It should not verify and should show a error but I would not risk downloading it. I doubt it would save a lot of time downloading the blockchain is quicker then it was many years ago it mostly gets slow when you have poor hardware or internet connection.

The spammers a small part of the blockchain that have tried to make their mark in history but by creating a node and running it 24/7 you are benefiting the community and making it more secure if everyone had this opinion and they shut down their nodes btc would be insecure and die over night. We have a lot of people running nodes and if you are limited on resources or you are not running it daily then you do not have to because there are many people to verify the network.

if your doing initial block download via TOR
expect it to be SUPER slow download (tor bottleneck)

Yes.

Who's crazy enough to share 397 GB compressed data? Wait, that would be me: tmp.loyce.club/bootstrap.tar.gz (http://tmp.loyce.club/bootstrap.tar.gz) (this file is scheduled to be deleted in 30 days). This includes blocks and chainstate, both updated until today. I had to figure out parallel compression (https://unix.stackexchange.com/questions/548434/multithreaded-xz-with-gzip-pv-and-pipes-is-this-the-most-efficient-i-can-ge) to speed it up.

Maybe, maybe not. It would be cool if you try both, and answer this question once and for all :)

You can (and should) have your own Bitcoin Core verify all blocks again. That undoes any potential time savings from downloading the bootstrap :P

Lol. You posted this while I was making it: compression, verification and checksum took some time.

Hmm wait, if you managed to compress it further it then its not a bad idea at all. For a full node i see no issue, since you just can't have something different from the rest, or else you yourself will be isolated from the network.

I was wondering what compression algorithm would work best with the blockchain, surely you tried xz and 7z among others?.
https://linuxreviews.org/Comparison_of_Compression_Algorithms

A torrent would be nice (don't want to overload your server).

when core has a milestone in the code that before X blockheight, blocks are treated as (assumevalid)

many many users 'could' compress up a chainfile or a utxoset state of that milestone height. get the file hash for integrity check and sign that they have verified the integrity (much like they sign for a hash of compiled core exe)

where people can then check the hash of a compressed file matches to know the content matches. which then people can download milestoned (assumevalid) blocks in a lump
and then build ontop of from said blockheight the natural peer way
(as long as there are alot of independant* reviewers signing the same hash for integrity)(*not the same social/cult-ural group)

Try it :) Just take a block-file, and compress it. I expected bzip2 to be the best, but it wasn't so I went back to gz (which is faster). I used pigz -9 -p8 (best compression on all cores). I shortly tried "-11", but that was around 20 times slower.

I tried torrents for another project, but it was annoying to keep working. Feel free to download it, the server is mostly idling and uses only 20% of it's allowed bandwidth. This server was donated for cool blockchain projects, so this is exactly what it was meant to do.

I want to make a parallelism, or maybe i can say a historical parallelism. Or analogy whatever you want to call.

When Bitcoin was born  we cant think in have the net in our hand, maybe we can have that in a laptop, but now you can have the full net in your hand in a smartphone.

500 gb its nothing for today tecnology. So imagine in ten years more where we can be. Maybe in our wrist in a smatwatch. Your wallet and the whole bitcoin core in your wrist.

Technology has changed quite a lot since 2009.

Yes, 500GB is nothing for today. And following Moore's Law, in ten years we should have devices holding up to 16TB (doubling every two years) that are as common as 500GB devices are today.

I mean, you can have a laptop today with more RAM (64GB) than a hard disk in 2009. Advances in technology have been amazing.

@LoyceV If you think that everyone should download and verify the blockchain on their own and not rely on 3rd party downloads may I ask why you are compressing it? Are you planning on releasing this and why?

I do not think you have bad intentions I just fear that newbies or people that do not know the risks might download this thinking it is a good thing to do and is normal. I think you should probably include a warning saying that this is not normal and you should download & verify the blockchain yourself. I guess for someone who wants to look around the software with a full blockchain but not use it for integrity could find a use for downloading the entire blockchain from a quicker source then direct but that is all I can think of.


yes but it will slow down at some point. The whole point of moores law is that there is a quick growth at the early stages of technology and huge advances are made but the longer that it is developed and exists the slower because you are not able to condense the components as much and not able to make them faster. I do not think we are there yet but I am not qualified to speculate when this will be because I do not know all the details but I would expect it to slow down at some point and I will give a example of m2 SSD look how small these components are it is hard to believe that they can make them faster then they are because on most operating systems it will boot up in seconds and the size is tiny and can fit in a closed fist. Making them smaller will probably not yield any benefit to because they are already very small.

i remember the moores law of the 1990's

i had a desktop. it had 3.5gb hard drive and 256mb ram and cost $1000
techy guys were saying there was a hard limit of 4gb on hard drives
they also said processors can only do 4ghz
and floppy disks can only be 1.4mb
and cd can only be 700mb
phonelines can only go upto 56k
"coz moores law has run out"

and yet
4tb beat old harddrives 1000x
microSD beats floppy by 350,000x
blueray beats CD by 71x
i9 cpu with 64bit 24 cores and 32 threads
1gbit fibre internet

i have a 2 tb nvme drive just for games. so i honestly cant see storage capacity for something as important as verifying that your stash is actually yours as being an issue.

anyway my working copy of the blockchain is on a dedicated 1 tb nvme. it gets backed up several different ways (both on and off site) as having a local copy of the full chain is that important to me.

my first computer was about $620 and only had 8mb of ram. and 200mb hdd. i got it from this mail order place that was advertising in a computer magazine. i must have call that guy a million times before i finally bought it. kind of disappointed with their shipping though. two big boxes full of these pink styrofoam peanuts and it looked like they just threw my 14 inch ibm monitor in there like a basketball shot. cord unravelled and all.  :o i immediately installed windows 3.1.

funny thing is i don't think machines today really are much more responsive than back then. sure they can run more powerful apps but for typing an email or something. you don't need that. had to save up that $620 too by the way.

my point was.
when people shout moores law and say we are at the end

i recall when i was last at the edges of the end of moores law in computing over 25 years ago
.. until it wasnt the end and life moved on

Because I can :) I mean, why not? I believe in options and freedom to choose.

I just did ;)

Because BADecker asked for it.

If someone downloads 397 GB without knowing what he's doing, he can't be helped (https://bitcointalk.org/index.php?topic=5112400.msg49842861#msg49842861).

There is a warning:

It may not even be faster. Bitcoin Core downloads from multiple sources, and some of them are probably much closer than my server.

I recall that same period too. The excitement of zip drives (I invested pretty heavily in that dealing with clients and showing them graphics and stuff on the move), the deals I thought were crazy in those days to buy new drives. The memory disks for Playstation that we thought were amazing... this was back when we were doing our best to keep file sizes low, compressing images on websites. Even had all kinds of golden rules for uploading things so people don't die on dialup waiting to load our sites.

Now we're uploading things to the best resolution possible, and my kids are eating up hundreds of gbs of data on their phones daily.

So yeah, I don't think we're at the edge, or that edge is getting farther out than is possible for most.

Big deal 500gb =D

I didn't know there's a procedure for it, I just compressed blocks and chainstate.

Even if it saves a few GB, that's only a few percent anyway.

This took over 2 hours on all CPU cores already.

Does faster drive (something like PCI-e 4.0 NVMe or regular even regular SATA SSD) mean anything in this case or is it only CPU bound? I assume that in this case the CPU and core clock (IPC performance too) is the most important aspect?

No. The CPU is already the limiting factor, stronger compression means there's even less to write (per second).

---

---

Back to the growth of the blockchain: the daily amount of data increased a lot in the past week:

I think that when the technology of data storage has evolved by changing scale, i.e. for example when the price of 10TB is at the current price of 1TB, it may be possible to increase the size of the blocks as this will not harm decentralisation, however we may be close to the theoretical limit in data storage

(facepalm)
1990's hit the theoretical limit of portable data storage of 1.4mb(floppies)
oh wait CD-rw's
1990's hit the theoretical limit of portable data storage of 700mb(CD)
oh wait DVD-rw's
DVD hit the limit.. oh wait blue ray
an along side all of that flash drives
portable hard drives

so theoretical limit.. is what, and when exactly?
and with a current 210gb bitcoin growth limit.
lets call it 4 years 6 months per terrabyte
how long is this deadline before
if you wish to call 10tb a hard technical limit of storage
thats 46years for geeks to invent new storage options

We have not shown any signs of being close to the theoretical limit but we are slowing down. Franky is right that we have evolved very quickly from things that we thought would be with us for a long time. CDs were very popular and people thought that it would be a method of storage for many years but they are now redundant so redundant that people do not have dvd players and do not have a cd tray in their computers.

I think the motivation for consumer storage might be the theoretical limit instead of the actual limit. hard drives wil continue to expand by a lot but they will be aimed at companies that need a lot of storage and cloud storage provides. The average consumer does not use more then 4TB atm and with other technologies slowing down and hitting their limit like games where the hardware is stronger then the things it relies on I think that could be what happens in the next 5-10 years. Consumer level hard drives might continue to be cheap but probably will not be made in smaller sizes corporate ones will probably continue to expand at a fast rate but will be more expensive to purchase.

Yeah, I would agree that whatever is being sold mainstream gives you a good indication of what the average user is going to be expecting to be the maximum storage.

For example, the new Mac mini with the M2 Pro comes with configuration options ranging from 512GB to 8TB. So that gives you an idea of what's mainstream these days. 512GB being the bare minimum to run a system, and if you need some extra space, you can get up to 8TB, without doing any installation yourself.

So, clearly 500GB of Bitcoin data is not too much these days.

for me is just a sense of pride, I don't know next. maybe when having the time I want to learn how to make a Lighting network node, and earning satoshi from that, or create a wallet and be a developer like you said.

and there is no harm if we have unused internet at home and pc because is not disturb also when running my pc with another program.

actually, I step to download a bit by bit, certainly not so felt long if in the future I make the blockchain program or etc.

well, set aside the pride and all. if you are not a developer, there's no reason for you to download such heavy data. am using electrum right now for my btc holdings. light and txs are fast as well. i was using the btc core before but yeah, it came to a point i ask myself why i was using it. if you have no valid or worthwhile reasons, just go for wallets that are not heavy especially for beginners and just regular users.

When talking about theoretical limits for data storage, I'm thinking about storage on a molecular level. You can't store much more than 1 bit per molecule. The closest thing I could find was 215 PB in 1 gram of DNA (https://www.science.org/content/article/dna-could-store-all-worlds-data-one-room), which is still several orders of magnitude away from current storage systems.

Is that because of technical limits, or because newer storage types focus more on access time than size?

its also not relying on one source supply of balance accounting
imagine if a litewallet didnt tell you a payment arrived at your address. reliance on a litewallet means you only know what they choose to tell you.
they can send you a tx, say its confirmed. but the utxo does not taint/track back to any coin mined nor any real coin existing on a blockchain. should they go down a deceitful route

running a full node means you get to see it all and know all coins have a track/taint back to their origins of creation(coinbase reward) that validates that all coins are real coins and not ghost coins made up

yes not everyone needs to know everything. but if your receiving regular payment from alot of random people.. just seeing a tx designated for you is not enough, you will want to ensure it is confirmed and has valid funding before releasing your part of the deal, 
because the more payments you receive the more risk of abuse if you did operate without a non independent checking method

Maybe AmoreJaz means if don't run a full node, and just use the core wallet just store the fund.
Sure, I was thinking before making the decision to download bitcoin core. because the core wallet is complicated, can not be used for transactions before the blockchain is completed, But cmon, core wallet is from satoshi nakamoto legacy, same as this forum we live. there is a sense of pride if we talk it on seminar or other forum if we used what satoshi create, bitcoin, core and bitcointalk forum.

For the record: you can send a transaction before Bitcoin Core is fully synced. The moment an input shows up in Bitcoin Core, you can send it. The tricky part is of course that it might have been spent already, and in that case your transaction will indeed be rejected by other nodes.

if your copying over a whole wallet file that also contains UTXO details of your spends. then yes. it can pop up instantly when initiating core, and will appear as pending confirmation until synced

but if just importing a private key you will have to wait for the section of a sync where by it recognises the association between the key pair to then display it mid sync while its building the blockchain

those that only got involved in say 2020 and just imported a private key/seed key, will still have to wait until 350gb+ is download to get to blockheight sync of 2020+ for their input to show up..

luckily my 2012 stash pops up on GUI display in the first ~100gb*
luckily my 2014 stash pops up on GUI display in the first ~150gb*
luckily my 201x stash's pop up on GUI display in the first ~300gb*
unluckily my 2022 play amount pops up after 400gb

* (im hoarding thus dont want to spend these)

Are you using a hot wallet for long-term holding?

my old stash is on legacy
.. am i doing a LukeJR.. nope

i have a few other features on my node i added myself so i just do "watch only" for the old stash thus not importing private, but importing public address just as a balance display

but if it was time to spend them. then yes i treat it as a hot wallet meaning get a (offline) new set of keypair. to use only public address as change address. and then spend old private key to destination address+ to the new public address(change) i import. and then wipe drive. and redo it all again.

its a hassle to spend old stash but that hassle is avoided by making it another reason to not wanna spend old stash.. making it complicated avoids easy temptation to spend

i have not put my private key of old stash in my node..ever

my 1frank funds are not my main stash. the 1frank funds was/are just a side 'dont give a crap' play money small fund. and i havnt bothered with that private key for 5 ish years which was the last time i had to do the hassle method of wiping hard drive and starting again due to treating it as a hot wallet in relation to those funds.
newer play money amounts i have(different keys) as display of balance using public keys on the node. but spend on a different device thats easier to wipe and start again.(lite wallet)

wow franky that's alot of stashes! you were smart to get involve in bitcoin so early  :o

luck.. not smarts
its a learning process. dont stop learning

i dont need to keep buying im self sustainable and live a good life thanks to bitcoin
i choose to buy more when there are undeniably cheap dips worthy of grabbing more

im no where near the desire to just sell it all. so i dont
its not "smart". its just not needing to.
yea i use small play money amounts as a general living/short term income accumulator but its like 0.0X% 0.0xxx of stashes so risk is low

but my main goal.. i want to ensure bitcoin functions as a financial payment network to protect my value..  and is not turned into a meme-stock storage network

from a technical side of data storage growth of 210gb a year is not disastrous growth. but how that data is used, is a disaster if majority of it is just memes. memes is very inefficient use of the available data growth and it limits/decreases of the financial utility of bitcoin

these major nft players are just spamming bitcoin with their 10,000 and 5,000 garbage nft collections non-stop i would imagine at some point. some of them seem to already be doing it. i say let the garbage stay on ethereum and ipfs/aws we don't need it on bitcoin. :o

it's all the devs fault for letting this loophole exist in the firstplace and then it's their fault again for not doing something about it. would a fix require a hard fork? if it's just a soft fork then they really have no excuses. but it seems like they don't even care and aren't even looking into how to put a plug on this hole.

devs snuck in a rule that a certain opcode if used can use upto the weight limit of 4mb
sneaking in(soft) that the opcode after block 7xx,xxx only uses 1 signature length (~80bytes) is still acceptable to old nodes that allow upto4mb. because 80bytes is still within that upto 4mb limit. thus old nodes would not reject such new snuck in adjustment to the limit

however it requires miners to stop adding in upto 4mb and to instead use the new rule of defined 80byte use
so its [lame buzzword] a MASF (miner assisted soft fork) but without causing a FORK(altcoin)

i prefer to call it miner assisted consensus strengthening

yep it wont cause a block re-org of old blocks. nor cause a altcoin. if all miners agreed as long as they join consensus to unite under a consenus agreement to all follow the rule..

only if its contentious where only some pools do and some dont. then there could be a fork where some pools continue to build on bulky tx in bulky blocks. and some build on lean tx in lean block. causing 2 different chains of blocks
thus can (as precedence exists) require that economic nodes (merchants and CEX) can agree they they will reject the bulky blocks at the 7xxx,xxx to scare pools into complying with the lean block design of new rules for the opcode to only be 80 bytes instead of 4mb

which is then a economic node [lame buzzword]assisted hard fork resulting in a miner assisted softfork to ensure there is no contentious split. by forcing compliance

in short using the same methods of 2017 mandatory activation.

my bitcoin folder seems about 5-10 GB left, or 10 GB left on block folders to 500 GB hits, maybe not a week or a couple of days we can reach that (500 GB) number, so if you here have an exact calculate it, when it happens?

what kind of benefit?

As far as I know, block data varies a bit (because of orphaned transactions). That's why your blk03436.dat is different than mine, you can't exchange them.

I checked my 2 Bitcoin Core installations (on different systems): The size of blocks is 510671104 and 509643300 kB (the difference is 1027804 kB). The oldest installation is the largest: it makes sense the more recent installation skipped all old orphaned blocks.

if it's useless, can we delete it manually on the block file?.
is it dangerous if I edit manually (delete) blocks (that almost 7K file)?

what is different blk and rev? (.dat) inside block folder

Or you could try a file system with compression, such as btrfs using zstd or something. Hopefully that spam bloat is compressible? Should help mitigate a bit the urge for replacing the hard drive.

the silliest advice i seen by core devs is that instead of trying to stop this dead weight meme bloat stuff.. they suggest if you dont want to receive it. downgrade to an old node of pre v0.12 which strips out the "witness" thus what you personally store is then not including the memes

however doing so makes you no longer a full node  because
a. you are not archiving full blockdata
b. your not validating latest ruleset
c. unable to offer initial blockdownload to other peers even if you are archiving all 7x0,000 blocks of stripped out witness/meme data

yeah that is really dumb advice franky. they're not really showing much concern about the situation if that's the best they have to offer.


even someone that doesn't even run a full node and just uses "apps" could understand all those issues.  :o

It's not just one file, it's part of other files. So you can't delete it. And it wouldn't make much difference: you'll save 0.2% disk space, which will be filled again 3 days later.

Bitcoin Core assumes the files on your disk are correct. If you delete some of it, you can expect unpredictable behaviour (such as crashes). It's not worth it.

dont get me wrong people that dont want to be full nodes can run their apps, software and wallets all they like
but when people who want to be full nodes are told core is a full node.
but then other idiots then tell people they can switch off x,y,z and then fool said victims into thinking they are then still a full node. is shameful advice

again if the idiots promoting these fool tactics actually just explained
"if you dont want to be a full node anylonger and want to switch some features off, you can do it within core, but understand you are no longer a full node by doing so"
or
"not upgrading downrates you to no longer be a full node"

then there would be no problems

..
its the maliciousness of making people think they are protecting the network by lulling people into no longer protecting the network, thus weakening the network protection via deceit

Well lzo is faster which is what i normally use. zstd can compress more but takes a little more writing, reading is very fast (intended for their datacenters, you know facebook etc). You can also change the compression level in the mount options.

ETF is explaining:
if an image was say 6mb and was compressed to get it down to 3mb to then put into a bitcoin block
its already using compression

meaning you then trying to compress something wont result in a 50% drop because its already in compressed format.. thus no gain

inshort
you cant compress a compression and keep compressing it
that 3mb will still be 3mb

EG
the 3mb was 6mb before being in a block. meaning it stops you from then compressing it after its in a block, becasue its already compressed

I already know this. And you should know, the likes of Facebook know this. Those algorithms are smart enough to not try compressing what can't be compressed anymore, and in addition to that btrfs will on its own by default test if something can be compressed, if not, it won't do it on a file by file basis.

I never said 50% did i? It depends on the data.

https://iili.io/HOHaRxn.png (https://freeimage.host/)

I just checked and now, my bitcoin folder has reached 500 GB of data, as you know, I never use it for any transaction yet (just open and download the blockchain), so the pure data is still the same since I created this thread.

the block folder almost hits also, just a bit of GB (less than 4 GB) data to reach 500. I can't wait for it.

this is of particular concern for beginners If want to buy a laptop, take your hand to choose at least 1 terabyte hard drive.

Or just prune the blockchain. There's a reason why that's the default, and for most users that will work just fine.

most beginners do not start as a full node. they have no great knowledge about the network at the beginning. most start by hoarding sats in centralised exchanges and only learn about full nodes later on once they have had time to learn

so its not a case of needing to buy a high spec laptop right from the start. nor does it require a high spec laptop once established. multiple terrabyte devices are the norm now, we are not in 2010 anymore


if you dont want to be a full node thats fine.. but understand it from the prospective of those that DO

people who want to be a full node.. actually want to be a full node
the reason they choose a full node instead of a light node is for reasons.. listen to those reasons

they actually want to archive the blockchain and provide it out to others to decentralise the blockchain.. the main point of being a full node is that. decentralised network, avoid central points of failure. etc etc

if you dont want to be a full node thats fine. but dont pretend that pruning keeps people at the same level as being a full node
if everyone pruned... no one would have full blockchain data from genesis to now anymore. = centralisation. and no more ability for new users to sync upto, if there are no full nodes left.. please understand that.. understand pruned nodes are a different level compared to full nodes


you have been around long enough now to atleast have heard about the useragent/services and how nodes treat each other differently depending on services they have enabled or chose to disable

learn the difference between full nodes that helps protect the network, other nodes, protocol, blockchain, decentralisation.. vs pruning which is just for personal validation and does not strengthen the wider network nor offer full service utility to peers

when those wanting to be full nodes (validate and archive data), want to discuss full node data. and you simply want to tell people to stop being a full node. but pretend they are still a full node at the same time.. you are missing alot of points about why bitcoin was invented, what functions blockchains provide why a distributed blockchain helps and why allowing the sharing of data helps

Calling a pruned node a "light" node is utterly confusing. A pruned node is a full node; the only difference is that it doesn't archive the entire chain. It verifies in the same manner non-pruned full nodes do.

It keeps them at the same level of protection, with minor discomforts (https://bitcointalk.org/index.php?topic=5414284.msg60980561#msg60980561).

This is a completely different point to make than "running a pruned node doesn't grant you the same benefits". Obviously, if everyone stopped archiving the blockchain, there would be no blockchain. But as I'm pointing out in the link above, there will always be individuals who will need to hold the entire chain. Regular consumers are not included.

And people who want to prune it, want a pruned node. You're missing the point: let them choose!

Don't worry, I have a backup ;) And I'm pretty sure you have one too, and so do thousands of other Bitcoin users.

One of the beautiful things of decentarlisation is having the freedom to choose.

Unacceptable.  People can only use Bitcoin in a manner that franky1 approves.   ::)

(but we're still not allowed to call him a fascist)

I was worried about this a while ago. I really don’t want to do prune mode, I’m happy running a full node. My current laptop will do the job for a while yet.

I’m still running trusty 0.17.0

I really don’t like upgrading, verifying my download always concerns me.

why would that concern you? are you worried that the blockchain is invalid or something?

THIS TOPIC is about the full archiving of bitcoin.
where people want to discuss archiving the full blockchain

yes people can and do have a choice but they should be informed about it, informed about the choice, informed of the differences..
.. something loyce and doomad fail to do. they pretend pruning is the same as being a full node, when its not

Maybe he is one of the perfect person. Same as me, sometimes I'm too lazy to verify my download because many steps we must go through gradually like Download the list of cryptographic checksums, Downloading the signatures to validity, and many other steps (11 steps  (https://bitcoincore.org/en/download/)) which we must follow or It could be complicated if never used before.

That's something they can improve: Tails (https://tails.boum.org/install/linux/index.en.html) for instance has a "Select your download..."-button, and verifies the file within the browser. I'm not sure how useful that is when the server gets compromised, but doesn't the same apply to Bitcoin Core's SHA256SUMS?

In my opinion In terms of storage, while a hard disk may work, using an SSD would significantly increase the speed and efficiency of downloading and storing the blockchain data. If your current 10 terabyte hard disk is slow, you may consider upgrading to an SSD or increasing your internet speed for faster syncing.

I don't think the button is to verify the authenticity of the software. If you notice, it says "make sure that it is safe and was not corrupted". It's possible that your Internet connection is unstable and gets interrupted, which if I remember correctly can result in file corruption in modern browsers.

The real question is why not save a step and verify the OpenPGP signature beforehand? It will prompt invalid if file is corrupted.

the other thing is.
if your downloading core from that one site.. but also downloading the signatures AND keys from that one site. then you are already failing. because IF that one site was compromised then all info on that one site is compromised. so obviously the keys would match the file because the corrupter/malicious hacker tailered all info available on the site to match his edited node download file

its better to get the node software from a trusted source(the site) and TEST that trust by looking for the signatures from the devs own servers/websites (decentralised sources) to make sure it all matches

OR save all that time by just grabbing the code from github and using a compiler that way you know what has been put into your node