My Bitcoin-QT blockchain died.

[soy]

I've been running Bitcoin-QT at night for months trying to build a blockchain.  I changed to 24/7 days ago and was up to July of last year.  Cold nights I started with Slush again after 2 years.  Running an S5, with a return of 45% of electric cost which makes it cheaper than propane.  I had an old address on Slush and tried to change it.  I use Firefox. Repeatedly for days with email exchanges with Slush the problem, mash the cancel the address change button (because it wouldn't change) and it would return to a login screen that doesn't work.  A Slush rep told me it might be my browser.  I upgraded Firefox to 64-bit from 32.  I soon got a BSOD on the Win7 machine.  Now after reverting to 32-bit Firefox, Bitcoin-QT won't run.  Rescan fails and am reindexing now.  And the Slush problem is unresolved.

Was this BSOD intentional by Mozilla?  BSOD seems to kill a blockchain with its abrupt termination making reindexing necessary.  Do they feel killing a blockchain is politically correct?  No wonder there's such a backlog of confirmations and long delays.  Nobody in their right mind is going to run Bitcoin-QT and contribute to the chain upkeep.

What's the solution?

soy

[1713532750]

1713532750

[dylben1234]

I don't know what exactly causes this. What exact bitcoin client are you using? Because bitcoin-qt is the name of the executable for BitcoinCore, BitcoinXT and BitCoin Unlimited. I know that if you switch from Core to an Unlimited Client like BitcoinUnlimited or BitcoinXT, using your same block source, you will need to reindex. And if you build and index the full blockchain with one, go to start the other, you'll be SOL and have to do it all over again. I had to reindex when I switched to the Unlimited Client 3 times. It kept getting to 99% and then quitting out. Luckily I have a fast PC and internet, so it was only a total of 24H wasted. I suspect data corruption in your blockchain data, try deleting and re-downloading the chain from scratch. If your set on becoming a full node, you can try running with these CLI options.... -par={TOTAL NUMBER OF CORES*THREADS} -discover=1 -dns=1 -forcednsseed=1 -listen=1 -listenonion=1 -maxconnections=128 -maxreceivebuffer=20000 -maxsendbuffer=4000 -peerbloomfilters=1 -timeout=2500 -datacarrier=1 -datacarriersize=172 -maxoutconnections=64

That will get you a ton of active connections to the network. Manually prune out any connections that don't provide any services (from the debug console.) That will allow you to maximize your download rate. -par set to the maximum number of logical cores will maximize your processing time. On some older OS's, that don't set affinities very well (like win7) I've found setting par to double your available logical cores can also help.

Now that network and CPU is taken care of, I find my biggest limiting factor is actually disk sequential read write speed. If you can, put the blockchain data on an SSD or RAID0 array. That will likely make it a lot faster to sync or index.

Once you've fully synced, you can use the -prune=<nBlocks> to delete the older blocks and save space while still maintaining a full node.

[ranochigo]

Was this BSOD intentional by Mozilla?  BSOD seems to kill a blockchain with its abrupt termination making reindexing necessary.

Probably not, I never had that problem. Have you think about the problem being with your computer? BSOD or improper shutdown can obviously corrupt the blockchain on your computer especially when you are synchronizing. Test your ram and your hardware for errors.

No wonder there's such a backlog of confirmations and long delays.  Nobody in their right mind is going to run Bitcoin-QT and contribute to the chain upkeep.

What's the solution?

The backlog of transaction is not related to the client itself. The problem is with the blocksize and the transaction volume.

[soy]

But more full nodes would be better or not?  Do installations of Bitcoin-QT contribute to the confirmation process or not?

[achow101]

Your browser has nothing to do with the operation of Bitcoin Core.

What are the specs of your machine? How does Bitcoin Core fail? What do you do that causes it to fail?

Please post your debug.log file.

But more full nodes would be better or not?  Do installations of Bitcoin-QT contribute to the confirmation process or not?

More full nodes are better, but they do not contribute to the creation of blocks (aka confirmations). More full nodes will not help confirmations be faster.

[cr1776]

But more full nodes would be better or not?  Do installations of Bitcoin-QT contribute to the confirmation process or not?

Just to be clear, is this Bitcoin Core? Assuming this is Bitcoin Core, what version are you running?

Bitcoin Core nodes, do help and are better, in general.

[soy]

How does Bitcoin Core fail? What do you do that causes it to fail?

I upgraded to 64-bit firefox and the BSOD that resulted caused an immediate shutdown of Bitcoin-QT.

But more full nodes would be better or not?  Do installations of Bitcoin-QT contribute to the confirmation process or not?

More full nodes are better, but they do not contribute to the creation of blocks (aka confirmations). More full nodes will not help confirmations be faster.

Unless the full node is solo mining I suppose but I'm not sure if Bitcoin-QT even does that anymore and the rarity of solo mining finding a block makes its effect negligible? 

So, Bitcoin-QT checks each block of the blockchain but doesn't discover new blocks unless mining and therefore doesn't contribute to block confirmation rate.

Reward only goes to the discover of the block or the pool.  I discovered a block for Slush soon after getting a KNC miner early on, the block worth $5k.

Would it be possible to program devices to only run confirmation on new blocks for a cut of the reward? 

[soy]

But more full nodes would be better or not?  Do installations of Bitcoin-QT contribute to the confirmation process or not?

Just to be clear, is this Bitcoin Core? Assuming this is Bitcoin Core, what version are you running?

Bitcoin Core nodes, do help and are better, in general.

Aren't all installations of Bitcoin-QT full nodes? My Bitcoin-QT is using QT version 5.6.1.

[Jet Cash]

I've been running Core on a notebook over public WiFi ( win 10 ) for well over a year now, and it's been remarkably stable. I upgrade as soon as there is a stable version, and I'm now on 0.14.0. I copied the blockchain onto an SSD, and I started another core node on an Ubuntu machine using that ( I haven't upgraded that to 0.14.0 yet, but I plan to) I've had a few hiccups over the months, and core always seems to cope. I make sure I shut down correctly, and if I have had a problem, then I shut down and restart once it has scanned my blockchain. It even copes with me forgetting to connect the SSD, although it tries to build a new blockchain, but if I restart with the SSD connected, then everything is fine. I'm impressed with the stability of the design and coding in core, and that's why I will stick with that.

If you are having problems with your blockchain, then I would check the hard drive to make sure that doesn't have problems, and also check to ensure you haven't let any malware onto your computer.

[soy]

I rebooted and chkdsk did find some irregularities.

I will try some of the suggestions.

Is Bitcoin-QT Bitcoin Core v0.13.0 (64-bit) a full node?

[soy]

Please correct me if I'm wrong in my understanding here.

Miners have a standard, the last block, that they must best by calculating a more efficient formula for content.  When a more efficient formula is found, discovered, miners dump what they have been working on and work on the next.  Miners in a pool get to share the reward for a discovered block and get some of the transaction fees.

Why not have programs that simply confirm that the block is good, better than the last, and get a cut?  Or would that reward, if calculated to be a little less than that for a new block search, cut into the pools' take making it unpopular regardless if it would speed up the process?

I understand miners don't run Bitcoin-QT.  I understand that Bitcoin-QT can run a mining operation.

[achow101]

Is Bitcoin-QT Bitcoin Core v0.13.0 (64-bit) a full node?

Yes. Any version of Bitcoin Core is a full node.

Miners have a standard, the last block, that they must best by calculating a more efficient formula for content.  When a more efficient formula is found, discovered, miners dump what they have been working on and work on the next.  Miners in a pool get to share the reward for a discovered block and get some of the transaction fees.

They aren't calculating a formula, or doing anything efficient (mining is a very inefficient operation). Basically miners are trying to find the right inputs to a specific formula (called sha256d) which result in an output that is less than the current target. The target is recalculated every 2016 blocks and done so in a way that everyone who is following the same chain calculates the same target.

Why not have programs that simply confirm that the block is good, better than the last, and get a cut?  Or would that reward, if calculated to be a little less than that for a new block search, cut into the pools' take making it unpopular regardless if it would speed up the process?

There are multiple problems with doing something like that. One is how would people who run such programs (aka full nodes) get paid? Miners are paid because they include a transaction in the block that they mined that pays the block reward to themselves. How would you pay full node operators? What stops people from gaming the system by running a bunch of fake nodes where only one actually validates the block and the rest just say they did?

[soy]

Yes. Any version of Bitcoin Core is a full node.


They aren't calculating a formula, or doing anything efficient (mining is a very inefficient operation). Basically miners are trying to find the right inputs to a specific formula (called sha256d) which result in an output that is less than the current target. The target is recalculated every 2016 blocks and done so in a way that everyone who is following the same chain calculates the same target.

Thanks.

So, rebuilding a blockchain with Bitcoin-QT does some crunching of every block download or reindexed.  How does that crunch differ from a dedicated miner's crunch?  An Antminer or a Mercury doesn't maintain a blockchain of course.

I can see that a large pool is in a race with other pools to discover blocks.  If the target is changed every 2016 blocks then dumping previous calculations would seem to be inefficient except at the 2016 point.  Calculation inputs would seem to need be saved so as to not repeat work.  One would think this would be approached in an other than random fashion.  Or is the hunt required to have random input tests?  And subsequent unique finds of the block's inputs might have a history of the random attempts to prove it isn't just copying but a unique confirmation?  Is this anywhere near what's happening?

soy

I wonder because the slow confirmation rate and the debate about what to do about it is the reason Bitcoin lost a couple of billion dollars the last few days.  And nobody is happy about that except pump and dumpers.

[cr1776]

But more full nodes would be better or not?  Do installations of Bitcoin-QT contribute to the confirmation process or not?

Just to be clear, is this Bitcoin Core? Assuming this is Bitcoin Core, what version are you running?

Bitcoin Core nodes, do help and are better, in general.

Aren't all installations of Bitcoin-QT full nodes? My Bitcoin-QT is using QT version 5.6.1.

That is the QT version.  It sounds like you are running Bitcoin Core, which would be something like version 0.14.0, 0.13.1, 0.13.0, 0.12.1 etc.  Anyway, there is a difference between Bitcoin Core, Bitcoin XT, Bitcoin Unlimited etc, so wanted to make sure we were giving advice for the right software.  Anyway, most (all?) versions of BTU and BXT also use QT for their user interface, so "Bitcoin-QT" could be something one of the above.  :-)   The software would be Bitcoin Core, Bitcoin XT, Bitcoin Unlimited etc. with Bitcoin Core being the reference implementation and the others forked.

But, yes, all running Bitcoin Core are full nodes.

[soy]

Yes, Bitcoin Core version v0.13.0 (64-bit)

[Jet Cash]

You might consider upgrading to 0.14.0. It seems to be sync'ing a lot faster for me.

[soy]

Done.

[soy]

Yes. Any version of Bitcoin Core is a full node.


They aren't calculating a formula, or doing anything efficient (mining is a very inefficient operation). Basically miners are trying to find the right inputs to a specific formula (called sha256d) which result in an output that is less than the current target. The target is recalculated every 2016 blocks and done so in a way that everyone who is following the same chain calculates the same target.

Thanks.

So, rebuilding a blockchain with Bitcoin-QT does some crunching of every block download or reindexed.  How does that crunch differ from a dedicated miner's crunch?  An Antminer or a Mercury doesn't maintain a blockchain of course.

I can see that a large pool is in a race with other pools to discover blocks.  If the target is changed every 2016 blocks then dumping previous calculations would seem to be inefficient except at the 2016 point.  Calculation inputs would seem to need be saved so as to not repeat work.  One would think this would be approached in an other than random fashion.  Or is the hunt required to have random input tests?  And subsequent unique finds of the block's inputs might have a history of the random attempts to prove it isn't just copying but a unique confirmation?  Is this anywhere near what's happening?

soy

I wonder because the slow confirmation rate and the debate about what to do about it is the reason Bitcoin lost a couple of billion dollars the last few days.  And nobody is happy about that except pump and dumpers.

Or are the inputs so complex in themselves, long and random, that rerunning the same data is so unlikely that it's not a serious consideration?

[achow101]

Thanks.

So, rebuilding a blockchain with Bitcoin-QT does some crunching of every block download or reindexed.  How does that crunch differ from a dedicated miner's crunch?  An Antminer or a Mercury doesn't maintain a blockchain of course.

A miner figures out what inputs result in a hash that is less than the target. A full node will take those inputs and check that they actually hash to less than the target. Full nodes also check that the transactions included in the block are valid transactions, among other things. Note that miners actually are a subset of full nodes. Miners need to be running full nodes in order to know what transactions and blocks are valid in order to mine a valid block. The mining hardware itself does not run full node software but rather the computer they are connected to that provides the work for the ASICs to do.

I can see that a large pool is in a race with other pools to discover blocks.  If the target is changed every 2016 blocks then dumping previous calculations would seem to be inefficient except at the 2016 point.  Calculation inputs would seem to need be saved so as to not repeat work.  One would think this would be approached in an other than random fashion.  Or is the hunt required to have random input tests?  And subsequent unique finds of the block's inputs might have a history of the random attempts to prove it isn't just copying but a unique confirmation?  Is this anywhere near what's happening?

The calculation that miners do is dependent on the block they are mining. Once a new block is found, they will have to change nearly all of the inputs in order to mine the next block. Otherwise they will be mining an alternative block to the one that has already been found, and that will likely result in them wasting hash power on a block that basically no one will accept.

[DannyHamilton]

- snip -
Is this anywhere near what's happening?
- snip -

No.

Solo miners (and mining pools) choose a list of valid unconfirmed transactions. They assemble those transactions into a list.  One of those transactions is the "coinbase" ( also known as "generation") transaction which pays the block subsidy plus all transaction fees wherever the solo miner (or pool) wants to. Then they calculate a merkle root of that list using the SHA256 hash function.  The merkle root is a 256 bit (32 byte) number that is unique to that list.  Changing anything in the list of transactions would result in a completely different merkle root, therefore the merkle root can act as proof that the list hasn't been modified.

Then the solo miner (or mining pool) builds an 80 byte block header containing:

• (4 bytes) version number
• (32 bytes) Double sha256 hash of the most recent block in the blockchain
• (32 bytes) Merkle root of the transaction list
• (4 bytes) timestamp
• (4 bytes) encoded current difficulty target
• (4 bytes) nonce

Those 80 bytes are the "input" to the SHA256 "formula".

Since each solo miner (or pool) will have chosen a different list of transactions, they will each have a completely different Merkle root. Therefore, they will each be working on a different block header.

The "nonce" is a 32 bit value that can be arbitrarily manipulated so that calculating the SHA256 hash of the 80 bytes will give a different result.

Next the solo miner (or pool participants) calculates the SHA256 hash of the block header, and the SHA256 hash of that result.  If the final result of the second hash is lower than the current target difficulty, then the block is "solved" and is broadcast to all peers.

If the result of the second hash is NOT lower than the current difficulty, then the miner (or pool participant) keeps changing the nonce and trying the double hash again until they either find a low enough result (which they then broadcast to their peers) or they receive a valid block from their peers.

If the solo miner (or pool) receives a valid solved block from a peer, then they add it to their blockchain, remove all the confirmed transactions from their memory pool and start the whole process over again from the beginning building a new list of unconfirmed transactions to confirm.