Bitcoin Forum

I've been running Bitcoin-QT at night for months trying to build a blockchain.  I changed to 24/7 days ago and was up to July of last year.  Cold nights I started with Slush again after 2 years.  Running an S5, with a return of 45% of electric cost which makes it cheaper than propane.  I had an old address on Slush and tried to change it.  I use Firefox. Repeatedly for days with email exchanges with Slush the problem, mash the cancel the address change button (because it wouldn't change) and it would return to a login screen that doesn't work.  A Slush rep told me it might be my browser.  I upgraded Firefox to 64-bit from 32.  I soon got a BSOD on the Win7 machine.  Now after reverting to 32-bit Firefox, Bitcoin-QT won't run.  Rescan fails and am reindexing now.  And the Slush problem is unresolved.

Was this BSOD intentional by Mozilla?  BSOD seems to kill a blockchain with its abrupt termination making reindexing necessary.  Do they feel killing a blockchain is politically correct?  No wonder there's such a backlog of confirmations and long delays.  Nobody in their right mind is going to run Bitcoin-QT and contribute to the chain upkeep.

What's the solution?

soy

I don't know what exactly causes this. What exact bitcoin client are you using? Because bitcoin-qt is the name of the executable for BitcoinCore, BitcoinXT and BitCoin Unlimited. I know that if you switch from Core to an Unlimited Client like BitcoinUnlimited or BitcoinXT, using your same block source, you will need to reindex. And if you build and index the full blockchain with one, go to start the other, you'll be SOL and have to do it all over again. I had to reindex when I switched to the Unlimited Client 3 times. It kept getting to 99% and then quitting out. Luckily I have a fast PC and internet, so it was only a total of 24H wasted. I suspect data corruption in your blockchain data, try deleting and re-downloading the chain from scratch. If your set on becoming a full node, you can try running with these CLI options.... -par={TOTAL NUMBER OF CORES*THREADS} -discover=1 -dns=1 -forcednsseed=1 -listen=1 -listenonion=1 -maxconnections=128 -maxreceivebuffer=20000 -maxsendbuffer=4000 -peerbloomfilters=1 -timeout=2500 -datacarrier=1 -datacarriersize=172 -maxoutconnections=64

That will get you a ton of active connections to the network. Manually prune out any connections that don't provide any services (from the debug console.) That will allow you to maximize your download rate. -par set to the maximum number of logical cores will maximize your processing time. On some older OS's, that don't set affinities very well (like win7) I've found setting par to double your available logical cores can also help.

Now that network and CPU is taken care of, I find my biggest limiting factor is actually disk sequential read write speed. If you can, put the blockchain data on an SSD or RAID0 array. That will likely make it a lot faster to sync or index.

Once you've fully synced, you can use the -prune=<nBlocks> to delete the older blocks and save space while still maintaining a full node.

Probably not, I never had that problem. Have you think about the problem being with your computer? BSOD or improper shutdown can obviously corrupt the blockchain on your computer especially when you are synchronizing. Test your ram and your hardware for errors.
The backlog of transaction is not related to the client itself. The problem is with the blocksize and the transaction volume.

But more full nodes would be better or not?  Do installations of Bitcoin-QT contribute to the confirmation process or not?

Your browser has nothing to do with the operation of Bitcoin Core.

What are the specs of your machine? How does Bitcoin Core fail? What do you do that causes it to fail?

Please post your debug.log file.

More full nodes are better, but they do not contribute to the creation of blocks (aka confirmations). More full nodes will not help confirmations be faster.

Just to be clear, is this Bitcoin Core? Assuming this is Bitcoin Core, what version are you running?

Bitcoin Core nodes, do help and are better, in general.

I upgraded to 64-bit firefox and the BSOD that resulted caused an immediate shutdown of Bitcoin-QT.




Unless the full node is solo mining I suppose but I'm not sure if Bitcoin-QT even does that anymore and the rarity of solo mining finding a block makes its effect negligible? 

So, Bitcoin-QT checks each block of the blockchain but doesn't discover new blocks unless mining and therefore doesn't contribute to block confirmation rate.

Reward only goes to the discover of the block or the pool.  I discovered a block for Slush soon after getting a KNC miner early on, the block worth $5k.

Would it be possible to program devices to only run confirmation on new blocks for a cut of the reward? 

Aren't all installations of Bitcoin-QT full nodes? My Bitcoin-QT is using QT version 5.6.1.

I've been running Core on a notebook over public WiFi ( win 10 ) for well over a year now, and it's been remarkably stable. I upgrade as soon as there is a stable version, and I'm now on 0.14.0. I copied the blockchain onto an SSD, and I started another core node on an Ubuntu machine using that ( I haven't upgraded that to 0.14.0 yet, but I plan to) I've had a few hiccups over the months, and core always seems to cope. I make sure I shut down correctly, and if I have had a problem, then I shut down and restart once it has scanned my blockchain. It even copes with me forgetting to connect the SSD, although it tries to build a new blockchain, but if I restart with the SSD connected, then everything is fine. I'm impressed with the stability of the design and coding in core, and that's why I will stick with that.

If you are having problems with your blockchain, then I would check the hard drive to make sure that doesn't have problems, and also check to ensure you haven't let any malware onto your computer.

I rebooted and chkdsk did find some irregularities.

I will try some of the suggestions.

Is Bitcoin-QT Bitcoin Core v0.13.0 (64-bit) a full node?

Please correct me if I'm wrong in my understanding here.

Miners have a standard, the last block, that they must best by calculating a more efficient formula for content.  When a more efficient formula is found, discovered, miners dump what they have been working on and work on the next.  Miners in a pool get to share the reward for a discovered block and get some of the transaction fees.

Why not have programs that simply confirm that the block is good, better than the last, and get a cut?  Or would that reward, if calculated to be a little less than that for a new block search, cut into the pools' take making it unpopular regardless if it would speed up the process?

I understand miners don't run Bitcoin-QT.  I understand that Bitcoin-QT can run a mining operation.

Yes. Any version of Bitcoin Core is a full node.

They aren't calculating a formula, or doing anything efficient (mining is a very inefficient operation). Basically miners are trying to find the right inputs to a specific formula (called sha256d) which result in an output that is less than the current target. The target is recalculated every 2016 blocks and done so in a way that everyone who is following the same chain calculates the same target.

There are multiple problems with doing something like that. One is how would people who run such programs (aka full nodes) get paid? Miners are paid because they include a transaction in the block that they mined that pays the block reward to themselves. How would you pay full node operators? What stops people from gaming the system by running a bunch of fake nodes where only one actually validates the block and the rest just say they did?

Thanks.

So, rebuilding a blockchain with Bitcoin-QT does some crunching of every block download or reindexed.  How does that crunch differ from a dedicated miner's crunch?  An Antminer or a Mercury doesn't maintain a blockchain of course.

I can see that a large pool is in a race with other pools to discover blocks.  If the target is changed every 2016 blocks then dumping previous calculations would seem to be inefficient except at the 2016 point.  Calculation inputs would seem to need be saved so as to not repeat work.  One would think this would be approached in an other than random fashion.  Or is the hunt required to have random input tests?  And subsequent unique finds of the block's inputs might have a history of the random attempts to prove it isn't just copying but a unique confirmation?  Is this anywhere near what's happening?

soy

I wonder because the slow confirmation rate and the debate about what to do about it is the reason Bitcoin lost a couple of billion dollars the last few days.  And nobody is happy about that except pump and dumpers.

That is the QT version.  It sounds like you are running Bitcoin Core, which would be something like version 0.14.0, 0.13.1, 0.13.0, 0.12.1 etc.  Anyway, there is a difference between Bitcoin Core, Bitcoin XT, Bitcoin Unlimited etc, so wanted to make sure we were giving advice for the right software.  Anyway, most (all?) versions of BTU and BXT also use QT for their user interface, so "Bitcoin-QT" could be something one of the above.  :-)   The software would be Bitcoin Core, Bitcoin XT, Bitcoin Unlimited etc. with Bitcoin Core being the reference implementation and the others forked.

But, yes, all running Bitcoin Core are full nodes.

Yes, Bitcoin Core version v0.13.0 (64-bit)

You might consider upgrading to 0.14.0. It seems to be sync'ing a lot faster for me.

Done.

Or are the inputs so complex in themselves, long and random, that rerunning the same data is so unlikely that it's not a serious consideration?

A miner figures out what inputs result in a hash that is less than the target. A full node will take those inputs and check that they actually hash to less than the target. Full nodes also check that the transactions included in the block are valid transactions, among other things. Note that miners actually are a subset of full nodes. Miners need to be running full nodes in order to know what transactions and blocks are valid in order to mine a valid block. The mining hardware itself does not run full node software but rather the computer they are connected to that provides the work for the ASICs to do.

The calculation that miners do is dependent on the block they are mining. Once a new block is found, they will have to change nearly all of the inputs in order to mine the next block. Otherwise they will be mining an alternative block to the one that has already been found, and that will likely result in them wasting hash power on a block that basically no one will accept.

No.

Solo miners (and mining pools) choose a list of valid unconfirmed transactions. They assemble those transactions into a list.  One of those transactions is the "coinbase" ( also known as "generation") transaction which pays the block subsidy plus all transaction fees wherever the solo miner (or pool) wants to. Then they calculate a merkle root of that list using the SHA256 hash function.  The merkle root is a 256 bit (32 byte) number that is unique to that list.  Changing anything in the list of transactions would result in a completely different merkle root, therefore the merkle root can act as proof that the list hasn't been modified.

Then the solo miner (or mining pool) builds an 80 byte block header containing:

• (4 bytes) version number
• (32 bytes) Double sha256 hash of the most recent block in the blockchain
• (32 bytes) Merkle root of the transaction list
• (4 bytes) timestamp
• (4 bytes) encoded current difficulty target
• (4 bytes) nonce

Those 80 bytes are the "input" to the SHA256 "formula".

Since each solo miner (or pool) will have chosen a different list of transactions, they will each have a completely different Merkle root. Therefore, they will each be working on a different block header.

The "nonce" is a 32 bit value that can be arbitrarily manipulated so that calculating the SHA256 hash of the 80 bytes will give a different result.

Next the solo miner (or pool participants) calculates the SHA256 hash of the block header, and the SHA256 hash of that result.  If the final result of the second hash is lower than the current target difficulty, then the block is "solved" and is broadcast to all peers.

If the result of the second hash is NOT lower than the current difficulty, then the miner (or pool participant) keeps changing the nonce and trying the double hash again until they either find a low enough result (which they then broadcast to their peers) or they receive a valid block from their peers.

If the solo miner (or pool) receives a valid solved block from a peer, then they add it to their blockchain, remove all the confirmed transactions from their memory pool and start the whole process over again from the beginning building a new list of unconfirmed transactions to confirm.

Making a list of unconfirmed transactions - are the same unconfirmed transactions transmitted to every full node?  Since different pools will pick different unconfirmed transactions for the list which they will crunch, it would seem like that would dictate all partially solved blocks be dumped lest an unconfirmed transaction will be in multiple blocks or else that's acceptable.  Or since once a block is solved then all inputs are dumped by everyone and restarted with new unconfirmed transactions.  Those unconfirmed transactions in the solved block are now considered confirmed?  Then subsequent confirmations are due to the solved block passing checks.  So, what's to prevent a pool from ignoring solved blocks except for its solution.  Wouldn't that increase their take of solved blocks?

They might be, but there is no guarantee that they will be.

It doesn't matter.  The solo miner (or pool) just chooses from the list of unconfirmed transactions that they have.


There is no such thing as a "partially solved block".  A block is either solved, or not solved.


Correct.  When a miner (or pool) received a solved block, they remove all the transactions that are in that block from their list of unconfirmed transactions.  Then they build a new list, build a new block header, and start the process of trying all the nonce values again.


The block is solved.  The transactions are confirmed.


No.  subsequent "confirmations" just means that additional blocks have been added to the blockchain on top of the block that first confirmed the transaction.  Each new block added to the blockchain is called a "confirmation" for ALL transactions that are already in the blockchain.

Since the blockchain is currently 458142 blocks long, this means that the very first transaction that paid Satoshi Nakamoto the block reward from the very first block now has 458142 "confirmations".


Each block references the block before it with a hash of that block (see my list of what's in the header).  That's what turns a collection of blocks into an actual blockCHAIN.  All nodes and all other miners will use the longest chain (technically they use the chain with the most proof-of-work, but effectively that typically means the chain with the most solved blocks). If they receive more than one chain of the same length, then they use the first one that they receive.

Lets imagine that you have a blockchain that is 3 blocks long:
Block_A -> Block_B -> Block_C

All the miners and pools (including you) are building a Block_D that has in the header a "Double sha256 hash of Block_C".

Now lets say you receive a Block_D from your peers (lets call it Block_D₁ so we don't get confused later).  All the other miners and nodes on the network have now added Block_D₁ to their blockchain.
Block_A -> Block_B -> Block_C -> Block_D₁

The entire network is now working on Block_E

Meanwhile, you choose to ignore it and continue to work on your own Block_D (lets call it Block_D₂ so we don't get confused later).

Now lets say you get very lucky and solve Block_D₂ before anybody else on the entire network solves Block_E.

Now there are two competing blockchains:
Block_A -> Block_B -> Block_C -> Block_D₁
Block_A -> Block_B -> Block_C -> Block_D₂

The whole network is all building Block_E using the hash of Block_D₁
You (and maybe a few miners that had network issues and failed to hear about Block_D₁ are building your own Block_E using the hash of Block_D₂

Since the entire rest of the global network has MUCH more hash power than you and your few peers, AND they have been working on their Block_E longer than you, there is a very good chance that they are going to solve a Block_E that uses the hash of Block_D₁ long before you are going to solve a Block_E that uses the hash of Block_D₂.  This means you will now have to ignore BOTH the Block_D AND the Block_E that the rest of the network is using. None of the network will ever accept your Block_E since they will already have a Block_E from their peers.  This will continue so long as you continue to ignore all the block that the rest of the network is using.

In other words, you will have created a FORK of the bitcoin blockchain that ONLY you know about, and that ONLY you are using.  Any bitcoins created in your fork will not be recognized as valid in the fork that everybody else is using.  Sure, you'll have lots of soyBitcoins, but nobody accepts soyBitcoins and they don't have any value.  You've wasted a lot of money on mining equipment and electricity to create a fork that nobody else wants to use.  That's going to get very expensive, and is a pretty big waste of money with no purpose.  If you had instead started working on Block_E when everyone else did then you might have solved that block first and received actual bitcoins on the blockchain that everybody uses. This would have been a much better use of your time and money.

Which would you prefer, spend a lot of money mining a bunch of worthless forked chain coins that nobody will see as valid and will be unspendable on the real blockchain, or spend that money mining actual bitcoins on the actual blockchain that you can spend and use and have actual value.

Now, there is an exception to this scenario.  If you control more hash power than the entire rest of the world combined, then you will be able to add blocks to your chain faster than the world can add blocks to their chain.  In this case, your chain will always be longer and the entire world will use your chain.  You will get ALL of the block rewards, and have exclusive control over which transactions get confirmed. This is commonly called a 51% attack (since the rest of the world controls 49% or less of the hash power and you control 51% or more.  Its a VERY expensive attack to acquire that much hash power, and then you risk everyone abandoning the bitcoin concept entirely, so you may spend a LOT of money and still not profit from it.

You had me at each additional block adds a confirmation to those before.  No additional work being done on transactions in that block.  This was the first I encountered that explanation.  Thanks DannyHamilton.

Well, the blockchain got as far as February 2017 while reindexing, a little over a month to go.  Then, apparently, there was an unexpected termination last night.  I've restarted, again, reindexing the blockchain from 4+ years.  Second or third time this year.

I suspect one not only needs a fully dedicated system but also have that fully dedicated system on an uninterruptible power supply.  If it wasn't a hickup in power then somebody got in a did me dirty interrupting the process without proper shutdown.  I had carefully shut down the process earlier in the week for a Windows update and changed the reboot for updates to no.

I bet this kind of malfunction causes quite a few early users, like those who earned bitcoins with USB single ASCI devices and put them in Bitcoin-QT wallets, then lost their blockchains but figured they could just rebuild at some later date, to later get frustrated and abandon them. 

Who benefits from lost bitcoins?  They add a false stability to a small degree I suppose.  Might be problematic for regulators (the taxman) if considered more than dust.

soy

and the block they had been trying to solve is aborted. How many blocks are aborted in the average day compared with the number of orphaned blocks?

So, my Bitcoin-QT started again, I look some hours later and although last I looked it was down to 3 years, it was back to processing blocks on disk 4 years 25 weeks.

I understand pruning is now possible.  Is it possible to prune before the entire blockchain is downloaded?  I'm surprised some developers of this pos haven't been taken behind the stores and stomped.

Anyway to set this Bitcoin-QT to nothing in nothing out until at least it's up to its last previous state, around Feb. of this year?  It feels like I'm getting exploited to hell with gigabytes out and unnecessary repeats of 4 years worth of blocks to check.

One thing is for sure.  This Bitcoin-QT is about as far from eco-friendly as it can get.  Wasted energy.  btc=$946 and falling.

No, not as far as I know (which isn't too far! LOL)
You must download the entire blockchain before you are able to prune it.

I've been running Core and wallet 13.0 / QT 5.6.1 on an HP laptop with two 1 terabyte disks and 16GB RAM for many months. It is all fed from my network which includes other laptops, all television, and four S9 miners, three S7 miners over GB modem/router/switches. My download speed from cable provider is 60MG/s. Not sure of upload. But I usually always have at least 20-30 peer connections.

I've had no problems after the initial download except once when the db became corrupted for some reason - probably Windows update. (Running Win10, which I have grown to dislike very much!)
The blockchain repaired itself in about a day and a half.

As mentioned before, whatever browser you are using has absolutely no effect on the Core/QT application, its installation and syncing up with the complete block chain.

If you are still having problems, stop all other tasks or application on the PC in question and don't surf the web while downloading the blockchain.

Not sure if I'm replying to the right person as I have a slight buzz from watching NCCAA Basketball all day long. (My Gamecocks are still in it!)  ;D

Best of luck with getting your Core up and running!!!  :)

No, the full Core node doesn't have the capability of mining anymore. Not sure of what version that was taken out. It was before my time in Running a Core full node.
Best wishes! :)

I haven't experienced the difficulties you describe, even with unexpected and uncontrolled shutdowns. Perhaps you should consider shutting down and creating a backup of the blockchain regularly. That way if you have a crash again you can restore and not have to start from the beginning again.


Everyone that doesn't lose their bitcoins.  The exchange rate between bitcoins and anything you may want to exchange them for depends on the supply of those bitcoins and the demand for them.  If bitcoins are permanently lost, then the supply shrinks. This drives the exchange rate up at any given demand to maintain balance.


More likely than not, the block they were working on was unsolvable.  Most blocks that most miners are working on are.

Another way to look at it is that EVERY hash is effectively a brand new block. Therefore there are 0 blocks ever "aborted" (aside from computer crashes that prevent the completion of computing a single hash).


Yes.


It's open source.  There's nothing preventing you (or anyone you hire) from doing a better job if you don't like the job the current participants are doing. The software is free.  You've gotten more than you paid for.  If you've chosen to exchange cash for bitcoins, then can I suggest caveat emptor.


This is not true.  Pruning occurs as the blockchain downloads.

[/quote]

Thanks for clarification. As I said, I didn't know too far!
I have always had the full blockchain and never have pruned it or needed to. So I was assuming that you must have the entire blockchain available before you are able to prune it.
I stand corrected. (Which is quite often! :) )

Okay, thanks.  I had been having problems with Google Calendar opening by Task Scheduler in the mornings.  Yesterday I unscheduled it.  Should have had an easier time but when I got on this AM there was the BSOD.  Perhaps because I failed to close Google News last night.  Restarted today and thankfully it went back to processing at 2 years 4 weeks behind.

So, it is possible to prune before the chain finishing?  I'll give it a try.  Thanks.

To be honest I'm skeptical.  So prune=550 will only keep about 2 days of the blockchain on he drive.  At 2 years 4 weeks behind I suspect all this does is chop off work that has been completed while new stuff is added to the other end.  No difference in the time it should take to get a functioning wallet.

That's what I was thinking, that you have to download the data in order to get to the last 550MB. Else how would it know that it is only going to keep approximately the last 2 days worth of the blockchain. It may discard all the data up until the last few days, but it still - in my thinking - has to go through the previous data.

From my understanding, a pruned node is not considered a full node and is unable to serve up historical blocks to other nodes. "Keeping all blocks is a service to the network, as you'll be able to provide all blocks for synchronizing nodes or requests of thin clients."

However it works and whether it will take just as long to get it downloaded and pruned or not, good luck with it. :)

I agree.  And since drive space isn't an issue, 2TB drive, next time it shuts down I'll restart without the -prune.  The prune switch seems to make no difference in network traffic judging by a glance at the debug network traffic.

If there was a way and I knew how to do it, I'd let you sync off my blockchain. I usually always have 20-30 peers syncing, giving to and taking from my blockchain. I have no issues with network traffic.

Hey, just thought of something. Sometimes malicious users will insert viruses or malware in the blockchain. Perhaps you are getting some of that crap and your anti-virus is flagging it and causing the Core node to stop syncing.
Not sure how Core or AV applications handle that.

I do recall I had to allow an exception for QT through my firewall port. But I have other apps that will prevent any malicious code from running. Anyway, from what I understand, malicious code in the blockchain can't infect your machine.

Just an idea.  8)

Also, if you haven't already done so, I would suggest reading this page https://bitcoin.org/en/full-node# (https://bitcoin.org/en/full-node#) and see if  there is anything that you are overlooking or answers to possible problems.
Good luck!

Correct.  You'll still need to download the entire blockchain, and it won't speed up the process. It will just delete the completed work as new blocks are added to the other end, so it will use less drive space.

It doesn't wait until the entire blockchain has been downloaded and stored on the drive before it starts pruning the completed stuff.

I'm not certain, but I suspect that once you have pruning turned on, it may start over downloading from the beginning again if you turn it off (since those early blocks may already have been pruned).

Yes, I know for a fact that you will have to re-download the entire blockchain if you turn off pruning.

No, it picked up where it left off, 2+ years to go.  So, the probability that the whole blockchain must first be downloaded before pruning starts is high.

I've torrent downloaded the blockchain before and built from that but it didn't seem any faster.

With only a month or two to go when it almost finished the other day, all those blocks, reindexing seems not to use those but downloads new or am I mistaken?

Well, turning pruning on didn't cause a restart.  I'll try stopping and starting without pruning now because if what you say is correct, I don't want to lose the blockchain at the end were I to go to no pruning then.  Although I can see that if one has already deleted a large part of the blockchain then yes, that would have to be downloaded with pruning off.

"You need to rebuild the database using -reindex to go back to unpruned mode.  Do you want to rebuild the block database now?"  So, from 1 year some months to go, it's back to 4+ years. ...my mistake, 8 years 10 weeks behind....

So, I went back to unpruned mode, recrunching the whole 9 yards, and was quite far along on Thursday but it was raining.  So, I shut it down rather than chance a power outage.  Then I left it down until Monday, today, because I always back up my systems on Saturday and also that most failures have occurred on Fri, Sat or Sun.  Today I needed to do some shopping and started Bitcoin-QT before I left.  A cell passed through and when I came back in my systems were down.

I fired them up.  Starting Bitcoin-QT, it went through a check and then said Bitcoin Core not responding.  I just let it set.  Lo and behold, a few minutes later it opened and commenced processing blocks where it left off.  I'm happy.

I wonder.  The many times I've restarted from scratch this year, are there old blockchains abandoned and taking up space?

soy

Okay, I've had the blockchain back for a week or so.  My Win7 machine has started locking up.  My C: drive is 2TB, backup on another drive.  I see I have used up most of C since starting to rebuild the blockchain with many restarts.  Are there duplicate blocks from earlier attempts that I may delete?