Me panicing because of a Core crash. Now I have to redownload the chain. Lame.

[ClownSpider]

FUCK!!!
OK, I have ***+ BTC.  
On 7-22 I spent 0.********** on Purse.io  Transaction accepted and completed on record, goods delivered.

Status: conflicted
Date: 7/22/15 05:**
To: 16aWEopdE*******************
Debit: -0.***** BTC
Transaction fee: -0.00001908 BTC
Net amount: -0.**** BTC
Transaction ID: *******************

On 7-20 I spent 0.************* BTC ALSO on Purse.io Transaction accepted and completed on record, goods delivered.
Status: conflicted
Date: 7/20/15 13:56
To: 16aWEo*****************
Debit: -0.********* BTC
Transaction fee: -0.00004079 BTC
Net amount: -0.********* BTC
Transaction ID: a3e523d1*********************

Today I open Bitcoin Core 10.2 and it takes a long time, then says ERROR, and crashed.
I open again.  Blockchain data fucked, must redownload everything.
Error?  How?  Did blockchain.info rewrite the blockchain logs to backdate and steal?  HOW?

Because I go to blockchain.info to check my only known account, sometimes I use blockchain.info only to look at my account balance by searching my BTC receiving address.  Is this address a secret??

Suddenly my account is $0.00!!!!!!!
It says all my money was sent to another account on 2015-07-18!!
Then another, then another, then another... etc  and then parked at a new wallet that is unknown to me.

HOW?  How could I be broke on the 18th, when I made transactions on the 20th and 22nd?
My money was still mine and I was still using it to buy stuff on purse.io on the 20th and 22nd.
According to Blockchain.info I have been at ZERO since the 18th!!

Now my transactions say "conflicted", but my blockchain is not finished downloading and will not be for many days.

I spent the money on Purse, received the goods, and paid.

Did someone rewire the logs to make it say I went broke on the 18th and just make the 20th and 22nd transactions conflict?
WTF is going on?
Or does the blockchain just randomly reassign me new wallet addresses and I will see this after the blockchain updates?
How the fuck?
And why the heck does it need to redownload the blockchain when it still has 46.8GB of blocks?
This is stupid that the BTC Core crashes so easy then requires a full re-download!  Seriously?
Whose dumb idea was that?

I am freaking out because blockchain.info has ALWAYS shown my balance just fine, until now.

[1714178214]

1714178214

[1714178214]

1714178214

[1714178214]

1714178214

[1714178214]

1714178214

[Newar]

Both "conflicted" tx are confirmed on www.blocktrail.com.

When you use one receiving address* and then make a tx from Bitcoin Core it will look like some of the coins go to unknown addresses. Read about "change": https://en.bitcoin.it/wiki/Change

It sounds like you need to wait for the DB to rebuild and rescan. Please report back when it's done.


*(for future reference, it's better practice to not use the same address over and over again)

[shorena]

FUCK!!!

I know its a lot of coins, but its easier to help you if you are calm.

-snip-
And why the heck does it need to redownload the blockchain when it still has 46.8GB of blocks?

It does not. I assume that your local database got corrupted. This happens when the client does not shut down properly, e.g. due to a crash. Its rescanning the local data to rebuild its database of TX that are interesting to you because they correspond to your addresses.

This is stupid that the BTC Core crashes so easy then requires a full re-download!  Seriously?
Whose dumb idea was that?

If you want to find a reason for the crash consult the debug.log file. I dont know why it "crases so easy" for you. Im running bitcoin core for over a year on several different systems and machines and the only time I had the problem you describe was when we had a power outage.

I am freaking out because blockchain.info has ALWAYS shown my balance just fine, until now.

As Newar said blockchain.info can only show you how much coins have been received on a given address. Your balance is more than that.

[achow101]

Error?  How?  Did blockchain.info rewrite the blockchain logs to backdate and steal?  HOW?

They can't do that.

I am fairly certain that the addresses where the Bitcoin went are change addresses. They are still your address. Bitcoin Core will send your extra Bitcoin to another address that you control so it follows the principle of one input one output for each address.

Now my transactions say "conflicted", but my blockchain is not finished downloading and will not be for many days.

Let it download. Bitcoin Core was confused (don't know why) about your transactions. Let it redownload and reindex the blockchain. All of your Bitcoin will still be there and still be spendable. Don't panic.

[pedrog]

Your coins are still there, like explained above.

Now that you got a good scare time to learn how to backup and don't have a lot of coins in your wallet in your desktop, you should distribute those coins between a few addresses.

[gablay12]

I have had a similar situation.If you have a back up, do not panic.Relax 

[ClownSpider]

Ok Thank you guys!
I was in a panic and very unhappy.
I do have a backup.
I did create a new wallet and moved BTC, then backed up wallet.dat.

The fact that I spent money after the 18th was comforting because I knew I wasnt really broke on that day.
This is the first time my only receiving address shows as empty, and as if it had forwarded the BTC elsewhere for no reason.

*(for future reference, it's better practice to not use the same address over and over again)

Why is it better to use a different address every time?  Doesnt that just spam the blockchain with more crap data and collectively result in making it take a week to download because everyone thinks they need to keep making up new BTC addresses for each transaction?

Chill out. I'm 80% sure you're alright and the coins are safe, 95% sure if you have a few backups.

3) do a rescan with bitcoin-qt.exe -rescan         (this takes HOURS)

My tipjar: 1GoodRkQnQBsTPnZDvpxCcZYw7KR98arjb

Thanks.
I will try a rescan, but it wants to redownload the "chainstate" folder.
I am not sure if it is possible to rescan that, if the core deleted it to rewrite it.
Wallet file is still here, and I did make a backup, encrypted and put someplace else.

I hate that the BTC Core is so stupid it has to rewrite perfectly good data just because one small part went bad with the BTC crash.  It cant just find and rewrite that one part, or go back one day or at least back to the day of the most recent transaction when everything must have been ok, and start from that time, it has to go back to day 1!  wtf. 

BTC Core needs to assume the date of the last confirmed transaction was good, and only go back to that date.  Not back 6+ years. 

[errornone]

yeah, that is a change address.

The private key of the wallet you made that payment is the one which controls this new address with the bitcoins.

[ClownSpider]

Ok, well now I cannot buy anything for a week now that the blockchain is redownloading the entire thing because of one little crash.
Glad the money is hopefully still mine.
But I cant use it now.
One day the blockcain will be so large that any crash may take a month to download.
I have very fast internet, but the blockchain data is a very slow download.

[tommorisonwebdesign]

I have this problem a lot. Do not use the same addresses over and over. This will prevent crashes like the many ones I have experienced. I am sure glad I was wise enough to create a backup.

[ClownSpider]

I have this problem a lot. Do not use the same addresses over and over. This will prevent crashes like the many ones I have experienced. I am sure glad I was wise enough to create a backup.

This is sad that BTC is so unstable that you can't just use one address.  It spams and fattens up the chain for no reason, making it slower to download every time someone has a crash.  Very bad for BTC to have this problem.  No wonder the mainstream will not adopt this broken tech.  It's like in BETA forever.

[achow101]

Why is it better to use a different address every time?  Doesnt that just spam the blockchain with more crap data and collectively result in making it take a week to download because everyone thinks they need to keep making up new BTC addresses for each transaction?

Nope, it doesn't spam the blockchain with more crap. It doesn't make a difference to that at all. The reason for using different addresses for everything is for security. It makes it more difficult for people to track your payments to places and it makes it easier for you to track payments to you. Also, by using multiple addresses, it is more difficult for people to know how much Bitcoin you actually have.

I hate that the BTC Core is so stupid it has to rewrite perfectly good data just because one small part went bad with the BTC crash.  It cant just find and rewrite that one part, or go back one day or at least back to the day of the most recent transaction when everything must have been ok, and start from that time, it has to go back to day 1!  wtf. 

It can't because LevelDB (the database type used by Bitcoin Core) is not all that great with recovery. Once it corrupts, the data may still be readable, but parts of it won't be, and it does not know where. The corruption would not only be just at the end where the latest transactions are, but could be throughout depending on what was open at the time. The corruption also prevents Bitcoin Core from being able to open and read the database as it should. While humans can open the file and read it(with the right tools), the computer is not able to interpret the data since it comes linearly. Humans can jump around and look at the data to find stuff that we need, but the computer processes things linearly. If it receives data it doesn't recognize, then it will crash and error out. The corruption causes this erroring and it can't be fixed without rebuilding the whole database.

BTC Core needs to assume the date of the last confirmed transaction was good, and only go back to that date.  Not back 6+ years. 

Since the data is read linearly, it won't ever be able to get to the point where it can read good data. Due to the corruption, it will error out and take the safe route of having to reindex the whole thing.

[achow101]

I have this problem a lot. Do not use the same addresses over and over. This will prevent crashes like the many ones I have experienced. I am sure glad I was wise enough to create a backup.

This is sad that BTC is so unstable that you can't just use one address.  It spams and fattens up the chain for no reason, making it slower to download every time someone has a crash.  Very bad for BTC to have this problem.  No wonder the mainstream will not adopt this broken tech.  It's like in BETA forever.

It doesn't spam and fatten up the chain, and it is still technically in alpha.

Bitcoin is in alpha development still. That is why version 1.0 still doesn't exist. It isn't stable and ready for mainstream adoption yet, which is why it is still labeled alpha by the core developers.

[ClownSpider]

Since the data is read linearly, it won't ever be able to get to the point where it can read good data. Due to the corruption, it will error out and take the safe route of having to reindex the whole thing.

Bitcoin is in alpha development still. That is why version 1.0 still doesn't exist. It isn't stable and ready for mainstream adoption yet, which is why it is still labeled alpha by the core developers.

wow ok.  Sounds like nobody thought it would get this big.
They need to make the next version use a better database system that is not linearly.
Simple fix then to a very big problem.

[achow101]

wow ok.  Sounds like nobody thought it would get this big.
They need to make the next version use a better database system that is not linearly.
Simple fix then to a very big problem.

Well the linear stuff is not just the database, but computers in general. They can't just jump around and process multiple things in different orders like humans do. What Bitcoin Core can do is migrate to a more ACID (Atomicity, Consistency, Isolation, Durability) database. That would allow it to recover better from crashes and not have to reindex every time the database corrupts. The only problem is that changing databases would require a huge rewrite of the code to support a different database and then that new version would most certainly not be backwards compatible. It would have to reindex the first time.

[ClownSpider]

Well the linear stuff is not just the database, but computers in general. They can't just jump around and process multiple things in different orders like humans do. What Bitcoin Core can do is migrate to a more ACID (Atomicity, Consistency, Isolation, Durability) database. That would allow it to recover better from crashes and not have to reindex every time the database corrupts. The only problem is that changing databases would require a huge rewrite of the code to support a different database and then that new version would most certainly not be backwards compatible. It would have to reindex the first time.

I wouldn't even call this jumping around.  I would call it going back to a confirmed transaction and looking for the error after that point.  The next version would hopefully be backwards compatible, read the current DB/chain on the system and write it locally into the new format, so that it does not need to redownload. 

The download is the problem.  6 days, because the bandwidth of bitcoin's network is so slow.
So I cant buy anything for 5 more days.  =(

Still better than those mother fuckers at paypal that do chargebacks 25 days after the fact.

[achow101]

I wouldn't even call this jumping around.  I would call it going back to a confirmed transaction and looking for the error after that point.  The next version would hopefully be backwards compatible, read the current DB/chain on the system and write it locally into the new format, so that it does not need to redownload. 

For a computer, that is jumping around. It is not in order that it is reading the file in. In order to go back to a confirmed transaction, it first needs to get through the corrupted data, which it will have no idea what that data is, before it can reach any good data. Even so, depending on what it was reading, it may be corrupted in multiple places in the database. The corruption might also might not be of the transactions, but of the file and database structure itself. If that was corrupted (which is probably the case since bad transactions can just be redownloaded) then having good transactions means nothing as it cannot read them or understand the format of the data. Just to be safe, it will reindex. Reindexing guarantees that the data it reads is not corrupted. Otherwise, the data might be corrupted and such a corruption can negatively affect the entire program.

The download is the problem.  6 days, because the bandwidth of bitcoin's network is so slow.
So I cant buy anything for 5 more days.  =(

Reindexing does not mean it is downloading. It might appear that way, but it is in fact reading the data off of the blk files and indexing them into a database in ldb files. If you were stupid enough to delete the blk files then it will be redownloading. Even so, it isn't the bandwidth of Bitcoin's network, it is your own internet bandwidth that limits the download. The bottleneck for you right now is your CPU and RAM. Reindexing is all on the CPU and the RAM. If you don't have a good CPU and enough RAM, then it will take a long time. For me, I can have a reindex done in about 15 hours. Some people have them done in 1 or 2 hours, others a few days. But the problem is on your computer, not the network.

[ClownSpider]

Reindexing does not mean it is downloading. It might appear that way, but it is in fact reading the data off of the blk files and indexing them into a database in ldb files. If you were stupid enough to delete the blk files then it will be redownloading. Even so, it isn't the bandwidth of Bitcoin's network, it is your own internet bandwidth that limits the download. The bottleneck for you right now is your CPU and RAM. Reindexing is all on the CPU and the RAM. If you don't have a good CPU and enough RAM, then it will take a long time. For me, I can have a reindex done in about 15 hours. Some people have them done in 1 or 2 hours, others a few days. But the problem is on your computer, not the network.

Fortunately I did not delete the block files, all 46.8GB and growing.
Perhaps the BTC network data passing through my system is new stuff and not part of the reprocessing of all blocks.
My CPU has an average of 88% CPU idle.  I have a quad core MBP with 16GB Ram, my CPU is not slow.
BTC Core is using under 350MB of Ram... and an average of 40% CPU.  Not using much.
My bandwidth at home is 120 MB/s up and down. 

How can I speed this up if it is all CPU, it should be done quickly.

Some command line settings I should try?

[philipma1957]

Reindexing does not mean it is downloading. It might appear that way, but it is in fact reading the data off of the blk files and indexing them into a database in ldb files. If you were stupid enough to delete the blk files then it will be redownloading. Even so, it isn't the bandwidth of Bitcoin's network, it is your own internet bandwidth that limits the download. The bottleneck for you right now is your CPU and RAM. Reindexing is all on the CPU and the RAM. If you don't have a good CPU and enough RAM, then it will take a long time. For me, I can have a reindex done in about 15 hours. Some people have them done in 1 or 2 hours, others a few days. But the problem is on your computer, not the network.

Fortunately I did not delete the block files, all 46.8GB and growing.
Perhaps the BTC network data passing through my system is new stuff and not part of the reprocessing of all blocks.
My CPU has an average of 88% CPU idle.  I have a quad core MBP with 16GB Ram, my CPU is not slow.
BTC Core is using under 350MB of Ram... and an average of 40% CPU.  Not using much.
My bandwidth at home is 120 MB/s up and down.  

How can I speed this up if it is all CPU, it should be done quickly.

Some command line settings I should try?

ssd's are one  way.  but not this time.  the slower rebuild is the best way to go.



like it or not you have enough coins (48)  

 to consider a few pc's  that mainly hold your coins and do nothing else.  48 coins is close to  14,500 usd.

 A pc with just bitcore  will speed thing up…    I have a pc with  an older quad core 16gb ram and a 250 gb ssd .

It can do a reindex in under 10 hours.

You could  consider having   2 wallets with blockchain.info   one with 2 or less coins  one with 4 to 8 coins.

and then the rest of the coins in a safe wallet.

[ClownSpider]

ssd's are one  way.  but not this time.  the slower rebuild is the best way to go.



like it or not you have enough coins (48)  

 to consider a few pc's  that mainly hold your coins and do nothing else.  48 coins is close to  14,500 usd.

 A pc with just bitcore  will speed thing up…    I have a pc with  an older quad core 16gb ram and a 250 gb ssd .

It can do a reindex in under 10 hours.

You could  consider having   2 wallets with blockchain.info   one with 2 or less coins  one with 4 to 8 coins.

and then the rest of the coins in a safe wallet.

Good Advice.
I have switched to SSD again.  I wanted it on an external I could just leave offline.  I will need a SSD external next.
This reindexing crashed on me 2 days into it, so I had to start over again.
The BTC Core is not even using 100% CPU, mostly just 36%.  So it is not even taking advantage of the CPU, and only using less than 350MB ram when I have 16GB.  Very frustrating.

Yes I should put my eggs in different baskets.
Money Management basics.