Problem with a lot of hdd writes on big wallet.dat

[kimon]

Hello All!

I have Bitcoin Core 0.11.1 Linux x64 and my wallet.dat size is about 480Mb (~52000 transactions).

So the problem is a lot of data writing on disk. By "iotop -o -a" I see that bitcoind writes about 4.2Gb on disk in 15 minutes. It's too much.
I think wallet.dat writes on disk each time when new transaction comes or something similar.

Is is possible to turn on something caching for that wallet.dat will not writtes to disk so often?
Or is it possible to write any changes to the temporary file, and save wallet.dat no more than once per hour?

Because the problem is too large volume disk writes.

Please help!

[1715292841]

1715292841

[1715292841]

1715292841

[1715292841]

1715292841

[1715292841]

1715292841

[1715292841]

1715292841

[1715292841]

1715292841

[2112]

How do you know this is wallet.dat I/O only? I would've thought that the majority of I/O is related to the LevelDB chain state storage, which frequently does large reorganizations of its internal storage levels, especially with txindex=1.

[kimon]

I have txindex=0.

I propose that's wallet.dat I/O because I see portions of increasing in "iotop -to -a" output.
And each portion is about 480 Mb. Exactly as wallet.dat size. That's why.

[2112]

I have txindex=0.

I propose that's wallet.dat I/O because I see portions of increasing in "iotop -to -a" output.
And each portion is about 480 Mb. Exactly as wallet.dat size. That's why.

To me it looks like somebody is repeatedly calling "backupwallet" on your server.

Anyway, here's the quickest way to find out:

1) create DB_CONFIG file with one line "set_lg_dir database"
2) start the bitcoind with -privdb=0
3) use db_stat to monitor it live
4) download (and read or at least skim) the documentation from docs.oracle.com

The easiest way to fix the large wallet.dat problems is by adding two more physical disk drives to your server: one for wallet.dat one for its log directory ("database" by default, but it is best to change it). Adding SSD drives doesn't help much because it uses https://en.wikipedia.org/wiki/Write-ahead_logging and this is exactly pessimal application for flash memory: all writes and no reads.

[kimon]

I did 1 and 2.
What command line arguments need to use for db_stat?
Only this works for me  "db_stat -d wallet.dat" (on shutdown bitcoind)

53162   Btree magic number
9       Btree version number
Little-endian   Byte order
multiple-databases      Flags
2       Minimum keys per-page
4096    Underlying database page size
1007    Overflow key/data size
1       Number of levels in the tree
1       Number of unique keys in the tree
1       Number of data items in the tree
0       Number of tree internal pages
0       Number of bytes free in tree internal pages (0% ff)
1       Number of tree leaf pages
4050    Number of bytes free in tree leaf pages (1% ff)
0       Number of tree duplicate pages
0       Number of bytes free in tree duplicate pages (0% ff)
0       Number of tree overflow pages
0       Number of bytes free in tree overflow pages (0% ff)
0       Number of empty pages
0       Number of pages on the free list

And can you to point me a documentation from docs.oracle.com that I should read? There is a lot of documents here.
Thank you!

[2112]

I did 1 and 2.
What command line arguments need to use for db_stat?
Only this works for me  "db_stat -d wallet.dat" (on shutdown bitcoind)

multiple-databases      Flags

And can you to point me a documentation from docs.oracle.com that I should read? There is a lot of documents here.
Thank you!

The general online documentation is at:

http://docs.oracle.com/cd/E17076_04/html/index.html

but Bitcoin Core is using older version (4.8.30?)

http://www.oracle.com/technetwork/database/berkeleydb/downloads/index-082944.html

The entire older version is in the source archives: e.g.

http://download.oracle.com/berkeley-db/db-4.8.30.zip

I don't know the flags to the version of the BerkeleyDB included in your Linux distro. If it isn't exactly the same as your bitcoind then it isn't going to work until you rebuild all the db_* utilities with the exact same flags as used to build bitcoind.

One thing that you obviously missed is that you quoted an output from the "list of databases" inside "wallet.dat". The actual Bitcoin wallet is inside that file within a database named "main".

The other thing that you missed is that if you get all versions and build flags of BerkeleyDB right you will be able to monitor everything inside wallet.dat live, without shutting down bitcoind. That is the whole point of the exercise!

Edit: Oh, and I just noticed one more thing: it is about 5am in Ukraine. Did you actually sleep tonight? This isn't a 5 minute job that any zombie can do, you need to be really awake and conscious.

Edit2: In another past post you've mentioned that you are using the built-in account functionality. The "accounts" in Bitcoin Core never worked right. They are now officially depreciated. Nobody is going to fix any performance bugs in that code. You can't easily fix those bug yourselves because the "accounts" were designed wrong and fixing them would require major rewrite. The only people who ever used built-in accounts were fraudsters and people who have absolutely no understanding of accounting. I presume you were just completely unfamiliar with accounting and that was why you used them.
 

[kimon]

I don't know the flags to the version of the BerkeleyDB included in your Linux distro. If it isn't exactly the same as your bitcoind then it isn't going to work until you rebuild all the db_* utilities with the exact same flags as used to build bitcoind.

I compiled db-4.8.30 by this (https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md)

configure --enable-cxx --disable-shared --with-pic
make install

But I don't know about flags with my bitcoind was compiled. I just got bitcoind here https://bitcoin.org/bin/bitcoin-core-0.11.1/bitcoin-0.11.1-linux64.tar.gz

One thing that you obviously missed is that you quoted an output from the "list of databases" inside "wallet.dat". The actual Bitcoin wallet is inside that file within a database named "main".

Yes, "db_dump -l wallet.dat" ouput only "main".

The other thing that you missed is that if you get all versions and build flags of BerkeleyDB right you will be able to monitor everything inside wallet.dat live, without shutting down bitcoind. That is the whole point of the exercise!

It seems the flags is right. So what command line argument need for db_stat that it will show everything inside wallet.dat live?

Edit: Oh, and I just noticed one more thing: it is about 5am in Ukraine. Did you actually sleep tonight? This isn't a 5 minute job that any zombie can do, you need to be really awake and conscious.

Sometimes when I getting exciting challenge, I can not fall asleep for a long time )

Edit2: In another past post you've mentioned that you are using the built-in account functionality. The "accounts" in Bitcoin Core never worked right. They are now officially depreciated. Nobody is going to fix any performance bugs in that code. You can't easily fix those bug yourselves because the "accounts" were designed wrong and fixing them would require major rewrite. The only people who ever used built-in accounts were fraudsters and people who have absolutely no understanding of accounting. I presume you were just completely unfamiliar with accounting and that was why you used them.

So what the right way to use bitcoind in multiuser system without "accounts"?
Keep all users and they addresses in my one database (like mysql) and then just do "bitcoind sendfrom" for sending and catch "walletnotify" event for receiving? Is it right?

The only people who ever used built-in accounts were fraudsters and people who have absolutely no understanding of accounting. I presume you were just completely unfamiliar with accounting and that was why you used them.

When I was making design my project I haven't found any materials that would explain to me the right way, so I had to deal with Bitcoin Core API such as it is. Forgive me my illiteracy.

[2112]

Sometimes when I getting exciting challenge, I can not fall asleep for a long time )

Just don't try to drive a car. Take a public transport like bus or metro. I don't want you to kill yourself by falling asleep while driving.

I just realized that my previous advice isn't good for you. I think that you are just one-man-company with programming skills at the level of a beginner. My advice was directed at somebody who is more skilled.

I see that 4 months ago your wallet.dat was half the current size. From the questions you asked in the last message I think the limit of what you could successfully program yourselves is as follows:
Write a BerkeleyDB program that deletes the old inactive accounts and transactions from your online wallet and saves them for a separate not-really-online wallet. This program will not need to work with bitcoind live, it is better to shut it down, run this program, and restart your online wallet. The not-really-online wallet can be run on a separate machine and will not really affect the responsiveness of your web front end.
This board is full of the incorrect advice for the desperate people in your situation. But I'm not going to throw you to sharks.

The English way to describe your situation is: "up shit creek without a paddle". I like the old Russian poet's advice more:

Бeлeeт пapyc oдинoкий B тyмaнe мopя гoлyбoм!.. Чтo ищeт oн в cтpaнe дaлeкoй? Чтo кинyл oн в кpaю poднoм?..   Игpaют вoлны - вeтep cвищeт, И мaчтa гнeтcя и cкpыпит... Увы, - oн cчacтия нe ищeт И нe oт cчacтия бeжит! Пoд ним cтpyя cвeтлeй лaзypи, Haд ним лyч coлнцa зoлoтoй... A oн, мятeжный, пpocит бypи, Кaк бyдтo в бypяx ecть пoкoй!

A lonely sail is flashing white Amidst the blue mist of the sea!... What does it seek in foreign lands? What did it leave behind at home?.. Waves heave, wind whistles, The mast, it bends and creaks... Alas, it seeks not happiness Nor happiness does it escape! Below, a current azure bright, Above, a golden ray of sun... Rebellious, it seeks out a storm As if in storms it could find peace!

Edit: corrected formatting with dummy first table column.