Bitcoin Forum
September 27, 2016, 05:15:54 PM *
News: Due to DDoS attacks, there may be periodic downtime.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: RFC: remove DB_PRIVATE flag  (Read 1372 times)
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 25, 2010, 01:03:13 AM
 #1

Wallet backups and other DB examination are easily possible in a safe, atomic, transactional fashion...   if and only if DB_PRIVATE flag is removed.

Code:
--- a/db.cpp
+++ b/db.cpp
@@ -77,7 +77,6 @@ CDB::CDB(const char* pszFile, const char* pszMode) : pdb(NULL)
                              DB_INIT_MPOOL |
                              DB_INIT_TXN   |
                              DB_THREAD     |
-                             DB_PRIVATE    |
                              DB_RECOVER,
                              S_IRUSR | S_IWUSR);


What, if any, problems arise from doing this?  Obviously, this does not cover the non-db4 databases such as the block data file.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
1474996554
Hero Member
*
Offline Offline

Posts: 1474996554

View Profile Personal Message (Offline)

Ignore
1474996554
Reply with quote  #2

1474996554
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1474996554
Hero Member
*
Offline Offline

Posts: 1474996554

View Profile Personal Message (Offline)

Ignore
1474996554
Reply with quote  #2

1474996554
Report to moderator
1474996554
Hero Member
*
Offline Offline

Posts: 1474996554

View Profile Personal Message (Offline)

Ignore
1474996554
Reply with quote  #2

1474996554
Report to moderator
satoshi
Founder
Sr. Member
*
qt
Offline Offline

Activity: 364


View Profile
August 26, 2010, 12:33:28 AM
 #2

Can you provide more details about what removing DB_PRIVATE does?

I can't remember if I had a specific reason for DB_PRIVATE, or if I just copied the flags from some example code.  Does removing DB_PRIVATE make it safe for other processes to open the database simultaneously?  That may be an improvement, depending what the side effects are.  Does it substantially reduce performance by making it have to write out every change immediately or do other coordination?  Are there additional locking or coordination files then?  What else changes?  You could test by timing an initial block download with and without DB_PRIVATE, preferably -connect-ing to a local machine so network isn't a factor.

Apparently, DB_PRIVATE doesn't do what you would hope it would do, which is prevent other processes from being able to open the database.  It still lets them, it just screws up if they do.  Another option, if there's a way, would be to make it lock the database files so they can't be accessed by other processes.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 26, 2010, 12:39:24 AM
 #3


DB_PRIVATE enables a few optimizations by making the assumption that only one process will access the db4 database.  Notably, this flag enables db4 to use pthreads-style mutex locking rather than heavy, operating-system-provided flock and shared memory.  Ref: DB_ENV->open documentation.

The general motivation is that db4 databases can be safely accessed in parallel with bitcoin client, assuming (a) DB_PRIVATE is removed and (b) bitcoin properly uses db4 transactions.  db4 transactions may even be employed to wrap around non-db4 data such as blk0001.dat, if the code is properly architected.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
satoshi
Founder
Sr. Member
*
qt
Offline Offline

Activity: 364


View Profile
September 12, 2010, 06:00:39 PM
 #4

Trying it without the DB_PRIVATE flag in rev 153.  We need to keep an eye on what's different.

On Windows at least, it creates six __db.001 - __db.006 files with sizes from 24K to 4MB.  It doesn't delete them on exit, it just leaves them behind.

The docs say it uses memory mapped files.  I assume they have the same file permissions as the database files, so the same user access restrictions apply.

Tests on Windows private LAN download of 78500 blocks:
with DB_PRIVATE     20 minutes 51 seconds
without DB_PRIVATE   20 minutes 51 seconds

I wasn't expecting them to come out exactly the same.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
September 12, 2010, 08:24:48 PM
 #5


Great, thanks!  Removing DB_PRIVATE should enable useful tools like gavin's bitcointools.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!