Bitcoin Forum
July 09, 2024, 03:01:40 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: using persistent volumes for certain files in data dir  (Read 157 times)
pyramation (OP)
Newbie
*
Offline Offline

Activity: 1
Merit: 0


View Profile
January 25, 2018, 10:08:16 PM
 #1

hey guys!

* TLDR; this is an attempt to make it faster to download the blockchain on servers running core or forks. so if you know a better solution, please point me to it.

I had an idea, but wanted to see if there's something already done for this that I'm unaware of. Imagine you want to deploy servers that are running core or forks of it. They all have similar data dir:

-rw-------   1 root root 342320183 Jan 25 22:03 blk0001.dat
drwx------   3 root root       102 Jan 25 05:12 database
-rw-------   1 root root         0 Jan 25 05:12 db.log
-rw-------   1 root root  28267003 Jan 25 22:03 debug.log
-rw-r--r--   1 root root        77 Jan 25 05:13 <yourfavoritecoinhere>.conf
-rw-------   1 root root     72026 Jan 25 22:03 peers.dat
-rw-------   1 root root       144 Jan 25 17:06 smsg.ini
drwx------  17 root root       578 Jan 25 19:30 smsgDB
drwx------ 134 root root      4556 Jan 25 22:03 txleveldb
-rw-------   1 root root     69632 Jan 25 21:24 wallet.dat

I was wondering if, there may be a way to have some of the blockchain related files actually coming from a persistent volume or stateful set, so you can basically mount (on a single kubernetes cluster) the same blockchain across multiple instances. It would be "centralized" in some sense, but only on your cluster - but the main point is actually to be able to spawn these services rapidly *without* having to download the entire blockchain.

Please let me know if this idea is worth pursuing of if there is a tried and true way people have been doing this! thanks Smiley
ScripterRon
Full Member
***
Offline Offline

Activity: 136
Merit: 120


View Profile
January 26, 2018, 12:16:48 PM
Merited by achow101 (2), ABCbits (2), Welsh (1)
 #2

The problem is database locking.  Bitcoin Core assumes it has complete control of its data directory.  So sharing a directory with multiple instances of Bitcoin Core would quickly result in data corruption.  The best you can do is periodically checkpoint the database directory (be sure to first stop the node using the directory) and then use the checkpoint to initialize the database directory for a new instance of Bitcoin Core.  I have done this several times to reduce the time it takes to create a new instance on a different system.
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!