Bitcoin Forum
March 29, 2024, 07:52:39 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Linux SysAdmin Help Debugging Slow Blockchain DL  (Read 187 times)
bagpuss (OP)
Newbie
*
Offline Offline

Activity: 2
Merit: 8


View Profile
September 22, 2019, 01:36:54 PM
Merited by bones261 (4), LoyceV (2), hugeblack (1), o_e_l_e_o (1)
 #1

I'm setting up a full bitcoin node on a raspberry pi and an external HDD - it's taken two weeks so far and still going. I'd like to learn how to debug where the bottleneck is. Any linux sysadmin folk can spare some time to guide me would be much appreciated.


RPi 3B+, 4GB RAM, Raspbian 4.19.66
USB3, to 2TB HDD
50Mbps up/down link

TOP
Tasks: 150 total,   1 running, 149 sleeping,   0 stopped,   0 zombie
%Cpu(s):  4.4 us,  2.0 sy,  0.0 ni, 74.0 id, 19.5 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   3906.0 total,     66.9 free,   1175.5 used,   2663.6 buff/cache
MiB Swap:    100.0 total,      0.4 free,     99.6 used.   2572.3 avail Mem

IOTOP
Total DISK READ:      1087.31 K/s | Total DISK WRITE:         0.00 B/s
Current DISK READ:    1093.92 K/s | Current DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND                        
 1686 be/4 pi       1087.31 K/s    0.00 B/s  0.00 % 88.99 % bitcoin-qt [bitcoin-msghand]
    1 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % init splash

IOSTAT -MX
Linux 4.19.66-v7l+ (xxx)      22/09/19        _armv7l_        (4 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           6.65    0.00    2.20   22.27    0.00   68.88

Device            r/s     w/s     rMB/s     wMB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util
mmcblk0          0.43    0.25      0.01      0.01     0.12     0.10  22.18  27.57    2.35   63.74   0.00    26.21    46.69   1.65   0.11
sda             83.01    4.43      0.94      1.06     0.01     5.06   0.01  53.35   17.57  137.77   1.92    11.56   244.82   9.16  80.12

1711698759
Hero Member
*
Offline Offline

Posts: 1711698759

View Profile Personal Message (Offline)

Ignore
1711698759
Reply with quote  #2

1711698759
Report to moderator
Each block is stacked on top of the previous one. Adding another block to the top makes all lower blocks more difficult to remove: there is more "weight" above each block. A transaction in a block 6 blocks deep (6 confirmations) will be very difficult to remove.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1711698759
Hero Member
*
Offline Offline

Posts: 1711698759

View Profile Personal Message (Offline)

Ignore
1711698759
Reply with quote  #2

1711698759
Report to moderator
LoyceV
Legendary
*
Online Online

Activity: 3262
Merit: 16316


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
September 22, 2019, 03:24:35 PM
 #2

From what I've seen, the HDD is most likely the bottleneck.
If you can use an SSD, that should give a large improvement. If you have one that's not big enough, it already helps a lot if you put only the "chainstate" directory on it.
You could also complete the download on a normal PC, then transfer the data directory to the Raspberry Pi when it's done.

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
bob123
Legendary
*
Offline Offline

Activity: 1624
Merit: 2481



View Profile WWW
September 22, 2019, 04:50:15 PM
Merited by hugeblack (1)
 #3

 TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND                        
 1686 be/4 pi       1087.31 K/s    0.00 B/s  0.00 % 88.99 % bitcoin-qt [bitcoin-msghand]


The bottleneck definitely is the hard drive.


If you don't want to get a SSD or don't want to use another computer to sync, you could still increase the syncing speed by increasing the database cache (dbcache parameter).

With 4GB of ram, 2GB dbcache would be a good value.
Either put it into the config file or start it with the parameter:

Code:
-dbcache=2048



bagpuss (OP)
Newbie
*
Offline Offline

Activity: 2
Merit: 8


View Profile
September 22, 2019, 04:54:25 PM
 #4

Thanks. How can the iops be so far off the stated 120 Mbps capability - 1Mbps is surely not the top of this drives performance?

https://www.amazon.com/gp/product/B00TKFEE5S/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1
ABCbits
Legendary
*
Offline Offline

Activity: 2828
Merit: 7327



View Profile
September 22, 2019, 05:24:22 PM
Merited by suchmoon (4)
 #5

Thanks. How can the iops be so far off the stated 120 Mbps capability - 1Mbps is surely not the top of this drives performance?

https://www.amazon.com/gp/product/B00TKFEE5S/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

120 MB/s is maximum speed (sequential speed), not random speed (which is far lower than sequential speed)

See https://stackoverflow.com/a/2471684 if you need detailed technical explanation

.
.BLACKJACK ♠ FUN.
█████████
██████████████
████████████
█████████████████
████████████████▄▄
░█████████████▀░▀▀
██████████████████
░██████████████
████████████████
░██████████████
████████████
███████████████░██
██████████
CRYPTO CASINO &
SPORTS BETTING
▄▄███████▄▄
▄███████████████▄
███████████████████
█████████████████████
███████████████████████
█████████████████████████
█████████████████████████
█████████████████████████
███████████████████████
█████████████████████
███████████████████
▀███████████████▀
█████████
.
Carlton Banks
Legendary
*
Offline Offline

Activity: 3430
Merit: 3068



View Profile
September 23, 2019, 01:07:23 AM
Last edit: September 23, 2019, 10:57:55 AM by Carlton Banks
Merited by suchmoon (4), malevolent (4), bones261 (4), hugeblack (1)
 #6

120 MB/s is maximum speed (sequential speed), not random speed (which is far lower than sequential speed)

indeed. very few workloads can actually make use of the sequential maximum speed, most are either random reads, or a mixture of random reads and writes. Not least Bitcoin, the blockchain is written sequentially, but you cannot possibly hope to verify the data at 120MB/s, lol (the Bitcoin devs have said that they've tested Bitcoin in "ideal" conditions with high end hardware, and cannot get blocks to verify faster than 50MB/s, although that was maybe 1 year ago).

The major disk performance bottleneck is computing the chainstate, which is a much more random workload than the blockchain, and what slows down anything that's RAM or disk speed restrained (such as a rasPi). Bitcoin chainstate (i.e. UTXO set) is constantly doing short burst of reads from the blocks and writes to the chainstate database. SSDs are best for that kind of usage pattern.




Make the chainstate (and the whole blockchain syncing) go faster thusly:

  • set dbcache= to at least 1GB (default is 450MB if you don't set it)
  • put your chainstate folder on an SSD, make a symlink to it in your .bitcoin directory (use the ln -s command)
  • put your swapfile on an SSD, make it equal or double your RAM, then disable the swapfile on the SD card (only if you're running from an SD card, of course)

you can even put the whole /dev/mmcblk0p2 partition (i.e. root partition) on the SSD, and leave /boot (i.e. /dev/mmcblk0p1 partition) on the SD card. But the biggest performance gains will come from the 3 steps above.

Vires in numeris
seoincorporation
Legendary
*
Offline Offline

Activity: 3108
Merit: 2855


Top Crypto Casino


View Profile
September 25, 2019, 04:17:05 AM
Last edit: September 27, 2019, 01:40:09 PM by seoincorporation
 #7


TOP
Tasks: 150 total,   1 running, 149 sleeping,   0 stopped,   0 zombie
%Cpu(s):  4.4 us,  2.0 sy,  0.0 ni, 74.0 id, 19.5 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   3906.0 total,     66.9 free,   1175.5 used,   2663.6 buff/cache



You could drop the cache ram with the next command as Root:

Code:
sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

This way you clean the buff/cache ram and get memory fo somer other new tasks.

source: https://www.commandlinefu.com/commands/view/9509/clear-filesystem-memory-cache

And LoyceV is right, an SSD would help a lot  Wink

█████████████████████████
████▐██▄█████████████████
████▐██████▄▄▄███████████
████▐████▄█████▄▄████████
████▐█████▀▀▀▀▀███▄██████
████▐███▀████████████████
████▐█████████▄█████▌████
████▐██▌█████▀██████▌████
████▐██████████▀████▌████
█████▀███▄█████▄███▀█████
███████▀█████████▀███████
██████████▀███▀██████████
█████████████████████████
.
BC.GAME
▄▄░░░▄▀▀▄████████
▄▄▄
██████████████
█████░░▄▄▄▄████████
▄▄▄▄▄▄▄▄▄██▄██████▄▄▄▄████
▄███▄█▄▄██████████▄████▄████
███████████████████████████▀███
▀████▄██▄██▄░░░░▄████████████
▀▀▀█████▄▄▄███████████▀██
███████████████████▀██
███████████████████▄██
▄███████████████████▄██
█████████████████████▀██
██████████████████████▄
.
..CASINO....SPORTS....RACING..
█░░░░░░█░░░░░░█
▀███▀░░▀███▀░░▀███▀
▀░▀░░░░▀░▀░░░░▀░▀
░░░░░░░░░░░░
▀██████████
░░░░░███░░░░
░░█░░░███▄█░░░
░░██▌░░███░▀░░██▌
░█░██░░███░░░█░██
░█▀▀▀█▌░███░░█▀▀▀█▌
▄█▄░░░██▄███▄█▄░░▄██▄
▄███▄
░░░░▀██▄▀


▄▄████▄▄
▄███▀▀███▄
██████████
▀███▄░▄██▀
▄▄████▄▄░▀█▀▄██▀▄▄████▄▄
▄███▀▀▀████▄▄██▀▄███▀▀███▄
███████▄▄▀▀████▄▄▀▀███████
▀███▄▄███▀░░░▀▀████▄▄▄███▀
▀▀████▀▀████████▀▀████▀▀
adroitful_one
Legendary
*
Offline Offline

Activity: 1568
Merit: 1041

1GhxHtabWhEpdb7e7oEJ2vd542n33BwTHR


View Profile
September 27, 2019, 02:10:14 AM
 #8

Which version of the Raspberry Pi are you using? If it's not a newer one with usb 3.0, that will really slow you down. I was mining burst with an 8 gb external on my solar in my camper just because I had everything to do it and wanted to play with it. It took several minutes to read the thing. So, if at all possible, upgrade that to a usb 3.0 and it will help tremendously.

Edit:Nevermind.. Seen the Pi 3 thing. I would use a desktop to sync the wallet first if at all possible. Should be fine after that.
Carlton Banks
Legendary
*
Offline Offline

Activity: 3430
Merit: 3068



View Profile
September 27, 2019, 09:14:53 AM
 #9

Which version of the Raspberry Pi are you using? If it's not a newer one with usb 3.0, that will really slow you down. I was mining burst with an 8 gb external on my solar in my camper just because I had everything to do it and wanted to play with it. It took several minutes to read the thing. So, if at all possible, upgrade that to a usb 3.0 and it will help tremendously.

it's not so much USB 3 as the new disk protocol that makes the Pi4 better.

USB 3 or 2 on their own only lets disks queue up 1 instruction simultaneously, whereas this new thing allows multiple disk commands to be sent in parallel over USB. You need a SATA hub (or an external drive) that talks the new protocol though, otherwise it won't work. any old sata disk will work over the protocol though, it's backwards compatible (at least back as far as SATA-II, you'd think, does anyone still have a SATA-I disk that works ? Huh)

trouble is, I forgot the name of this new USB disk protocol. UFA? totally forgot, sorry everyone Cheesy

Vires in numeris
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!