Bitcoin Forum
December 11, 2024, 07:37:30 PM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: BitCoin core writes too fragmented data  (Read 695 times)
CyberDain (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 13, 2017, 04:24:28 PM
Last edit: November 13, 2017, 04:34:59 PM by CyberDain
 #1

at most 100000 fragments after full sync

https://i.imgur.com/iPDBITz.png
can you fix it?
achow101
Moderator
Legendary
*
Offline Offline

Activity: 3570
Merit: 6927


Just writing some code


View Profile WWW
November 13, 2017, 04:49:23 PM
 #2

This is not something that Bitcoin Core can control. It is entirely based upon the filesystem and how it allocates space to store data. Bitcoin Core itself has no control over that and cannot control where data is physically located on disk.

CyberDain (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 13, 2017, 05:20:27 PM
 #3

disagree. Writing many files simultaneously produce huge amount of fragmented files, that cause slow sync.
achow101
Moderator
Legendary
*
Offline Offline

Activity: 3570
Merit: 6927


Just writing some code


View Profile WWW
November 13, 2017, 05:34:53 PM
 #4

disagree. Writing many files simultaneously produce huge amount of fragmented files, that cause slow sync.
The files are not written simultaneously.

CyberDain (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 13, 2017, 06:13:14 PM
 #5

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
LoyceV
Legendary
*
Offline Offline

Activity: 3528
Merit: 17817


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
November 13, 2017, 07:55:13 PM
 #6

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
uTorrent allocates space upfront, for a file with a known file size. Bitcoin Core writes files year after year, it writes wherever your disk has some space available at that moment. And since you use your computer for other tasks too, the "empty spots" on your disk change location. It would make no sense to reserve 500 GB upfront when you first install Bitcoin Core, to prevent fragmentation in the years to come.

The easiest solution is simply not to worry about it, your computer doesn't need to defragment files all the time.
But if you really want to, move Bitcoin Core to a different disk, and it will be completely sequential.

The sync speed - in my experience - mainly depends on your system's memory, and how efficiently your computer uses it. For comparison: I get about 1 block per second, on an i3 with hdd, Linux and 12 GB RAM.

▄▄███████████████████▄▄
▄█████████▀█████████████▄
███████████▄▐▀▄██████████
███████▀▀███████▀▀███████
██████▀███▄▄████████████
█████████▐█████████▐█████
█████████▐█████████▐█████
██████████▀███▀███▄██████
████████████████▄▄███████
███████████▄▄▄███████████
█████████████████████████
▀█████▄▄████████████████▀
▀▀███████████████████▀▀
Peach
BTC bitcoin
Buy and Sell
Bitcoin P2P
.
.
▄▄███████▄▄
▄████████
██████▄
▄██
█████████████████▄
▄███████
██████████████▄
███████████████████████
█████████████████████████
████████████████████████
█████████████████████████
▀███████████████████████▀
▀█████████████████████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀

▀▀▀▀███▀▀▀▀
EUROPE | AFRICA
LATIN AMERICA
▄▀▀▀











▀▄▄▄


███████▄█
███████▀
██▄▄▄▄▄░▄▄▄▄▄
████████████▀
▐███████████▌
▐███████████▌
████████████▄
██████████████
███▀███▀▀███▀
.
Download on the
App Store
▀▀▀▄











▄▄▄▀
▄▀▀▀











▀▄▄▄


▄██▄
██████▄
█████████▄
████████████▄
███████████████
████████████▀
█████████▀
██████▀
▀██▀
.
GET IT ON
Google Play
▀▀▀▄











▄▄▄▀
achow101
Moderator
Legendary
*
Offline Offline

Activity: 3570
Merit: 6927


Just writing some code


View Profile WWW
November 13, 2017, 10:51:32 PM
 #7

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
No, you just don't understand how Bitcoin Core works. It is not like uTorrent where the file sizes are known and it can allocate space accordingly. Rather what happens is that space is allocated for each block as it arrives. The blockchain is not one massive file that Bitcoin Core must download. It is a bunch of individual blocks that are recorded to disk separately. This occurs for both initial sync and during normal operation because it does not know the full blockchain size or even block sizes beforehand. It just allocates space as it needs it. Since it is constantly appending to files and requesting more space as it appends to each file. This is why it cannot use a continuous space: it does not know how much it will need and cannot request it all. Furthermore, as LoyceV points out, because Bitcoin Core is designed to run continuously and must write new blocks to disk as they are found, it will result in some fragmented data since it reserves space as it needs it, and presumably you are doing other stuff on the computer that also reserves space as it needs it.

CyberDain (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 14, 2017, 04:05:50 AM
 #8

I supposed size of files are approximately known

https://i.imgur.com/Z3sALsK.png
CyberDain (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 19, 2017, 08:11:52 AM
 #9

so I think best practice is shrink HDD to 200gb for BTC db, then BTC will not write to middle of whole drive and decrease overall system performance.
LoyceV
Legendary
*
Offline Offline

Activity: 3528
Merit: 17817


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
November 19, 2017, 08:24:58 AM
 #10

so I think best practice is shrink HDD to 200gb for BTC db, then BTC will not write to middle of whole drive and decrease overall system performance.
Why do you think some fragmentation reduces your overall system performance? It only influences sequential reads, which is something you barely do, unless you're copying large files.
If you create a separate partition on the same disk, the read arm of the hdd has to travel to that location anway, in those cases is can be faster if the data is all closer together.
Your 200 GB partition will be filled soon anyway, after which you'll need to increase it again.

If you really want to increase your system performance, install a SSD.

▄▄███████████████████▄▄
▄█████████▀█████████████▄
███████████▄▐▀▄██████████
███████▀▀███████▀▀███████
██████▀███▄▄████████████
█████████▐█████████▐█████
█████████▐█████████▐█████
██████████▀███▀███▄██████
████████████████▄▄███████
███████████▄▄▄███████████
█████████████████████████
▀█████▄▄████████████████▀
▀▀███████████████████▀▀
Peach
BTC bitcoin
Buy and Sell
Bitcoin P2P
.
.
▄▄███████▄▄
▄████████
██████▄
▄██
█████████████████▄
▄███████
██████████████▄
███████████████████████
█████████████████████████
████████████████████████
█████████████████████████
▀███████████████████████▀
▀█████████████████████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀

▀▀▀▀███▀▀▀▀
EUROPE | AFRICA
LATIN AMERICA
▄▀▀▀











▀▄▄▄


███████▄█
███████▀
██▄▄▄▄▄░▄▄▄▄▄
████████████▀
▐███████████▌
▐███████████▌
████████████▄
██████████████
███▀███▀▀███▀
.
Download on the
App Store
▀▀▀▄











▄▄▄▀
▄▀▀▀











▀▄▄▄


▄██▄
██████▄
█████████▄
████████████▄
███████████████
████████████▀
█████████▀
██████▀
▀██▀
.
GET IT ON
Google Play
▀▀▀▄











▄▄▄▀
toastyhamster
Newbie
*
Offline Offline

Activity: 12
Merit: 0


View Profile
November 19, 2017, 04:24:50 PM
 #11

What's typical sync time?  Due to space constraints and for availability I'm experimenting running Core on a low spec Celeron with RAID space for media etc, currently on day 3 of sync with 2 more left apparently!

Fragmentation looks ok but I did start with loads of disk space which would have helped.
slate_main
Member
**
Offline Offline

Activity: 105
Merit: 11

BYTZ


View Profile
November 21, 2017, 06:09:26 PM
 #12

What software is that fascinating looking drive mapping/sector tool?
CyberDain (OP)
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 22, 2017, 10:56:24 AM
 #13

What software is that fascinating looking drive mapping/sector tool?
UltimateDefrag
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!