Bitcoin Forum
September 25, 2018, 02:19:49 PM *
News: ♦♦ New info! Bitcoin Core users absolutely must upgrade to previously-announced 0.16.3 [Torrent]. All Bitcoin users should temporarily trust confirmations slightly less. More info.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: BitCoin core writes too fragmented data  (Read 588 times)
CyberDain
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 13, 2017, 04:24:28 PM
 #1

at most 100000 fragments after full sync

http://i.imgur.com/iPDBITz.png
can you fix it?
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1537885189
Hero Member
*
Offline Offline

Posts: 1537885189

View Profile Personal Message (Offline)

Ignore
1537885189
Reply with quote  #2

1537885189
Report to moderator
1537885189
Hero Member
*
Offline Offline

Posts: 1537885189

View Profile Personal Message (Offline)

Ignore
1537885189
Reply with quote  #2

1537885189
Report to moderator
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1526
Merit: 1656


3F1Y9yquzvY6RWvKbw2n2zeo9V5mvBhADU


View Profile WWW
November 13, 2017, 04:49:23 PM
 #2

This is not something that Bitcoin Core can control. It is entirely based upon the filesystem and how it allocates space to store data. Bitcoin Core itself has no control over that and cannot control where data is physically located on disk.

CyberDain
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 13, 2017, 05:20:27 PM
 #3

disagree. Writing many files simultaneously produce huge amount of fragmented files, that cause slow sync.
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1526
Merit: 1656


3F1Y9yquzvY6RWvKbw2n2zeo9V5mvBhADU


View Profile WWW
November 13, 2017, 05:34:53 PM
 #4

disagree. Writing many files simultaneously produce huge amount of fragmented files, that cause slow sync.
The files are not written simultaneously.

CyberDain
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 13, 2017, 06:13:14 PM
 #5

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
LoyceV
Legendary
*
Offline Offline

Activity: 1246
Merit: 1995


Let's make Bitcointalk great again!


View Profile WWW
November 13, 2017, 07:55:13 PM
 #6

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
uTorrent allocates space upfront, for a file with a known file size. Bitcoin Core writes files year after year, it writes wherever your disk has some space available at that moment. And since you use your computer for other tasks too, the "empty spots" on your disk change location. It would make no sense to reserve 500 GB upfront when you first install Bitcoin Core, to prevent fragmentation in the years to come.

The easiest solution is simply not to worry about it, your computer doesn't need to defragment files all the time.
But if you really want to, move Bitcoin Core to a different disk, and it will be completely sequential.

The sync speed - in my experience - mainly depends on your system's memory, and how efficiently your computer uses it. For comparison: I get about 1 block per second, on an i3 with hdd, Linux and 12 GB RAM.

achow101
Moderator
Legendary
*
Offline Offline

Activity: 1526
Merit: 1656


3F1Y9yquzvY6RWvKbw2n2zeo9V5mvBhADU


View Profile WWW
November 13, 2017, 10:51:32 PM
 #7

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
No, you just don't understand how Bitcoin Core works. It is not like uTorrent where the file sizes are known and it can allocate space accordingly. Rather what happens is that space is allocated for each block as it arrives. The blockchain is not one massive file that Bitcoin Core must download. It is a bunch of individual blocks that are recorded to disk separately. This occurs for both initial sync and during normal operation because it does not know the full blockchain size or even block sizes beforehand. It just allocates space as it needs it. Since it is constantly appending to files and requesting more space as it appends to each file. This is why it cannot use a continuous space: it does not know how much it will need and cannot request it all. Furthermore, as LoyceV points out, because Bitcoin Core is designed to run continuously and must write new blocks to disk as they are found, it will result in some fragmented data since it reserves space as it needs it, and presumably you are doing other stuff on the computer that also reserves space as it needs it.

CyberDain
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 14, 2017, 04:05:50 AM
 #8

I supposed size of files are approximately known

https://i.imgur.com/Z3sALsK.png
CyberDain
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 19, 2017, 08:11:52 AM
 #9

so I think best practice is shrink HDD to 200gb for BTC db, then BTC will not write to middle of whole drive and decrease overall system performance.
LoyceV
Legendary
*
Offline Offline

Activity: 1246
Merit: 1995


Let's make Bitcointalk great again!


View Profile WWW
November 19, 2017, 08:24:58 AM
 #10

so I think best practice is shrink HDD to 200gb for BTC db, then BTC will not write to middle of whole drive and decrease overall system performance.
Why do you think some fragmentation reduces your overall system performance? It only influences sequential reads, which is something you barely do, unless you're copying large files.
If you create a separate partition on the same disk, the read arm of the hdd has to travel to that location anway, in those cases is can be faster if the data is all closer together.
Your 200 GB partition will be filled soon anyway, after which you'll need to increase it again.

If you really want to increase your system performance, install a SSD.

toastyhamster
Newbie
*
Offline Offline

Activity: 12
Merit: 0


View Profile
November 19, 2017, 04:24:50 PM
 #11

What's typical sync time?  Due to space constraints and for availability I'm experimenting running Core on a low spec Celeron with RAID space for media etc, currently on day 3 of sync with 2 more left apparently!

Fragmentation looks ok but I did start with loads of disk space which would have helped.
slate_main
Member
**
Offline Offline

Activity: 131
Merit: 11

Slate, Unrestricted global entertainment access


View Profile WWW
November 21, 2017, 06:09:26 PM
 #12

What software is that fascinating looking drive mapping/sector tool?

CyberDain
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile
November 22, 2017, 10:56:24 AM
 #13

What software is that fascinating looking drive mapping/sector tool?
UltimateDefrag
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!