Bitcoin Forum
December 13, 2017, 09:00:56 PM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: BitCoin core writes too fragmented data  (Read 583 times)
CyberDain
Newbie
*
Offline Offline

Activity: 6


View Profile
November 13, 2017, 04:24:28 PM
 #1

at most 100000 fragments after full sync

http://i.imgur.com/iPDBITz.png
can you fix it?
1513198856
Hero Member
*
Offline Offline

Posts: 1513198856

View Profile Personal Message (Offline)

Ignore
1513198856
Reply with quote  #2

1513198856
Report to moderator
1513198856
Hero Member
*
Offline Offline

Posts: 1513198856

View Profile Personal Message (Offline)

Ignore
1513198856
Reply with quote  #2

1513198856
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1513198856
Hero Member
*
Offline Offline

Posts: 1513198856

View Profile Personal Message (Offline)

Ignore
1513198856
Reply with quote  #2

1513198856
Report to moderator
1513198856
Hero Member
*
Offline Offline

Posts: 1513198856

View Profile Personal Message (Offline)

Ignore
1513198856
Reply with quote  #2

1513198856
Report to moderator
1513198856
Hero Member
*
Offline Offline

Posts: 1513198856

View Profile Personal Message (Offline)

Ignore
1513198856
Reply with quote  #2

1513198856
Report to moderator
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
November 13, 2017, 04:49:23 PM
 #2

This is not something that Bitcoin Core can control. It is entirely based upon the filesystem and how it allocates space to store data. Bitcoin Core itself has no control over that and cannot control where data is physically located on disk.

CyberDain
Newbie
*
Offline Offline

Activity: 6


View Profile
November 13, 2017, 05:20:27 PM
 #3

disagree. Writing many files simultaneously produce huge amount of fragmented files, that cause slow sync.
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
November 13, 2017, 05:34:53 PM
 #4

disagree. Writing many files simultaneously produce huge amount of fragmented files, that cause slow sync.
The files are not written simultaneously.

CyberDain
Newbie
*
Offline Offline

Activity: 6


View Profile
November 13, 2017, 06:13:14 PM
 #5

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
LoyceV
Legendary
*
Online Online

Activity: 966


Howdy


View Profile
November 13, 2017, 07:55:13 PM
 #6

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
uTorrent allocates space upfront, for a file with a known file size. Bitcoin Core writes files year after year, it writes wherever your disk has some space available at that moment. And since you use your computer for other tasks too, the "empty spots" on your disk change location. It would make no sense to reserve 500 GB upfront when you first install Bitcoin Core, to prevent fragmentation in the years to come.

The easiest solution is simply not to worry about it, your computer doesn't need to defragment files all the time.
But if you really want to, move Bitcoin Core to a different disk, and it will be completely sequential.

The sync speed - in my experience - mainly depends on your system's memory, and how efficiently your computer uses it. For comparison: I get about 1 block per second, on an i3 with hdd, Linux and 12 GB RAM.

achow101
Moderator
Legendary
*
Offline Offline

Activity: 1246


17kKQppUsngUiByDsce4JXoZEjjpvX9bpR


View Profile WWW
November 13, 2017, 10:51:32 PM
 #7

may be.
But DB files too fragmented.
and for example uTorrent writes clean continuous blocks from beginning of drive as you can see on picture,  even thousand files in torrents.
So I think this is bug of bitcoin core .
No, you just don't understand how Bitcoin Core works. It is not like uTorrent where the file sizes are known and it can allocate space accordingly. Rather what happens is that space is allocated for each block as it arrives. The blockchain is not one massive file that Bitcoin Core must download. It is a bunch of individual blocks that are recorded to disk separately. This occurs for both initial sync and during normal operation because it does not know the full blockchain size or even block sizes beforehand. It just allocates space as it needs it. Since it is constantly appending to files and requesting more space as it appends to each file. This is why it cannot use a continuous space: it does not know how much it will need and cannot request it all. Furthermore, as LoyceV points out, because Bitcoin Core is designed to run continuously and must write new blocks to disk as they are found, it will result in some fragmented data since it reserves space as it needs it, and presumably you are doing other stuff on the computer that also reserves space as it needs it.

CyberDain
Newbie
*
Offline Offline

Activity: 6


View Profile
November 14, 2017, 04:05:50 AM
 #8

I supposed size of files are approximately known

https://i.imgur.com/Z3sALsK.png
CyberDain
Newbie
*
Offline Offline

Activity: 6


View Profile
November 19, 2017, 08:11:52 AM
 #9

so I think best practice is shrink HDD to 200gb for BTC db, then BTC will not write to middle of whole drive and decrease overall system performance.
LoyceV
Legendary
*
Online Online

Activity: 966


Howdy


View Profile
November 19, 2017, 08:24:58 AM
 #10

so I think best practice is shrink HDD to 200gb for BTC db, then BTC will not write to middle of whole drive and decrease overall system performance.
Why do you think some fragmentation reduces your overall system performance? It only influences sequential reads, which is something you barely do, unless you're copying large files.
If you create a separate partition on the same disk, the read arm of the hdd has to travel to that location anway, in those cases is can be faster if the data is all closer together.
Your 200 GB partition will be filled soon anyway, after which you'll need to increase it again.

If you really want to increase your system performance, install a SSD.

toastyhamster
Newbie
*
Offline Offline

Activity: 11


View Profile
November 19, 2017, 04:24:50 PM
 #11

What's typical sync time?  Due to space constraints and for availability I'm experimenting running Core on a low spec Celeron with RAID space for media etc, currently on day 3 of sync with 2 more left apparently!

Fragmentation looks ok but I did start with loads of disk space which would have helped.
slate_main
Member
**
Offline Offline

Activity: 98


View Profile
November 21, 2017, 06:09:26 PM
 #12

What software is that fascinating looking drive mapping/sector tool?
CyberDain
Newbie
*
Offline Offline

Activity: 6


View Profile
November 22, 2017, 10:56:24 AM
 #13

What software is that fascinating looking drive mapping/sector tool?
UltimateDefrag
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!