Bitcoin Forum
November 01, 2024, 01:03:07 AM *
News: Bitcoin Pumpkin Carving Contest
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Bitcoind taking up 380megs or more.. ??  (Read 1331 times)
gateway (OP)
Hero Member
*****
Offline Offline

Activity: 552
Merit: 500


View Profile
July 31, 2013, 08:23:28 PM
 #1

Hi, im running the release  "version" : 80300, compiled from source from github on a ubuntu 12.04.x server

After all the data was downloaded the memory usage sits at around 380megs .. any reason this is so high w/o anything going on yet.. ?



and htop says..



deepceleron
Legendary
*
Offline Offline

Activity: 1512
Merit: 1036



View Profile WWW
August 02, 2013, 07:08:08 PM
 #2

You have the memory available. This is likely LevelDB using caches and write buffers to speed things up. Some may want Bitcoin to minimize it's RAM usage, while others may want Bitcoin to run as fast and low latency as possible, so there is no right answer; the default LevelDB memory parameter options are used.
grue
Legendary
*
Offline Offline

Activity: 2058
Merit: 1434



View Profile
August 02, 2013, 08:10:04 PM
 #3

https://i.minus.com/iwmmJsMV3BQnU.png

You have the memory available. This is likely LevelDB using caches and write buffers to speed things up. Some may want Bitcoin to minimize it's RAM usage, while others may want Bitcoin to run as fast and low latency as possible, so there is no right answer; the default LevelDB memory parameter options are used.
dbcache is set to 25 mb by default, so this can't be the cause.

It is pitch black. You are likely to be eaten by a grue.

Adblock for annoying signature ads | Enhanced Merit UI
deepceleron
Legendary
*
Offline Offline

Activity: 1512
Merit: 1036



View Profile WWW
August 02, 2013, 08:45:01 PM
 #4

https://i.minus.com/iwmmJsMV3BQnU.png

You have the memory available. This is likely LevelDB using caches and write buffers to speed things up. Some may want Bitcoin to minimize it's RAM usage, while others may want Bitcoin to run as fast and low latency as possible, so there is no right answer; the default LevelDB memory parameter options are used.
dbcache is set to 25 mb by default, so this can't be the cause.

Where?

Write buffer=max 4mb (per file), 129 sst files currently...

src\src\leveldb\include\leveldb\options.h ->

 
Code:
 // Amount of data to build up in memory (backed by an unsorted log
  // on disk) before converting to a sorted on-disk file.
  //
  // Larger values increase performance, especially during bulk loads.
  // Up to two write buffers may be held in memory at the same time,
  // so you may wish to adjust this parameter to control memory usage.
  // Also, a larger write buffer will result in a longer recovery time
  // the next time the database is opened.
  //
  // Default: 4MB
  size_t write_buffer_size;

  // Number of open files that can be used by the DB.  You may need to
  // increase this if your database has a large working set (budget
  // one open file per 2MB of working set).
  //
  // Default: 1000
  int max_open_files;

a quote
Quote
There are other sources of memory usage:
  8MB - the default cache
  4MB - write buffer (though should only build up if you do 4MB worth of writing)
  very large - Unix buffer cache usage for opened files (they are mmapped). If you want to trim this, try tweaking MmapLimiter in leveldb/util/env_posix.cc so it initializes the "allowed mmaps" to 0 instead of 1000.  Though this is mostly an accounting issue.  The files will be sitting in the buffer cache regardless I think.


I think Bitcoin instantly became one of the largest data sets of LevelDB, with 10GB x a lot of users.
grue
Legendary
*
Offline Offline

Activity: 2058
Merit: 1434



View Profile
August 05, 2013, 07:30:50 PM
 #5


-dbcache=<n>           Set database cache size in megabytes (default: 25)

It is pitch black. You are likely to be eaten by a grue.

Adblock for annoying signature ads | Enhanced Merit UI
gateway (OP)
Hero Member
*****
Offline Offline

Activity: 552
Merit: 500


View Profile
August 05, 2013, 08:19:09 PM
 #6


-dbcache=<n>           Set database cache size in megabytes (default: 25)

is that for cmd line or config, i do not see these settings in bitcoind help

Diapolo
Hero Member
*****
Offline Offline

Activity: 772
Merit: 500



View Profile WWW
August 05, 2013, 08:46:41 PM
 #7


-dbcache=<n>           Set database cache size in megabytes (default: 25)

is that for cmd line or config, i do not see these settings in bitcoind help

Cmd line or config, whatever you want.

Dia

Liked my former work for Bitcoin Core? Drop me a donation via:
1PwnvixzVAKnAqp8LCV8iuv7ohzX2pbn5x
bitcoin:1PwnvixzVAKnAqp8LCV8iuv7ohzX2pbn5x?label=Diapolo
zvs
Legendary
*
Offline Offline

Activity: 1680
Merit: 1000


https://web.archive.org/web/*/nogleg.com


View Profile WWW
August 07, 2013, 01:00:33 AM
 #8

After all the data was downloaded the memory usage sits at around 380megs .. any reason this is so high w/o anything going on yet.. ?

transactions stored in memory, orphans

kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1026



View Profile
August 07, 2013, 01:19:33 PM
 #9

a quote
Quote
There are other sources of memory usage:
  8MB - the default cache
  4MB - write buffer (though should only build up if you do 4MB worth of writing)
  very large - Unix buffer cache usage for opened files (they are mmapped). If you want to trim this, try tweaking MmapLimiter in leveldb/util/env_posix.cc so it initializes the "allowed mmaps" to 0 instead of 1000.  Though this is mostly an accounting issue.  The files will be sitting in the buffer cache regardless I think.

I think some people missed this, or don't understand how mmap works, or don't understand how unix systems account for memory usage.

Free memory is wasted memory.  Allocating memory is all about updating table entries, and it takes no more time to change a page from "buffer cache" to "heap" than it does to change it from "free" to "heap".

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
twobits
Sr. Member
****
Offline Offline

Activity: 574
Merit: 250



View Profile
August 08, 2013, 02:15:15 PM
 #10


-dbcache=<n>           Set database cache size in megabytes (default: 25)

dbcache is from the bdb only days, and sets the bdb cache size last I checked.

With that now just the wallet file, not much need to set this.


█████                █████      ███████             
█████                ███    █████████████       
█████                ██  █████████████████   
█████                █  ██████              ██████ 
█████                    ████                      ████ 
█████████████  █████                        ████
█████████████  █████                        ████
█████████████  █████                        ████
█████                    █████                             
█████                █  ██████              ███████
█████                ██  ███████████    █████ 
█████                ███    █████████    ████   
█████                █████      ███████    ██
███
███
███
███
███
███
███
███
███
HyperQuant.net
Platform for Professional Asset Management
███
███
███
███
███
███
███
███
███
WhitePaper
One-Pager
███
███
███
███
███
███
███
███
███
Telegram 
Facebook
Twitter
Medium
███
███
███
███
███
███
███
███
███
███
███
███
███
███
███
███
███
███
█████                █████      ███████             
█████                ███    █████████████       
█████                ██  █████████████████   
█████                █  ██████              ██████ 
█████                    ████                      ████ 
█████████████  █████                        ████
█████████████  █████                        ████
█████████████  █████                        ████
█████                    █████                             
█████                █  ██████              ███████
█████                ██  ███████████    █████ 
█████                ███    █████████    ████   
█████                █████      ███████    ██
grue
Legendary
*
Offline Offline

Activity: 2058
Merit: 1434



View Profile
August 08, 2013, 02:33:00 PM
 #11


-dbcache=<n>           Set database cache size in megabytes (default: 25)

dbcache is from the bdb only days, and sets the bdb cache size last I checked.

With that now just the wallet file, not much need to set this.


False.

http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.1/
release notes for 0.8.0
Quote
New/changed settings (command-line or bitcoin.conf file)
--------------------------------------------------------
dbcache : controls LevelDB memory usage.

It is pitch black. You are likely to be eaten by a grue.

Adblock for annoying signature ads | Enhanced Merit UI
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!