bitcoind 0.9.2 uses 800MB of RAM at 30 connections!

[monsterer]

Hi guys,

I just upgraded to bitcoind 0.9.2 hoping to save some memory from version 0.8.4 only to discover it uses more RAM!

What on earth is it doing that requires 800MB of non-virtual memory?

Is there any way to reduce this number, as its 20% of my 4GB total!

Cheers, Paul.

[DeathAndTaxes]

The memory pool is getting larger as is the UTXO (the UTXO isn't in memory but the db does cache it to improve performance - thanks gmaxwell).  Honestly memory usage is going to continue to rise.  I know this sounds like a cop-out but RAM is dirt cheap why not go to 8GB?  Any optimizations you make now are likely to be minimal and if Bitcoin continues to grow so will the memory pressure.

Still 800MB does seem high for 30 connections.  What OS? How many addresses & txns in the wallet?  Addresses more than txn but both seem to increase memory.

There may be a "luck of the draw" element as I have 128 connections right now and memory is <700 MB.  If by luck your peers are making a more diverse range of requests it could be causing the db to cache more heavily?  You can't control who connects to you but you can cap the number of connections.  16 is a good number as you are making available as many inbound slots are you are using outbound.

There is also a command line or configuration file option "dbcache=x" where x is the max db cache size in MBs.  By default it is 100 MB.  Warning I have only increased not decreased the value so I would test the effects on performance in a non-production environment.

[monsterer]

The memory pool is getting larger as is the UTXO (the UTXO isn't in memory but the db does cache it to improve performance - thanks gmaxwell).  Honestly memory usage is going to continue to rise.  I know this sounds like a cop-out but RAM is dirt cheap why not go to 8GB?  Any optimizations you make now are likely to be minimal and if Bitcoin continues to grow so will the memory pressure.

Still 800MB does seem high for 30 connections.  What OS? How many addresses & txns in the wallet?  Addresses more than txn but both seem to increase memory.

There may be a "luck of the draw" element as I have 128 connections right now and memory is <700 MB.  If by luck your peers are making a more diverse range of requests it could be causing the db to cache more heavily?  You can't control who connects to you but you can cap the number of connections.  16 is a good number as you are making available as many inbound slots are you are using outbound.

There is also a command line or configuration file option "dbcache=x" where x is the max db cache size in MBs.  By default it is 100 MB.  Warning I have only increased not decreased the value so I would test the effects on performance in a non-production environment.

10 transactions and around 100 addresses. Centos 6.

Increasing RAM is one way, but it'll add to my hosting bill. :|

Cheers, Paul.

[DeathAndTaxes]

Yeah I am at a loss unless it is just bad luck due to the peer mix.  I assumed it was a much larger wallet (thousands of keys and transactions).

[azeteki]

This interests me as I'm rather confused as to what seems to be causing various users to report different memory usage.

A few people on IRC seem to have ~300MB or less. I personally report over 700MB on mainnet.

If the low memory usage reports are indeed legitimate it would be useful to track this down.

[Cranky4u]

If developers think 800MB is acceptable and trending higher is expected, how in hell does the BTC community expect to become an everyday mobile app for commerce?

[DeathAndTaxes]

If developers think 800MB is acceptable and trending higher is expected, how in hell does the BTC community expect to become an everyday mobile app for commerce?

Bitcoind doesn't run on any mobile platform.  What mobile app runs a full node?  None?  Full nodes are the backbone of the network.

[Remember remember the 5th of November]

If developers think 800MB is acceptable and trending higher is expected, how in hell does the BTC community expect to become an everyday mobile app for commerce?

Bitcoind doesn't run on any mobile platform.  What mobile app runs a full node?  None?  Full nodes are the backbone of the network.

Why not trade performance(i.e the caching in memory) for...get this, performance! Well, performance of the OS, and let pool operators modify the client to fit their needs. This way, Bitcoin Core would use less RAM for regular people willing to run a full node.

[monsterer]

Is it possible to have linux limit the maximum ram usage on a per-process basis?

Wonder whether bitcoind would run ok with only 100MB of real ram and the rest virtual?

[Meuh6879]

decrease CACHE to 10Mo (or 5Mo).
start inline command with 12 connexions max.

https://en.bitcoin.it/wiki/Running_Bitcoin

[Newar]

Some more numbers:

0.9.2.1 --disable-wallet
65 connections

top:
220 MB or ~10%

smaps:
(http://superuser.com/questions/102005/how-can-i-display-the-memory-usage-of-each-process-if-i-do-a-ps-ef)
* Mem usage for PID 17843
-- Size:
132
-- Rss:
56
-- Pss:
56

I run it with some niceness, although that should not affect memory use.

Is it possible to have linux limit the maximum ram usage on a per-process basis?

ulimit?
http://linuxcommand.org/man_pages/ulimit1.html

ulimit [-SHacdflmnpstuv [limit]]
              Provides control over the resources available to the  shell  and
              to  processes started by it, on systems that allow such control.