Bitcoin Forum
December 02, 2016, 06:24:12 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: CreateNewBlock too slow (especially while bitcoin is under tx spam attack)  (Read 3442 times)
molecular
Donator
Legendary
*
Offline Offline

Activity: 2128



View Profile
March 23, 2011, 10:49:17 PM
 #1

I'm describing the problem here from my personal perspective, bitcoind running on my desktop, which has a slow atom cpu and is frequently under various other load conditions), poclbm running on dedicated miner with a 5970.

Problem is this: after running bitoind for a while, I see "Problems communicating with bitcoin RPC, data: None" messages on the miner. The freuqency of these increases over time (presumably with size of bitcoind's transaction cache). Getwork interval is at 5s.

It has been suggested to used git-version with -limitfreerelay=1. I tried that and it might have taken some pressure, but the problem still occurs.

After a couple of hours my miner doesn't do any work any more, because getwork continuously fails.

I'm quoting from #bitcoin-dev to elaborate the problem and possible workarounds and/or solutions:

Quote from: #bitcoin-dev
<molecular> ArtForz, am running into slight getwork troubles at 66 tx cache size already? does that make sense?
<ArtForz> nope
<tcatm> molecular: what's the trouble? getwork taking long time to return?
<ArtForz> do you have a really slow disk or high I/O load?
<molecular> yes
<molecular> getwork takes several seconds
<molecular> at some point it takes longer the my getwork interval (currently 5s)
<molecular> I have slowish cpu (atom)
<ArtForz> yeah, that might do it
<molecular> which is also under load from other shit since it's my desktop

this made sense to me, ArtForz kept analysing:

Quote from: #bitcoin-dev
<ArtForz> can you check if it's actually pegging the CPU?
<ArtForz> because here it seemed to be more I/O than CPU bound
<molecular> I can't see it using cpu in htop. trying to verify that
<molecular> there's some iotop app? what's it called?
<ArtForz> iotop ?

I'm emerging iotop on my desktop, while the chatter continues:

Quote from: #bitcoin-dev
<ArtForz> what stalls getwork is CreateNewBlock
<ArtForz> and I suspect *that* is more I/O than cpu bound
<ArtForz> I don't really know why though, it doesnt *look* like it does lots of I/O
<molecular> CreateNewBlock is in O(n) with n == <number of tx in cache>?
<ArtForz> yes
<molecular> ArtForz, why do you think IO might be the problem?
<molecular> it's all in RAM

Quote from: #bitcoin-dev
<ArtForz> yes, it does a fopen/fseek/fread/fclose for blk0001.dat
<tcatm> so that's probably what slows it down
<ArtForz> (if the input is from a tx thats already in a block)
<ArtForz> and it does a lookup in blkindex DB for every call, too
<molecular> "if (!txPrev.ReadFromDisk(txdb, txin.prevout, txindex))" <- you mean this, tcatm?
<tcatm> molecular: yep
<ArtForz> yeah, that sounds like it might cause slowdowns, especially if you dont have enough free memory to keep blk0001 cached
<tcatm> how does that code find the transaction in blk0001.dat? is there an index for txhash -> blkhash?
<ArtForz> yes
<ArtForz> blkindex.dat for txhash->offset

Quote from: #bitcoin-dev
<ArtForz> urrr... why the F are we not caching that stuff?
<molecular> slush, workaround will relieve the pain, but in the end we should fix stuff in bitcoin
<ArtForz> it's not like tx can magically appear in blocks while no new block comes along
<molecular> I think ArtForz might've just identified the root of the problem?
<ArtForz> could be...
<ArtForz> can't think of a elegant way to work around it though
<ArtForz> thats... really weird

Quote from: #bitcoin-dev
<ArtForz> well, we do 2 things really with that prev tx
<ArtForz> 1. check if it's in a block (can be cached between block updates)
<ArtForz> 2. if it is, get the value of the output referred to (same thing)

I'm stopping here and backposting link to #bitcoin-dev...

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480703052
Hero Member
*
Offline Offline

Posts: 1480703052

View Profile Personal Message (Offline)

Ignore
1480703052
Reply with quote  #2

1480703052
Report to moderator
Hal
VIP
Sr. Member
*
expert
Offline Offline

Activity: 314



View Profile
March 24, 2011, 11:57:54 PM
 #2

Looking at the getwork code, it should only call CreateNewBlock() every minute or so:
Code:
       if (pindexPrev != pindexBest ||
            (nTransactionsUpdated != nTransactionsUpdatedLast && GetTime() - nStart > 60))
        {
[...]
            nTransactionsUpdatedLast = nTransactionsUpdated;
            pindexPrev = pindexBest;
            nStart = GetTime();

            // Create new block
            pblock = CreateNewBlock(reservekey);
This will call CreateNewBlock() either on a new block coming in, or on a new transaction if it's been 60 seconds since the last call. This shouldn't affect every getwork call, unless it's actually taking 60 seconds to do CreateNewBlock(), which would be remarkable.

Hal Finney
ArtForz
Sr. Member
****
Offline Offline

Activity: 406


View Profile
March 25, 2011, 08:30:35 AM
 #3

Yes, only once every 60 seconds.
note that CreateNewBlock holds cs_main and cs_mapTransactions while doing its thing.
So it not only blocks RPC for that time, but also processing and sending of p2p messages and ... really, pretty much everything.
On a decent box with 1500 cached tx, it takes 1.7 seconds.
In extreme cases (lots of cached tx, slow cpu, low free memory so a decent % of reads actually have to hit disk), well north of 10 seconds.
And it's entirely avoidable.

bitcoin: 1Fb77Xq5ePFER8GtKRn2KDbDTVpJKfKmpz
i0coin: jNdvyvd6v6gV3kVJLD7HsB5ZwHyHwAkfdw
m0mchil
Full Member
***
Offline Offline

Activity: 171


View Profile
March 25, 2011, 02:53:12 PM
 #4

Here is a temporary workaround for pool operators http://github.com/m0mchil/bitcoin/tree/poolmode

I renamed CreateNewBlock to ProcessTransactions and call it on every new block or incoming transaction. If less than 60 seconds elapsed since last execution it does nothing. Then CreateNewBlock simply provides most recent snapshot of processed transactions. The bottleneck is moved from RPC thread to network thread.

Of course this will be obsolete when actual problem is fixed. As I understand it, now orphan transactions get verified (ECDSA) when their parents become available.

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!