MrSam
|
|
July 06, 2011, 02:34:51 PM |
|
If you want to make sure your blocks are more likely to get accepted quicker, find out the IPs of other large miner nodes and addnode those instead of just blindly making a ton of connections as this patch does. Won't those generally be the nodes that are accepting incoming connections? I tend to use https://en.bitcoin.it/wiki/Fallback_Nodes , but the wikibot doesnt seem to update it.
|
|
|
|
Matt Corallo
|
|
July 06, 2011, 02:39:26 PM |
|
B. Not even pool operators should be using this, a pool bitcoind should be accepting incoming connections, and if you do that, you will get plenty of connections really fast. Yes, but from lots of small nodes. That doesn't help you very much. True, but those nodes are probably only connected to other large nodes who listen, making two hops is really very fast anyway. If you want to make sure your blocks are more likely to get accepted quicker, find out the IPs of other large miner nodes and addnode those instead of just blindly making a ton of connections as this patch does. Won't those generally be the nodes that are accepting incoming connections? Yep, but if you blindly make a ton of connections, you are still fairly unlikely to make connections to other miners, better to just addnode the best ones and make 10 connections instead of making 30 and only getting a couple large miners.
|
|
|
|
JoelKatz
Legendary
Offline
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
|
|
July 06, 2011, 02:42:42 PM |
|
I can't find the problem. I've compiled it many times from scratch and every time the same result. Can you test it on your end?
Are you saying it works fine with just my patches but if you add the 'getblock' patches, then you have problems? Are you using the 'getblock*' RPC calls that the patch adds? It looks to me like they access mapBlockIndex without holding the appropriate lock. Can you try this change: (And if this fixes it, you may want to share this fix with whoever wrote those patches!) --- patch/rpc.cpp 2011-07-06 07:39:21.647871060 -0700 +++ rpc.cpp 2011-07-06 07:41:38.033681011 -0700 @@ -277,9 +277,13 @@ Value getblockbycount(const Array& param string blkname = strprintf("blk%d", height); + CBlock block; CBlockIndex* pindex; bool found = false; + CRITICAL_BLOCK(cs_main) + { + for (map<uint256, CBlockIndex*>::iterator mi = mapBlockIndex.begin(); mi != mapBlockIndex.end(); ++mi) { @@ -295,9 +299,9 @@ Value getblockbycount(const Array& param "getblockbycount height\n" "Dumps the block existing at specified height"); - CBlock block; block.ReadFromDisk(pindex); block.BuildMerkleTree(); + } return BlockToValue(block); } @@ -312,6 +316,10 @@ Value getblockbyhash(const Array& params uint256 hash; hash.SetHex(params[0].get_str()); + CBlock block; + + CRITICAL_BLOCK(cs_main) + { map<uint256, CBlockIndex*>::iterator mi = mapBlockIndex.find(hash); if (mi == mapBlockIndex.end()) @@ -319,9 +327,9 @@ Value getblockbyhash(const Array& params CBlockIndex* pindex = (*mi).second; - CBlock block; block.ReadFromDisk(pindex); block.BuildMerkleTree(); + } return BlockToValue(block); }
|
I am an employee of Ripple. Follow me on Twitter @JoelKatz 1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
|
|
|
talpan
|
|
July 06, 2011, 05:19:17 PM |
|
Thanks a lot for your help, so far it is all running perfect. But I think the problem was somewhere else, after a little bit more testing I got an exception. It had something to do with libboost, I did reinstall that and the problem was solved. I'll send you small donation(haven't that much, yet ) in the next days. And the patch is from jgarzik, I'll make a pull with your patch and of course your credits of course. regards, talpan
|
|
|
|
somebadger
Member
Offline
Activity: 170
Merit: 10
|
|
July 06, 2011, 11:13:00 PM |
|
fyi: renamed the repository to bitcoin4pools, new url is: https://github.com/somebadger/bitcoin4poolsAny addition suggestions are welcome, will start a new thread later today and reference this thread and give credit to JoelKatz
|
|
|
|
JoelKatz
Legendary
Offline
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
|
|
July 06, 2011, 11:57:26 PM |
|
I have some updates available at: http://davids.webmaster.com/~davids/updates.diff.txtThis includes a performance improvement for ToHex, some style cleanups, some tuning changes to hub mode to reduce outbound connections, and a huge change to 'getwork'. The change to 'getwork' is one I attempted before with not so good results, but I've rethought the way it's implemented, and this is much simpler. The idea is to update the 'skeleton block' used to build 'getwork' requests when a new transaction or block is received rather than waiting for the next 'getwork' request to come along to do it. The reason this makes a difference is that it means the 'getwork' request won't have to wait for the code handling the new block/transaction to release the 'cs_main' lock. Testing shows this really smooths out the 'spikes' in the 'getwork' response time. However, this is a risky change. It is invasive to not just the code to get a work unit but also the code to confirm a valid block. Some people may wish to apply just the ToHex change and the cleanups.
|
I am an employee of Ripple. Follow me on Twitter @JoelKatz 1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
|
|
|
MrSam
|
|
July 07, 2011, 12:36:52 AM |
|
I'll run it on testnet to see what it gives
|
|
|
|
TeraPool
Newbie
Offline
Activity: 42
Merit: 0
|
|
July 07, 2011, 04:15:20 AM |
|
fyi: renamed the repository to bitcoin4pools, new url is: https://github.com/somebadger/bitcoin4poolsAny addition suggestions are welcome, will start a new thread later today and reference this thread and give credit to JoelKatz Thanks for that! Please make a post of said new through in this post, as I am watching this post for some time now as I assume many other pool creators are.
|
|
|
|
Caesium
|
|
July 07, 2011, 10:13:50 AM |
|
JoelKatz, Managing to get segfaults when I send getwork to bitcoind with this set of patches now, backtrace below: (gdb) bt #0 0x00007f710a6e8394 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x0000000000447103 in boost::interprocess::interprocess_recursive_mutex::lock (this=<value optimized out>) at /usr/include/boost/interprocess/sync/posix/interprocess_recursive_mutex.hpp:53 #2 0x0000000000477d98 in Enter (this=0x0, nIndex=@0x9c1968, keypool=...) at util.h:245 #3 CCriticalBlock (this=0x0, nIndex=@0x9c1968, keypool=...) at util.h:260 #4 CWallet::ReserveKeyFromKeyPool (this=0x0, nIndex=@0x9c1968, keypool=...) at wallet.cpp:1041 #5 0x000000000047889c in CReserveKey::GetReservedKey (this=0x9c1960) at wallet.cpp:1117 #6 0x0000000000458562 in CreateNewBlock (reservekey=...) at main.cpp:2713 #7 0x000000000048f630 in GetWorkBlock ( pmidstate=0x7f70feff9ef0 "C\377\376p\177", pdata=0x7f70feff9e30 "#", phash1=0x7f70feff9eb0 "\250\223\f\003", hashTarget=...) at rpc.cpp:1359 #8 0x000000000048fca2 in FastGetWork (id=...) at rpc.cpp:1461 #9 0x000000000049d03b in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:2031 #10 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #11 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) ---Type <return> to continue, or q <return> to quit--- at rpc.cpp:1981 #12 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #13 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #14 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #15 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #16 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #17 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #18 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #19 0x000000000049d328 in ThreadRPCServer3 (parg=0x7f71002503d0) at rpc.cpp:1981 #20 0x00007f710a6e5d8c in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #21 0x00007f7109c9004d in clone () from /lib/x86_64-linux-gnu/libc.so.6 #22 0x0000000000000000 in ?? () (gdb)
This happening both with a fresh blockchain downloaded and a new wallet. Let me know what I can give you to help debug
|
|
|
|
JoelKatz
Legendary
Offline
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
|
|
July 07, 2011, 03:56:12 PM |
|
What build is this exactly?
|
I am an employee of Ripple. Follow me on Twitter @JoelKatz 1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
|
|
|
Jine (OP)
|
|
July 07, 2011, 03:59:25 PM |
|
wallet.cpp is in the github version - no the stable one.
|
Previous founder of Bit LC Inc. | I've always loved the idea of bitcoin.
|
|
|
Jine (OP)
|
|
July 07, 2011, 04:21:28 PM |
|
I have some updates available at: http://davids.webmaster.com/~davids/updates.diff.txtThis includes a performance improvement for ToHex, some style cleanups, some tuning changes to hub mode to reduce outbound connections, and a huge change to 'getwork'. The change to 'getwork' is one I attempted before with not so good results, but I've rethought the way it's implemented, and this is much simpler. The idea is to update the 'skeleton block' used to build 'getwork' requests when a new transaction or block is received rather than waiting for the next 'getwork' request to come along to do it. The reason this makes a difference is that it means the 'getwork' request won't have to wait for the code handling the new block/transaction to release the 'cs_main' lock. Testing shows this really smooths out the 'spikes' in the 'getwork' response time. However, this is a risky change. It is invasive to not just the code to get a work unit but also the code to confirm a valid block. Some people may wish to apply just the ToHex change and the cleanups. jine@bitcoins:~$ uptime 18:20:27 up 9 days, 6:45, 3 users, load average: 0.00, 0.19, 0.22
I say no more. We had some serious issues a while ago with a lot of disconnects, so i took a shot and applied this.... 0.00 load.. seriously?
|
Previous founder of Bit LC Inc. | I've always loved the idea of bitcoin.
|
|
|
Jine (OP)
|
|
July 07, 2011, 04:23:11 PM |
|
... and now it froze. Reverted back to old bitcoind... EDIT: received: addr (31 bytes) received: inv (37 bytes) got inventory: tx f9e2f6b050644cc94341 have sending: addr (31 bytes) received: addr (31 bytes) sending: inv (37 bytes) sending: addr (31 bytes) sending: inv (37 bytes) sending: inv (37 bytes) received: addr (31 bytes) received: addr (61 bytes) sending: addr (31 bytes) sending: inv (37 bytes) sending: inv (37 bytes) BitcoinMiner: proof-of-work found hash: 00000000000004f00d620b3a2fd5e4ba19043e2053e9f4060a3795d2d00c5155 target: 0000000000000abbcf0000000000000000000000000000000000000000000000 CBlock(hash=00000000000004f00d62, ver=1, hashPrevBlock=0000000000000a6772c7, hashMerkleRoot=7e05d00288, nTime=1310055675, nBits=1a0abbcf, nNonce=568783248, vtx=37) CTransaction(hash=14b69a7cab, ver=1, vin.size=1, vout.size=1, nLockTime=0) CTxIn(COutPoint(0000000000, -1), coinbase 04cfbb0a1a035fc601) CTxOut(nValue=50.07110000, scriptPubKey=044dcd4ee79a23daccd12df3450a23) CTransaction(hash=da5496d248, ver=1, vin.size=16, vout.size=2, nLockTime=0) CTxIn(COutPoint(9c55619ba7, 1), scriptSig=3045022100f6c19dcd06577d) CTxIn(COutPoint(8d7f3d9c1d, 1), scriptSig=304602210095f7f99816769e) CTxIn(COutPoint(3255f7943d, 0), scriptSig=304602210089bbaa23e5be3c) CTxIn(COutPoint(a8be29ad98, 1), scriptSig=3045022100bab4ddaaf7da9c) CTxIn(COutPoint(abdc5e1a70, 1), scriptSig=30450220442f40e8b65ef407)
... new block?
CTxOut(nValue=0.01000000, scriptPubKey=OP_DUP OP_HASH160 91d45c1f79fa) CTransaction(hash=a51043e11b, ver=1, vin.size=2, vout.size=2, nLockTime=0) CTxIn(COutPoint(6701c1b7a5, 0), scriptSig=30450220224680e1db9d648d) CTxIn(COutPoint(60cc6580e0, 0), scriptSig=304402207cb98fb1870604cd) CTxOut(nValue=0.00775252, scriptPubKey=OP_DUP OP_HASH160 c1e8b98c8c3d) CTxOut(nValue=0.02000000, scriptPubKey=OP_DUP OP_HASH160 401ce258491c) vMerkleTree: 14b69a7cab da5496d248 035078318f ea17127588 c0c344418b eef1e852f6 649507f180 ac905c95d4 0842b073a4 305372759d 9a6e764565 6794e2e09c d76296138f fb2fbfb31b 827e418d7f b142831de3 cd499e996d 5b43802900 68971974ef 29a1b4054b 1ca1ee01f1 f7abc8ab40 9f5768fc54 47394$ 07/07/11 16:21 generated 50.0711 keypool keep 10145 AddToWallet 14b69a7cab new AddToWallet 68971974ef update ThreadRPCServer method=getinfo ThreadRPCServer method=getinfo ThreadRPCServer method=getinfo ThreadRPCServer method=getinfo accepted connection 178.47.108.93:60078 ThreadRPCServer method=getinfo ThreadRPCServer method=getinfo ThreadRPCServer method=getinfo ThreadRPCServer method=getinfo ThreadRPCServer method=getinfo
* was unresponsive, so i runned ./bitcoind stop*
ThreadRPCServer method=stop DBFlush(false) addr.dat refcount=0 addr.dat flush ThreadSocketHandler exiting ThreadOpenConnections exiting ThreadRPCServer exiting blkindex.dat refcount=1 wallet.dat refcount=25 StopNode()
* restarted *
Bitcoin version 0.3.23-beta Default data directory /root/.bitcoin
|
Previous founder of Bit LC Inc. | I've always loved the idea of bitcoin.
|
|
|
JoelKatz
Legendary
Offline
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
|
|
July 07, 2011, 05:13:31 PM |
|
Getting the locking right on this patch (the one to precompute the skeleton block) is proving to be a pain.
|
I am an employee of Ripple. Follow me on Twitter @JoelKatz 1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
|
|
|
Jine (OP)
|
|
July 07, 2011, 05:22:47 PM |
|
Is it any other way to do it? Possibly by not requesting work at all? Generating it in pushpoold perhaps? I don't have any other ideas.
Both me and Eleuthria is talking about suicide on IRC (not serious tho... yet ;P) because of all the issues.. (Not only bitcoind-related, but server/ddos/stability/stupid users etc)
|
Previous founder of Bit LC Inc. | I've always loved the idea of bitcoin.
|
|
|
JoelKatz
Legendary
Offline
Activity: 1596
Merit: 1012
Democracy is vulnerable to a 51% attack.
|
|
July 07, 2011, 05:27:21 PM |
|
I've been there with the DDoS attacks, I know how awful that is. Giel de Nijs suggested a binary protocol between bitcoind and pushpool. This would eliminate all the HTTP/JSON/hex/base64 nonsense entirely. It would also permit pushpool to request work units in blocks of, say, 10, reducing the handoffs between the two programs. This would mean long polling could be implemented in a saner way than a signal as well. On the bright side, my current secret project will make all mining pools obsolete anyway and drastically reduce the feasibility of the 51% attack so there's some hope.
|
I am an employee of Ripple. Follow me on Twitter @JoelKatz 1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
|
|
|
|
Zagitta
Full Member
Offline
Activity: 302
Merit: 100
Presale is live!
|
|
July 07, 2011, 11:21:06 PM |
|
Is it any other way to do it? Possibly by not requesting work at all? Generating it in pushpoold perhaps? I don't have any other ideas.
Both me and Eleuthria is talking about suicide on IRC (not serious tho... yet ;P) because of all the issues.. (Not only bitcoind-related, but server/ddos/stability/stupid users etc)
That's an idea i've had myself and i'm trying to implement it in my own backend however i still haven't quit understood what exactly the data is calculated from (besides that the previous block is involved somehow) but once i get it, it should be pretty trival to implement considering the code for handling the block chain is already ported to C#: http://code.google.com/p/bitcoinsharp/So if anyone got a simple explenation or links that explains the getwork process please share
|
|
|
|
somebadger
Member
Offline
Activity: 170
Merit: 10
|
|
July 11, 2011, 01:29:54 AM Last edit: July 11, 2011, 10:57:50 PM by somebadger |
|
I have updated the repo, reverted back to .23 with patches and updates, plus --help additions. .24 had some issues will get back on those once i get some time. https://github.com/somebadger/bitcoin4pools/commits/v0.3.23-poolps. running now for 22hrs without a glitch
|
|
|
|
Caesium
|
|
July 12, 2011, 04:47:19 PM |
|
Looks good here too, cheers
|
|
|
|
|