Bitcoin Forum
December 05, 2016, 10:33:23 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 [679] 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 ... 744 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2029981 times)
idonothave
Full Member
***
Offline Offline

Activity: 234


View Profile
October 19, 2015, 05:33:44 AM
 #13561

question was: Have one p2pool process connecting to multiple bitcoind processes via RPC at the same time?
you have answered: No.
how should I undersand this: [BITCOIND_RPCUSERPASS [BITCOIND_RPCUSERPASS ...]]

Fallback option?

You might be able to connect to multiple bitcoinds with a load balancer, maybe.

you mean this one http://lnlb.sourceforge.net/ for example?
1480934003
Hero Member
*
Offline Offline

Posts: 1480934003

View Profile Personal Message (Offline)

Ignore
1480934003
Reply with quote  #2

1480934003
Report to moderator
1480934003
Hero Member
*
Offline Offline

Posts: 1480934003

View Profile Personal Message (Offline)

Ignore
1480934003
Reply with quote  #2

1480934003
Report to moderator
1480934003
Hero Member
*
Offline Offline

Posts: 1480934003

View Profile Personal Message (Offline)

Ignore
1480934003
Reply with quote  #2

1480934003
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480934003
Hero Member
*
Offline Offline

Posts: 1480934003

View Profile Personal Message (Offline)

Ignore
1480934003
Reply with quote  #2

1480934003
Report to moderator
1480934003
Hero Member
*
Offline Offline

Posts: 1480934003

View Profile Personal Message (Offline)

Ignore
1480934003
Reply with quote  #2

1480934003
Report to moderator
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 19, 2015, 05:59:39 AM
 #13562

you mean this one http://lnlb.sourceforge.net/ for example?

Maybe. I don't know enough about it. I would imagine that p2pool talking to bitcoind is simple enough that simple load balancing would be enough but I don't know enough to say for sure.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
yslyung
Legendary
*
Offline Offline

Activity: 1050


Mine Mine Mine


View Profile
October 19, 2015, 08:10:01 AM
 #13563

windpath, your site is partially working ? seems like it may not be updating.

yeah another block, more pls to cover the dry spell.
windpath
Legendary
*
Offline Offline

Activity: 938


View Profile WWW
October 19, 2015, 01:17:06 PM
 #13564

windpath, your site is partially working ? seems like it may not be updating.

yeah another block, more pls to cover the dry spell.

Thanks,

Not sure what went down, something crashed bitcoind, which is unusual on my server.

Restarted, will keep an eye on it today.

jtoomim
Hero Member
*****
Offline Offline

Activity: 555


View Profile WWW
October 19, 2015, 02:40:10 PM
 #13565

Not sure what went down, something crashed bitcoind, which is unusual on my server.
The congested mempool that we've had recently from the 14kB blocks is causing excessive memory consumption. From my investigation so far, it looks like there's a memory leak in getblocktemplate/CreateNewBlock. This makes bitcoind's memory consumption increase by as much as 1 GB per day. I've been restarting my bitcoind processes every 2 days or so.

Hosting bitcoin miners for $75 to $90/kW/month on clean, cheap hydro power.
http://Toom.im
jtoomim
Hero Member
*****
Offline Offline

Activity: 555


View Profile WWW
October 19, 2015, 02:41:47 PM
 #13566

This is what I'm talking about.



If it's shares over the last 24 hours, surely the payout should drop slowly as old shares expire (which it seems to do on Fri 16th to Sat 17th)?

That looks like the default payout address used by that node was changed. The graph shows the payout for the address that was being used at the time the data point was collected, not at the time the graph was generated.

Hosting bitcoin miners for $75 to $90/kW/month on clean, cheap hydro power.
http://Toom.im
jtoomim
Hero Member
*****
Offline Offline

Activity: 555


View Profile WWW
October 19, 2015, 02:45:33 PM
 #13567

yeah another block, more pls to cover the dry spell.

http://173.48.117.206:9332/static/graphs.html?Day

This one was found by a 5 TH/s miner.

Hosting bitcoin miners for $75 to $90/kW/month on clean, cheap hydro power.
http://Toom.im
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 19, 2015, 03:33:49 PM
 #13568

This is what I'm talking about.



If it's shares over the last 24 hours, surely the payout should drop slowly as old shares expire (which it seems to do on Fri 16th to Sat 17th)?

That looks like the default payout address used by that node was changed. The graph shows the payout for the address that was being used at the time the data point was collected, not at the time the graph was generated.

It does look that way but is definitely not the case. p2pool has been running since Oct16 with the same address...

user 1677  1668 15 Oct16 pts/3    09:28:11 python ./run_p2pool.py -a 1address

What happened was that someone was saturating my upstream by with a bitcoind connection (presumably downloading the blockchain). I ran a tc.sh script which was apparently over-conservative with the bandwidth settings (both for bitcoind and p2pool). I can understand why that would stop me getting new shares but not that precipitous drop.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
windpath
Legendary
*
Offline Offline

Activity: 938


View Profile WWW
October 19, 2015, 04:18:39 PM
 #13569

Not sure what went down, something crashed bitcoind, which is unusual on my server.
The congested mempool that we've had recently from the 14kB blocks is causing excessive memory consumption. From my investigation so far, it looks like there's a memory leak in getblocktemplate/CreateNewBlock. This makes bitcoind's memory consumption increase by as much as 1 GB per day. I've been restarting my bitcoind processes every 2 days or so.

We keep relatively strict limits for bitcoind's mempool on the node, granted below is only since this mornings restart, I'll keep an eye on it.

Code:
bitcoin-cli getmempoolinfo
{
    "size" : 1967,
    "bytes" : 1745856
}

jonnybravo0311
Hero Member
*****
Offline Offline

Activity: 994


Mine at Jonny's Pool


View Profile WWW
October 19, 2015, 06:12:56 PM
 #13570

Not sure what went down, something crashed bitcoind, which is unusual on my server.
The congested mempool that we've had recently from the 14kB blocks is causing excessive memory consumption. From my investigation so far, it looks like there's a memory leak in getblocktemplate/CreateNewBlock. This makes bitcoind's memory consumption increase by as much as 1 GB per day. I've been restarting my bitcoind processes every 2 days or so.

We keep relatively strict limits for bitcoind's mempool on the node, granted below is only since this mornings restart, I'll keep an eye on it.

Code:
bitcoin-cli getmempoolinfo
{
    "size" : 1967,
    "bytes" : 1745856
}
I'm inclined to believe the memory leak... although I've only seen it on my Linux (Ubuntu 14.04) node.  I watched bitcoind spool up to consume 8GB of RAM before it was shut down.  On my Mac, using the same codebase, I regularly only see about 400MB of RAM being consumed by the bitcoind.  Both Linux and Mac are built from source.

Anyway, back on topic... I wonder why my reject rate is always so high mining on your node windpath.  My ping time to your server is about 17ms, yet I'm constantly seeing 10% or higher DOA.  When I was running my own VPS node (with about the same ping time), I would typically see about 3-4%.  Yes, running my own local node when I'm at home is considerably lower (usually about 1% or less), and that's to be expected since my miners and my node are all hardwired into the same internal gigabit network.

I'm wondering if it is caused by the traffic on the node, and p2pool's inherent single-threaded execution.  Anyone have thoughts on it?

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
windpath
Legendary
*
Offline Offline

Activity: 938


View Profile WWW
October 19, 2015, 06:22:24 PM
 #13571

I'm inclined to believe the memory leak... although I've only seen it on my Linux (Ubuntu 14.04) node.  I watched bitcoind spool up to consume 8GB of RAM before it was shut down.  On my Mac, using the same codebase, I regularly only see about 400MB of RAM being consumed by the bitcoind.  Both Linux and Mac are built from source.

Anyway, back on topic... I wonder why my reject rate is always so high mining on your node windpath.  My ping time to your server is about 17ms, yet I'm constantly seeing 10% or higher DOA.  When I was running my own VPS node (with about the same ping time), I would typically see about 3-4%.  Yes, running my own local node when I'm at home is considerably lower (usually about 1% or less), and that's to be expected since my miners and my node are all hardwired into the same internal gigabit network.

I'm wondering if it is caused by the traffic on the node, and p2pool's inherent single-threaded execution.  Anyone have thoughts on it?

Yea, I'll keep an eye on it, so far mempool seems to be behaving normally.

As to the high reject rate on my node I suspect you are correct.

In addition to the # of miners a lot of folks seem to park their browsers on the stats pages which hit both p2pool and bitcoind to pull some of the data for the front end.

There are some relatively easy fixes like setting the data refresh to stop in the browser after say 30 minutes with a "I'm still here" button to restart it....

I'll look into it.

jonnybravo0311
Hero Member
*****
Offline Offline

Activity: 994


Mine at Jonny's Pool


View Profile WWW
October 19, 2015, 06:28:09 PM
 #13572

Yea, I'll keep an eye on it, so far mempool seems to be behaving normally.

As to the high reject rate on my node I suspect you are correct.

In addition to the # of miners a lot of folks seem to park their browsers on the stats pages which hit both p2pool and bitcoind to pull some of the data for the front end.

There are some relatively easy fixes like setting the data refresh to stop in the browser after say 30 minutes with a "I'm still here" button to restart it....

I'll look into it.
If I remember that UI correctly, there's a config file that defines the stats refresh rate, which defaults to like 10 seconds for the stats and 30 for the graphs (or something like that).  Of course, you've customized it so much that file has probably been rendered irrelevant at this point Tongue.

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
jtoomim
Hero Member
*****
Offline Offline

Activity: 555


View Profile WWW
October 19, 2015, 10:43:10 PM
 #13573

I'm inclined to believe the memory leak... although I've only seen it on my Linux (Ubuntu 14.04) node.  I watched bitcoind spool up to consume 8GB of RAM before it was shut down.  On my Mac, using the same codebase, I regularly only see about 400MB of RAM being consumed by the bitcoind.  Both Linux and Mac are built from source.

Are both bitcoind nodes being used for p2pool? My testing indicates that the use of getblocktemplate is necessary for the memory leaking to occur.

Hosting bitcoin miners for $75 to $90/kW/month on clean, cheap hydro power.
http://Toom.im
Meuh6879
Legendary
*
Offline Offline

Activity: 1078



View Profile
October 19, 2015, 11:10:32 PM
 #13574

[HS]
i have opened a thread for the memory (and CPU) problem.
https://github.com/bitcoinxt/bitcoinxt/issues/82

every day, i change the maxmempooltx after viewing the history of the CPU (less 100% freeze = increase the setting).

actually, it fluctuate from 800 to 1200.
[/HS]

This problem impact RPC server ... and so, the P2Pool process.

French ... but not so much   ---===---   P2P ... it's people at the end   ---===---   P2Pool (10,9 GH/s).
Comment miner des bitcoins ? Un tutoriel est là : https://bitcointalk.org/index.php?topic=1114415.0
Bitcoin change everything ... an explain of this fact : https://www.youtube.com/watch?v=joITmEr4SjY
jonnybravo0311
Hero Member
*****
Offline Offline

Activity: 994


Mine at Jonny's Pool


View Profile WWW
October 20, 2015, 03:32:01 AM
 #13575

I'm inclined to believe the memory leak... although I've only seen it on my Linux (Ubuntu 14.04) node.  I watched bitcoind spool up to consume 8GB of RAM before it was shut down.  On my Mac, using the same codebase, I regularly only see about 400MB of RAM being consumed by the bitcoind.  Both Linux and Mac are built from source.

Are both bitcoind nodes being used for p2pool? My testing indicates that the use of getblocktemplate is necessary for the memory leaking to occur.
Yes.  I was running a VPS (a DigitalOcean droplet) that I used as my backup p2pool node.  I had other miners besides my own on it pretty constantly.  It ran for quite a long time until a few weeks ago it started acting up.  The bitcoind process would die for no apparent reason.  Sometimes the p2pool process would just die off as well.  Finally spending time looking at the box, I watched as the bitcoind would slowly consume more and more memory, up until it took almost the entire 8G available, at which point it would invariably die off.

On my Mac, I also run a p2pool node that is local to my miners at my home.  That bitcoind, just like the one on my droplet, was compiled from source.  On my local node, I've never seen it utilize more than 1G of RAM.  More typically, it's around 400-500MB.  Granted, I have fewer miners on my local home node (2-3 at most with a total hash of about 5TH/s).  On my Linux node, I typically had 5-6 miners with a hash rate of about 15TH/s.

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
Polyatomic
Sr. Member
****
Offline Offline

Activity: 256


View Profile
October 20, 2015, 05:54:51 AM
 #13576

...
 I watched as the bitcoind would slowly consume more and more memory, up until it took almost the entire 8G available, at which point it would invariably die off.

What was the magic you used to produce the output, (8G <-- bitcoind is using this much ram.)
Somethings not right there man.

Linux will try to use alot of the computers memory to keep the system fast. 8 gigs of ram
i'm thinking is below average for power linux users. Is adding more of it an option for you.

Maybe i'll bootstrap a build and run the latest bitcoind and the p2pool to see what the story is.
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
October 20, 2015, 11:48:16 AM
 #13577

I'm having no issues with my node, in fact, I'm really happy with how it & p2pool are performing in general - especially since I reverted back to the standard bitcoind settings - latency has reduced nicely.

@ windpath: Are the luck stats displayed on your web GUI correct?:

Seven Days    162.74%
Thirty Days    148.03%

If they are, I'm pretty sure there isn't another pool out there that can touch us atm  Smiley
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 20, 2015, 01:48:56 PM
 #13578

This is what I'm talking about.



If it's shares over the last 24 hours, surely the payout should drop slowly as old shares expire (which it seems to do on Fri 16th to Sat 17th)?

Any other input on this? If shares are being dropped early, it seems like this is either a bug that is causing it or a bug that is being exploited by a malicious actor (?). Either way, not good. I'm willing to help with diagnosing this if anyone who knows what they're doing is interested. I suspect it's easy enough to duplicate.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266



View Profile
October 20, 2015, 02:01:00 PM
 #13579


Any other input on this? If shares are being dropped early, it seems like this is either a bug that is causing it or a bug that is being exploited by a malicious actor (?). Either way, not good. I'm willing to help with diagnosing this if anyone who knows what they're doing is interested. I suspect it's easy enough to duplicate.

I'm pretty sure it's not a p2pool issue, otherwise we'd all be suffering from it? So, that leaves either a malicious 3rd party or it's a problem with the users setup. My diagnostic skills are fairly limited I'm afraid - so I won't be much help. If it were me who was having this problem, I'd consider ensuring my rig has all the right security settings set up correctly first (firewall, ports, iptables etc), then I'd start from scratch & check all my configs, redownload/recompile everything & see if the issue persists.

Not very helpful or fun to do, I know - but that's what I'd do....... Tongue
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 20, 2015, 02:51:03 PM
 #13580


Not very helpful or fun to do, I know - but that's what I'd do....... Tongue

But my understanding is that even if one fully disconnects from p2pool, the shares you've found should count for 24 hours from their time of creation. Once those shares are in the p2pool net, it shouldn't matter my setup.

Personally, I'm a casual miner and this has only cost me a couple of $ worth of Bitcoin so I'm not going to sweat it too much unless it keeps happening but I'm willing to give back in terms of assisting troubleshooting.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
Pages: « 1 ... 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 [679] 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 ... 744 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!