Bitcoin Forum
January 23, 2019, 09:26:35 PM *
News: Latest Bitcoin Core release: 0.17.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 ... 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 [525] 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 ... 814 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2567600 times)
mdude77
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001



View Profile
August 31, 2014, 12:25:22 PM
 #10481

Nice results on the RelayNode mod... But how does it compare to having the standalone java node and p2pool instance versus the combined python implementation?  Pros/cons? Would pypy execution make any appreciable difference for the modified p2pool instance? 

Right now I'm running the mainline p2pool via pypy and Matt's java Relay separately and it's functioning fine, so I'm interested if there would be any gain to switch to the combined implementation.

The biggest gain I'd say to using the combined version is that you won't need Java on your machine.

I'm going to switch my public node to using the combined version, see if it helps.

M

I mine at Kano's Pool because it pays the best and is completely transparent!  Come join me!
1548278795
Hero Member
*
Offline Offline

Posts: 1548278795

View Profile Personal Message (Offline)

Ignore
1548278795
Reply with quote  #2

1548278795
Report to moderator
1548278795
Hero Member
*
Offline Offline

Posts: 1548278795

View Profile Personal Message (Offline)

Ignore
1548278795
Reply with quote  #2

1548278795
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1548278795
Hero Member
*
Offline Offline

Posts: 1548278795

View Profile Personal Message (Offline)

Ignore
1548278795
Reply with quote  #2

1548278795
Report to moderator
Duce
Full Member
***
Offline Offline

Activity: 175
Merit: 100


View Profile
August 31, 2014, 04:37:52 PM
 #10482

Nice results on the RelayNode mod... But how does it compare to having the standalone java node and p2pool instance versus the combined python implementation?  Pros/cons? Would pypy execution make any appreciable difference for the modified p2pool instance? 

Right now I'm running the mainline p2pool via pypy and Matt's java Relay separately and it's functioning fine, so I'm interested if there would be any gain to switch to the combined implementation.

The biggest gain I'd say to using the combined version is that you won't need Java on your machine.

I'm going to switch my public node to using the combined version, see if it helps.

M

I see that you changed your pool to Matt's fork (python included) as well. Did you try to use your extended node status and found that it did not work? I too have seen an improvement as others with Matt's fork but lost the ability to run a parallel version. I am really asking to verify that it is not just me? This is by no means a discredit to Matt's efforts as the loss of a "pretty" display does not take away from the improved performance.
mdude77
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001



View Profile
August 31, 2014, 06:16:21 PM
 #10483

Nice results on the RelayNode mod... But how does it compare to having the standalone java node and p2pool instance versus the combined python implementation?  Pros/cons? Would pypy execution make any appreciable difference for the modified p2pool instance? 

Right now I'm running the mainline p2pool via pypy and Matt's java Relay separately and it's functioning fine, so I'm interested if there would be any gain to switch to the combined implementation.

The biggest gain I'd say to using the combined version is that you won't need Java on your machine.

I'm going to switch my public node to using the combined version, see if it helps.

M

I see that you changed your pool to Matt's fork (python included) as well. Did you try to use your extended node status and found that it did not work? I too have seen an improvement as others with Matt's fork but lost the ability to run a parallel version. I am really asking to verify that it is not just me? This is by no means a discredit to Matt's efforts as the loss of a "pretty" display does not take away from the improved performance.

Not sure what you mean by extended node status?

I still have a pretty display, as far as I can tell.  Not the best, but better than the default.

M

I mine at Kano's Pool because it pays the best and is completely transparent!  Come join me!
Duce
Full Member
***
Offline Offline

Activity: 175
Merit: 100


View Profile
August 31, 2014, 07:35:33 PM
 #10484

Nice results on the RelayNode mod... But how does it compare to having the standalone java node and p2pool instance versus the combined python implementation?  Pros/cons? Would pypy execution make any appreciable difference for the modified p2pool instance? 

Right now I'm running the mainline p2pool via pypy and Matt's java Relay separately and it's functioning fine, so I'm interested if there would be any gain to switch to the combined implementation.

The biggest gain I'd say to using the combined version is that you won't need Java on your machine.

I'm going to switch my public node to using the combined version, see if it helps.

M

I see that you changed your pool to Matt's fork (python included) as well. Did you try to use your extended node status and found that it did not work? I too have seen an improvement as others with Matt's fork but lost the ability to run a parallel version. I am really asking to verify that it is not just me? This is by no means a discredit to Matt's efforts as the loss of a "pretty" display does not take away from the improved performance.

Not sure what you mean by extended node status?

I still have a pretty display, as far as I can tell.  Not the best, but better than the default.

M
I can't get anything but the default to work, I built the node twice to make sure I did not do something wrong but it does not seem to pull the data in. I must be missing something. When I access the non-default front end it shows up on the web but does not display any data, it worked before this build. Well I would rather let it run than waste the time to playing with another front end.
mdude77
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001



View Profile
August 31, 2014, 07:39:32 PM
 #10485

Nice results on the RelayNode mod... But how does it compare to having the standalone java node and p2pool instance versus the combined python implementation?  Pros/cons? Would pypy execution make any appreciable difference for the modified p2pool instance? 

Right now I'm running the mainline p2pool via pypy and Matt's java Relay separately and it's functioning fine, so I'm interested if there would be any gain to switch to the combined implementation.

The biggest gain I'd say to using the combined version is that you won't need Java on your machine.

I'm going to switch my public node to using the combined version, see if it helps.

M

I see that you changed your pool to Matt's fork (python included) as well. Did you try to use your extended node status and found that it did not work? I too have seen an improvement as others with Matt's fork but lost the ability to run a parallel version. I am really asking to verify that it is not just me? This is by no means a discredit to Matt's efforts as the loss of a "pretty" display does not take away from the improved performance.

Not sure what you mean by extended node status?

I still have a pretty display, as far as I can tell.  Not the best, but better than the default.

M
I can't get anything but the default to work, I built the node twice to make sure I did not do something wrong but it does not seem to pull the data in. I must be missing something. When I access the non-default front end it shows up on the web but does not display any data, it worked before this build. Well I would rather let it run than waste the time to playing with another front end.

All I did was:

- rename my existing p2pool folder
- git clone the fork
- move the data folder from the old p2pool folder to the new one
- move/copy the webstatic folder from the old p2pool folder to the new one

And of course modified my startup script for p2pool to use the relay code.

M

I mine at Kano's Pool because it pays the best and is completely transparent!  Come join me!
PatMan
Hero Member
*****
Offline Offline

Activity: 924
Merit: 1000


Watch out for the "Neg-Rep-Dogie-Police".....


View Profile WWW
August 31, 2014, 07:50:01 PM
 #10486


All I did was:

- rename my existing p2pool folder
- git clone the fork
- move the data folder from the old p2pool folder to the new one
- move/copy the webstatic folder from the old p2pool folder to the new one

And of course modified my startup script for p2pool to use the relay code.

M

Same here, no problems at all.

"When one person is deluded it is called insanity - when many people are deluded it is called religion" - Robert M. Pirsig.  I don't want your coins, I want change.
Amazon UK BTC payment service - https://bitcointalk.org/index.php?topic=301229.0 - with FREE delivery!
http://www.ae911truth.org/ - http://rethink911.org/ - http://rememberbuilding7.org/
Duce
Full Member
***
Offline Offline

Activity: 175
Merit: 100


View Profile
August 31, 2014, 07:58:33 PM
 #10487


All I did was:

- rename my existing p2pool folder
- git clone the fork
- move the data folder from the old p2pool folder to the new one
- move/copy the webstatic folder from the old p2pool folder to the new one

And of course modified my startup script for p2pool to use the relay code.

M

Same here, no problems at all.
Wow thanks guys, I didn't think about just copying the old web-static folder over but relied on just making a new one. I did rename the existing p2pool folder so I do have the old files. That was easy, thanks again!
hamburgerhelper
Member
**
Offline Offline

Activity: 83
Merit: 10


View Profile
August 31, 2014, 08:16:23 PM
 #10488

Nice to see Matt being so responsive to the folks who are Java-averse. That's the way you build a user base!

Has anyone who ran p2pool + RelayNodeClient.jar compared the resource utilization with the 100% python version (Matt's fork of p2pool)? I'm interested in either a decrease in RAM or CPU utilization. I've been running the Java client on 2GB of RAM w/2 CPUs and haven't noticed trouble. I'm looking for some hard data to convince me that the switch to pure python is worth it.

Thanks for helping me be lazy and still run a bitchin p2pool node.
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 501


View Profile
August 31, 2014, 09:08:16 PM
 #10489

Nice results on the RelayNode mod... But how does it compare to having the standalone java node and p2pool instance versus the combined python implementation?  Pros/cons? Would pypy execution make any appreciable difference for the modified p2pool instance? 

Right now I'm running the mainline p2pool via pypy and Matt's java Relay separately and it's functioning fine, so I'm interested if there would be any gain to switch to the combined implementation.
The only thing I can think of is that, because Python is single-threaded in practice (both pypy and CPython have this global lock (the GIL) and they run multiple threads which contend for it, actually decreasing performance over just running a single thread and multiplexing between them....), you may end up in a situation where you're waiting for the relay network thread to finish processing something, but because all of its processing is incredibly light-weight, I highly doubt it would make more than a few nanoseconds difference.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 501


View Profile
August 31, 2014, 09:09:11 PM
 #10490

Nice results on the RelayNode mod... But how does it compare to having the standalone java node and p2pool instance versus the combined python implementation?  Pros/cons? Would pypy execution make any appreciable difference for the modified p2pool instance? 

Right now I'm running the mainline p2pool via pypy and Matt's java Relay separately and it's functioning fine, so I'm interested if there would be any gain to switch to the combined implementation.
The only thing I can think of is that, because Python is single-threaded in practice (both pypy and CPython have this global lock (the GIL) and they run multiple threads which contend for it, actually decreasing performance over just running a single thread and multiplexing between them....), you may end up in a situation where you're waiting for the relay network thread to finish processing something, but because all of its processing is incredibly light-weight, I highly doubt it would make more than a few nanoseconds difference.
Of course if you dont already have a physical CPU core spare running the Java version, you probably save on process context switching anyway...

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
kgb2mining
Member
**
Offline Offline

Activity: 112
Merit: 10


View Profile
September 01, 2014, 01:40:34 AM
 #10491

I'm not very sure on the CPU overhead - I have a 16-core machine and honestly it didn't break a sweat before and isn't now.  However, java introduced about a 2.5GB RAM overhead which is completely gone now when using the pure-python version.

I'll just echo what PatMan said - even if there's no real gain performance-wise between the two, there is definitely a maintenance overhead gain with everything now streamlined and that is enough for me.  I don't need to worry about the java process on its own, don't have to worry on reboots or scripting it up to restart, etc.  It's all happy under one roof.

Yesterday I was tinkering around with adding peering nodes and I think I tinkered too much, my efficiency never got back above 100%.  I fixed most of what I was doing and since restart about 12 hours ago I have 7/0/0 shares and 110%+ efficiency again.  GBT is hovering around .16-.18 on average, memory usage is small at around 500MB, and my bandwith usage is only around 50KB/s.
hamburgerhelper
Member
**
Offline Offline

Activity: 83
Merit: 10


View Profile
September 01, 2014, 03:19:55 AM
 #10492

Thanks for the info kgb2mining.

It looks like the java version is using about 70 MB of RAM and 0.2% CPU. Since I already have the startup scripts, and upgrading is so simple, I think I'll just keep running the java version. It gives me some flexibility when it comes to upgrading (like, I can just upgrade the RelayNodeClient without restarting p2pool).

Interesting about your p2pool peer experimentation. Would you mind sharing your peer list? Has anyone found that there is an optimal number of peers? Several of these peers are gone now, but here's my list:

-n p2pool.org
-n minefast.coincadence.com
-n 14.17.121.234
-n 58.22.92.36
-n 115.201.192.188
-n 125.126.130.134
-n 107.170.178.16
-n 50.251.148.42
-n 115.202.27.30
-n 182.41.211.14
-n 50.248.204.210
-n 192.71.218.197
-n 107.170.116.123
-n 173.160.157.222
-n 82.196.8.44
kgb2mining
Member
**
Offline Offline

Activity: 112
Merit: 10


View Profile
September 01, 2014, 04:12:03 AM
 #10493

Heh, I went for the "scorched earth" peer list approach to see what happens... Wink

I went to the p2pool node info page, sorted by country, exported to CSV, then some grep, awk, cut and echo'd the 84 U.S. nodes that resulted.  Currently it appears I've active peering connections with 60+ of them outbound (hovering between 62-64 average connected).

I'm of the mindset that it doesn't hurt to peer with as many nodes as you are able to reliably communicate with, since you want to get your shares out to the network as fast as possible, before everyone else.  If you are able to announce your share outbound quickly to as many nodes as you can, you might just win that race.  I could be completely off in my thinking, and hurting myself by having so many peers connected.  Time should tell tho I think, and if my efficiency suffers, then it might be better to approach it more methodically.
hamburgerhelper
Member
**
Offline Offline

Activity: 83
Merit: 10


View Profile
September 01, 2014, 04:39:50 AM
 #10494

Heh, I went for the "scorched earth" peer list approach to see what happens... Wink

I went to the p2pool node info page, sorted by country, exported to CSV, then some grep, awk, cut and echo'd the 84 U.S. nodes that resulted.  Currently it appears I've active peering connections with 60+ of them outbound (hovering between 62-64 average connected).

I'm of the mindset that it doesn't hurt to peer with as many nodes as you are able to reliably communicate with, since you want to get your shares out to the network as fast as possible, before everyone else.  If you are able to announce your share outbound quickly to as many nodes as you can, you might just win that race.  I could be completely off in my thinking, and hurting myself by having so many peers connected.  Time should tell tho I think, and if my efficiency suffers, then it might be better to approach it more methodically.

Ok, this was my line of thinking. But you took it waaaay further. For that, I applaud you!

I'm beefing up my peers with a slightly different strategy: sort the node list by uptime and use the oldest nodes with reasonable latency. I figure that the oldest nodes will have the deepest roots into the p2pool network. Ok, I hope we just didn't ruin our competitive advantage here...
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 501


View Profile
September 01, 2014, 05:07:58 AM
 #10495

Ok, this was my line of thinking. But you took it waaaay further. For that, I applaud you!

I'm beefing up my peers with a slightly different strategy: sort the node list by uptime and use the oldest nodes with reasonable latency. I figure that the oldest nodes will have the deepest roots into the p2pool network. Ok, I hope we just didn't ruin our competitive advantage here...
This kind of approach rarely works. In fact, by simply decreasing peering within the relay network, relay times improved. Mostly, using this approach means your peaks (remember when a new block is found you get huge bandwidth peaks, even though amortized across even a second, you wont see all that much bandwidth) will be significantly higher, leading to strange things on the network, mostly increased packet loss. This means instead of getting a block pretty quick, you'll have to wait for the packet to timeout and get a resend. I would recommend finding only a handful of peers which are both geographically local and have high hashpower so you'll get blocks they found directly from the horse's mouth.

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
hamburgerhelper
Member
**
Offline Offline

Activity: 83
Merit: 10


View Profile
September 01, 2014, 05:24:09 AM
 #10496

Ok, this was my line of thinking. But you took it waaaay further. For that, I applaud you!

I'm beefing up my peers with a slightly different strategy: sort the node list by uptime and use the oldest nodes with reasonable latency. I figure that the oldest nodes will have the deepest roots into the p2pool network. Ok, I hope we just didn't ruin our competitive advantage here...
This kind of approach rarely works. In fact, by simply decreasing peering within the relay network, relay times improved. Mostly, using this approach means your peaks (remember when a new block is found you get huge bandwidth peaks, even though amortized across even a second, you wont see all that much bandwidth) will be significantly higher, leading to strange things on the network, mostly increased packet loss. This means instead of getting a block pretty quick, you'll have to wait for the packet to timeout and get a resend. I would recommend finding only a handful of peers which are both geographically local and have high hashpower so you'll get blocks they found directly from the horse's mouth.

Matt, we're referring to p2pool peers, not bitcoin peers. I think that the p2pool blocks (the "shares") are very very small in comparison to bitcoin blocks. Since my node is the origin of the newly found share (the share is outbound to the rest of the p2pool network) it can only get out as fast as the maximum outbound bandwidth of my node - it seems to me that I want to try to hit that max (which in my case is 250 Mbps) so that there are many spokes to my hub (this seems to me to be hub/spoke topology in the first set of share transmissions, then it turns into a graph as the share gets retransmitted by my peer nodes).

Am I totally off base here? I'm not an expert on p2pool, I just read some stuff.
norgan
Sr. Member
****
Offline Offline

Activity: 308
Merit: 250

Decentralize your hashing - p2pool - Norgz Pool


View Profile WWW
September 01, 2014, 06:15:17 AM
 #10497

Hey all,

Been away on holidays and been lurking here to see what's going in. Just a quick Q, I have a miner on my node that has almost 100% DOA. IS that going to have any negative impacts on my node?

Miner, tech geek, operator of NorgzPool - Sydney Australia P2Pool Node creator of p2pool fancy front end

Tips: 1NorganBbymShTN2MMpfGzRYJF8mcPeXjv Exchange BTC locally in Australia or Donate to p2pool miners
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 501


View Profile
September 01, 2014, 06:15:39 AM
 #10498

Matt, we're referring to p2pool peers, not bitcoin peers. I think that the p2pool blocks (the "shares") are very very small in comparison to bitcoin blocks. Since my node is the origin of the newly found share (the share is outbound to the rest of the p2pool network) it can only get out as fast as the maximum outbound bandwidth of my node - it seems to me that I want to try to hit that max (which in my case is 250 Mbps) so that there are many spokes to my hub (this seems to me to be hub/spoke topology in the first set of share transmissions, then it turns into a graph as the share gets retransmitted by my peer nodes).

Am I totally off base here? I'm not an expert on p2pool, I just read some stuff.
Indeed, the relay network is a bit different. As many of its peers still use standard bitcoin p2p connections, full blocks get sent over the network, making peaks much higher. Still, you have to think about more than just your node. If you found the share, maxing your uplink isnt a bad idea, but if someone else found the share, you still want their share to propagate quickly (at least to you...), and for that you shouldn't fill their connection count. Of course you also have to account for bitcoind being on the same uplink (what if your share was a block?).

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 501


View Profile
September 01, 2014, 06:17:01 AM
 #10499

In any case, this thread is moving on, I'm gonna unsubscribe. If anyone has any further relay network questions, dont hesitate to reach out via bitcoin-peering@my full name.com

Bitcoin Ubuntu PPA maintainer - donate to me personally: 1JBMattRztKDF2KRS3vhjJXA7h47NEsn2c
http://bitcoinrelaynetwork.org maintainer
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
K1773R
Legendary
*
Offline Offline

Activity: 1792
Merit: 1008


/dev/null


View Profile
September 01, 2014, 06:49:22 AM
 #10500

In any case, this thread is moving on, I'm gonna unsubscribe. If anyone has any further relay network questions, dont hesitate to reach out via bitcoin-peering@my full name.com
You should create a own thread for it.

[GPG Public Key]  [Devcoin Builds]  [BBQCoin Builds]  [Multichain Blockexplorer]  [Multichain Blockexplorer - PoS Coins]  [Ufasoft Miner Linux Builds]
BTC/DVC/TRC/FRC: 1K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM AK1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: NK1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: LKi773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: EK1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: bK1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
Pages: « 1 ... 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 [525] 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 ... 814 »
  Print  
 
Jump to:  

Bitcointalk.org is not available or authorized for sale. Do not believe any fake listings.
Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!