Bitcoin Forum
March 19, 2024, 10:46:56 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 ... 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 [675] 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 ... 814 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2591571 times)
notbatman
Legendary
*
Offline Offline

Activity: 2212
Merit: 1038



View Profile
October 06, 2015, 02:49:10 PM
 #13481

CPU usage results for pypy vs python using branch jtoomim_performance of p2pool after 21.7 hours:

Regular python:
68 minutes 3 seconds, 5.22% average

pypy:
39 minutes 20 seconds, 3.82% average

That's 42% lower CPU usage for pypy.

This is multiplicative with the jtoomim_performance benefit, so using the jtoomim_performance branch with pypy has about 65% lower CPU usage than using the head branch with regular python.

Test conditions: Core i7 4790k processor, 12 peers, 76 to 80 TH/s load (SP30s). Both nodes were run on the same server at the same time, with a local BitcoinXT bitcoind.

That's a big improvement - impressive  Smiley

Tying it out now.

Yes it is, I just can't figure why he's pushing XT? Anybody with the technical skills to re-code p2pool with a 42% improvement in speed should have the good sense to stay away from this XT nonsense.
1710845216
Hero Member
*
Offline Offline

Posts: 1710845216

View Profile Personal Message (Offline)

Ignore
1710845216
Reply with quote  #2

1710845216
Report to moderator
1710845216
Hero Member
*
Offline Offline

Posts: 1710845216

View Profile Personal Message (Offline)

Ignore
1710845216
Reply with quote  #2

1710845216
Report to moderator
1710845216
Hero Member
*
Offline Offline

Posts: 1710845216

View Profile Personal Message (Offline)

Ignore
1710845216
Reply with quote  #2

1710845216
Report to moderator
Once a transaction has 6 confirmations, it is extremely unlikely that an attacker without at least 50% of the network's computation power would be able to reverse it.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1710845216
Hero Member
*
Offline Offline

Posts: 1710845216

View Profile Personal Message (Offline)

Ignore
1710845216
Reply with quote  #2

1710845216
Report to moderator
Richy_T
Legendary
*
Offline Offline

Activity: 2380
Merit: 2100


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 06, 2015, 07:29:05 PM
 #13482


read this: https://bitcointalk.org/index.php?topic=882402.0
there is howto set cgminer fixed after restart for S4. with another bitmain devices it should be similar

Thanks. I don't think those instructions are directly relevant but they point in the right direction. Looks like I'll need to cook up my own cramfs

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 07, 2015, 10:36:47 AM
 #13483

Yes it is, I just can't figure why he's pushing XT? Anybody with the technical skills to re-code p2pool with a 42% improvement in speed should have the good sense to stay away from this XT nonsense.

This isn't really the right thread or forum for that discussion. I'll give a brief bullet-point answer. If you want to have a conversation about this, we should probably move it to a different forum or a separate thread so as not to derail this one.

1. Because I think that large blocks are technically feasible, and will not make Bitcoin insecure, inaccessible, or excessively centralized.
2. Because I think that Bitcoin can handle much larger blocks (up to about 32 MB) and transaction volumes than we currently have without any code changes (except the block size limit) or exotic hardware.
3. Because I think that there are enough possible optimizations in both code and semi-exotic hardware to bring us safely up to 8 GB blocksizes in 5 years if we try hard, and in 20 years if we are lazy.
3. Because I think that BIP101 is better than BIP100 or any other proposal out there due to its simplicity, predictability, and because I think its growth rate is reasonable and unlikely to require later changes. BIP100 would be a distant second place in my opinion if it were implemented.
4. Because I think that large blocks are the most feasible method of paying miners enough in fees once the block subsidy drops below 12.5 BTC.
5. Because I do not think that consensus among developers is a reasonable method for making technical decisions for large or controversial projects.
6. Because I do not think that consensus among any group is a reasonable method for making political decisions, much less consensus among a subset of developers.
7. Because I like the way Gavin thinks about technical and political problems.
8. Because, although I think his conduct could be more gracious and polite, I think that Mike Hearn has been right about a lot of technical and political issues lately.

You can also see some of my comments on the issue here:

https://www.reddit.com/user/jtoomim/

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
yslyung
Legendary
*
Offline Offline

Activity: 1500
Merit: 1002


Mine Mine Mine


View Profile
October 07, 2015, 11:52:35 AM
 #13484

saw the update on irc ... thx Wink

here is the link to DL latest commit https://github.com/p2pool/p2pool

updating mine shortly.

The main branch only includes the BIP101 change and the payout_address web interface change. The performance modification has not been merged into the main branch yet. If you want to test out the performance improvements, you have to use the jtoomim_performance branch, which forrestv is currently testing out.

https://github.com/p2pool/p2pool/tree/jtoomim_performance

git pull
git checkout jtoomim_performance

Edit/note: if you use pypy instead of regular python to run p2pool, the performance benefit is likely to be greater than 40%, since most of the rest of the slowdowns that I've seen would be amenable to pypy acceleration, but the problem I fixed was not. I may do some benchmarks to see if this is true in a while.

just downloaded, running now to test it out ! thx for the upgrade Wink running smoothly.

can we have more blocks ?
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 07, 2015, 01:24:01 PM
 #13485

just downloaded, running now to test it out ! thx for the upgrade Wink running smoothly.

can we have more blocks ?
So greedy. We just gave you one seven hours ago.

(It was from one of our nodes running jtoomim_performance on pypy on BitcoinXT, in case you're curious.)

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
notbatman
Legendary
*
Offline Offline

Activity: 2212
Merit: 1038



View Profile
October 07, 2015, 02:51:52 PM
 #13486

Yes it is, I just can't figure why he's pushing XT? Anybody with the technical skills to re-code p2pool with a 42% improvement in speed should have the good sense to stay away from this XT nonsense.

This isn't really the right thread or forum for that discussion. I'll give a brief bullet-point answer. If you want to have a conversation about this, we should probably move it to a different forum or a separate thread so as not to derail this one.

1. Because I think that large blocks are technically feasible, and will not make Bitcoin insecure, inaccessible, or excessively centralized.
2. Because I think that Bitcoin can handle much larger blocks (up to about 32 MB) and transaction volumes than we currently have without any code changes (except the block size limit) or exotic hardware.
3. Because I think that there are enough possible optimizations in both code and semi-exotic hardware to bring us safely up to 8 GB blocksizes in 5 years if we try hard, and in 20 years if we are lazy.
3. Because I think that BIP101 is better than BIP100 or any other proposal out there due to its simplicity, predictability, and because I think its growth rate is reasonable and unlikely to require later changes. BIP100 would be a distant second place in my opinion if it were implemented.
4. Because I think that large blocks are the most feasible method of paying miners enough in fees once the block subsidy drops below 12.5 BTC.
5. Because I do not think that consensus among developers is a reasonable method for making technical decisions for large or controversial projects.
6. Because I do not think that consensus among any group is a reasonable method for making political decisions, much less consensus among a subset of developers.
7. Because I like the way Gavin thinks about technical and political problems.
8. Because, although I think his conduct could be more gracious and polite, I think that Mike Hearn has been right about a lot of technical and political issues lately.

You can also see some of my comments on the issue here:

https://www.reddit.com/user/jtoomim/

Ahh I see you're pushing XT for political reasons; backing MitGavin and GoogleHern.
Richy_T
Legendary
*
Offline Offline

Activity: 2380
Merit: 2100


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 07, 2015, 05:57:03 PM
 #13487


You can also see some of my comments on the issue here:

https://www.reddit.com/user/jtoomim/



There was zero reason for him to inject the block size discussion into this thread, particularly the way he chose to do so. I have a side but I see no need to denigrate those who have a different opinion.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
btcscot
Newbie
*
Offline Offline

Activity: 30
Merit: 0


View Profile
October 08, 2015, 08:16:38 AM
 #13488

well im giving up on trying this vps and i have just booted my server 2x E5440 @ 2.83ghz 12gb of ram and 146gb hdd

going to install ubuntu and duck dns so my ip stays connected to the domain i have a good firewall in the house so im not worried about folk trying to hack in for my £0.01 in btc hahaha

hopefully will be up before i go away on holiday if not atleast i will have the ssh port and the 9332 9333 8333 ports open 
yslyung
Legendary
*
Offline Offline

Activity: 1500
Merit: 1002


Mine Mine Mine


View Profile
October 08, 2015, 09:35:07 AM
 #13489

just downloaded, running now to test it out ! thx for the upgrade Wink running smoothly.

can we have more blocks ?
So greedy. We just gave you one seven hours ago.

(It was from one of our nodes running jtoomim_performance on pypy on BitcoinXT, in case you're curious.)

no ! me NO greedy, want to harvest for the dry season. dry spell for over 4 days is no good Sad
Richy_T
Legendary
*
Offline Offline

Activity: 2380
Merit: 2100


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 08, 2015, 09:03:25 PM
 #13490

From looking through the update code on the Antminer, it looks like it's a fairly straightforward process...

1)Obtain latest firmware
2)Unpack firmware
3)Uncompress RamFS image
4)Substitute custom cgminer for original cgminer
5)Recompress RamFS image
6)zip only new image into tgz file
7)Use upgrade page on the web interface to update
8)Cry over bricked antminer (?)

If anyone is interested in this and it works, I could put it for download somewhere.

Any other changes that might be worth slipping in?

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
jonnybravo0311
Legendary
*
Offline Offline

Activity: 1344
Merit: 1023


Mine at Jonny's Pool


View Profile WWW
October 09, 2015, 01:14:45 PM
 #13491

For which Antminer?  The S7?

Also, step 8 seems a bit off... you sure you want to tell people to brick their Antminers? Tongue

Interesting random fact: 9 of the last 20 p2pool blocks are BIP101.

Jonny's Pool - Mine with us and help us grow!  Support a pool that supports Bitcoin, not a hardware manufacturer's pockets!  No SPV cheats.  No empty blocks.
Richy_T
Legendary
*
Offline Offline

Activity: 2380
Merit: 2100


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 09, 2015, 02:46:33 PM
 #13492

For which Antminer?  The S7?

Also, step 8 seems a bit off... you sure you want to tell people to brick their Antminers? Tongue

Interesting random fact: 9 of the last 20 p2pool blocks are BIP101.

No, this is for the S5. 4.9.0 cgminer is working fine but I want to make it survive a reboot.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 09, 2015, 07:34:50 PM
 #13493

From looking through the update code on the Antminer, it looks like it's a fairly straightforward process...

1)Obtain latest firmware
2)Unpack firmware
3)Uncompress RamFS image
4)Substitute custom cgminer for original cgminer
5)Recompress RamFS image
6)zip only new image into tgz file
7)Use upgrade page on the web interface to update
8)Cry over bricked antminer (?)

If anyone is interested in this and it works, I could put it for download somewhere.

Any other changes that might be worth slipping in?

I think you can also just use the smit1237 firmware. If I remember correctly, if you want to use a custom cgminer with smit1237's FW, you just stick the new cgminer into /config (which is non-volatile) and that becomes the new default.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
Richy_T
Legendary
*
Offline Offline

Activity: 2380
Merit: 2100


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
October 09, 2015, 08:17:34 PM
 #13494


I think you can also just use the smit1237 firmware. If I remember correctly, if you want to use a custom cgminer with smit1237's FW, you just stick the new cgminer into /config (which is non-volatile) and that becomes the new default.

If that's the one in the link, that was for the S4 (though they may be compatible?). I'll look into it a bit more. I may want to add some extra stuff in in any case.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 09, 2015, 09:40:48 PM
 #13495

If that's the one in the link, that was for the S4 (though they may be compatible?). I'll look into it a bit more. I may want to add some extra stuff in in any case.

smit1237's firmware for S5s is here: https://www.nicehash.com/?p=software

Specifically, https://www.nicehash.com/sw/SD-S5-20150415_nicehash_perf_graphs.tar.gz

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 11, 2015, 03:55:47 PM
 #13496

I have been seeing a lot of performance problems with pypy recently. One of my nodes exceeded 4 GB of RAM usage by pypy/p2pool alone. I recommend using regular python, as it seems to give more reliable performance.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
p3yot33at3r
Sr. Member
****
Offline Offline

Activity: 266
Merit: 250



View Profile
October 11, 2015, 04:39:22 PM
 #13497

I have been seeing a lot of performance problems with pypy recently. One of my nodes exceeded 4 GB of RAM usage by pypy/p2pool alone. I recommend using regular python, as it seems to give more reliable performance.

I was wondering myself, but thought it was just me..... Tongue
idonothave
Full Member
***
Offline Offline

Activity: 238
Merit: 100


View Profile
October 12, 2015, 06:56:20 AM
 #13498

I have been seeing a lot of performance problems with pypy recently. One of my nodes exceeded 4 GB of RAM usage by pypy/p2pool alone. I recommend using regular python, as it seems to give more reliable performance.

I was wondering myself, but thought it was just me..... Tongue

I have also noticed that and temporarily got back to python but in my opinion the problem was in combination pypy / high GBT latency so I have added to my bitcoin.conf lines

Code:
blockmaxsize=250000 # default: 250000
blockprioritysize=27000 # default: 27000
mintxfee=0.0002 # default: 0.0001
minrelaytxfee=0.0002 # default: 0.0001

then latency went down around 0.6 and I switched back to pypy without any problems again but I think it is not permanent solution and if there is anybody experienced with bitcoin.conf tuning I would welcome to have one recommended template of bitcoin.conf usable here
jtoomim
Hero Member
*****
Offline Offline

Activity: 818
Merit: 1005


View Profile WWW
October 12, 2015, 09:55:55 AM
 #13499

I suspect the performance issue is due to the p2pool's known_txns_var (the equivalent of the p2pool mempool) getting large over time, and pypy taking a lot more space to store this for some reason. In any case, the problems that I've seen appear to only manifest themselves after running p2pool for a while, like a few days. Restarting p2pool clears the caches and reduces memory usage, improving performance.

When a node creates a share that includes large low-fee transactions, it forces all other p2pool nodes to download and store those transactions. Setting minrelaytxfee to something reasonably high like 0.0005 and blockprioritysize to something low like 0 may help performance for all p2pool nodes, because your node would force fewer low-fee spam txns onto those nodes.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
idonothave
Full Member
***
Offline Offline

Activity: 238
Merit: 100


View Profile
October 12, 2015, 10:27:58 AM
 #13500

I suspect the performance issue is due to the p2pool's known_txns_var (the equivalent of the p2pool mempool) getting large over time, and pypy taking a lot more space to store this for some reason. In any case, the problems that I've seen appear to only manifest themselves after running p2pool for a while, like a few days. Restarting p2pool clears the caches and reduces memory usage, improving performance.

When a node creates a share that includes large low-fee transactions, it forces all other p2pool nodes to download and store those transactions. Setting minrelaytxfee to something reasonably high like 0.0005 and blockprioritysize to something low like 0 may help performance for all p2pool nodes, because your node would force fewer low-fee spam txns onto those nodes.

someone should hold leadership with this to say, guys, to put this and this is the best way for now... and even why not to vote for major changes and p2pool bitcoin.conf best template form
you as a provider of quite big part of p2pool hashrate would be that
Pages: « 1 ... 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 [675] 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 ... 814 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!