Bitcoin Forum
October 24, 2017, 04:33:49 AM *
News: Latest stable version of Bitcoin Core: 0.15.0.1  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 [797] 798 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2491917 times)
jtoomim
Hero Member
*****
Offline Offline

Activity: 735


View Profile WWW
September 21, 2017, 05:02:47 PM
 #15921

pypy cpu usage is lower than 5%, but it seams as anything is verry slow.
That sounds like you might be out of RAM and have swapping to disk. Can you check? Task manager -> Performance -> Memory. In Use, Available, Committed, Paged Pool.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
1508819629
Hero Member
*
Offline Offline

Posts: 1508819629

View Profile Personal Message (Offline)

Ignore
1508819629
Reply with quote  #2

1508819629
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1508819629
Hero Member
*
Offline Offline

Posts: 1508819629

View Profile Personal Message (Offline)

Ignore
1508819629
Reply with quote  #2

1508819629
Report to moderator
Xantus
Jr. Member
*
Offline Offline

Activity: 37


View Profile
September 21, 2017, 08:11:49 PM
 #15922

pypy cpu usage is lower than 5%, but it seams as anything is verry slow.
That sounds like you might be out of RAM and have swapping to disk. Can you check? Task manager -> Performance -> Memory. In Use, Available, Committed, Paged Pool.
no, system look fine, Memory is 50%free, CPU usage is now with pypy somethink around 15% (5%CPU for pypy avergage, and 20%peek)
but other strange thing happend, since i Wrote this morning, i esay let it run and go to work. now i am here again, take a look and p2pool is a bit online  Huh
i cant connect the web interface from an other computer, and the miner cant connect to p2pool node. also i got no p2pool incomming connection. but it got 6 outgoing connection.
does this linux box have an own software firewall ? - i think that is an network problem ...


Log file from p2pool on pypy in an ubuntu bash box on windows 10 :
www.c-xantus.de/log3

System Preformance (using ubuntu bash and p2pool on pypy):
jtoomim
Hero Member
*****
Offline Offline

Activity: 735


View Profile WWW
September 24, 2017, 02:48:59 AM
 #15923

Xantus, maybe what's happening is that there just aren't any exceptions in Windows Firewall for this. Native Windows apps usually trigger a request to open ports in the firewall when needed, but I maybe WSL pypy p2pool isn't doing that. Try disabling Windows Firewall for an hour to see if that helps. If it does, then you can manually add a firewall rule to allow incoming tcp connections on port 9332 and 9333 (or whatever you're using).

Edit: Pages like this suggest that you do indeed need to open up ports on the Windows Firewall if you want to allow WSL processes to serve as servers.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
Xantus
Jr. Member
*
Offline Offline

Activity: 37


View Profile
September 24, 2017, 08:32:53 PM
 #15924

well, that is strange,
i tested that. its true, i get more connection ...

but i also see ofen this :
"Warning: LOST CONTACT WITH BITCOIND for" is displayed ... perhaps these linux bash box dont runs cleanly.
it fells like it freeze every few minutes:

and after restart now in bench mode (i also put in new tcp rule for in- and outgoing tcp connection on port 9332 and 9333) :
without miner connected : www.c-xantus.de/log4
with miner connected : www.c-xantus.de/log5

by the way, P2Pool is using at linux bash with pypy 1,3Gb Memory an betwen 0 and 15% cpu
jtoomim
Hero Member
*****
Offline Offline

Activity: 735


View Profile WWW
September 25, 2017, 04:33:06 AM
 #15925

"Warning: LOST CONTACT WITH BITCOIND for" is displayed ... perhaps these linux bash box dont runs cleanly.
That's what happens when your CPU gets overloaded and stalled. Let's say you have enough incoming traffic in your TCP buffers to keep p2pool's CPU busy for 1.1 minutes. P2pool first sees that it has a queued need to get a new block template, so p2pool sends a getblocktemplate request to bitcoind. Bitcoind may respond immediately, but the response goes to the end of that 1.1 minute queue. P2pool then works through its queue of stuff, then gets to the bitcoind request, and process it, but notices that it took 1.1 minutes and so complains about it.

Same thing for web UI requests. When you point your browser to localhost:9332, that request goes into p2pool's queue of jobs to work on, and p2pool doesn't get around to it until e.g. 1.1 minutes has passed.

Quote
by the way, P2Pool is using at linux bash with pypy 1,3Gb Memory an betwen 0 and 15% cpu
The 15% CPU is the problem. 15% is above the 12.5% threshold for fully utilizing a single core on your machine, which means that the work in p2pool's to-do queue is snowballing.

The --bench output is somewhat helpful, but it seems that the bulk of the load is happening in a function that I didn't put benchmarking code into. In your log files, I frequently see line pairs like this one (note the timestamps):

Code:
2017-09-24 21:52:37.814726    0.000 ms for 7 txs in p2p.py:handle_remember_tx (0.000 ms/tx)
2017-09-24 21:52:58.864085    0.000 ms for 11 txs in handle_losing_tx (0.000 ms/tx)
...
2017-09-24 21:52:59.239189 > >>> Warning: LOST CONTACT WITH BITCOIND for 1.6 minutes! Check that it isn't frozen or dead!
The "Lost contact" message means that your CPU was stalled for 1.6 minutes. 1.6 minutes before the "Lost contact" message at 12:52:59 would be 12:51:20 or something like that. That means that your CPU was busy between 21:52:37 and 21:52:58. However, we don't know what was happening at that time, since the code that was running was something that I didn't add benchmarking code to.

Can you add this to your p2pool startup command line and send me the resulting profile1.log file? Try to run it for about an hour. If you run it for too much less than that, and the profile data will be dominated by the start-up share loading time.

Code:
pypy -m cProfile -o "profile1.log" run_p2pool.py [other options]

If anyone is curious, you can analyze those cProfile log files with a python script that looks like this:
Code:
import pstats, sys

if not len(sys.argv) > 1:
   print "Usage: cumtime.py input_file [-t] [lines]"
   sys.exit()

p = pstats.Stats(sys.argv[1])
if '-t' in sys.argv:
    p.sort_stats('tottime')
else:
    p.sort_stats('cumtime')

lines = 100
if len(sys.argv) > 2:
    try:
        lines = int(sys.argv[2])
    except:
        pass

p.print_stats(lines)

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
davidadrake
Newbie
*
Offline Offline

Activity: 29


View Profile
October 04, 2017, 03:09:19 AM
 #15926

I haven't been on p2pool-bitcointalk for a while, but I remember reading through the posts every day, and jtoomin, I believe had the version of p2pool to run. On p2pool.org, it shows it has been 83 days since the last block, so are they not running jtoomin's fork?

Is there a list somewhere of pools that are running the fork? Sorry if this has been answered somewhere, but I looked through a bunch of pages without finding any info.

Thanks,

Dave
Xantus
Jr. Member
*
Offline Offline

Activity: 37


View Profile
October 04, 2017, 05:48:24 AM
 #15927


That's what happens when your CPU gets overloaded and stalled. Let's say you have enough incoming traffic in your TCP buffers to keep p2pool's CPU busy for 1.1 minutes. P2pool first sees that it has a queued need to get a new block template, so p2pool sends a getblocktemplate request to bitcoind. Bitcoind may respond immediately, but the response goes to the end of that 1.1 minute queue. P2pool then works through its queue of stuff, then gets to the bitcoind request, and process it, but notices that it took 1.1 minutes and so complains about it.

Same thing for web UI requests. When you point your browser to localhost:9332, that request goes into p2pool's queue of jobs to work on, and p2pool doesn't get around to it until e.g. 1.1 minutes has passed.

Quote
by the way, P2Pool is using at linux bash with pypy 1,3Gb Memory an betwen 0 and 15% cpu
The 15% CPU is the problem. 15% is above the 12.5% threshold for fully utilizing a single core on your machine, which means that the work in p2pool's to-do queue is snowballing.

Hey big Programmer jtoomim,

i dont belive in these CPU overload thinks you talking about. first time you sayed quad core ist with 25% overloaded, now it is 12,5% an i use it with python2,7 an see load peaks of this singel task by 48% without any connection lost message. there is an other big problem a displayed but no one is intereted.

this linux think hangs and freeze for a few sec everey round about 30sec. and the tcp-connections in these linux box in an problem, perhaps because timeouts between linux box and system network interface.

i run now p2pool on phyton 2.7 an waste a day after start up until it reduce his mining difficult and hoping ther will be a good running windows version with Segwit in the future.

@davidadrake
i have no list, but for now these is a node : "212.43.81.114:9332"
p2pool.org i the "non segwit" version, jtoomim goes his own "segwit way" for now. - korregt me if i talking wrong things!
Cryptonomist
Newbie
*
Offline Offline

Activity: 21


View Profile
October 04, 2017, 07:52:33 AM
 #15928

I haven't been on p2pool-bitcointalk for a while, but I remember reading through the posts every day, and jtoomin, I believe had the version of p2pool to run. On p2pool.org, it shows it has been 83 days since the last block, so are they not running jtoomin's fork?

Is there a list somewhere of pools that are running the fork? Sorry if this has been answered somewhere, but I looked through a bunch of pages without finding any info.

Thanks,

Dave

Hello,

There are a few websites that give the results from a node scanner. For example http://p2pool.jir.dk/bitcoin/ gives publicly reachable nodes in the P2pool main network and in the jtoomim network. Nodes with a version number 17 or 16 are nodes of the main network. Nodes with a version number of 15.0-33 are nodes from the jtoomim network.
Another useful site is https://p2pool.coinpool.pw/. This site also gives the publicly reachable nodes for both networks.
P2pool.org has also a node scanner, but this one only shows results for the p2pool main network. All the information shown on the p2pool.org website is currently that of the p2pool main network and not of the jtoomim network.
If you want to know the ip address of nodes that are not publicly reachable, the only way to do that is run your own node and look to which nodes it makes a connection.

I hope this answers your question.
BitTeo
Sr. Member
****
Offline Offline

Activity: 266



View Profile
October 04, 2017, 09:52:16 AM
 #15929

pypy cpu usage is lower than 5%, but it seams as anything is verry slow.
That sounds like you might be out of RAM and have swapping to disk. Can you check? Task manager -> Performance -> Memory. In Use, Available, Committed, Paged Pool.

How many set virtual memory with 16gb of ram in windows 10 ? .

██
█║█
║║║
║║║
█║█
██
'BTC MULTI-WALLET SOON'
▬▬▬ Download WHITEPAPER ▬▬▬

                    ▄██▄
                  ▄██████▄
                ▄██████████
              ▄██████████▀   ▄▄
            ▄██████████▀   ▄████▄
          ▄██████████▀    ████████▄
         ██████████▀      ▀████████
         ▀███████▀   ▄███▄  ▀████▀   ▄█▄
    ▄███▄  ▀███▀   ▄███████▄  ▀▀   ▄█████▄
  ▄███████▄      ▄██████████     ▄█████████
  █████████    ▄██████████▀    ▄██████████▀
   ▀█████▀   ▄██████████▀    ▄██████████▀
     ▀▀▀   ▄██████████▀    ▄██████████▀
          ██████████▀    ▄██████████▀
          ▀███████▀      █████████▀
            ▀███▀   ▄██▄  ▀█████▀
                  ▄██████▄  ▀▀▀
                  █████████
                   ▀█████▀
                     ▀▀▀
e i d o o
██

███▀▀
▐▐▌
▐▌
▐▌
▐▐▌
███▄▄
▀▀███
▐▌▌
▐▌
▐▌
▐▌▌
▄▄███
jtoomim
Hero Member
*****
Offline Offline

Activity: 735


View Profile WWW
October 04, 2017, 06:51:50 PM
 #15930

On p2pool.org, it shows it has been 83 days since the last block, so are they not running jtoomin's fork?
Correct. The most recent block found on jtoomimnet was found on September 18th. Jtoomimnet currently has around 2.6 PH/s and an expected time per block of around 22 days. Mainnet has around 0.7 PH/s and an expected time per block of around 90 days.


Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
belcher
Sr. Member
****
Offline Offline

Activity: 241


View Profile
October 04, 2017, 10:52:31 PM
 #15931

Has anybody thought about in detail how adding the uncles scheme to p2pool would work?

It was mentioned in the original bitcointalk thread about uncles in 2013 here and here. And also by jtoomim a few months ago here

But I haven't been able to find more details (or even mentions) than those. I've got a couple of ideas but it's good to read everything that's been written about it beforehand.


If uncles was applied to bitcoin then special care would have to be taken to make sure a transaction in an uncle doesn't conflict with a transaction in the main chain. This doesn't apply to p2pool's sharechain because the shares can't conflict with each other in the same way, each share just gets added to the total work that the hasher did. This is quite a cool property of applying uncles to p2pool.

If uncles are added then I think the share interval could be reduced even more to (say) 15 seconds. There'd be no more orphans at all. This would reduce the difficulty (by a constant factor but still) which would mean if p2pool gets really big then small hashers can still solve shares fairly often.


Another problem with the existing algorithm for choosing which share to base work off of is that all shares are treated the same, regardless of their difficulty, which makes low-diff shares have an advantage for selfish mining attacks. Large-diff shares contribute more work to p2pool, so they should have a greater chance of winning an orphan race, ceteris paribus. But you can't just pick the highest-diff share of all shares in an orphan race, since that would cause a race-to-the-top scenario where the optimal difficulty to use for your shares is the highest diff that a competitor uses plus one. That would also allow a selfish miner to reliably win orphan races against siblings: instead of working on a child share of the best known share, you could just orphan it by creating a sibling with +1 difficulty. So it needs to be probabilistic.

Another way of solving this apart from being probabilistic is to limit the difficulty that hashers can mine on. Only allow p2pool hashers to mine 1x difficulty, 10x difficulty, 100x difficulty and 1000x difficulty. Then it becomes impossible for a bad hasher to just mine difficulty + 1, they can only mine difficulty x 10.

Of course, with the scheme of adding uncles to p2pool there totally stop being orphans so these kind of difficulty games don't lead to extra profit.

1HZBd22eQLgbwxjwbCtSjhoPFWxQg8rBd9
JoinMarket - CoinJoin that people will actually use.
PGP fingerprint: 0A8B 038F 5E10 CC27 89BF CFFF EF73 4EA6 77F3 1129
Polyatomic
Sr. Member
****
Offline Offline

Activity: 257


View Profile
October 05, 2017, 09:12:58 AM
 #15932

If uncles are added then I think the share interval could be reduced even more to (say) 15 seconds.
For what its worth, average time between blocks on the p2pool share chain was 10 seconds once upon a time.
belcher
Sr. Member
****
Offline Offline

Activity: 241


View Profile
October 05, 2017, 03:20:03 PM
 #15933

If uncles are added then I think the share interval could be reduced even more to (say) 15 seconds.
For what its worth, average time between blocks on the p2pool share chain was 10 seconds once upon a time.


I suppose it was increased because of orphans and DOA shares. If so then using uncles should allow share intervals to go lower because there wouldn't be a race to publish shares anymore.

1HZBd22eQLgbwxjwbCtSjhoPFWxQg8rBd9
JoinMarket - CoinJoin that people will actually use.
PGP fingerprint: 0A8B 038F 5E10 CC27 89BF CFFF EF73 4EA6 77F3 1129
Xantus
Jr. Member
*
Offline Offline

Activity: 37


View Profile
October 06, 2017, 04:21:30 AM
 #15934

How many set virtual memory with 16gb of ram in windows 10 ? .
3,6Gb memory free, 450Mb virtual Memory (see picture a few posts up)
jtoomim
Hero Member
*****
Offline Offline

Activity: 735


View Profile WWW
October 06, 2017, 05:48:13 PM
 #15935

If uncles was applied to bitcoin then special care would have to be taken to make sure a transaction in an uncle doesn't conflict with a transaction in the main chain.
Actually, you'd simply ignore all transactions in uncles except the coinbase transaction, which gets programmatically modified to reduce the amounts.

Quote
If uncles are added then I think the share interval could be reduced even more to (say) 15 seconds.
I'd really rather not, for CPU and RAM performance reasons in addition to orphan rate reasons. It takes a medium-slow CPU on Python2.7 up to 4 seconds to process a share and issue new work with 1 MB blocks. With 4 MB blocks, you would generally be unable to run p2pool on medium-slow CPUs with a 15 second share interval, as it would take 16 seconds to process each share. Share variance is currently insignificant compared to block variance on p2pool. Increasing the share interval to 60 seconds on average is more likely to be a good idea than decreasing it.

While there are a few major changes to p2pool's architecture that would drastically reduce p2pool's performance load, those changes are quite large and may never be done. It would certainly involve more code than just adding uncles.

Quote
There'd be no more orphans at all. This would reduce the difficulty (by a constant factor but still) which would mean if p2pool gets really big then small hashers can still solve shares fairly often.
There would be uncles instead. Uncles have a smaller reward for the person who mined them than normal shares, so there's still a potential fairness penalty to having high uncle rates. The game theory of uncles is complicated, and if you don't set the rules right, it can be in a miner's best interest to not mention other people's shares as uncles and orphan them instead. Giving the uncled shares a hit in revenue is necessary to making sure there's an incentive to include them.

Quote
Another way of solving this apart from being probabilistic...
A probably better solution is to just follow the share with the lowest hash, regardless of its difficulty. This inherently prefers high-diff shares, but is fair in that low-diff shares have a chance to win against a high-diff share if the low-diff share is particularly lucky in its hash. Basically, it is effectively equivalent to the algorithm I proposed in the post that you linked to, but where the source of randomness is the hashes themselves.

Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power.
http://Toom.im
davidadrake
Newbie
*
Offline Offline

Activity: 29


View Profile
October 12, 2017, 03:16:11 AM
 #15936

Are the blocks that are mined on jtoomin's fork empty, like they were on the main p2pool?

I haven't been on p2pool since around Christmas I think, but I remember them being mostly empty. I think when I was on, a few people were testing jtoomin's fork, but I thought that it was supposed to get merged with the main p2pool. Is that not the case anymore?

I found it hard to find, as there's no info about his fork anywhere. It's not really accessible, you have to know exactly what you're looking for - I had to ask on here. Maybe at least something on the first page, or are you trying to keep it small - I thought it was nice when the main p2pool was around 10PH.

It would be really cool if there was something like p2pool.org if it's not getting merged.

just some of my thoughts.
Xantus
Jr. Member
*
Offline Offline

Activity: 37


View Profile
October 12, 2017, 04:09:01 AM
 #15937

Are the blocks that are mined on jtoomin's fork empty, like they were on the main p2pool?

I haven't been on p2pool since around Christmas I think, but I remember them being mostly empty. I think when I was on, a few people were testing jtoomin's fork, but I thought that it was supposed to get merged with the main p2pool. Is that not the case anymore?

I found it hard to find, as there's no info about his fork anywhere. It's not really accessible, you have to know exactly what you're looking for - I had to ask on here. Maybe at least something on the first page, or are you trying to keep it small - I thought it was nice when the main p2pool was around 10PH.

It would be really cool if there was something like p2pool.org if it's not getting merged.

just some of my thoughts.
p2pool is stange thing since this fork...  but, there is now other way  Undecided but empty blocks does on jtoomim fork of p2pool not happening. Blocks are full. only the pool software is gone  be a strange think, mainly on windows ...
seams to run better on Linux.
frodocooper
Full Member
***
Offline Offline

Activity: 124


View Profile
October 12, 2017, 11:30:49 AM
 #15938

Are the blocks that are mined on jtoomin's fork empty, like they were on the main p2pool?
No.

I think when I was on, a few people were testing jtoomin's fork, but I thought that it was supposed to get merged with the main p2pool. Is that not the case anymore?
Mainnet P2Pool must support altcoins, and jtoomimnet currently does not work properly with altcoins. The merging of jtoomimnet into mainnet is therefore on hold until jtoomimnet is made compatible with altcoins.
davidadrake
Newbie
*
Offline Offline

Activity: 29


View Profile
October 16, 2017, 10:23:31 AM
 #15939

I opened up the webpage for the p2pool jtoomim node that I am connected to, and it says there was a block found an hour ago, but there is nothing in my wallet. I checked another node, and the addresses listed did not receive anything either. It used to be instant, or do the coins need to mature now? It is a cold storage wallet, not an exchange. That's about $400 I should have received ~ .070 BTC

My wallet: https://blockchain.info/address/1AdPjxC3u2K32pyRFYaLAUp59qXXGP4fpd
Block: https://blockchain.info/block/000000000000000000817d56511e1e3f2269250ee49f9fa651fc2c625c360138
Node I'm connected to:https://btc.coinpool.pw/#
Another node that has addresses that show no transactions:http://low-doa.mine.nu:9334/static/
Gws24
Sr. Member
****
Offline Offline

Activity: 468


View Profile
October 16, 2017, 12:17:22 PM
 #15940

That block was mined by Antpool, not p2pool. Looks like the pool is falsely reporting a block found or the pool is mining in antpool and not p2pool and it found a block for antpool. If the latter is the case and your not the pool's owner then you've been scammed.

Pages: « 1 ... 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 [797] 798 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!