[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[daemondazz]

There is definitely something strange with Terracoin, I continue having troubles with TRC and P2pool but today the Terracoin wallet is unable to synchronize with the network from about 5 hours ago and it's stuck in block 175050.  

Currently on block 175055. Have you updated the latest build?

[K1773R]

rename thread, 2000/2500 GH/s

[forrestv]

rename thread, 2000/2500 GH/s

Done, thanks!

[roy7]

TRC was flooded with hashpower, I think they 51%'ed it as well to orphan other miners blocks.  The difficulty was jacked up through through the roof, then then left.

https://bitcointalk.org/index.php?topic=261986.0

This is the downside of short interval difficulty adjustment.  The two week one on bitcoin may be annoying but the short ones are open to far more abuse.

It wasn't just a flood of hash power, it was a bug in the difficulty adjustment someone was exploiting so they could mine blocks for their attack at 5K difficulty even if everyone else was on a far higher difficulty. Can't win that battle. Hash power coming and going isn't a problem for TRC because of the EMA difficulty adjustments, which were added earlier this year when hash power could mess things up.

[Krellan]

Just upgraded p2pool from 13.1 to 13.2.

It's downloading the sharechain again now, and seemingly having a rough time of it: there's lots of Python exceptions all over the place, about connections lost.  My Internet connection is reasonably good, though, so unsure why it would be doing this (13.1 was fine).

What's the preferred way to upgrade from 13.1 to 13.2?  Is there a way to do it without losing the downloaded sharechain?  I just did a rough upgrade, unpacking a new directory and then running Python in there, as if I had never used p2pool before.

Also, what version is best to use?  The tarball release of 13.2 from github?  The master branch from github, cloned with "git clone"?
 
Thanks!
Josh

[mdude77]

Just upgraded p2pool from 13.1 to 13.2.

It's downloading the sharechain again now, and seemingly having a rough time of it: there's lots of Python exceptions all over the place, about connections lost.  My Internet connection is reasonably good, though, so unsure why it would be doing this (13.1 was fine).

What's the preferred way to upgrade from 13.1 to 13.2?  Is there a way to do it without losing the downloaded sharechain?  I just did a rough upgrade, unpacking a new directory and then running Python in there, as if I had never used p2pool before.

Also, what version is best to use?  The tarball release of 13.2 from github?  The master branch from github, cloned with "git clone"?
 
Thanks!
Josh

When I upgrade, I usually unzip the new version into a new folder, move the data folder from the one to the new one, rename the old folder, then name the new folder the proper name (so my script still works).

As for the connections lost .. p2pool doesn't appear to be multithreaded.  When downloading the sharechain it gets bogged down in processing the shares that other things don't get processor power, so when it's done processing the shares, timeouts/errors pop up all over the place.

I always use the latest .zip from github.

M

[Krellan]

This might be the wrong place to ask, but I'm wondering if anybody else has done this.

I don't want my port 9332 open to the world (p2pool Web stats and RPC interface).

I have a Web server on my machine, so tried to redirect it, so that this would work:

URL typed into browser of https://mysite.example.com/whatever/p2pool/ --> proxy to http://127.0.0.1:9332/ on the inside

Although I wanted to keep port 9332 closed from the outside world, I still want to be able to peek at it remotely, and so have added https and HTTP basic authentication to it (old style username/password) on my Web server's configuration.

This works, but unfortunately it's fooled by the JavaScript and HTML in the page, having lots of links to places like "/static" and "/local_stats" and such.

I know my Apache proxy is working, because I get p2pool's redirections, such as when you go to port 9332 without a location and it redirects you to the "/static" webpage: https://mysite.example.com/static

This doesn't work on my setup, because I need to have p2pool be under a path, not at the toplevel.

So, I tried the "mod_proxy_html" Apache module.  This seems to be failing to parse the HTML generated by p2pool, and rewriting it correctly, because then, after doing that, I no longer get toplevel redirects, but I get p2pool to tell me "No such resource/No such child resource".  This text is an error from p2pool, not my Web server, so I know that at least the proxy is getting through to p2pool and making the internal connection to port 9332 correctly.  However, something's getting garbled in my attempt to rewrite the HTML, so it's not getting forwarded to the correct URL, I think.  Unfortunately my debugging trail goes cold there, I have no visibility into the internal workings of mod_proxy_html unfortunately.

Has anybody else confronted this problem?

I wonder if it would be possible to rewrite the links in p2pool's Web interface to be relative, not absolute, paths?  Would there be a technical problem with this, or would it be a good improvement to have?

Thanks!
Josh

[Krellan]

When I upgrade, I usually unzip the new version into a new folder, move the data folder from the one to the new one, rename the old folder, then name the new folder the proper name (so my script still works).

As for the connections lost .. p2pool doesn't appear to be multithreaded.  When downloading the sharechain it gets bogged down in processing the shares that other things don't get processor power, so when it's done processing the shares, timeouts/errors pop up all over the place.

I always use the latest .zip from github.

Nice!

Is the "data" folder the only folder that should be copied from the old to the new, or are there more?  Does the p2pool program write to any files outside the "data" folder?  I should save a before-and-after copy so that I can compare and see what it changes.

Good news, it's settled down now, and starts to work.  It now sees the rest of the pool.  Thanks for letting me know that the timeouts/errors are normal when downloading the initial sharechain.

Josh

[K1773R]

This might be the wrong place to ask, but I'm wondering if anybody else has done this.

I don't want my port 9332 open to the world (p2pool Web stats and RPC interface).

I have a Web server on my machine, so tried to redirect it, so that this would work:

URL typed into browser of https://mysite.example.com/whatever/p2pool/ --> proxy to http://127.0.0.1:9332/ on the inside

Although I wanted to keep port 9332 closed from the outside world, I still want to be able to peek at it remotely, and so have added https and HTTP basic authentication to it (old style username/password) on my Web server's configuration.

This works, but unfortunately it's fooled by the JavaScript and HTML in the page, having lots of links to places like "/static" and "/local_stats" and such.

I know my Apache proxy is working, because I get p2pool's redirections, such as when you go to port 9332 without a location and it redirects you to the "/static" webpage: https://mysite.example.com/static

This doesn't work on my setup, because I need to have p2pool be under a path, not at the toplevel.

So, I tried the "mod_proxy_html" Apache module.  This seems to be failing to parse the HTML generated by p2pool, and rewriting it correctly, because then, after doing that, I no longer get toplevel redirects, but I get p2pool to tell me "No such resource/No such child resource".  This text is an error from p2pool, not my Web server, so I know that at least the proxy is getting through to p2pool and making the internal connection to port 9332 correctly.  However, something's getting garbled in my attempt to rewrite the HTML, so it's not getting forwarded to the correct URL, I think.  Unfortunately my debugging trail goes cold there, I have no visibility into the internal workings of mod_proxy_html unfortunately.

Has anybody else confronted this problem?

I wonder if it would be possible to rewrite the links in p2pool's Web interface to be relative, not absolute, paths?  Would there be a technical problem with this, or would it be a good improvement to have?

Thanks!
Josh

why running apache public instead of p2pool? thats like having a more secure system (p2pool) and replace it with somethin vulnerable (apache2)...

[Mobius]

Just upgraded p2pool from 13.1 to 13.2.

It's downloading the sharechain again now, and seemingly having a rough time of it: there's lots of Python exceptions all over the place, about connections lost.  My Internet connection is reasonably good, though, so unsure why it would be doing this (13.1 was fine).

What's the preferred way to upgrade from 13.1 to 13.2?  Is there a way to do it without losing the downloaded sharechain?  I just did a rough upgrade, unpacking a new directory and then running Python in there, as if I had never used p2pool before.

Also, what version is best to use?  The tarball release of 13.2 from github?  The master branch from github, cloned with "git clone"?
 
Thanks!
Josh

best is github version

[AtomSea]

My friend and I started a p2pool BTC node and every 65 minutes or so we get about 4 minutes of downtime.
See the graph here:
http://108.161.129.247:9332/static/

What could be causing these downspikes?

Thanks for the great idea/community everyone!

[daemondazz]

My friend and I started a p2pool BTC node and every 65 minutes or so we get about 4 minutes of downtime.
See the graph here:
http://108.161.129.247:9332/static/

What could be causing these downspikes?

Thanks for the great idea/community everyone!

Check for cron jobs that fire at 15 past every hour, it could be that the server you are running it on is a bit underpowered in some way (most likely disk IO bandwidth) and it can't keep up.

[lenny_]

My friend and I started a p2pool BTC node and every 65 minutes or so we get about 4 minutes of downtime.
See the graph here:
http://108.161.129.247:9332/static/

What could be causing these downspikes?

Thanks for the great idea/community everyone!

I would check also if your HDD haven't got any bad sectors. Check your system logs and scan your HDD.

[embii]

We nailed the problem.   

We are prepping a completely virtual environment that will allow us to spin up multiple loadbalanced p2p nodes depending on load.
The problem was caused by virtual clock drift.  The system time was drifting approximately 5 minutes every hour which caused an automatic re sync of the time.  The time re sync appeared as a total drop in the charts when actually no drops were occurring.  We have corrected the time drift issue and are no longer seeing the drops.

Thanks for all your suggestions.

EMBTCII

[zvs]

someone decided to DDoS my server, so my 3ghash mining operation was offline for a few minutes, before redirecting to another pool

the network is saved

[notme]

Wow, 3.75 TH/s at the moment.  Good job p2pool!

[ebereon]

Wow, 3.75 TH/s at the moment.  Good job p2pool!

yeah, i like that too, thats because of 13.2 which is fully ASIC combatible. I hope everyone is running p2pool with a minimum of --give-author 0.1, Better 0.5! Forrest did a good job with version 13.2!

I run 2 Avalons on my public node very stable. ava.0 is a original batch#2 which is restarting from time to time but without huge hashrate impact and ava.1 is simply connected to a notebook.

If someone want to test p2pool with your ASIC befor making an own node, my public node is in my sig (100mbit)  

[freshzive]

Is it ok to run cgminer using the --balance command with P2Pool + one non p2pool?

Trying to even out payouts a little more using this method, but it looks like P2Pool may be large enough to get rid of that serious variance here soon

[Krak]

Is it ok to run cgminer using the --balance command with P2Pool + one non p2pool?

Trying to even out payouts a little more using this method, but it looks like P2Pool may be large enough to get rid of that serious variance here soon

I wouldn't recommend it; p2pool basically uses its own separate block chain for shares. According to the third line in the cgminer readme:

Do not use on multiple block chains at the same time!

[zarethernet]

You need to run a proxy on STRATUM p2pool LTC, tell me what to do. That's after the launch.
To connect with the striatum is not possible.

python mining_proxy.py --pow-algo scrypt --host 127.0.0.1 --port 9327 --stratum-port 3333 --getwork-port 9999 --no-midstate --real-target --custom-user pool --custom-password x --verbose
2013-07-31 21:44:08,350 DEBUG stats logger.get_logger # Logging initialized
2013-07-31 21:44:08,360 DEBUG protocol logger.get_logger # Logging initialized
2013-07-31 21:44:08,360 DEBUG socket_transport logger.get_logger # Logging initialized
2013-07-31 21:44:08,415 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,415 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,433 WARNING proxy jobs. # C extension for midstate not available. Using default implementation instead.
2013-07-31 21:44:08,433 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,434 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,435 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,435 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,436 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,436 DEBUG proxy logger.get_logger # Logging initialized
2013-07-31 21:44:08,438 ERROR proxy mining_proxy.main # Stratum host/port autodetection failed
Traceback (most recent call last):
File "mining_proxy.py", line 155, in main
new_host = (yield utils.detect_stratum(args.host, args.port))
AttributeError: 'module' object has no attribute '_parse'
2013-07-31 21:44:08,439 INFO proxy mining_proxy.main # Stratum proxy version: 1.3.0
2013-07-31 21:44:08,439 INFO proxy mining_proxy.main # Trying to connect to Stratum pool at 127.0.0.1:9327
2013-07-31 21:44:08,440 INFO proxy mining_proxy.main # Setting PoW algo: scrypt
2013-07-31 21:44:08,441 INFO stats stats.print_stats # 1 peers connected, state changed 1 times
2013-07-31 21:44:08,441 DEBUG protocol protocol.connectionMade # Connected 127.0.0.1
2013-07-31 21:44:08,441 DEBUG protocol protocol.connectionMade # Resuming connection: []
2013-07-31 21:44:08,441 INFO proxy mining_proxy.on_connect # Connected to Stratum pool at 127.0.0.1:9327
2013-07-31 21:44:08,441 INFO proxy mining_proxy.on_connect # Authorizing custom user pool, password x
2013-07-31 21:44:08,442 DEBUG protocol protocol.writeJsonRequest # < {"params": ["pool", "x"], "id": 1, "method": "mining.authorize"}
2013-07-31 21:44:08,442 INFO proxy mining_proxy.on_connect # Subscribing for mining jobs
2013-07-31 21:44:08,442 DEBUG protocol protocol.writeJsonRequest # < {"params": [], "id": 2, "method": "mining.subscribe"}
2013-07-31 21:44:08,461 DEBUG protocol protocol.lineReceived # > {u'result': None, u'jsonrpc': u'2.0', u'id': 1, u'error': None}
2013-07-31 21:44:08,461 DEBUG protocol protocol.lineReceived # > {u'result': [[u'mining.notify', u'ae6812eb4cd7735a302a8a9dd95cf71f'], u'', 2], u'jsonrpc': u'2.0', u'id': 2, u'error': None}
2013-07-31 21:44:08,464 INFO proxy mining_proxy.main # -----------------------------------------------------------------------
2013-07-31 21:44:08,464 INFO proxy mining_proxy.main # PROXY IS LISTENING ON ALL IPs ON PORT 3333 (stratum) AND 9999 (getwork)
2013-07-31 21:44:08,464 INFO proxy mining_proxy.main # -----------------------------------------------------------------------
2013-07-31 21:44:08,465 DEBUG protocol protocol.lineReceived # > {u'jsonrpc': u'2.0', u'params': [14.52613027417101], u'method': u'mining.set_difficulty', u'id': 940918514}
2013-07-31 21:44:08,465 INFO proxy client_service.handle_event # Setting new difficulty: 14.5261302742
2013-07-31 21:44:08,465 DEBUG protocol protocol.writeJsonResponse # < {"error": null, "id": 940918514, "result": null}
/usr/lib/python2.6/site-packages/stratum-0.2.13-py2.6.egg/stratum/protocol.py:201: DeprecationWarning: Returning a value other than None from stratum.protocol.ClientProtocol.dataReceived is deprecated since Twisted 11.0.0.
return self.lineLengthExceeded(self._buffer)
2013-07-31 21:44:08,489 INFO proxy mining_proxy.on_disconnect # Disconnected from Stratum pool at 127.0.0.1:9327
2013-07-31 21:44:08,489 INFO stats stats.print_stats # 0 peers connected, state changed 1 times
2013-07-31 21:44:08,489 DEBUG socket_transport socket_transport.clientConnectionLost # [Failure instance: Traceback (failure with no frames): : Connection to the other side was lost in a non-clean fashion.
]
2013-07-31 21:44:11,332 INFO stats stats.print_stats # 1 peers connected, state changed 1 times
2013-07-31 21:44:11,332 DEBUG protocol protocol.connectionMade # Connected 127.0.0.1
2013-07-31 21:44:11,332 DEBUG protocol protocol.connectionMade # Resuming connection: []
2013-07-31 21:44:11,332 INFO proxy mining_proxy.on_connect # Connected to Stratum pool at 127.0.0.1:9327
2013-07-31 21:44:11,333 INFO proxy mining_proxy.on_connect # Authorizing custom user pool, password x
2013-07-31 21:44:11,333 DEBUG protocol protocol.writeJsonRequest # < {"params": ["pool", "x"], "id": 1, "method": "mining.authorize"}
2013-07-31 21:44:11,333 INFO proxy mining_proxy.on_connect # Subscribing for mining jobs
2013-07-31 21:44:11,333 DEBUG protocol protocol.writeJsonRequest # < {"params": [], "id": 2, "method": "mining.subscribe"}
2013-07-31 21:44:11,337 DEBUG protocol protocol.lineReceived # > {u'result': None, u'jsonrpc': u'2.0', u'id': 1, u'error': None}
2013-07-31 21:44:11,338 DEBUG protocol protocol.lineReceived # > {u'result': [[u'mining.notify', u'ae6812eb4cd7735a302a8a9dd95cf71f'], u'', 2], u'jsonrpc': u'2.0', u'id': 2, u'error': None}
2013-07-31 21:44:11,338 DEBUG protocol protocol.lineReceived # > {u'jsonrpc': u'2.0', u'params': [14.52613027417101], u'method': u'mining.set_difficulty', u'id': 512789334}
2013-07-31 21:44:11,339 INFO proxy client_service.handle_event # Setting new difficulty: 14.5261302742
2013-07-31 21:44:11,339 DEBUG protocol protocol.writeJsonResponse # < {"error": null, "id": 512789334, "result": null}
2013-07-31 21:44:11,340 INFO proxy mining_proxy.on_disconnect # Disconnected from Stratum pool at 127.0.0.1:9327
2013-07-31 21:44:11,340 INFO stats stats.print_stats # 0 peers connected, state changed 1 times
2013-07-31 21:44:11,340 DEBUG socket_transport socket_transport.clientConnectionLost # [Failure instance: Traceback (failure with no frames): : Connection to the other side was lost in a non-clean fashion.