[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[KyrosKrane]

sign up to BizSpark or dreamspark and get access to free Azure credit. enough to run a dual core 3.5gb sever for the whole moth for free.

http://www.microsoft.com/bizspark/default.aspx
https://www.dreamspark.com/

That doesn't solve any issues, the blockchain download alone to run p2pool is greater than 3.5 gb

Edit: I'm sorry, I'm tired and didn't realize you were talking about RAM.  You could also use a service like linode.com.  $20/mth gets you 2gb or RAM and a dual core for a month, where Azure becomes $111 to $134 /mth after your free month.  You can also get 4gb of RAM and 4 cores from linode.com for $40/mth.  There is also digitalocean.com.

Another option I've mentioned before is a company called CrownCloud.net. They're a bitcoin-friendly hosting provider. They don't allow cloud mining, but they do allow you to run a bitcoin node and pool software, and they also take payment in bitcoin. Two of their packages in particular stand out. One's an OpenVPS package in Frankfurt, Germany, and the other is in their LA location. Both offer 3GB RAM, 50GB HDD, and plenty of CPU and bandwidth for $30 annually. Perfect for hosting a full bitcoin node and p2pool.  Those particular packages are frequently out of stock; I was able to nab one by following them on Twitter and waiting for them to announce more capacity.

[1714021935]

1714021935

[1714021935]

1714021935

[norgan]

sign up to BizSpark or dreamspark and get access to free Azure credit. enough to run a dual core 3.5gb sever for the whole moth for free.

http://www.microsoft.com/bizspark/default.aspx
https://www.dreamspark.com/

That doesn't solve any issues, the blockchain download alone to run p2pool is greater than 3.5 gb

Edit: I'm sorry, I'm tired and didn't realize you were talking about RAM.  You could also use a service like linode.com.  $20/mth gets you 2gb or RAM and a dual core for a month, where Azure becomes $111 to $134 /mth after your free month.  You can also get 4gb of RAM and 4 cores from linode.com for $40/mth.  There is also digitalocean.com.

Another option I've mentioned before is a company called CrownCloud.net. They're a bitcoin-friendly hosting provider. They don't allow cloud mining, but they do allow you to run a bitcoin node and pool software, and they also take payment in bitcoin. Two of their packages in particular stand out. One's an OpenVPS package in Frankfurt, Germany, and the other is in their LA location. Both offer 3GB RAM, 50GB HDD, and plenty of CPU and bandwidth for $30 annually. Perfect for hosting a full bitcoin node and p2pool.  Those particular packages are frequently out of stock; I was able to nab one by following them on Twitter and waiting for them to announce more capacity.

Guys, look into both of those programs. I've had $160US/month of Azure credit for 3 years now. It's not just one month free!
They have dropped the allowance now to $110us/month but still great for any student or person with registered business or tax id.
you get quite a bit of storage included in that. I a whole bunch of machines under my msdn account but I can tell you that the $110 allowance would be plenty of ram, cpu, storage and bandwidth to run a node quite happily.

[ceslick]

I will set one up here in cloud crown and see how it goes. Would appreciate any support from miners there once complete

[MineForeman.com]

sign up to BizSpark or dreamspark and get access to free Azure credit. enough to run a dual core 3.5gb sever for the whole moth for free.

http://www.microsoft.com/bizspark/default.aspx
https://www.dreamspark.com/

That doesn't solve any issues, the blockchain download alone to run p2pool is greater than 3.5 gb

Edit: I'm sorry, I'm tired and didn't realize you were talking about RAM.  You could also use a service like linode.com.  $20/mth gets you 2gb or RAM and a dual core for a month, where Azure becomes $111 to $134 /mth after your free month.  You can also get 4gb of RAM and 4 cores from linode.com for $40/mth.  There is also digitalocean.com.

Another option I've mentioned before is a company called CrownCloud.net. They're a bitcoin-friendly hosting provider. They don't allow cloud mining, but they do allow you to run a bitcoin node and pool software, and they also take payment in bitcoin. Two of their packages in particular stand out. One's an OpenVPS package in Frankfurt, Germany, and the other is in their LA location. Both offer 3GB RAM, 50GB HDD, and plenty of CPU and bandwidth for $30 annually. Perfect for hosting a full bitcoin node and p2pool.  Those particular packages are frequently out of stock; I was able to nab one by following them on Twitter and waiting for them to announce more capacity.

Guys, look into both of those programs. I've had $160US/month of Azure credit for 3 years now. It's not just one month free!
They have dropped the allowance now to $110us/month but still great for any student or person with registered business or tax id.
you get quite a bit of storage included in that. I a whole bunch of machines under my msdn account but I can tell you that the $110 allowance would be plenty of ram, cpu, storage and bandwidth to run a node quite happily.

The actual setup of the p2pool servers is the easy bit.  If I were doing it I would use Amazon EC2 with SSD's etc... (Auzure is just insane for a project like this) Load Balances to mitagate DDOS, DNS to direct the user to the closes server ... all the good stuff.  My real life job has me doing something similar.

The real issue is getting community support behind it in order to pay for it.  If I could work out how to do that I would spin up a set of servers tomorrow.  Perhaps a crowdfunding campaign?

Neil

[nreal]

What should be edited in the code to be able to increase nodes minimium share difficulty, it would be much easier than change it with every miner. (walletaddress/100 000000)...100M

[norgan]

The actual setup of the p2pool servers is the easy bit.  If I were doing it I would use Amazon EC2 with SSD's etc... (Auzure is just insane for a project like this) Load Balances to mitagate DDOS, DNS to direct the user to the closes server ... all the good stuff.  My real life job has me doing something similar.

The real issue is getting community support behind it in order to pay for it.  If I could work out how to do that I would spin up a set of servers tomorrow.  Perhaps a crowdfunding campaign?

Neil

why is Azure insane for a project like this? I was suggesting it for individuals to run nodes. All you need is a business or to be a student. it's a good deal and I have nodes running on the Singapore dc's with great stats.

Azure has full load balancing and all the good stuff that Amazon has. To be quite frank they are almost identical but I've managed to get the Azure credit with free hosting and Amazon provide nothing like that.
 

[KyrosKrane]

The actual setup of the p2pool servers is the easy bit.  If I were doing it I would use Amazon EC2 with SSD's etc... (Auzure is just insane for a project like this) Load Balances to mitagate DDOS, DNS to direct the user to the closes server ... all the good stuff.  My real life job has me doing something similar.

The real issue is getting community support behind it in order to pay for it.  If I could work out how to do that I would spin up a set of servers tomorrow.  Perhaps a crowdfunding campaign?

Neil

I think we're talking about two different things. The setups we're suggesting so far are suitable for smaller installs, like a pool suited for a few users, or one user with a goodly-sized farm. If we're talking about pools that can handle industrial-strength mining farms, or a "centralized" p2pool node with thousands of users, then it becomes a whole other ballgame. In that case, you're correct, it would take a much more resilient configuration.

[PatMan]

The problem, as I've said on many occasions, is not a lack of nodes - there are literally hundreds of public nodes & hundreds more private nodes. Even if there were millions of nodes p2pool would be in exactly the same situation - all the nodes in the world won't make a difference if there is no hardware capable of mining on them efficiently/effectively. The number of nodes will increase with the number of miners who are able to use them - why would anyone want to set-up a node (and pay for the privilege) if nobody can mine on it?

I'm afraid the fate of p2pool is in the hands of hardware manufacturers, as it's obvious the p2pool dev doesn't give a shit. If Bitmain can't/don't manage to release a firmware that works with p2pool, then it will only be a matter of time before the older asics that currently work with p2pool become unprofitable to run & get switched off, as is already happening with the S1's depending on the cost of electricity in any particular country. This is what has happened to me, after mining with p2pool for over 2 years, and it will happen to everyone else soon enough unless p2pool &/or hardware manufacturers bang their heads together & work something out.

It's sad that p2pool is relying completely on third parties for it's existence, expecting hardware manufacturers to change their set-ups to accommodate p2pool instead of p2pool developing to accommodate more hardware. I hold forrestv 100% responsible for this situation, even more so after his last cry baby statement. It's not as if he wasn't warned about this 15 months ago - but absolutely nothing has been done & he continues to ignore everyone. It's sad really.

[nreal]

Dont count the rejects, count the shares you get.
Dont mine with minimal settings with 1th hardware - its for usb block erupters, use minimium walletaddress/10 000000 and see what happens in a week or two...

Remember to relax

[JakeTri]

At the moment my conclusion is a proxy isn't going to help getting S2s working with p2pool.

I believe the problem is the constant "drop EVERYTHING you are doing and restart" messages that p2pool sends every 30 seconds.  Then it rejects all the old work coming p2pool leading to large DOA.

My recommendations for p2pool to become more S2 friendly:

1 - Sending new work every 30 seconds is fine.  It's apparently not necessary to force the miner to restart "clean jobs" = true.  It also needs to gracefully accept work from prior job IDs for a period of time like other pools do.

2 - Start at a higher pseudo difficulty than 1.

3 - Allow a higher pseudo difficulty than 1000.

4 - Send the actual errors back for rejected work, instead of just "false".

If I understood python I'd take a stab at 2 through 4.  1 might be a big deal.

Here are 2 patches for p2pool that address point 2 and 3 from your list.

First a simple patch that allow higher than 1000 pseudo-difficulty.
https://github.com/jaketri/p2pool/commit/05b630f2c8f93b78093043b28c0c543fafa0a856

And another patch that add "--min-difficulty" parameter to p2pool. For my setup I use 256 as start pseudo difficulty.

https://github.com/jaketri/p2pool/commit/5f02f893490f2b9bfa48926184c4b1329c4d1554

Thanks, that's good.  Now why isn't it in the main code?

M

See pull request #187 ( https://github.com/forrestv/p2pool/pull/187 ) created March 3, 2014 for p2pool main code.

I continue to play a little bit with point 1 and 4 from your initial list and created 2 more experimental patches:

First patch is to request work restart only once for each block.
https://github.com/jaketri/p2pool/commit/0b0c3473c0a67f24f4bc9b95de002770c69828ef

Second patch is to accept any valid work even stale or DOA and let the share verification decide if share is valid or not.
https://github.com/jaketri/p2pool/commit/875c3483b82d50ce77f153f46ce7c20e193f17c0

This 2 patches help a lot for my Bitfury miner (without them I have about 25-30% DOA) but unfortunately I do not have an S2 to test.

Here is my experimental p2pool node running on port 19332 with this patches applied in case anyone want to do a quick test with S2 and let me know if they see any improvement.

p2pool.25u.com:19332

Keep in mind that p2pool node with above patches will not reject any valid work but the shares may get rejected later when adding them to share chain.

Note: On same address but on the standard p2pool port 9332 is a p2pool server without this patches if you want to see results with and without the patches.

[HellDiverUK]

I'm hopefully going to have a p2pool node up and running again in the next week or so, it'll be on a VPS with a static IP, rather than hanging off my home ADSL line as before.  Details to follow.

[raskul]

installing blockchain on iceland.centralcavern.uk

[bryonp]

See pull request #187 ( https://github.com/forrestv/p2pool/pull/187 ) created March 3, 2014 for p2pool main code.

I continue to play a little bit with point 1 and 4 from your initial list and created 2 more experimental patches:

First patch is to request work restart only once for each block.
https://github.com/jaketri/p2pool/commit/0b0c3473c0a67f24f4bc9b95de002770c69828ef

Second patch is to accept any valid work even stale or DOA and let the share verification decide if share is valid or not.
https://github.com/jaketri/p2pool/commit/875c3483b82d50ce77f153f46ce7c20e193f17c0

This 2 patches help a lot for my Bitfury miner (without them I have about 25-30% DOA) but unfortunately I do not have an S2 to test.

Here is my experimental p2pool node running on port 19332 with this patches applied in case anyone want to do a quick test with S2 and let me know if they see any improvement.

p2pool.25u.com:19332

Keep in mind that p2pool node with above patches will not reject any valid work but the shares may get rejected later when adding them to share chain.

Note: On same address but on the standard p2pool port 9332 is a p2pool server without this patches if you want to see results with and without the patches.

[/quote]

I am hooked in there with you....lets see.......... this is a single S2

[MineForeman.com]

why is Azure insane for a project like this? I was suggesting it for individuals to run nodes. All you need is a business or to be a student. it's a good deal and I have nodes running on the Singapore dc's with great stats.

Sorry, I meant running a distributed pool of p2pool servers on a MSDN credit Azure account is insane .

I have some doubts that Azure would be the right platform p2pool in general though, there are better (technically that is) alternatives out there.  I don't know if you have used EC2 but there is a reason that the "big players" use it.

Neil

[MineForeman.com]

The problem, as I've said on many occasions, is not a lack of nodes - there are literally hundreds of public nodes & hundreds more private nodes.

Agreed, but I am meaning a set of nodes that have the "P2Poll Community Approved" stamp on them.

At the moment it is not even clear that you can use other peoples nodes.  Even if you do work it out there are literally tones of them run by anonymous people of dubious ethics (I mean from the outsiders point of view).  There is also no way of being able to trust them with your hash'es, they could be charging a 10% fee and have "tweeked" their code.

Neil

[sconklin321]

What should be edited in the code to be able to increase nodes minimium share difficulty, it would be much easier than change it with every miner. (walletaddress/100 000000)...100M

I think the main reason it's set with your worker name is so that each miner mining on a node can set their share value without having to override what the node operator wants to mine at to get the default value.

[norgan]

why is Azure insane for a project like this? I was suggesting it for individuals to run nodes. All you need is a business or to be a student. it's a good deal and I have nodes running on the Singapore dc's with great stats.

Sorry, I meant running a distributed pool of p2pool servers on a MSDN credit Azure account is insane .

I have some doubts that Azure would be the right platform p2pool in general though, there are better (technically that is) alternatives out there.  I don't know if you have used EC2 but there is a reason that the "big players" use it.

Neil

my azure node runs more efficiently than my VMware one locally. the whole EC2 v Azure argument is mute since they are both built for the same kind of performance and scalability. it really comes down to interface and any specific features. I've spent a lot of time with Azure and migrated our company to it. "The big players use it" (EC2) because they were up first and they committed to it, Azure is the underdog that has really proven to deliver a very rich featureset and give AWS a run for their money. Even Amazon admit that MS do SQL better on Azure than they can.

Anyway, this is not exactly a hosting thread so i'll leave it at that and you can decide for yourself.

[mdude77]

At the moment my conclusion is a proxy isn't going to help getting S2s working with p2pool.

I believe the problem is the constant "drop EVERYTHING you are doing and restart" messages that p2pool sends every 30 seconds.  Then it rejects all the old work coming p2pool leading to large DOA.

My recommendations for p2pool to become more S2 friendly:

1 - Sending new work every 30 seconds is fine.  It's apparently not necessary to force the miner to restart "clean jobs" = true.  It also needs to gracefully accept work from prior job IDs for a period of time like other pools do.

2 - Start at a higher pseudo difficulty than 1.

3 - Allow a higher pseudo difficulty than 1000.

4 - Send the actual errors back for rejected work, instead of just "false".

If I understood python I'd take a stab at 2 through 4.  1 might be a big deal.

Here are 2 patches for p2pool that address point 2 and 3 from your list.

First a simple patch that allow higher than 1000 pseudo-difficulty.
https://github.com/jaketri/p2pool/commit/05b630f2c8f93b78093043b28c0c543fafa0a856

And another patch that add "--min-difficulty" parameter to p2pool. For my setup I use 256 as start pseudo difficulty.

https://github.com/jaketri/p2pool/commit/5f02f893490f2b9bfa48926184c4b1329c4d1554

Thanks, that's good.  Now why isn't it in the main code?

M

See pull request #187 ( https://github.com/forrestv/p2pool/pull/187 ) created March 3, 2014 for p2pool main code.

I continue to play a little bit with point 1 and 4 from your initial list and created 2 more experimental patches:

I don't think #1 is fixable on the p2pool side.  Since I wrote the above, I've learned that p2pool must force a restart for it's share chain, as each share found every ~30 secs is equiv to a block in the BTC chain.  While I can forcible override the jobid to get it by p2pool's edits, I'm pretty sure any shares of greatest size will be rejected by the pool because it's using the wrong data.  (I assume that means it would show up stale.)

M

[JakeTri]

At the moment my conclusion is a proxy isn't going to help getting S2s working with p2pool.

I believe the problem is the constant "drop EVERYTHING you are doing and restart" messages that p2pool sends every 30 seconds.  Then it rejects all the old work coming p2pool leading to large DOA.

My recommendations for p2pool to become more S2 friendly:

1 - Sending new work every 30 seconds is fine.  It's apparently not necessary to force the miner to restart "clean jobs" = true.  It also needs to gracefully accept work from prior job IDs for a period of time like other pools do.

2 - Start at a higher pseudo difficulty than 1.

3 - Allow a higher pseudo difficulty than 1000.

4 - Send the actual errors back for rejected work, instead of just "false".

If I understood python I'd take a stab at 2 through 4.  1 might be a big deal.

Here are 2 patches for p2pool that address point 2 and 3 from your list.

First a simple patch that allow higher than 1000 pseudo-difficulty.
https://github.com/jaketri/p2pool/commit/05b630f2c8f93b78093043b28c0c543fafa0a856

And another patch that add "--min-difficulty" parameter to p2pool. For my setup I use 256 as start pseudo difficulty.

https://github.com/jaketri/p2pool/commit/5f02f893490f2b9bfa48926184c4b1329c4d1554

Thanks, that's good.  Now why isn't it in the main code?

M

See pull request #187 ( https://github.com/forrestv/p2pool/pull/187 ) created March 3, 2014 for p2pool main code.

I continue to play a little bit with point 1 and 4 from your initial list and created 2 more experimental patches:

I don't think #1 is fixable on the p2pool side.  Since I wrote the above, I've learned that p2pool must force a restart for it's share chain, as each share found every ~30 secs is equiv to a block in the BTC chain.  While I can forcible override the jobid to get it by p2pool's edits, I'm pretty sure any shares of greatest size will be rejected by the pool because it's using the wrong data.  (I assume that means it would show up stale.)

M

This is also my understanding and I when p2pool server is forcing a work restart it will be a dead time until miner is able to get a share based on new data.
I was thinking that miner start using the new work data with or without a work restart but with the work restart it will just drop the old work. So if p2pool server does not issue a work restart it is possible for the miner to find valid shares based on old work until the new work data is used.

If my theory is wrong and p2pool server does not issue work restart every ~30 sec then miners should get more orphan and/or dead shares. To test this I started this experimental p2pool server and so far from 13 found shares I got only 2 orphans and 0 dead. This does not prove anything yet but so far either my miner was very very lucky or the results are going somewhere.

[nreal]

Increasing share period to 2 min, all problems solved  

 SHARE_PERIOD=30, # seconds

 SHARE_PERIOD=120, # seconds

Why not? What is wrong with that? Then Bitcoind GetBlockTemplate Latency wouldnt mean as much as it does now either - could accept more transactions -> more income.....