[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[runeks]

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.

What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?

[rjk]

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.

What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?

For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.

[NothinG]

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.

What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?

For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.

if optimized.

[DeathAndTaxes]

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.

What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?

For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.

The integration into client would be worthwhile IMHO.  Honestly any future alt-chain should consider that from day 1.  Making the p2pool an integral part of protocol.

[runeks]

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.

What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?

For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.

I wasn't aware of Gavin wanting to merge p2pool with bitcoind. That's interesting. Personally, I would say keep them separate. I like the "Write programs that do one thing and do it well"-approach. bitcoind seems like a building block for block chain-related applications to me (though a libbitcoin would be better). Integrating a specific block chain-related application into it seems like a strange move to me. But hey, I'm no lead developer of Bitcoin. I can see the logic in making it easier for people to solo mine though, possibly leading people away from pooled mining.

Regarding the pure C implementation to improve efficiency; have you had any problems with the Python implementation underperforming? What kind of cheap hardware are you referring to? I suspect p2pool would run fine on an ARM processor, though I haven't tried. Or are you talking about even slower hardware, like - for example - the Atmel CPU that is in a Bitforce Single (so that it can run without a host computer)?

[rjk]

Regarding the pure C implementation to improve efficiency; have you had any problems with the Python implementation underperforming? What kind of cheap hardware are you referring to? I suspect p2pool would run fine on an ARM processor, though I haven't tried. Or are you talking about even slower hardware, like - for example - the Atmel CPU that is in a Bitforce Single (so that it can run without a host computer)?

One example was earlier in this thread, I believe. Someone wanted to use an old P3 with 384mb RAM and probably a similarly old hard drive. It was causing all kinds of issues, maxing out the CPU because it was swapping so much. I dunno if the memory requirement is because of inefficiencies in python or p2pool, or whether the same issue would happen when implemented in C/C++, but I do know that many applications do generally get better all around performance when written in a compiled language, not an interpreted language.

[kano]

It's only 2 lines of code (and 2 braces) in init.cpp and 1 line added to the unix makefile to add a whole new module thread onto bitcoind ...
(I do that every time I upgrade my bitcoind - I got sick of editing my changes in every time )

So hoping for Gavin to do anything is no reason at all to decide for or against doing it.

[DeepBit]

Blocks in shorter chains (or invalid chains) are called "orphan blocks", and while they are stored, they are not used for anything. When a block becomes an orphan block, all of its valid transactions are re-added to the pool of queued transactions and will be included in another block. The 50 BTC reward for the orphan block will be lost, which is why a network-enforced 100-block maturation time for generations exists.

I think it's actually a 120-block mature time, though.

120 is not network-enforced, your client is just too careful :)

[TurdHurdur]

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.

You want someone to write twisted in C?

[O_Shovah]

Just as info.

I have been running bitcoind  bitcoin-qt  p2pool and multiple miners on different ARM boards.

While the miners and bitcoind and Bitcoin-QT run without problems and little Cpu and modest RAM usage on a 600Mhz ARM
p2pool consumes to much CPU power.

The consumption of CPu power is largely scaling with the amount of shares distributed by the miners connected.
I found a miner with ~ 400Mh/s to be to CPU consuming for a 600Mhz and a 1000Mhz ARM CPU.

If the CPU power isn't sufficient it triggers the effect that it works for a while ~45 min.
Afther that the amount of "Timed out while getting work from p2pool" raises significantly.
Other the whole time the p2pool client starts lagging behind the network's time/share count
and finally ends up going unsync afther ~ 60 min.

I will try on my pandaboard these days.

[Ente]

Oh, I would absolutely love a single package/installer/binary which installs/runs everything to start p2poolmining! And yes, the current approach with python, modules, p2pool, bitcoind, blockchain and sharechain seems too complicated to pack into one block.. So yes, I see your point in a complete redesign. I dont know enough about the inner workings of all that (monolithic? libraries? dynamic linking?) to decide for or against c, python, modular, monolithic etc.

However I would, at this point, prefer to not integrate p2pool into satoshi client. Maybe integrate a bitcoin client/daemon into p2pool, but the default should be two independent programs.
The current events with rc1, rc2, mandatory updates in quick order and many unstable or not working combinations give me a bad feeling about merging the two to a single, standard client..

What would be the way to make a new, better p2pool version? One program which has everything included (graphs, dependencies)? It wouldnt make a big difference if its C or python then. There is that python-to-exe version too, for windows, apparently?

People would still need bitcoind (if you dont integrate that). People still need a miner (if you dont integrate that). People still need the radeon "runtime". It sounds like a gigantic project to deliver an all-in-one download-doubleclick-mine solution..

Ente

[kjj]

Oh, I would absolutely love a single package/installer/binary which installs/runs everything to start p2poolmining! And yes, the current approach with python, modules, p2pool, bitcoind, blockchain and sharechain seems too complicated to pack into one block.. So yes, I see your point in a complete redesign. I dont know enough about the inner workings of all that (monolithic? libraries? dynamic linking?) to decide for or against c, python, modular, monolithic etc.

However I would, at this point, prefer to not integrate p2pool into satoshi client. Maybe integrate a bitcoin client/daemon into p2pool, but the default should be two independent programs.
The current events with rc1, rc2, mandatory updates in quick order and many unstable or not working combinations give me a bad feeling about merging the two to a single, standard client..

What would be the way to make a new, better p2pool version? One program which has everything included (graphs, dependencies)? It wouldnt make a big difference if its C or python then. There is that python-to-exe version too, for windows, apparently?

People would still need bitcoind (if you dont integrate that). People still need a miner (if you dont integrate that). People still need the radeon "runtime". It sounds like a gigantic project to deliver an all-in-one download-doubleclick-mine solution..

Ente

Ahem...

[Ente]

Ahem...

[p2pcoin - a self contained p2pool node - boots from CD, USB or network]

Definitely true, and valuable work for all miners!
The question is, how many % of the total network is mining on dedicated hardware? The new members surely will simply use their regular, only computer, which happens to have a fat gpu for gaming. I dont expect many of them to boot a linux every time they want to mine, and to not mine when they use their computer for other stuff.

Still, I love the linux-bitcoin-remixes, staying far away from M$ for many years now!

Ente

[runeks]

Just as info.

I have been running bitcoind  bitcoin-qt  p2pool and multiple miners on different ARM boards.

While the miners and bitcoind and Bitcoin-QT run without problems and little Cpu and modest RAM usage on a 600Mhz ARM
p2pool consumes to much CPU power.

The consumption of CPu power is largely scaling with the amount of shares distributed by the miners connected.
I found a miner with ~ 400Mh/s to be to CPU consuming for a 600Mhz and a 1000Mhz ARM CPU.

If the CPU power isn't sufficient it triggers the effect that it works for a while ~45 min.
Afther that the amount of "Timed out while getting work from p2pool" raises significantly.
Other the whole time the p2pool client starts lagging behind the network's time/share count
and finally ends up going unsync afther ~ 60 min.

I will try on my pandaboard these days.

Interesting. Have you tried an alternative Python implementation like PyPy? I think that would be what I would try first, but also - in part - because I think PyPy is cool .

Other than that, I think it might be worth the effort to dig into the Python code and take a look at what it is that consumes so much CPU power/RAM. In my experience with Python, a very minor - and seemingly unimportant change - can easily cause a 10x speedup. Simply because the Python code itself is so opaque, in the sense that not many people know what a certain line of Python code actually translate into machine code wise. I can't help think that the current Python code *can* be made to run on older hardware, it just requires some CPU- and RAM-profiling of the code, and making the according adjustments. This has - understandably - not been forrestv's highest priority so far.

[O_Shovah]

Just as info.

I have been running bitcoind  bitcoin-qt  p2pool and multiple miners on different ARM boards.

While the miners and bitcoind and Bitcoin-QT run without problems and little Cpu and modest RAM usage on a 600Mhz ARM
p2pool consumes to much CPU power.

The consumption of CPu power is largely scaling with the amount of shares distributed by the miners connected.
I found a miner with ~ 400Mh/s to be to CPU consuming for a 600Mhz and a 1000Mhz ARM CPU.

If the CPU power isn't sufficient it triggers the effect that it works for a while ~45 min.
Afther that the amount of "Timed out while getting work from p2pool" raises significantly.
Other the whole time the p2pool client starts lagging behind the network's time/share count
and finally ends up going unsync afther ~ 60 min.

I will try on my pandaboard these days.

Interesting. Have you tried an alternative Python implementation like PyPy? I think that would be what I would try first, but also - in part - because I think PyPy is cool .

Other than that, I think it might be worth the effort to dig into the Python code and take a look at what it is that consumes so much CPU power/RAM. In my experience with Python, a very minor - and seemingly unimportant change - can easily cause a 10x speedup. Simply because the Python code itself is so opaque, in the sense that not many people know what a certain line of Python code actually translate into machine code wise. I can't help think that the current Python code *can* be made to run on older hardware, it just requires some CPU- and RAM-profiling of the code, and making the according adjustments. This has - understandably - not been forrestv's highest priority so far.

Well tonights tests will show if a dual core ARM is sufficient.

As far as i understand it PyPy is a language substitute for python.
Wich would meant i would have to port most of the software ?
I guess im lacking the time to do so.
I would also like a implementation nearer to hardware but that will take some efforts.
(assembler and C ftw )

[runeks]

No you shouldn't have to port anything. PyPy should just work with normal Python code. It's simply a different implementation of the standard Python interpreter. No difference in the language it interprets, just how it does it.

For one thing it has a built-in Just-in-time compiler powered by LLVM, which means that instead of the writers of a Python implementation needing to worry about how to create an efficient Just-in-time compiler, the work of people from the LLVM project, who are already experts in constructing such a compiler, can be leveraged.

[twmz]

No you shouldn't have to port anything. PyPy should just work with normal Python code. It's simply a different implementation of the standard Python interpreter. No difference in the language it interprets, just how it does it.

For one thing it has a built-in Just-in-time compiler powered by LLVM, which means that instead of the writers of a Python implementation needing to worry about how to create an efficient Just-in-time compiler, the work of people from the LLVM project, who are already experts in constructing such a compiler, can be leveraged.

This made me curious, so I am running p2pool with pypy as we speak.  It seems to be working fine, but I'll watch it for a day or two and see if there are any abnormalities.

[Sukrim]

Other approach: If we had many programming hours (bounty for additional programmers?) to throw at the current p2pool: What would we want changed and updated? Only the graphing part? I dont see any fundamental problems to be solved?

I do. Simplicity. Yes, I realize that it's already pretty simple, but for the average Joe that just wants to point his miner somewhere and be done with it, it's not.

I'd love to see a BOINC project with bitcoind (or better: libbitcoin) + P2Pool + cgminer included as "worker" instance.

The BTC blockchain could be bootstrapped via a (monthly changed) torrent file (see http://www.coregrid.net/mambo/images/stories/TechnicalReports/tr-0139.pdf) included in the worker program.

BOINC "points" would only be there for fancy stats or so...

The advantage would be, that monthly the most recent mining, pool + bitcoin software as well as a verified snapshot of the blockchain automatically get pushed to the miners (considered you trust the BOINC server operator). Surely nothing for the most paranoid out there (there it might be needed to distribute the stuff as source code + build script, which is not something the average Windows user will be able to pull off easily) but it should be enough to be able to download the BOINC client, sign up for a project + enter a payout address.

[kano]

Actually BOINC and Bitcoin seem to be completely opposite ideals IMO.

[kano]

Actually BOINC and Bitcoin seem to be completely opposite ideals IMO.

https://bitcointalk.org/index.php?topic=63245.0

So ... was that agreement or disagreement?

... https://bitcointalk.org/index.php?topic=63245.msg747009#msg747009

Basically Bitcoin mining is for making Bitcoins and Bitcoins have a monetary value and also a high cost to generate.

Projects like Folding and SETI are purely for science sake
There may be some financial gain to be made from them, but the basis is knowledge not money.
In these 2 cases: knowledge to solve medical problems or gain more knowledge about space.
Research

Most BOINC projects obviously are research, looking at the BOINC page ... a small few may not be but that is unclear.