Bitcoin Forum
March 19, 2024, 08:36:30 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 ... 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 [66] 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 ... 814 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2591571 times)
runeks
Legendary
*
Offline Offline

Activity: 980
Merit: 1008



View Profile WWW
March 12, 2012, 01:46:16 AM
 #1301

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.
What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?
"Governments are good at cutting off the heads of a centrally controlled networks like Napster, but pure P2P networks like Gnutella and Tor seem to be holding their own." -- Satoshi
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1710837390
Hero Member
*
Offline Offline

Posts: 1710837390

View Profile Personal Message (Offline)

Ignore
1710837390
Reply with quote  #2

1710837390
Report to moderator
1710837390
Hero Member
*
Offline Offline

Posts: 1710837390

View Profile Personal Message (Offline)

Ignore
1710837390
Reply with quote  #2

1710837390
Report to moderator
1710837390
Hero Member
*
Offline Offline

Posts: 1710837390

View Profile Personal Message (Offline)

Ignore
1710837390
Reply with quote  #2

1710837390
Report to moderator
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
March 12, 2012, 01:51:39 AM
 #1302

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.
What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?
For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.


Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
NothinG
Hero Member
*****
Offline Offline

Activity: 560
Merit: 500



View Profile
March 12, 2012, 02:14:39 AM
 #1303

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.
What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?
For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.



if optimized.

DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
March 12, 2012, 02:24:19 AM
 #1304

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.
What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?
For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.



The integration into client would be worthwhile IMHO.  Honestly any future alt-chain should consider that from day 1.  Making the p2pool an integral part of protocol.
runeks
Legendary
*
Offline Offline

Activity: 980
Merit: 1008



View Profile WWW
March 12, 2012, 02:28:28 AM
 #1305

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.
What would be the motivation? As far as I can tell, forrestv's Python implementation works fine. What would be the reason to put in this relatively large amount of work to get the p2pool protocol implemented in another language?
For one thing, having a stable implementation in C++ would make it more appealing to the bitcoind developers to include it all in one binary. Gavin has expressed interest in possible integration, but I am fairly certain it wouldn't be considered unless it was written in the same language as the rest of bitcoind.

In addition, a pure C implementation has the potential to be far more efficient and would run better on cheap hardware, if optimized.


I wasn't aware of Gavin wanting to merge p2pool with bitcoind. That's interesting. Personally, I would say keep them separate. I like the "Write programs that do one thing and do it well"-approach. bitcoind seems like a building block for block chain-related applications to me (though a libbitcoin would be better). Integrating a specific block chain-related application into it seems like a strange move to me. But hey, I'm no lead developer of Bitcoin. I can see the logic in making it easier for people to solo mine though, possibly leading people away from pooled mining.

Regarding the pure C implementation to improve efficiency; have you had any problems with the Python implementation underperforming? What kind of cheap hardware are you referring to? I suspect p2pool would run fine on an ARM processor, though I haven't tried. Or are you talking about even slower hardware, like - for example - the Atmel CPU that is in a Bitforce Single (so that it can run without a host computer)?
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
March 12, 2012, 02:33:50 AM
 #1306

Regarding the pure C implementation to improve efficiency; have you had any problems with the Python implementation underperforming? What kind of cheap hardware are you referring to? I suspect p2pool would run fine on an ARM processor, though I haven't tried. Or are you talking about even slower hardware, like - for example - the Atmel CPU that is in a Bitforce Single (so that it can run without a host computer)?
One example was earlier in this thread, I believe. Someone wanted to use an old P3 with 384mb RAM and probably a similarly old hard drive. It was causing all kinds of issues, maxing out the CPU because it was swapping so much. I dunno if the memory requirement is because of inefficiencies in python or p2pool, or whether the same issue would happen when implemented in C/C++, but I do know that many applications do generally get better all around performance when written in a compiled language, not an interpreted language.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
kano
Legendary
*
Offline Offline

Activity: 4438
Merit: 1794


Linux since 1997 RedHat 4


View Profile
March 12, 2012, 03:34:34 AM
 #1307

It's only 2 lines of code (and 2 braces) in init.cpp and 1 line added to the unix makefile to add a whole new module thread onto bitcoind ...
(I do that every time I upgrade my bitcoind - I got sick of editing my changes in every time Tongue)

So hoping for Gavin to do anything is no reason at all to decide for or against doing it.

Pool: https://kano.is - low 0.5% fee PPLNS 3 Days - Most reliable Solo with ONLY 0.5% fee   Bitcointalk thread: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code - k for kano
The ONLY active original developer of cgminer. Original master git: https://github.com/kanoi/cgminer
DeepBit
Donator
Hero Member
*
Offline Offline

Activity: 532
Merit: 501


We have cookies


View Profile WWW
March 12, 2012, 03:51:51 AM
 #1308

Quote
Blocks in shorter chains (or invalid chains) are called "orphan blocks", and while they are stored, they are not used for anything. When a block becomes an orphan block, all of its valid transactions are re-added to the pool of queued transactions and will be included in another block. The 50 BTC reward for the orphan block will be lost, which is why a network-enforced 100-block maturation time for generations exists.
I think it's actually a 120-block mature time, though.
120 is not network-enforced, your client is just too careful :)

Welcome to my bitcoin mining pool: https://deepbit.net ~ 3600 GH/s, Both payment schemes, instant payout, no invalid blocks !
Coming soon: ICBIT Trading platform
TurdHurdur
Full Member
***
Offline Offline

Activity: 216
Merit: 100


View Profile
March 12, 2012, 07:37:57 AM
 #1309

Should we start a bounty to get this recoded in C/C++? Not sure if there is one already, I remember mention of it earlier. Either pure C, or C++ in the same coding style and using the same libs as bitcoind.

You want someone to write twisted in C?
O_Shovah
Sr. Member
****
Offline Offline

Activity: 410
Merit: 252


Watercooling the world of mining


View Profile
March 12, 2012, 07:57:55 AM
 #1310

Just as info.

I have been running bitcoind  bitcoin-qt  p2pool and multiple miners on different ARM boards.

While the miners and bitcoind and Bitcoin-QT run without problems and little Cpu and modest RAM usage on a 600Mhz ARM
p2pool consumes to much CPU power.

The consumption of CPu power is largely scaling with the amount of shares distributed by the miners connected.
I found a miner with ~ 400Mh/s to be to CPU consuming for a 600Mhz and a 1000Mhz ARM CPU.

If the CPU power isn't sufficient it triggers the effect that it works for a while ~45 min.
Afther that the amount of "Timed out while getting work from p2pool" raises significantly.
Other the whole time the p2pool client starts lagging behind the network's time/share count
and finally ends up going unsync afther ~ 60 min.

I will try on my pandaboard these days.

Ente
Legendary
*
Offline Offline

Activity: 2126
Merit: 1001



View Profile
March 12, 2012, 09:10:06 AM
 #1311

Oh, I would absolutely love a single package/installer/binary which installs/runs everything to start p2poolmining! And yes, the current approach with python, modules, p2pool, bitcoind, blockchain and sharechain seems too complicated to pack into one block.. So yes, I see your point in a complete redesign. I dont know enough about the inner workings of all that (monolithic? libraries? dynamic linking?) to decide for or against c, python, modular, monolithic etc.

However I would, at this point, prefer to not integrate p2pool into satoshi client. Maybe integrate a bitcoin client/daemon into p2pool, but the default should be two independent programs.
The current events with rc1, rc2, mandatory updates in quick order and many unstable or not working combinations give me a bad feeling about merging the two to a single, standard client..

What would be the way to make a new, better p2pool version? One program which has everything included (graphs, dependencies)? It wouldnt make a big difference if its C or python then. There is that python-to-exe version too, for windows, apparently?

People would still need bitcoind (if you dont integrate that). People still need a miner (if you dont integrate that). People still need the radeon "runtime". It sounds like a gigantic project to deliver an all-in-one download-doubleclick-mine solution..

Ente
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1024



View Profile
March 12, 2012, 11:08:36 AM
 #1312

Oh, I would absolutely love a single package/installer/binary which installs/runs everything to start p2poolmining! And yes, the current approach with python, modules, p2pool, bitcoind, blockchain and sharechain seems too complicated to pack into one block.. So yes, I see your point in a complete redesign. I dont know enough about the inner workings of all that (monolithic? libraries? dynamic linking?) to decide for or against c, python, modular, monolithic etc.

However I would, at this point, prefer to not integrate p2pool into satoshi client. Maybe integrate a bitcoin client/daemon into p2pool, but the default should be two independent programs.
The current events with rc1, rc2, mandatory updates in quick order and many unstable or not working combinations give me a bad feeling about merging the two to a single, standard client..

What would be the way to make a new, better p2pool version? One program which has everything included (graphs, dependencies)? It wouldnt make a big difference if its C or python then. There is that python-to-exe version too, for windows, apparently?

People would still need bitcoind (if you dont integrate that). People still need a miner (if you dont integrate that). People still need the radeon "runtime". It sounds like a gigantic project to deliver an all-in-one download-doubleclick-mine solution..

Ente

Ahem...

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
Ente
Legendary
*
Offline Offline

Activity: 2126
Merit: 1001



View Profile
March 12, 2012, 11:23:57 AM
 #1313

[p2pcoin - a self contained p2pool node - boots from CD, USB or network]

Definitely true, and valuable work for all miners!
The question is, how many % of the total network is mining on dedicated hardware? The new members surely will simply use their regular, only computer, which happens to have a fat gpu for gaming. I dont expect many of them to boot a linux every time they want to mine, and to not mine when they use their computer for other stuff.

Still, I love the linux-bitcoin-remixes, staying far away from M$ for many years now!

Ente
runeks
Legendary
*
Offline Offline

Activity: 980
Merit: 1008



View Profile WWW
March 12, 2012, 01:15:06 PM
 #1314

Just as info.

I have been running bitcoind  bitcoin-qt  p2pool and multiple miners on different ARM boards.

While the miners and bitcoind and Bitcoin-QT run without problems and little Cpu and modest RAM usage on a 600Mhz ARM
p2pool consumes to much CPU power.

The consumption of CPu power is largely scaling with the amount of shares distributed by the miners connected.
I found a miner with ~ 400Mh/s to be to CPU consuming for a 600Mhz and a 1000Mhz ARM CPU.

If the CPU power isn't sufficient it triggers the effect that it works for a while ~45 min.
Afther that the amount of "Timed out while getting work from p2pool" raises significantly.
Other the whole time the p2pool client starts lagging behind the network's time/share count
and finally ends up going unsync afther ~ 60 min.

I will try on my pandaboard these days.
Interesting. Have you tried an alternative Python implementation like PyPy? I think that would be what I would try first, but also - in part - because I think PyPy is cool Smiley.

Other than that, I think it might be worth the effort to dig into the Python code and take a look at what it is that consumes so much CPU power/RAM. In my experience with Python, a very minor - and seemingly unimportant change - can easily cause a 10x speedup. Simply because the Python code itself is so opaque, in the sense that not many people know what a certain line of Python code actually translate into machine code wise. I can't help think that the current Python code *can* be made to run on older hardware, it just requires some CPU- and RAM-profiling of the code, and making the according adjustments. This has - understandably - not been forrestv's highest priority so far.
O_Shovah
Sr. Member
****
Offline Offline

Activity: 410
Merit: 252


Watercooling the world of mining


View Profile
March 12, 2012, 02:59:14 PM
 #1315

Just as info.

I have been running bitcoind  bitcoin-qt  p2pool and multiple miners on different ARM boards.

While the miners and bitcoind and Bitcoin-QT run without problems and little Cpu and modest RAM usage on a 600Mhz ARM
p2pool consumes to much CPU power.

The consumption of CPu power is largely scaling with the amount of shares distributed by the miners connected.
I found a miner with ~ 400Mh/s to be to CPU consuming for a 600Mhz and a 1000Mhz ARM CPU.

If the CPU power isn't sufficient it triggers the effect that it works for a while ~45 min.
Afther that the amount of "Timed out while getting work from p2pool" raises significantly.
Other the whole time the p2pool client starts lagging behind the network's time/share count
and finally ends up going unsync afther ~ 60 min.

I will try on my pandaboard these days.
Interesting. Have you tried an alternative Python implementation like PyPy? I think that would be what I would try first, but also - in part - because I think PyPy is cool Smiley.

Other than that, I think it might be worth the effort to dig into the Python code and take a look at what it is that consumes so much CPU power/RAM. In my experience with Python, a very minor - and seemingly unimportant change - can easily cause a 10x speedup. Simply because the Python code itself is so opaque, in the sense that not many people know what a certain line of Python code actually translate into machine code wise. I can't help think that the current Python code *can* be made to run on older hardware, it just requires some CPU- and RAM-profiling of the code, and making the according adjustments. This has - understandably - not been forrestv's highest priority so far.

Well tonights tests will show if a dual core ARM is sufficient.

As far as i understand it PyPy is a language substitute for python.
Wich would meant i would have to port most of the software ?
I guess im lacking the time to do so.
I would also like a implementation nearer to hardware but that will take some efforts.
(assembler and C ftw Cheesy )

runeks
Legendary
*
Offline Offline

Activity: 980
Merit: 1008



View Profile WWW
March 12, 2012, 03:27:47 PM
 #1316

No you shouldn't have to port anything. PyPy should just work with normal Python code. It's simply a different implementation of the standard Python interpreter. No difference in the language it interprets, just how it does it.

For one thing it has a built-in Just-in-time compiler powered by LLVM, which means that instead of the writers of a Python implementation needing to worry about how to create an efficient Just-in-time compiler, the work of people from the LLVM project, who are already experts in constructing such a compiler, can be leveraged.
twmz
Hero Member
*****
Offline Offline

Activity: 737
Merit: 500



View Profile
March 12, 2012, 03:57:56 PM
 #1317

No you shouldn't have to port anything. PyPy should just work with normal Python code. It's simply a different implementation of the standard Python interpreter. No difference in the language it interprets, just how it does it.

For one thing it has a built-in Just-in-time compiler powered by LLVM, which means that instead of the writers of a Python implementation needing to worry about how to create an efficient Just-in-time compiler, the work of people from the LLVM project, who are already experts in constructing such a compiler, can be leveraged.

This made me curious, so I am running p2pool with pypy as we speak.  It seems to be working fine, but I'll watch it for a day or two and see if there are any abnormalities.

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
Sukrim
Legendary
*
Offline Offline

Activity: 2618
Merit: 1006


View Profile
March 12, 2012, 10:44:17 PM
 #1318

Other approach: If we had many programming hours (bounty for additional programmers?) to throw at the current p2pool: What would we want changed and updated? Only the graphing part? I dont see any fundamental problems to be solved?

I do. Simplicity. Yes, I realize that it's already pretty simple, but for the average Joe that just wants to point his miner somewhere and be done with it, it's not.
I'd love to see a BOINC project with bitcoind (or better: libbitcoin) + P2Pool + cgminer included as "worker" instance.

The BTC blockchain could be bootstrapped via a (monthly changed) torrent file (see http://www.coregrid.net/mambo/images/stories/TechnicalReports/tr-0139.pdf) included in the worker program.

BOINC "points" would only be there for fancy stats or so...

The advantage would be, that monthly the most recent mining, pool + bitcoin software as well as a verified snapshot of the blockchain automatically get pushed to the miners (considered you trust the BOINC server operator). Surely nothing for the most paranoid out there (there it might be needed to distribute the stuff as source code + build script, which is not something the average Windows user will be able to pull off easily) but it should be enough to be able to download the BOINC client, sign up for a project + enter a payout address.

https://www.coinlend.org <-- automated lending at various exchanges.
https://www.bitfinex.com <-- Trade BTC for other currencies and vice versa.
kano
Legendary
*
Offline Offline

Activity: 4438
Merit: 1794


Linux since 1997 RedHat 4


View Profile
March 13, 2012, 12:39:38 AM
 #1319

Actually BOINC and Bitcoin seem to be completely opposite ideals IMO.

Pool: https://kano.is - low 0.5% fee PPLNS 3 Days - Most reliable Solo with ONLY 0.5% fee   Bitcointalk thread: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code - k for kano
The ONLY active original developer of cgminer. Original master git: https://github.com/kanoi/cgminer
kano
Legendary
*
Offline Offline

Activity: 4438
Merit: 1794


Linux since 1997 RedHat 4


View Profile
March 13, 2012, 01:59:56 AM
 #1320

Actually BOINC and Bitcoin seem to be completely opposite ideals IMO.

https://bitcointalk.org/index.php?topic=63245.0
So ... was that agreement or disagreement?

... https://bitcointalk.org/index.php?topic=63245.msg747009#msg747009

Basically Bitcoin mining is for making Bitcoins and Bitcoins have a monetary value and also a high cost to generate.

Projects like Folding and SETI are purely for science sake
There may be some financial gain to be made from them, but the basis is knowledge not money.
In these 2 cases: knowledge to solve medical problems or gain more knowledge about space.
Research

Most BOINC projects obviously are research, looking at the BOINC page ... a small few may not be but that is unclear.

Pool: https://kano.is - low 0.5% fee PPLNS 3 Days - Most reliable Solo with ONLY 0.5% fee   Bitcointalk thread: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code - k for kano
The ONLY active original developer of cgminer. Original master git: https://github.com/kanoi/cgminer
Pages: « 1 ... 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 [66] 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 ... 814 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!