Bitcoin Forum
December 03, 2016, 04:44:54 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 [402] 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 ... 744 »
  Print  
Author Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool  (Read 2028760 times)
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 14, 2014, 06:05:00 AM
 #8021

Yes, we've covered that before, but there's no point in considering "absolute terms". Few people consider anything related to BTC in "absolute terms". In fact I can't think of many situations in which there are useful BTC related charts which are not log-linear.

Bitcoin in absolute terms are what you spend (or save). In fact the number you are quoting, standard deviation divided by mean, has no units at all. It's a number that doesn't exist in the real world, only on graphs.
1480740294
Hero Member
*
Offline Offline

Posts: 1480740294

View Profile Personal Message (Offline)

Ignore
1480740294
Reply with quote  #2

1480740294
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480740294
Hero Member
*
Offline Offline

Posts: 1480740294

View Profile Personal Message (Offline)

Ignore
1480740294
Reply with quote  #2

1480740294
Report to moderator
1480740294
Hero Member
*
Offline Offline

Posts: 1480740294

View Profile Personal Message (Offline)

Ignore
1480740294
Reply with quote  #2

1480740294
Report to moderator
1480740294
Hero Member
*
Offline Offline

Posts: 1480740294

View Profile Personal Message (Offline)

Ignore
1480740294
Reply with quote  #2

1480740294
Report to moderator
murraypaul
Sr. Member
****
Offline Offline

Activity: 476


View Profile
March 14, 2014, 12:38:30 PM
 #8022

In case you are, or in case there are other readers as confused as I, you can think about it like this: the number of p2Pool shares a miner submits per time period is an approximately Poisson distributed random number. This means that the standard deviation of this number is proportional to the square root of the mean. As the share submission rate increases, the standard deviation only increases as the square root of the mean, so a miner with a greater share submission rates has, proportionally, a much lower standard deviation.
The key word you are using there, which contributes much to the confusion, is proportionately. In fact the standard deviation, measured in units of dollars, btc, or anything else is higher for the larger miner. It grows more slowly than the mean, but it still grows. A small miner has a small standard deviation in absolute terms.

A standard deviation in absolute terms makes no sense conceptually.
Small miners suffer from much higher variance on p2pool than large miners.
You don't care because they aren't earning much anyway, they do.
p2pool may be a poor choice for small miners anyway, because the payouts they do end up receiving will be in many tiny chunks which will end up costing them more to spend than a single payout from a normal pool.

BTC: 16TgAGdiTSsTWSsBDphebNJCFr1NT78xFW
SRC: scefi1XMhq91n3oF5FrE3HqddVvvCZP9KB
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1946


Poor impulse control.


View Profile WWW
March 14, 2014, 01:56:05 PM
 #8023

Yes, we've covered that before, but there's no point in considering "absolute terms". Few people consider anything related to BTC in "absolute terms". In fact I can't think of many situations in which there are useful BTC related charts which are not log-linear.

Bitcoin in absolute terms are what you spend (or save). In fact the number you are quoting, standard deviation divided by mean, has no units at all. It's a number that doesn't exist in the real world, only on graphs.


Now you're just being wilfully silly.

A person that has one bitcoin that loses one bitcoin proportionally loses much more than a person that has a thousand bitcoins and also loses one bitcoin. In absolute terms, however, they've lost the same amount. I doubt either of them will be thinking of the loss in absolute terms.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 14, 2014, 06:08:14 PM
 #8024

A person that has one bitcoin that loses one bitcoin proportionally loses much more than a person that has a thousand bitcoins and also loses one bitcoin. In absolute terms, however, they've lost the same amount. I doubt either of them will be thinking of the loss in absolute terms.

Yes, I agree. Risk relative to assets does matter.

But there is no direct relationship between your hash rate and your assets, so risk relative to hash rate does not necessarily matter. Where I already said I agree is that someone who is poor (you said poor country but the wealth of the individual matters more) should be more concerned about smaller risks.

In other words, if you want to manage risk in "relative" terms, the correct denominator to use is assets, not hash rate. What this means in practice for hobby miners is they need not be overly concerned about variance.
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 14, 2014, 06:21:53 PM
 #8025

p2pool may be a poor choice for small miners anyway, because the payouts they do end up receiving will be in many tiny chunks which will end up costing them more to spend than a single payout from a normal pool.

This is a good point. Maybe it should be possible to store deferred payouts in the share chain up to some minimum and apply them to a future block, the way eligius does. There is definitely value in coinbase payouts and the pool not having a wallet (though in practice eligius does have a wallet) while at the same time not paying out a tiny slice of every single block to every single miner.

I don't know if this can be made hop proof though.
smoothrunnings
Hero Member
*****
Offline Offline

Activity: 546


View Profile
March 14, 2014, 11:50:32 PM
 #8026

Is there anyway to disable the wallet function in P2Pool and have it instead forward the percentage of shares the note is to receive to a different wallet?

I have hit a stumbling block; over a week a go I initiated a transfer of over 1/2 a bitcoin and Bitcoin-QT told me I need to account for 0.0004 of the fee, so I did that and ever since the transfer has been in limbo, it looks like bitcoin-qt gave me the wrong fee to account for it if I understand the blockchain correctly.

https://blockchain.info/tx-index/7bbbc7b5fac8c2973e9c62a17cead946b9cf1832aac47479799ce566fafabcc1

Also I need to figure out if can fix this issue or have I lost those coins? Sad

Thanks,



correct fee for that is 0.0004, the fee on link provided is 0.00004

chances of getting into a block that is less than 10k (or 27k if it's priority)  in size is slim

This goes back to my initial problem with bition-qt/bitcoind when it comes to sent transactions. I am trying to transfer 0.31779405 bitcoins and I am getting this dialog from bitcoin-qt, its not hard to imagine that with my 0.59xxx bitcoins the number was 0.00004 not 0.0004 ...


KyrosKrane
Sr. Member
****
Offline Offline

Activity: 292


View Profile WWW
March 15, 2014, 08:02:04 AM
 #8027

Is there anyway to disable the wallet function in P2Pool and have it instead forward the percentage of shares the note is to receive to a different wallet?

I have hit a stumbling block; over a week a go I initiated a transfer of over 1/2 a bitcoin and Bitcoin-QT told me I need to account for 0.0004 of the fee, so I did that and ever since the transfer has been in limbo, it looks like bitcoin-qt gave me the wrong fee to account for it if I understand the blockchain correctly.

https://blockchain.info/tx-index/7bbbc7b5fac8c2973e9c62a17cead946b9cf1832aac47479799ce566fafabcc1

Also I need to figure out if can fix this issue or have I lost those coins? Sad

Thanks,



correct fee for that is 0.0004, the fee on link provided is 0.00004

chances of getting into a block that is less than 10k (or 27k if it's priority)  in size is slim

This goes back to my initial problem with bition-qt/bitcoind when it comes to sent transactions. I am trying to transfer 0.31779405 bitcoins and I am getting this dialog from bitcoin-qt, its not hard to imagine that with my 0.59xxx bitcoins the number was 0.00004 not 0.0004 ...



I'm reasonably sure there's a bug somewhere in bitcoin-qt that causes transaction fees to be off by a factor of ten. Basically it adds an extra zero after the decimal point. I've had three transactions hit with this issue, where the fee that bitcoin-qt asks for is exactly 1/10th what the rest of the network is expecting. This causes my transactions to take many hours (in one case, over a day) until they get confirmed.  That said, this is probably off-topic for this thread.

Tips and donations: 1KyrosREGDkNLp1rMd9wfVwfkXYHTd6j5U  |  BTC P2Pool node: p2pool.kyros.info:9332
roy7
Sr. Member
****
Offline Offline

Activity: 434


View Profile
March 15, 2014, 12:53:12 PM
 #8028

Switch to Electrum. Smiley

RoyalMiningCo: Pools retired. Was fun!
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 17, 2014, 03:02:31 AM
 #8029

I opened a recruiting thread: https://bitcointalk.org/index.php?topic=518920

Hope it can make a difference.
phillipsjk
Legendary
*
Offline Offline

Activity: 1008

Let the chips fall where they may.


View Profile WWW
March 17, 2014, 08:17:48 AM
 #8030

Well I think the results are in: pypy is not really faster than python c modules on my machine (FreeBSD 9.2).



The left grouping is python2.7, the grouping on the right is pypy(2.0 I believe).

CPU usage is hard to compare directly.  It appears to be proportional to the amount of data relayed on the network though. It takes time for the number of active connections to recover after a restart. The drop-off near the end was caused by an Internet outage.

I am assuming the large spikes in the pypy side of the graph are caused by disk cache misses. The disk is a SSD, so a lag of an extra second would probably have to correspond to over 500 seeks (assuming 2ms each). P2Pool being run by pypy appears to use about 3x the memory as P2Pool run by python (1200 vs 400 MB). My memory usage graph is blank for some reason.



As you can see, the DOA rate does not change much. If anything overall hash-rate goes down a bit as the number of connections go up. My machine is limited to 16 connections.

When going back to python, I used the following options:
Code:
-O     : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE

The first option was used just in case it helps. The second option was intended to undo/ignore my handywork in this post.



James' OpenPGP public key fingerprint: EB14 9E5B F80C 1F2D 3EBE  0A2F B3DE 81FF 7B9D 5160
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 17, 2014, 09:30:40 AM
 #8031

Somewhat odd results there. Are you short on RAM, maybe swapping? I get a pretty big drop in latency and CPU usage using pypy on a low end dual-core AMD box, but I also see the increased memory usage (but I have 8 GB so not an issue). I don't see latency spikes at all.

Does FreeBSD do huge pages? I found that helps on Linux, probably even more with pypy given the bigger memory footprint, though I didn't measure that.

IYFTech
Hero Member
*****
Offline Offline

Activity: 686


WANTED: Active dev to fix & re-write p2pool in C


View Profile
March 17, 2014, 12:23:22 PM
 #8032

I opened a recruiting thread: https://bitcointalk.org/index.php?topic=518920

Hope it can make a difference.

That went well.........



Don't be upset with a p2pool user because he pointed out the blindingly obvious - be upset about the lack of development/bug fixing/improvements by the "devs" of p2pool who are taking donations for nothing Wink

PS: If you are having to restart your node every week I suggest you take another look at your setup/hardware - something ain't right there dude.

-- Smiley  Thank you for smoking  Smiley --  If you paid VAT to dogie for items you should read this thread:  https://bitcointalk.org/index.php?topic=1018906.0
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 17, 2014, 12:31:06 PM
 #8033

As I said, if you don't like the job the developers are doing, tell them about it here. That's appropriate.  Better yet offer specific suggestions or volunteer to do some of the work yourself, or sponsor it.

Crashing a thread asking miners to try p2pool (and offering to help them do so) with complaints about the p2pool developers is not appropriate. Especially while doing nothing constructive about any of the issues you or I have identified.

PatMan
Hero Member
*****
Offline Offline

Activity: 924


Watch out for the "Neg-Rep-Dogie-Police".....


View Profile WWW
March 17, 2014, 01:40:21 PM
 #8034

As I said, if you don't like the job the developers are doing, tell them about it here. That's appropriate.  Better yet offer specific suggestions or volunteer to do some of the work yourself, or sponsor it.

Crashing a thread asking miners to try p2pool (and offering to help them do so) with complaints about the p2pool developers is not appropriate. Especially while doing nothing constructive about any of the issues you or I have identified.



Think I'll chime in here - been a while since I posted anything on the p2pool thread, mainly because I gave up with p2pool a while ago due to the exact reasons IYFTech pointed out in your doomed thread begging for miners.

Offering to help new users setup p2pool correctly when you are obviously having difficulty setting up your own node is unwise. Having to reboot every week means you have done something wrong, even with all the bugs in p2pool, you should not have to do this.

I used to mine at p2pool in the pre-asic age when it worked pretty well. Things started going tits up when stratum was implemented, but this was finally sorted out after many months of wrangling due to a few so called "gurus" denying that there was anything wrong, when there obviously was. Next up was the asic problem, nothing worked with p2pool apart from a select few items that had to be fiddled with & coaxed into playing nicely with p2pool (including special firmware) while all the time the same "gurus" denied there was ever a problem & it was every body else's fault. If you scroll back through the p2pool pages you will see this, every mention of any problem with p2pool was instantly dismissed by the "gurus" who would accuse miners of not knowing what they were doing & even told to go mine somewhere else. Great PR eh? Although I notice that things have slightly improved in this respect as most of the "gurus" seem to have moved on themselves......

I got so frustrated with not only the lack of development with p2pool but also the unwillingness of certain "guru" users to even admit that p2pool could be improved, that I even tried to set up a more user friendly version over 10 months ago:

https://bitcointalk.org/index.php?topic=213051.0

My problem was not finding someone capable of coding such a program, it was finding a coder who understood the bitcoin protocol/mining process & implementing that into the software - but after much discussion & searching I was reluctantly forced into dropping the project. What IYFTech said in your thread about ease of use, GUI, wizard installer etc is 100% correct & is exactly what I was trying to achieve & I can't help but think that if I had been able to get the project off the ground p2p mining would be in a much healthier state than it is now, as would the network in general due to the decentralized nature of p2p mining, which is the one thing everyone seems to agree on.

Nothing changes with p2pool. It is a classic case of someone having a good idea a few years ago & then resting on their laurels expecting it to be fine forever more, while watching the donations roll in. It's the same old story here, always has been, always will be. Adapt & improve or die - it's as simple as that.

Peace  Grin

Edit: The issues that you say you have identified have been known about for well over a year - the problem is that nothing has been done about it.

"When one person is deluded it is called insanity - when many people are deluded it is called religion" - Robert M. Pirsig.  I don't want your coins, I want change.
Amazon UK BTC payment service - https://bitcointalk.org/index.php?topic=301229.0 - with FREE delivery!
http://www.ae911truth.org/ - http://rethink911.org/ - http://rememberbuilding7.org/
IYFTech
Hero Member
*****
Offline Offline

Activity: 686


WANTED: Active dev to fix & re-write p2pool in C


View Profile
March 17, 2014, 02:02:03 PM
 #8035

+1

Dude, great minds & all that........ Cheesy Cheesy

-- Smiley  Thank you for smoking  Smiley --  If you paid VAT to dogie for items you should read this thread:  https://bitcointalk.org/index.php?topic=1018906.0
phillipsjk
Legendary
*
Offline Offline

Activity: 1008

Let the chips fall where they may.


View Profile WWW
March 17, 2014, 04:59:09 PM
 #8036

Somewhat odd results there. Are you short on RAM, maybe swapping? I get a pretty big drop in latency and CPU usage using pypy on a low end dual-core AMD box, but I also see the increased memory usage (but I have 8 GB so not an issue). I don't see latency spikes at all.

There is a little bit of swap use, but the only things using more than about 16MB are Bitcoind and P2Pool. Together they use about 2GB of my 3.3GB of available memory. By "cache miss", I meant the OS had to go to disk at all (the Bitcoin blockchain does not fit in RAM).

Looking at that graph a second time, it appear that the first spike starts to ramp up over an hour or so. Not sure what would cause that (maybe a peer downloading the block-chain?). If your machine has 8GB of RAM, it is probably newer than mine. Maybe pypy JIT overhead is negligible on newer hardware.

One other possibility is that pypy did lower latency, making my machine the first choice for information on the network. The resulting work then increased latency (Bitcoind is running "niced" for some reason). Edit: The spike in data-out after the latency spike may indicate this if the time-scale was not days. May need to re-test if I get a new CPU/motherboard.

Keep in mind the the python packages on FreeBSD appear to use c-optimized modules by default. This may diminish any JIT performance advantages.

Edit: While we are making feature requests: I would like to see a p2pool block template latency graph, if that is possible. Would take a lot of guess-work out of my testing.


James' OpenPGP public key fingerprint: EB14 9E5B F80C 1F2D 3EBE  0A2F B3DE 81FF 7B9D 5160
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 17, 2014, 07:52:22 PM
 #8037

Offering to help new users setup p2pool correctly when you are obviously having difficulty setting up your own node is unwise. Having to reboot every week means you have done something wrong, even with all the bugs in p2pool, you should not have to do this.

I never said reboot. And for that matter, there is nothing bad that happens specifically after a week. I have noticed there are some resource leaks that come up, so to avoid problems, I set things up to restart p2pool (only, I don't reboot or restart bitcoind) once per week, before any problems occur. Others have reported the exact same resource leaks. There is nothing wrong with my setup. My node works wonderfully (consistently >100% efficiency).

There is a bit of irony here, as you talk about "gurus" claiming there is nothing wrong with p2pool and accusing miners of not knowing what they are doing then you show up here and do the exact same thing.

Most of the ASIC problems you identified last year don't exist any more. This was mostly solved by the ASIC miners working better. As you correctly state, there hasn't been

I will be reopening a new recruiting thread, and it will be self moderated. If you have something to add in terms of recruiting miners to p2pool and therefore helping bitcoin, you and IYFTech are welcome to join. General complaints about p2pool, especially when you haven't even tried using it recently with current hardware are unwelcome. "It's not as good as my vaporware idea of a project" is even more useless.


smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 17, 2014, 08:06:16 PM
 #8038

There is a little bit of swap use, but the only things using more than about 16MB are Bitcoind and P2Pool. Together they use about 2GB of my 3.3GB of available memory. By "cache miss", I meant the OS had to go to disk at all (the Bitcoin blockchain does not fit in RAM).

Getblocktemplate itself shouldn't have to go to disk (the mempool is in memory, obviously). Yes the rest of bitcoind does hit disk and maybe this slows down GBT (due to locks), but that shouldn't be a major issue with an SSD. One guess would be maybe with the increased pypy memory usage (not sure why that happens) you have less available RAM for caching so you hit disk (even SSD) more, slowing things down. Perhaps that offsets the CPU gains from pypy.

Quote
Looking at that graph a second time, it appear that the first spike starts to ramp up over an hour or so. Not sure what would cause that (maybe a peer downloading the block-chain?). If your machine has 8GB of RAM, it is probably newer than mine. Maybe pypy JIT overhead is negligible on newer hardware.

Ah, that could be. One thing I do is block incoming connections on my bitcoind node, which has the effect of avoiding blockchain downloads (since the standard client only does them on outgoing connections -- to them, incoming to you). Downloading the block chain is really bad with p2pool, especially if you have any ISP bandwidth issues, but probably for other performance reasons, as you point out.

Quote
Edit: While we are making feature requests: I would like to see a p2pool block template latency graph, if that is possible. Would take a lot of guess-work out of my testing.

How would this be different from the graph that is currently displayed?

Regarding my hardware it isn't that the system is so new (or fast), it is just that I had some extra 4 GB modules sitting around so I used them. It is actually an nettop type system with a dual-core but otherwise severely underpowered AMD. It may be that I was more CPU limited without pypy compared to a more conventional PC so I saw more improvement.


phillipsjk
Legendary
*
Offline Offline

Activity: 1008

Let the chips fall where they may.


View Profile WWW
March 17, 2014, 09:06:15 PM
 #8039

Most of the ASIC problems you identified last year don't exist any more. This was mostly solved by the ASIC miners working better. As you correctly state, there hasn't been

You can see from my graph that it is not fixed for me. The fix was to extend the block-target to 30 seconds. With a 10 second block target, I would have >65% DOA shares on average because the round time on my ASIC is 12.8 seconds (and does not support longpolling). As it is, I expect at least 22% DOA. In practice, I am getting 30% or more.

Quote
How would this be different from the graph that is currently displayed?

A P2Pool getblock template lantency graph would tell me how long P2Pool spends processing it's own blocks. I suspect that it runs several seconds on my machine. I could artificially lower the bitcoind getblocktemplate time by:
  • Un-nice-ing  bitcoind. This will actually slow down P2Pool
  • Closing bitcoind to incoming connections as suggested. I want to contribute to the Bitcoin network as much as possible. Mining is just a bonus.
Edit: If I assume the P2Pool getblocktemplate processing always takes the same amount to time (thus makes a graph pointless), I can calculate it from my hash-rate graph. 30% DOA - 22% Expected DOA = 8% of 30 seconds of round time being CPU latency.
30secondsx8%=2.4 seconds - 0.7s  Bitcoind latency = 1.7 seconds P2Pool latency
--some of that may be due to limited network bandwidth too.
--My machine is dual-core as well, so that simple math may not hold.

Quote
Regarding my hardware it isn't that the system is so new (or fast), it is just that I had some extra 4 GB modules sitting around so I used them.

My machine is one of those ought's machines that supports 64bit processing, but only has 32 bits of address space. I suspect if pypy worked with no problems for you; your python modules were probably not c-optimized.


James' OpenPGP public key fingerprint: EB14 9E5B F80C 1F2D 3EBE  0A2F B3DE 81FF 7B9D 5160
smooth
Legendary
*
Offline Offline

Activity: 1246



View Profile
March 17, 2014, 09:46:18 PM
 #8040

Most of the ASIC problems you identified last year don't exist any more. This was mostly solved by the ASIC miners working better. As you correctly state, there hasn't been

You can see from my graph that it is not fixed for me. The fix was to extend the block-target to 30 seconds. With a 10 second block target, I would have >65% DOA shares on average because the round time on my ASIC is 12.8 seconds (and does not support longpolling).

Are you using one of the new ASICs that I mentioned earlier? The real fix is to natively support longpolling or stratum (or shorter rounds), as the recent ASICs do. Otherwise, your ASIC will just not work well with any pool or coin with fast rounds.

As much a supporter of p2pool I might be, I'd suggest you move that hardware elsewhere if it can't be fixed. When I was running older ASICs I tried p2pool and then stopped using it. Wasn't fixable given the latency inherent in the hardware.

Quote
My machine is one of those ought's machines that supports 64bit processing, but only has 32 bits of address space. I suspect if pypy worked with no problems for you; your python modules were probably not c-optimized.

Which modules? I'm using whatever python module come from apt-get on ubnutu 13.10? (Pretty much the default install instructions for p2pool if I recall correctly.) Do you know if there is problem with those?


Pages: « 1 ... 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 [402] 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 ... 744 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!