Nice!  I stand corrected.  Didn't know you were lurking on this forum here

I checked back a few weeks on the blog entries, didn't see anything, and just assumed it hadn't been listed.  Didn't check all the way back to the start.

Very good charts/graphs.  Nice data crunching!

Josh

Nice!  Sounds like a great idea.

Wow, that's great to know!  Good to know that there is a signature of sorts.

Curious if it would be a good feature for p2pool to add more of an obvious signature to its blocks?  I'm not that familiar with the block layout of Bitcoin, but going by previous blocks, there's certainly a facility there to embed some arbitrary text.  I would keep it very simple, just "P2Pool", so as not to take up much space.

Or, give the winning miner the opportunity to insert some text of their choice into their share/block (perhaps paid for out of their share, to make up for the loss of giving up space in the block that would otherwise hold more profitable transaction fees)?  I can see this being seriously misused, though, maybe best not to offer this feature.

Josh

We're in luck.  There was talk on the p2pool thread about that, just recently.

The p2pool donation address appears as the second-to-last input in the coinbase transaction of every p2pool solved block.

1Kz5QaUPDtKrj5SqW5tFkn7WZh8LmQaQi4

This address always appears as the second-to-last entry in the coinbase transaction.

BTW, the very last entry, the "Unable to decode input address", contains a special hash which links up the p2pool sharechain with the Bitcoin blockchain.  It changes all the time, so it's probably not as useful as a signature, though.

It would be great if p2pool indicated its solved blocks in a more obvious way, but these are the cards we have been dealt.

Thanks.  It happened on September 4, as per the timestamp on the screenshot photo.

Unfortunately, I can't find how to send a report.

In the phone's "Application manager", on the main "App info" screen that was in my earlier screenshot, the button that was previously labeled "Report" is now labeled "Uninstall"!  I tried to go to "Battery" but Bitcoin doesn't show up in the list (it is being very well-behaved now, because I'm on a fast Wi-Fi connection so it can complete its periodic sync quickly).  So, I can't send a report from the Battery screen, which I should have done from that screenshot!

I was able to send another kind of report by going to the "Active apps" tab of "Application manager", selecting Bitcoin, then the service "BlockchainServiceImpl" had a "Report" button.  I did this.  Don't know if it will help, though.

Is there a way to force Android to always enable the "Report" button, for apps of interest?

Josh

A few days ago, I found it taking a mere 64% of my battery!

Here's screenshots:



Unfortunately, didn't know about the "Report Issue" feature at the time this happened.

I've rebooted the phone several times since then, though, and it didn't happen again!

Josh

Nice, I just noticed P2Pool has appeared in this list for the first time:

http://organofcorti.blogspot.com/2013/09/september-1st-2013-weekly-pool-and.html

Any thoughts on perhaps spending 7-or-so bytes per block, to put "P2Pool" in the coinbase signature, to make it easier for P2Pool mined blocks to be identified as such, for purposes of making these charts?  It's an exciting time for P2Pool, as it is now over 1% of the entire Bitcoin network, and this will draw more attention (and thus hopefully more people joining the pool and adding hashpower).

Josh

Hi there.

Is it possible to identify p2pool mined blocks at blockchain.info?

Currently, they show up as individual IP addresses, or as Deepbit, so p2pool doesn't get credit when compared against the other pools.

As an example, Block 255951 and Block 255934 were mined by p2pool, but they are showing up as Deepbit and an IP address, respectively.

Josh

Nice, up and running!  My Erupters are also much happier now, it seems.  Twinkling like fireflies in summertime.


Cool!!  Thanks for accepting my patches.

Josh

OK, I cleaned up what I originally had, and got it working.

It now works correctly when peer sends pong with nonce of zero (overlapping pongs will confuse the timing, as noted in the rather nice comment in the existing ping receiver).

I added a ratelimit to RPC-requested ping, so that it will limit to 1/second per peer.  The regular automated keepalive ping is immune to this.

The "pingtime" field in getpeerinfo is newly added, and has the pingtime in decimal seconds.  I assume that Bitcoin users are well used to small decimal numbers with many digits by now

It's now clean enough that I feel it's a pull request candidate:

https://github.com/bitcoin/bitcoin/pull/2937

Some more thoughts:

1) If ping is still outstanding, and peer is taking a very long time to respond, this isn't indicated anywhere.  The pingtime field contains the previously completed ping's time, and updates atomically (it doesn't change until the next ping is fully completed).  So, a naive consumer of the data could be fooled into thinking a badly lagged node is still responsive.  Thought about indicating the current waiting time in the pingtime field instead, if the current waiting time is greater than the previously completed ping's time.  Good idea, or not?

2) If peer is flooding me with pings, I do not ratelimit those.  Reason is, if network is lagged and it suddenly gets better, a lot of previous requests will come through all at once, having been bunched up in buffers.  So, peer could be obeying ratelimit on their end, but by the time it got to me, it would wrongly look to me as if peer is flooding.  Also, if I ratelimit and drop excessive pings, it would mean the first ping gets responded to, but the later pings don't.  The nonce system expects the opposite: it throws away previous nonces when it rolls up a new nonce for a later ping.  That would mean that the ping would essentially be lost in the shuffle.  This is pedantic, anyway, and some kind of (generous) rate limit is probably a good idea as a sanity check.  Since ping commands are limited to 1/second, perhaps limit incoming ping responses (pongs) to 2/second or 3/second?

3) Similarly, if peer is flooding me with pongs, I just swallow them, since they have no reply.  The nonce checking code does not allow a previous nonce to be reused, so duplicated pongs will merely be dropped on the floor.  I wonder if should be more aggressive against peers who are flooding, perhaps Misbehaving() them?

Josh

OK, that makes sense.  I thought that p2pool paid out each successful share based on the difficulty of that share itself (how close to zero the hash was), not based on the target difficulty.  Interesting how there's still so much variance in the payout table, then: each payout is slightly different, they don't quantize themselves into recognizable patterns.

Since there's only 8640 shares, I would think that it would be easy to see groupings in the payout table, but instead, each number is slightly different.  Does the target difficulty continuously update itself and change a little each time?  Maybe that is why.

Interesting.  I thought p2pool already paid out on a prorated basis, based on how difficult your share was, regardless of what the target was?  If you just barely limp in over the minimum required difficulty, you get paid a small amount, but if you find a massively high difficulty share (such as a share that's good enough to become a real Bitcoin block) you get paid that much more.  You don't get paid the same amount per share, I have noticed.

Didn't know it took into account the ability to optionally choose a higher minimum required difficulty, and increased the payout ratio proportionally, to compensate you for your loss from all the little low shares that you've now chosen to throw away instead of submit.  What's the ratio that it uses?

Let's say I submit 6 shares, at these difficulties: 1, 1, 1, 1000, 1000, 1000.

That will pay me about 3003 difficulties worth of work.  Now, let's say I change my minimum difficulty to 500.  Instead of submitting 6 shares, I submit only 3 shares, at these difficulties: 1000, 1000, 1000.  This only pays me about 3000 difficulties worth of work, not 3003.  So I have lost a little income here.  In case there's something I'm missing.

Anybody have the exact formula?

Josh

That's something I don't get: how voluntarily increasing your difficulty will help you.

Wouldn't you want to claim as many shares as possible, even if they are small amounts?

Since each share you find increases your payout allocation within p2pool's 8640-share window, it should always be beneficial to add more shares, right?

It seems to me that all you would be doing by increasing your difficulty would be to forfeit some potential earnings from weak shares, by throwing those shares away instead of submitting them to the sharechain, unless there's something I'm missing.

Maybe if p2pool is CPU limited, or you're running remote miners that are network-limited between them and p2pool, and you have so many miners that you're constantly spamming p2pool with 1-difficulty shares, that might make a difference.  In that case, wouldn't it just be better to voluntarily increase your difficulty to something that is still well below the minimum required for a valid share (perhaps a difficulty of 1000 or so), which would cut down traffic dramatically from your miners, but still allow you to claim every share in the sharechain that you are entitled to?  Or, am I missing something here?

Josh

Good point.

The current thinking is to ratelimit outbound ping requests to maximum of one ping per second.  If faster than that, outbound pings would not be sent.  Similarly, inbound pings would be dropped (no pong would be sent) if they come in too fast.  User action is necessary to generate these additional pings, anyway, they would never be done automatically at such a high rate.  The existing keepalive formula, despite its hairiness/shortcomings, remains unchanged by my patch.

As for not pinging inbound connections, I think it would still be best to ping all connections, that way, ping times can be measured/compared across all peers.  It makes the data more useful, to collect it from everybody, not just a subset.

Good to know about bitcoinj already having a good implementation of ping nonces.

I added ping time measurement to bitcoind, and it works, using the "ping" and "pong" commands (ping sends a unique nonce, pong echoes that nonce).

I added the ping time as an additional output to getpeerinfo.  The ping time is in seconds, as a decimal number, so it often appears as a very small decimal number with many places, looking rather appropriate for Bitcoin

Now, I can see, in the table of connected peers, the ping time for each peer.  The ping command is handled in the normal command handling function, it isn't given additional priority or anything like that, so the ping time also includes all command processing backlog time, revealing which nodes are heavily congested with commands to be processed (such as nodes trying to catch up on blockchain synchronization).

I also added a new RPC command, "ping", to request that a ping command be sent out to all peers.  This is in additional to the usual case of sending out the ping command as a keepalive.

It's not ready for pull request yet, but to take a look at it, it's here:

https://github.com/Krellan/bitcoin/commit/fbc91c10d818432070ef8bac56b07707f43ef0b1

Interesting observations:

1) The reference bitcoind sends ping commands with 0 (zero) as a nonce, however, during my testing I encountered other peers that sent me valid nonces (nonzero, looks random enough to me).  So, there must be other patches floating around that also add valid nonces to the ping command?

2) I noticed a lot of nonce mismatches.  Upon further inspection, there are a lot of peers that send a pong reply with 0 (zero) as the nonce, regardless of what I had sent to it in the original ping command.  I wonder what causes this?  The reference bitcoind doesn't do this, it is correct, and it looks like a bug in some other client.

3) In my implementation I only keep track of one outstanding nonce/timestamp at a time.  There's nothing stopping the user from sending another ping while still waiting for the first to complete, though.  I wonder if it is worth it to complicate the implementation by using a vector so I can remember more than one nonce/timestamp at a time per node, or only consider the most recent ping attempt to be the only valid ping?

4) I'm using a unique random nonce for each peer, which is rather wasteful of valuable random number entropy.  Perhaps use the same nonce for all peers in a single ping command, or would that open up other problems?

5) Adding the new RPC command "ping" opens up an opportunity for a user with RPC access to attempt to DoS the rest of the Bitcoin network, by sending out ping commands as fast as possible.  I need to ratelimit this.  Any suggestions on what a reasonable ratelimit would be?

6) Similarly to the above, if a peer is sending me ping commands too quickly, I should also ratelimit my pong replies.  Any suggestions for that?  There are other things as well, such as sending me unsolicited pong commands (without a corresponding ping).  If peer is doing these, should I punish them with Misbehaving() or just silently ignore them?

Thanks!

This is a rather cool feature, I'm happy about it, and it was inspired by the "pings" webpage of p2pool.

Josh

You guys are NOT UNDERSTANDING the fundamental problem.

Checking the "Restrict background data" box:

1) It will only turn off cellular data, and will not affect Wi-Fi data.

2) The problem isn't data, it's that the app keeps waking up, taking battery life.

3) Due to unfixed bugs in the app, if it can't reach its peers, it gets unhappy, and spins in a loop, taking 100% CPU.  This will further drain battery life.

4) The app shouldn't be doing things behind the user's back anyway, but this is a philosophical design issue, one that we absolutely and fundamentally disagree on.  You believe that apps should be free to wake up whenever they want, hook whatever events they want, make whatever connections they want, consume finite resources (battery) as much as they want, and at whatever times they want.  We don't.  Thus, this makes it impossible to further reason or compromise about this issue.

Josh

Thank you!

Somebody agrees with me!

I'm not the only one who thinks that this app running in the background, burning up data plan and battery, is not the best idea!

It's a great app, don't get me wrong on that, I just wish it would have a way of not running until it was time for the user to actually want to use it.

A single checkbox, that would be strictly obeyed by the app, would make all the difference:

[ ] Run in background

Uncheck this, and it would never run except when in the foreground.  No hooking the reboot event, to run then.  No hooking the charger connection event, to run then.  No running on a periodic timer, either.  Simply do not run at all, unless the app is in the foreground.  That's really it.  That's all we want.

Your words are effective.  I can't seem to communicate to the developers, they are deflecting my requests, and patronizing me by merely adding a link to the "Data Usage" settings screen.

Yes, I enabled the connection bars.  However, we're at a fundamental misunderstanding here.  Hiding the connection bars will only mask the problem, making it less visible.  The problem is that this app wakes up in the background and does its stuff, and the user has no way to disable this unwanted behavior.  Do you see this?  Do you realize this?

Before I deleted it, it had used 5% of my battery.  That's not just in my head!

Ouch, I just noticed something new: the synchronization now starts happening on its own, on a timer, even when the device is running on battery!

WHY are you doing this?  What is your motivation?

Bitcoin Wallet is such a good app, why do you add these malware-like things in the background?  Why do you hook reboots?  Why do you restart the network synchronizations on a timer?  Can't I just run the app when it's open, and leave it at that?  Please behave like a well-behaved app, and do not consume resources in the background.  Simply shut down.  Do nothing.  When the user leaves the app, close the connections, and allow your process to exit.

I looked at Battery usage and Bitcoin Wallet has taken a full 5% of my battery.  That might not sound like a lot, but I'm trying to save as much as I can.

This latest update is worse, instead of better, because now instead of acting only at reboot and only when the charger is first connected, it now acts on its own in the background, without any way for the user to stop this from happening.

Damn it, it just popped up again, and I thought I had just killed it!  What's the timer frequency?  It seems to be firing every few minutes or so.

Reluctantly, I just sent a payment to zero out my wallet, and I am now DELETING this fine app.  It makes me sad, because this app is truly great.. when it's up and running in the foreground!

I don't understand why making network connections in the background, on a timer, and at startup are not concerns of yours.  It seems that many people would be leery of an app that consumes their network and battery even when the app is closed, and not wanted.  At the minimum this should be a configurable option.

That's really cool!

Glad you took the idea and ran with it.

It's about what I had in mind