Coinbase spread is insane now. They are buying Bitcoin for $71 and selling it at $137!

My last trade was executed at:

    2013-04-11 13:57:48 UTC

At the time, trading lag was about 1 minute - not as bad as yesterday.

According to the banner, trading will re-open at:

    2013-04-12 02:00:00 UTC

So we're looking to 12 hours of down time, right in the middle of the US-day.

I would have preferred throttling the number of trades per account to some number per hour, rather than stop all trading across the board.

I can only imagine the huge crush of trades that will be placed exactly at 02:00 UTC today - likely will hammer their servers even harder than
they were yesterday.

I just finished recovering credits for accounts that were mining during our two outages:

122.5 minutes in duration 2012-10-26 15:23:19 to 2012-10-26 17:25:48. (GMT)
Shares earned from Sat Oct 27 2012 11:57:00 GMT+0000 (UTC) to Sat Oct 27 2012 19:49:00 GMT+0000 (UTC).

I also fixed the root cause of our backend server crashes.

For those of you that were showing missing credits - you should be back to normal now - plus a little bit.  I added a 30% bonus on all shares reported during these credit outages just to make sure everyone is made whole.

If you still see a discrepancy not in your favor, please let us know and we can look more closely at your account.

Going to bed now...

- Mike

I had a critical server go down twice this week - for a total of 9 hours (ouch - I know - our monitoring alerts also failed).  I found the problem with the server, and I was able to capture all the earned shares (though your online stats do not yet reflect the shares earned during in those two gaps - I hope to get that back up some time Sunday).

The server that went down was responsible for recording credits into our credits database.  I was able to recover data from our raw share reporting data store - it's just going to take me a little while to migrate it back into our credits database where you can see it again.

I hope you'll be patient with me.  I promise to recover the data that was lost - you'll be credited for every share earned (and we'll kick in some bonus credits to thank you for your patience).

We do.  The only thing we check is work age.  We measure how old the work we gave you is.  If it's stale, we still give you credit if it falls within our 60 second grace period.  So PLEASE DO send stale shares to us (but please also refresh your getworks each 60 seconds).  If you do both of those, I see no reason why miners on our pool would see anything but 0% stales.

Point taken.


Our server is configured to allow longpolls to be held open for up to 90 minutes; after that the connection is dropped (and the client should see the dropped connection so it can re-establish it).  It's possible we have some other bug in our pool, but our median work age runs about 45 seconds - so it seems the majority of miners are having no problem getting recent work.  Regardless of long-poll connections dropping, our headers clearly indicate (to me) that the work should be refreshed each 60 seconds.  So even if we do have a bug where miners are silently having their long-polls dropped, they should be unilaterally fetching new getwork every 60 seconds.

As to the policy of accepting older work - we wanted to remove any consideration of communication latency being a barrier to adopting our pool.  We assume the risk of latency we introduce into the system - as a miner, you get credit for all work you do (as long as you get reasonably fresh work from us - i.e., no more than 60 seconds old).  We felt that places the incentives on us to improve our pool performance, but doesn't penalize miners when we have latency issues.

The spec seems to imply (to me) BOTH the allowed values of the timestamp AND a deadline time for which they will be accepted.  As a practical matter, we even accept work with larger timestamp rolls as long as the block being worked on is still current.

An update on our LONGPOLL experiment.

I've removed the extra LONGPOOL flushes - your LONGPOLLs will now only update when we know about a new Block to mine on.

I agree that this was a misuse of LONGPOLL - but let me explain my reason for trying it.

Our pool has a guarantee of accepting work - even if it is stale - for a fixed amount of time from when we generated the work for the miner.  Right now, this
is set to 60 seconds.  This information is implied by our getwork headers:


Notice that we specify a 60 second expire time in X-Roll-Ntime.  That's telling miners that not only can they increment the timestamp 60 times, but also
that they should NOT be submitting this work more than 60 seconds in the future (if they do - they risk getting a stale).  I don't know if other pools
have the same kind of strong guarantee of accepting stale work, but we've implemented a "Grace Period" - we will accept even stale work for 60 seconds
after the end of a block - but only if you received the work within the last 60 seconds.

We notice many miners do not update their work frequently.  According to the semantics of the X-Roll-Ntime extension, a miner should cease mining
on blocks older than expire seconds old.  My aggressive sending of LONGPOLL flushes was my attempt to get miners to update their work each minute.

Here's a snapshot of our internal metrics.



Before the change (at 12:15 PST today), you'll see that we were flushing each 60 seconds, which results in relatively few stale shares when each new block is
found (called "generation start" in the chart).

After the change, you can see no more frequent long-poll flushes, but relatively more stales at the start of a new block.

There are a few miners on our pool working on very old work - note the work age graph (darker bands indicate older work shares).  A properly written miner on our
pool should not be submitting work much more than 60 seconds from the time it is received.  We are seeing shares on work that is over an hour old in the extreme
(a very buggy miner), but also over 5 minutes from several miners on our pool.

If you're getting a significant number of stales while mining on our pool - let us know what mining client (and version) you're running; we'd like to eliminate stales completely
from our pool.

I can't access any of the wiki pages most of today (Google's cache is helpful, here).

Do we need a mirror for this site?  Seems like it has been down quite often, and, as a central repository of technical information it's a bit frustrating to have it be unavailable.

I wrote a Munin plugin for the getinfo data you might like to graph with Munin.

https://github.com/mckoss/contrib/blob/master/plugins/other/bitcoind_



You can graph: wallet balance, peer connections, block number, and difficulty.

I was feeling nervous about the lengthy delay in the order fulfillment coming up against the PayPal 45 day deadline for order dispute.  So I asked BFL for a refund via PayPal - I just got it back, no questions asked.  That might make others in the order queue feel more confident that BFL is a reputable company.

I wanted to test that the transfer systems actually worked to get money out.  So, in December, I was able to withdraw a significant amount of money via:

Bitcoinica -> MtGox -> Dwolla -> MY BANK

It all worked and just took a few business days to get real $ back into my bank account.  I feel satisfied that it's not all a house of cards where money goes in and can never get out.  The longest delay was a few days waiting for "manual approval" at Bitcoinica for a large $ withdrawal (Zhou was traveling, I think, which slowed down customer support on Bitcoinica in December).

Too bad I sold my BTC at $4 to do this test :-(

My pleasure - and thanks for the input!

I've uploaded a new Bitcoin Primer PDF - based on @Clark's design, but now more compatible with different devices and smaller (using standard PDF fonts).

Good point.  Is this better?

Thanks.  You might also like to see this version:

http://wiki.pageforest.com/#a-bitcoin-primer

You can edit online and see changes in real-time, and save your own version on the Pageforest Wiki site.  I used this to edit the original when I was transforming it to Markdown so I could see formatting changes in real-time.

Now I just need to find a way to convert to LaTex or similar, to apply the style sheet used by Clark's formatted version to create a two-column printable PDF.

I've migrated the source of this document from Google Docs to be a Markdown-formatted file AND moved the master source file to GitHub.  If you have any further corrections, you can edit it yourself and send me a pull request!

Thanks again for all the input!

Good points.  Fixed the 404, BTW - I was updating the links to Clark's excellently formatted version.  But I restored the updated plain-formatting PDF as well.

Thanks!

Thanks!  Glad you like it.