With respect to detailed questions about the issue, we're currently being somewhat vague— simply because it's helpful to give innocent users as much of a head-start on trouble makers as possible.  

At the current time we don't know that the issue is exploitable. The class of issue and the overall paranoid design of the reference client make it hard to tell for sure. It is probably hard to exploit if it is exploitable at all.  Because of the potential seriousness the issue has been dealt with promptly and as if it were exploitable. (I'm not answering the specific timing questions because they may identify the issue too clearly).

If the issue turns out to be practically exploitable we'd much rather learn of it as a purely academic fact— academic because almost all impacted users had already applied fixes—  a few weeks from now, than learn about that in the form of hundreds of wallets being stolen through an exploit in the next few days.

I always encourage people to review the git history, but if you spot the fix for this issue— please don't point it out yet (and I will remove posts that do), if you like you can contact me privately and I'll gladly tell everyone how smart you were later. —  Reviewing the commits is generally good advice it's always good to have more eyes on the code, and even if you don't satisfy your curiosity with respect to this issue you may learn something else useful.

An empty string would be fine. It doesn't really matter. The point of H2 is just so it would be easy for people to do the proof-of-memory for you but only so long as you take the transactions they selected. If there aren't any transactions then H2 could be anything, so long as it was standardized.

When someone who is clearly experienced with the system suggests something that sounds pointless I usually assume I've misread the suggestion and try again.  I've found this to be a useful technique. I think it's one you might consider adopting, because we've clearly failed to communicate here.

I'm unsure where I failed on my part: I already emphasized including the _INPUTS_, not the content of the prior block, as the mechanism and that the particular motivation miners who do not have the the set of unspent transaction— creating an incentive to mine no txn, or I'd try to restate it to make it more clear.

I don't consider botsnets problematic on their face.  Why should I?

"What is to stop the botnet from holding blockchain info on his own computer and communicating it to the bots via the net"

It makes the botnet easier to track, block, discover, and the owner easier to catch.  But if they want to do that great!  

I suspect you're missing the point of my message:  It's arguable that there is an economic incentive to be anti-social: You can avoid the cost of maintaining the blockchain while mining so long as you don't mine any transactions.  We can largely eliminate that misincentive. I don't actually think such a change is required now, I'm just pointing out that we're not helpless.



You actually don't know that— "they" might not even exist.  There could be some bug in popular getmemorypool mining software causing txn to not get mined, for example.  Alternatively, the miner(s) here might happily mine txn with large enough fees.  You're identifying them only on the basis of them not having txn, so if the party in question even exists you'd be intentionally ignoring the blocks where they do mine txn with fees, so you can't say they don't.

Some people have hypothesized the that the recent five-fold increase of the number of 1tx transactions (now about 16% of blocks) is due to a botnet mining without a copy of the blockchain (which might get noticed by the owners of the stolen computing power) and also without using a pool (which creates a central point for tracking the botnet).

I've not seen any real evidence for this, nor do I yet share the concern people have had over it—  

But if this is really the case, we can prevent this kind of behavior:

Take the _inputs_ (not their IDs, the data) to the transactions in the previous block.  Hash them, giving H1.

Take the inputs to the txn you are currently mining, hash them, giving H2.

Take the hash of all your outputs in the coinbase giving H3.

Add to your coinbase H(H(H1||H2)||H3).

Make including this a protocol rule.

Now the botnet can only mine if it had access to verify the transactions in the prior block (and the block its mining).

Every stick needs a little carrot too:

Make every full node also give a getmemorypool style command on the network port which gives out a set of transactions, along with H(H1||H2).  A botnet that wants to mine without having a copy of the blockchain can trust random nodes to provide the proof of memory, but if it does so it must take the exact set of transactions that node provides.

You could go further and commit to random txn in the open set instead of just recent inputs, but that isn't required just to shift the incentives away from memoryless mining.

They bound some corner case DOS attacks (I send you a million block difficulty 1 fork, filling up your disk space), and reduce isolation attacks (new person installs software and an evil ISP keeps them from connecting to the real bitcoin network) to the problem of getting an authentic copy of the software or the problem of plausably maintaining a fork at recent difficulties.

Don't read too much more into them than that.

Yes, the 120 confirmations help a lot— as does the fact that eligius only pays out .66 BTC or more (unless you stop mining).

The metric is similar to coin-days-destroyed... the greater the input value * age the greater the txn priority.

Different implementations pick different checkpoints (or none at all— they aren't strictly necessary— they just close a couple of DOS attacks and reduce the risk of attacks via network isolation).

In the reference client the checkpoints are in checkpoints.cpp.    0.5.2 (the most recent full release) has its highest at 140700.  0.6.0 will release with its highest checkpoint at 168000. (the network is currently at 170201, we'll probably be at >173500 by the time 0.6.0 is released).  For the reference client new checkpoints are just introduced periodically by developers at whim but always thousands of blocks back so that they're never placed anywhere near competition between chains.

Based on your comments I'm quite confident that you're actually running 0.6.0rc1. What you are seeing is exactly symptomatic of rc1. You can get rc2 from http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.6.0/test/   if you really did install rc2, are you sure you restarted after doing so?

_any_ miner lying can lower the difficulty slightly, so everyone gets a marginal return for doing it. This makes larger drifts (which at still limited to half a percent one time reduction in difficulty at most) harder, yes, but it makes smaller drifts much more likely.


If I understand correctly you're proposing:

10./(2016./sum([1./GAPi for i in range(2016)])) = adj factor

So, if GAP is all 10 you have 1.  ... All 5 you have 2. Etc.

What happens when every third is a false tick that goes back 10 minutes, then the next tick is correct.

The gaps then look like [10, -10, 30, ...].   The arithmetic mean gives the right value.

10./(2016./sum([1./(10+(-20*(x%3==1))+(20*(x%3==2))) for x in range(2016)])) = 0.1111x   !

Shall we try for a geometric mean so that we can get an imaginary adjustment next?

This actually isn't desirable.  In the current system miners have no economic incentive to lie about their timestamps (no marginal gain from doing so), except for the window boundary block, whos permissible times are constrained by the surrounding blocks and the actual time, allowing at most a small one time difficulty shift (assuming the warping bug is fixed).  Including other blocks in the time computation would create an incentive to lie constantly for every miner, and for the same final response time a much larger allowable skew.

Thats not to say that such a change wouldn't have also been a viable approach— it probably would have been— I just don't see that it's clearly better.

Your summary is not very accurate or useful.


None of the development is going on in secret. The discussions are out in the public and are fairly transparent. If you don't have the time to follow and understand all of it, then thats a tradeoff you're choosing to make.  If there are reasonable ways to improve the transparency then I'm sure offers to help improve communications would be appreciated by all.

Or what do you expect? Certified mail on every commit?  You don't get to demand personal service.

(and frankly— a bit of unsolicited advice: the screeching bold faced text from an account with 25 posts just screams sockpuppet/troll — you might want to consider moderating your tone before you get written off as someone who's just trying to create controversy for fun)

Er! Danger Will Robinson! Danger!

The selection function is fairly greedy but the objective is non-convex, this can lead to surprising results.  You can currently end up paying more fees than you need. Consider the following sequence:

Say you want to send 1.0 BTC.
Selection pass runs, finds 1.0 BTC using 100 .01 BTC inputs that all have more than six confirms.
Software then decides that a fee of .01 (random figure) is needed based on the size of the transaction.
Selection runs again looking for 1.01, but can't find that much using inputs with >=6 confirms, so it considers 1 confirm transactions.
Selects inputs of 1.0,0.01 from the set of transactions with at least one confirm, and pays fees that are not really required.

Adding more stages makes this worse.   I have a patch here that first only considers inputs from the set which are high enough priority that any transaction composed solely of them will be high priority, somewhat like your posed your proposed patch but it sets the threshold to be max(144/value,1) confirms.  In my testing I found that it pretty frequently caused stupid decisions at least for wallets with distributions like mine.

If you want to iterate it should be over the entire CreateTransaction, not just inside select coins (and, in fact, all the SelectCoins iteration should be removed). I'm hesitant to write code to do this because any change in this area desperately needs test cases. This code is far too risky to change lightly.

I'll try to find some time in the next couple days to elaborate on my thoughts... I'll also clean up the stuff I wrote before I realized the optimization attack would be an issue.

(it may also be better to describe it as running arbitrary stochastic algorithms — not just MCMC, because it can run things that people wouldn't generally think of as MCMC at all too.)


What made me sad about this is that it ultimately means the system can't subsist entirely on 'useful' work— that there will always be substantial busywork overhead because useful work (potentially optimization-attack work) must always net destroy coins or it will be economically beneficial to produce such work in order to gather the resulting coins.  This may indeed still leave the system attractive vs something like litecoin (where the reduced security model relative to bitcoin may be tolerable simply because of the different nature of use) but still ultimately limits its effectiveness over a more straight forward pay-for-computation model...

There are advantages over straight pay-for-computation, in that you've removed some of the risks (e.g. risk I'll pay for computation, and they won't do it— or the risk that I'll do computation and they won't pay— the risk that I ask for N units and they really do M), etc... though there are other ways to solve these problems.

In any case— even if your system goes bust as a blockchain technology— all the LUA sandbox, best-solution-prize, turning random solutions into distinguished points, stuff could be applied to create an interesting distributed computation tool.

Then again, who knows: Other than people paying for bitcoin mining remote pay for idle time computation has basically gone nowhere. I probably had UID 10 on CPUshare... but the usage and tools just didn't seem to materialize. I'm shocked that someone hasn't added password/wpa cracking kernels to the RPC bitcoin miners, since they could all also are very agreeable to distinguished point POW, and run great on GPUs.... but it just hasn't happened.

... more after I've had a chance to think for a bit longer

This can be worked around quite reasonably.

If there are things that the new GUI does poorly, please open issues on them: https://github.com/bitcoin/bitcoin/issues

There are many people willing to work on and maintain the QT interface. If there are objective weaknesses that are bugging you they will probably be fixed. (On matters of taste— well, no promises... Its no good making one user happy only to make another unhappy).

No one is maintaining the WX gui anymore.  No one is willing to do so. It's even a pain to build.

I have a simple protocol here for private messages associated with transactions which can only be read by the reviver. This would allow inband broadcast green addresses without compromising privacy or bloating the blockchain:

Say we have a TXN(Me->You) with a TXID.  I want to tell you that it was really RealME who created this transaction, even though no one knows who Me is, and I want to do this in a way you can prove to others that RealMe claims to be Me should I screw you... but we don't want anyone else to know otherwise.

So I first send TXN(Me->You) on the network.

I then form a message to be sent secretly to you  Message={TXID,SIGN(RealMe,TXID)}

I then compute Ek = ECDH(Me_private,You_public) and then compute Em = Enc(Ek,Message).

Then I sign Em,  Sig_EM = SIGN(Me,TXID||Em) and finally I broadcast {TXID,Em,Sig_EM}.

Relaying nodes are able to associate the message to a particular recent transaction and only allow one (or a small number) of messages to prevent DOS.  But other than telling that the sender of TXN(Me->You) send a message they learn nothing more.

The recipient of the transaction would see they received a message. They could repeat the procedure, decode the message, and find the signature in it proving that RealMe claims responsibility for the transaction.   Should I cheat them by reneging on the transaction, they could publish the signature to prove my untrustworthyness.

The problem here is that the addresses we send funds to don't include a public key... only the hash of the public key. So I probably don't know You_public to perform the ECDH.   This could be resolved by making an address type with an extra 256 bit compressed public key tacked on the end.  This key would only be used to receive private messages associated with the address.  (This would also have the benefits of working with P2SH transactions, even complicated multisig ones so long as you don't mind that there is a single address which can be used to decode messages).

Such a system would enable some unfortunate additional parasitic applications of the Bitcoin network... but better to have them only ephemerally sending a limited amount of traffic rather than imposing on the blockchain for all time. By rate limiting these messages based on transactions they shouldn't make the situation any worse.

I'm not generally in favor of this at this time.  "Green addresses" add irrelevant bloat to the block chain (no one but the recipient cares about that data, certainly not people 10 years from now) and compromise the privacy of Bitcoin users— even people who aren't directly the users of green addresses.  They also encourage poor key management practices (e.g. keep all your funds on a single green address) or even more chain bloat (by doing two step transactions).

I also understand from talking to Magicaltux that green addresses have not been much of a success.

We added the signmessage functionality so that people could use a well known public key (your green address if you will) to sign any data they want, including transaction IDs, without putting anything in the blockchain or publishing it.    For example, when you made a withdraw from a trusted bank, the bank could use its well known key to signmessage the relevant transaction ID.  It could then either send this signature directly to the recipient if you provided a payment URL,  or provide it to the user as a "confirmation value" which they could carry over to the other side.

Unfortunately MTGOX did not implement the former because they don't want their customers telling them that they're sending funds to $QUESTIONABLE_PLACE, and didn't implement the latter because it's not fully automatic.

I know nothing about the MCMC problems used for physics, but I sure can't see how this can be used for MC sampling for bayesian inference, since you actually need the whole chain data after the bootstrapping. Perhaps it would be helpful if you spelled out some example problems and gave the setups for them.

As someone who uses optimization frequently the idea of being able to buy cycles to run arbitrary stochastic algorithms is a very exciting one. 

In particular, I often use optimization ~interactively while problem solving. While my average utilization is fairly low, I would really like enormous amounts of computation in short bursts. If I could buy EC2 in 5 minute chunks I'd regularly be buying 1000 CPUs worth at a time.  So a distributed system which allowed me to shift computation in time in order to get more of it at once would be _VERY_  useful to me.


I have some vague concerns that the result of the Lua execution engine (with software floating point!) will be that such a system could not be competitive with centralized computational services— simply because you're going to take some large slowdown to start with. You don't seem to have addressed resource exhaustion except for speed, e.g. running the systems out of memory, or simply requiring great amounts in order to disadvantage nodes with smaller amounts creating a race where all participants must have as much ram as the most and making it hard to reason about the returns on investing resources into this system. Yes, participants can use the escape hatch out, but at a 10x disadvantage to ones that have enough ram to keep going.

For the same reason, I'm unhappy with the alternative block regular hash— how is such a system to be competative with 50% deadweight loss (on top of the other losses)?

Bleh.  I wrote several paragraphs of text with excited ideas about this and that— and also arguing that you didn't need the second hash function because if you already had 51% computing power you'd be better off using it to run optimized versions of your own computing work then using it to trick the rest of the network into running inefficient versions of your work.

When then made me realize that your system has a pretty debilitating "Optimization attack"— I'm sure this isn't news to you, thus the every other block optimization... but I don't really think it helps much. I think your paper should spell this attack out clearly.

Say an attacker forms some A() which is just a trivial function e.g. A(x,y)=x, he also forms an A()', say A(x,y)'= sha256^1000(x) * 0 + x which is identical to A() in results but which is much more expensive to compute.  He announces A()' to the network, but detects it on his own systems and runs A() instead. He now has a enormous speed advantage on all other nodes.

The alternative block idea prevents this from being trivially translated into an overpowering attack, though the security is still enormously reduced compared to blockchain cryptocurrencies. But I don't think thats the worse result.

The worst part of this is that even ignoring overpowering attacks, I believe it will be economically advantageous to perform this attack so long as getting the network to use some A() always costs strictly more than the reward. This means that currency will only enter the system through execution of the dummy A that cost no one anything to run, which (assuming constant computing power) means another 2x overhead: for every unit of real work you do, you must have a unit of dummy work to create the coin to run it.

This ... on top of the standing halving required to make overpowering even a little non-trivial.. and the halving (at best!) of using a highly sandboxed execution engine (rather than optimized machine code)....  As a result this system probably has a peak computing efficiency of 12.5%, while offering substantially less security than existing cryptocurrency in a number of regards (the >25% attack, the fact that the POW can be expensive to validate— burdening regular nodes not just mining ones, the fact that the POW might be insecure to validate, again burdening regular nodes and not just mining ones).   For people who don't need to store up computing for burst loads this system could pretty much never be more cost effective then simply computing their desired work directly.

Have I gone off in space here?



In any case, in the chance that this isn't as useless as I now fear, before finalizing your system I'd like to encourage you to take the chance to pick up some protocol improvements which can't easily be implemented in Bitcoin. The anti-timewarping fix would be one obvious one, less trivially, you should probably change to Ed25519 from the secp256k1 curve. There are a number of proposed changes at https://en.bitcoin.it/wiki/Hardfork_Wishlist though many of them would be inapplicable to you or a bit too blue-sky.

I've unsticked this and updated the subject to indicate that this isn't a pool (and thus its hashrate is doublecounted against the pools its actually using) but instead a hopping proxy service, as it's name is potentially misleading.