Thats certantly the case today but we make no promise to maintain that in the future, if changing it serve some useful end. The block files are not really a user facing interface. Headers first will make it write to them out of order (but still append only), but pruning may delete whole blocks out from under you and also in the future we may implement things like compression which changes the format.

1000 nodes run by one party is very harmful to the network compared to just running one system with more capacity, consolidating node control like that reduces the diversity of voices available to other hosts. Done maliciously it would be called a sybil, done innocently its just a waste.

RFC 6979 is hash-function agile, though it suggests that you use the same hash for its hmac as you use for message hashing in your signature. I believe it's being used by some Bitcoin wallets already, and although the spec is wordy the implementation is pretty straight forward.

It's my general preference to stick with a construction that has has more review absent a good reason not to. E.g. Just do 6979 but with the HMAC-SHA512 used for derivation elsewhere. This way one of the answers to 'oh no I'm giving my private key to this!' is "It's a standard"... obviously this is no excuse for us to not review it too, but it will increase other people's confidence if the specific construction has been widely reviewed.

I also think it's a good idea to have a single spec which defines everything you need in signing, see my BIP 32.5 mailing list post.

It would be good to review what other things are doing. My strong preference would just be to give a simplified explanation of 6979 using HMAC-SHA512,  plus the low-S rule, plus canonical DER encoding. Sipa's canonical encoding BIP can just explain enforcing the canonical encoding rules.  Hopefully we can get everyone behaving consistently, particularly for hardware wallets its important that the _exact_ behavior be verifiable so that you can preclude the possibility of kleptographic backdoors.

Please do not use a #@$@ number without an assignment. Just call it BIP-oleganza-backup for the moment, until the text is ready.  Otherwise we get a mess of number collisions and people calling things by colliding numbers they picked and not wanting to change them. (this isn't nitpicking, it's happened multiple times)

Otherwise— this sounds useful! Should it perhaps specify more of the storage service? e.g. how much data can you expect to store, how would such a service be compensated? how would you know which service(s) you're using?

The last in particular seems to be a tough question... but in general we should probably try to specify a "minimum interoperable unit", and I'm not sure if the message alone is terribly interesting.

WRT the spec.  The IV really should be non-determinstic, it's already stored in the encrypted message. With a constant IV an observer can tell with AES block precision where the first modification to an updated copy was (and perhaps some more elaborate attacks, e.g. it would be trivially insecure if the cipher mode selected was CTR—). There is no need for the IV to be deterministic that I'm aware of... If you're worried about embedded device RNG quality, you could recommend that the IV be constructed as H(time||other-random||pubkey).

You appear to have no length encoded for the plaintext. AES-CBC is only capable of encoding an integral number of blocks, so something must encode the plaintext length.  I might suggest it use self-descriptive padding, e.g. there is always at least 1 byte of padding, and last byte says how many bytes of padding there are (up to 16, though perhaps some applications might want more padding to close a size sidechannel?).  Another style of self-descriptive padding I've seen used is to pad with a 0 bit and then all ones until the end, and the receiver drops all trailing 1s and the last 0 (has the advantage of fewer decodings being invalid).

The signature encoding can be made constant length, and probably should be, doing so will save at least one byte (and probably several, depending on how you were planning on having a variable length signature encoding).

Is there a reason to keep the AuthFingerprint?  It can be derived from the message itself and the signature (e.g. how bitcoin's signed message works), omitting it would save ~19 bytes.

Is there a particular motivation for using a digital signature instead of using a MAC?  One reason I could see is that you might want to have multiple servers synchronizing their data without individually talking to the user, like the PGP SKS keyserver— but for that case you'd want to add a sequence number (so you know if an update you're getting is a newer message or not).

Should these encrypted data chunks have a good-until date coded in them? I'd say it could be provided out of band, but not if we wanted it to be authenticated by the signatures (for the imagined synchronization network).

[Hm. Wow, a synchronizing server would be super cool for this, if we had a good way of avoiding abuse.]

Except you already proved it bad by conventional cryptographic standards in https://bitcointalk.org/index.php?topic=698460.msg7927510#msg7927510 where you find a collision. A good hash-function is not efficiently distinguishable from a random oracle (other than in the trivial sense where it has a compact implementation).

I've struggled a bit watching along worrying that this thread is a train-wreak of everything wrong with amateur cryptography: There are many reasonably well studied and designed functions in this space already (hash functions, and hashcash as proof of work), that benefit from decades of knoweldge and experience in building constructs which are actually irreducible for the purposes they're designed for... This is precisely the kind of effort people are counseled against, and for good reason.

Being difficult to analyze mathematically is pretty much the polar opposite what we want in a cryptographic building block— there is just an extreme danger of a complete break lurking right behind an intellectual speed-bump.  It's often said that anyone can come up with a scheme that they can't crack themselves, but it's perhaps more interesting to note that almost anyone can come up with a scheme with a trivial weakness but which is hard enough to analyze that no one will find it until someone's money (or life!) is on the line.

OTOH, I'm happy people have been having fun; and most of the other POW navel gazing I've seen is even more worthless... I've drafted and canned a couple messages for this thread, struggling with the difficulty in conveying a kind of underlying understanding of what responsible work looks like for cryptographic primitives— telling you to time travel back to 1994 and read sci.crypt with me is not likely to be helpful....

—  but when you get to the point of making claims that the function might actually be useful in cryptographic applications, I feel the need to speak up before another one of these incidents (which was caused by this thread, which went without adequate criticism) happens (read all the messages, not just my first response).

Have fun, by all means— but cryptography is a subtle and difficult science and art. Building a good system is an engineering discipline and having any confidence of security requires formalizing your work and putting in effort which is orders of magnitude more difficult than the basic implementation tinkering. If you don't find that kind of hard core analysis interesting yourself, then sadly it's likely never going to be done for your function. I'd say you were on the right path when you attacked and found a severe cryptographic flaw in your approach, but then something went wrong when you discarded it and continued like it never happened...  If BCT were a forum that I expected competent cryptographic review in, I might also say that something has gone wrong that it's taken this long to get a less than supportive message in this thread.

You were on to something with your attack— why not dig into it more and instead of adding complexity (which might just be obfuscating weaknesses instead of removing them), just assume this is a fun learning experience and see how many other ways you can break the original construct or what other kinds of seemingly-okay-but-really-broken functions you can build in this space?  I think more than anything else doing cryptanalysis and finding holes in functions has increased my appreciation for the enormous challenge that doing good work in this space involves. (I suppose I could say that In one sense there is no science of cryptography except cryptanalysis).

But this is just my curmudgeonly view, offered for your consideration.

Don't confuse blockchain.info's completely broken "Shared Send"— they provide no privacy at all for reasons unrelated with this thread. The privacy implications of well constructed CoinJoins are discussed in some depth in the initial post and some other posts in this thread.

Presumably because he purchased the account, note the two year gap in activity and total change in character.

In any case, this is offtopic for the technical discussion area. Any such system wouldn't be Bitcoin. Cheers.

Thats pretty inefficient and not very private.  It would be straightforward enough to just allow a suffix on addresses e.g. 

B123456789BCDFG-00001  and encrypt the suffix using the negotiated  ephemeral key and include it in the aux data.

Virtually all merchants (and, by definition, all competent merchants) use unique addresses in Bitcoin. This is widely supported, widely practiced, etc.   The fact that you can screw it up and a few people do is no excuse to make an uncommon mistake mandatory.

"Stealth addresses" as currently proposed break what already works pretty severely— using a different address per user makes scanning have a quadratic cpu costs in the number of payments you receive. Bytecoin/monero/etc. have payment IDs which address the issue, though the don't have a way to serialize them with the address— so they get accidentally left out a lot, which seems like an easy-to-address shortcoming.

Making stealth addresses "tip jar only" makes them far less interesting. There is no reason that all payments couldn't be done with this kind of address— assuming it was well constructed, and users would be much less private if only a narrow usecase uses them.

There are a great many things that can be done wrong— including the fact that a poorly done reusable-address proposal can preclude adoption of a good and usable one ("We've already got that, low priority to add another one",  "We're not going to spent time on that— as you can see it's not widely used"). An early flaw was already avoided where it would be incompatible with having a third party scan for you or hardware wallets... lots of stuff to possibly get wrong in this space.

Already we seem to have not learned from the experience with this kind of address in Bytecoin/Monero/etc.—  Payment IDs are an important feature but also a usability challenge with the way they're implemented in BCN/XMR/.... Right now with the way dark wallet implements this if you ask two different parties to pay you with the same address and only one of them does, you cannot tell which one paid you.

I provided feedback along these lines too: https://github.com/bitcoin/bips/pull/17#issuecomment-34442152

What ended up getting proposed there is effectively a fairly insecure brainwallet scheme. I'd personally not recommend people deploy it.

Petertodd claimed to be working on a BIP and reserved a number but hasn't proposed any text. https://github.com/bitcoin/bips/blob/master/bip-0063.mediawiki

New address types should not see widespread use without extensive review for cryptographic and privacy weaknesses and functionality footguns. This really needs a clear specification.

They should be— they're much closer in security model than you're giving them credit for— though this is a bit off-topic.  They can be silently and very subtly changed to give bad random numbers to signing operations, for example, thus constantly leak keys for months without being noticed.  You cannot argue that they're strictly superior to centralized wallets, since the centralized kind may be using things like HSM spending limits, offline keys, and insurance policies to make fairly strong guarantees on potential losses.  (not that I favor either… I just think the "but you have the private keys!" argument is highly misleading about the true security model)

Access to control such automatic updates would be begging to be kidnapped at gunpoint. Not only would it be a systematic risk to the entire system, it would be a risk to the centralized parties with the access to control it.

Classical centeralized updates are undesirable to the highest degree imaginable, they would undermine the very motivations of the system— if you're going to trust someone to freely change the software, you might as well just have them maintain a ledger themselves: it would be much more efficient!

There are tools that could be used to lubricate updates, without creating so much risk— but the ultimate answer is that updates should not and must not be urgent: Even if you could somehow make automatic updates themselves safe, they're still not acceptable— it's not like people with their own patches and update procedures can freely accept the software changing out from under them at any time. Any uncontrolled change in behavior might undermine the user's own infrastructure and cause funds loss... so it's simply not acceptable to expect that everyone is going to run the same software at all times.

The smallest block hash yet found is an estimate of the total work done by the network.  Also, the sum of the difficulties (well *2^32) on the blocks so far is an estimate, which is the one returned in the bitcoin core logs or in the getblockchaininfo rpc.

00000000000000000000b7de9e5c19e52be073156924b7cf235efb27ae8a202a

math.log((2**256)/0x00000000000000000000b7de9e5c19e52be073156924b7cf235efb27ae8a202a,2) = 80.47746080768307

"Apparent difficulty" of 391,895,084,984,304.

And pretty much just on track with the regular block measured work: UpdateTip: new best=00000000000000000000b7de9e5c19e52be073156924b7cf235efb27ae8a202a  height=313338  log2_work=79.978295  tx=43580048  date=2014-07-31 13:29:05 progress=0.999999

This is the first block solution we've found which is a >80 bit partial preimage of 0.

This is a hardfork of the P2P protocol, but not the blockchain.  If you setup a protocol adapter (e.g. a node hacked to change its version handshake to bring the old behavior back) a prior release should still sync the chain... though it's been a while since I've done this.

Sort of a mixed bag there, you can actually take a pre BIP-50 node and fully sync the blockchain, I last did this with 0.3.24 a few months ago. It just will not reliably handle reorgs involving large blocks unless you change the BDB config too. So it's debatable if this is a hard fork either, since it's quasi-non-deterministic. There were prior bugs fixed where older versions would get stuck and stop syncing the chain before that too...

So I think by a really strong definition of creating a blockchain which violates the rules mandated by prior versions we have never had a hardfork.

No, these are.

Unfortunately, bounties don't seem to work, especially if you're looking to (dis)prove security with them— I think it sad, because your ideas are probably the least ill-advised in the memory-hard workfunction space.  (Though, seeing that you're now using siphash internally, I might worry some about your approximation-freseness...)

I say "least ill-advised" with no intended insult to your work: I'm concerned that the whole notion of memory hardness as a useful part of a work-function my be ill-advised:  If anyone is looking for a grant for research on the implications of how "memory hardness" moves costs from energy to die-area and may or may not create differential advantages for attackers or defenders due die-area related costs being amortized during long term operation in either the POW-blockchain model or the password strengthening KDF model, I can very likely find funding for that (esp for the KDF model).

It is becoming conventional wisdom that memory hard KDFs are "well advised", but the analysis in the scrypt paper ignores operating costs— I talked to Colin Percival, and it seems he didn't really have enough visibility into the hardware low levels to even begin to estimate energy impact.  Experience on modern process mass produced computing hardware (e.g. GPUs) seems to show power costs exceeding retail, much less manufacturing costs, in only a couple months of operation, so I'm personally pretty worried that memory hard functions go in the wrong direction, creating amortization advantages for attackers compared to functions whos large scale execution end up thermally bound— but it's an informal concern and I have no idea where the breakpoints are where this would start to strongly favor attacks (/centralization). At a minimum existing "cost" arguments for memory hard functions in some applications are ignoring a majority of the cost (energy), so more research here could be pretty interesting.

Being created by people who are not yet qualified to write software in this space. You're likely to go broke if you create custom transaction processing software at your current level of expertise, if not due to transaction grinding than due to something else. When you build some kind of automatic transacting system you're effectively building a cryptographic protocol, small details can result in complete insecurity.

It is perfectly possible to transact in Bitcoin without using the blockchain for every transaction (in a decenteralized way too— though for buying sodapop federated solutions may be much less costly).
Yep, though note that that post was also from before the million byte limit was added to Bitcoin, along with many other protections against lost of decentralization and against denial of service... it's an argument to the general feasibility of this class of approach, and indeed— it's fine. We're still not yet to a point where sending 100GB/day is "not a big deal", nor is demand for Bitcoin transactions anything like that (and, arguably if Bitcoin currently required 100GB/day now we never would reach that level of demand— because such a costly system at this point would be completely centralized and inferior to traditional banks and Visa). Visa's 2008 transaction volume is also a long way from handling the total transaction volume of the worlds small cash transactions, as you seemed to be envisioning— yet Bitcoin _can_ accommodate that, but not if you continue to believe you can shove all the worlds transactions constantly in to a single global broadcast network.