Soft forks

[domob]

A soft fork means that old code will accept the new rules

nope, that's wrong

Yeah I agree with Carlton here. The segwit soft fork meant that legacy blocks are still accepted after the fork but this doesn't work the other way... From what I can tell the signed part of segwit transactions just get ignored by full nodes before v13 and they just take the 150 byte transactions (as an example from the standard size) because the longest chain is running with them (and there isn't a reasonable rival).

I sorry to say that, but this is still wrong.  See here for what a soft fork is: https://en.bitcoin.it/wiki/Softfork

I.e. it means that only additional restrictions are added (in the case of segwit, that the witness must be valid in addition to the old-style signature).  This means that old nodes accept all new blocks, but it also means that transactions that would be accepted by old nodes may be invalid according to the new rules.

Now it could be that in the particular case of segwit mining, you are fine running old code, as long as noone maliciously sends you transactions that are valid according to old rules but invalid according to new ones.  But that would then be just by chance and not a general property of soft forks.  Also, at least according to my understanding (I'm not a miner and haven't looked at the details recently), aren't segwit blocks supposed to have a new commitment in the coinbase, which old mining software wouldn't create?

[1714654284]

1714654284

[1714654284]

1714654284

[1714654284]

1714654284

[1714654284]

1714654284

[gmaxwell]

There is confusion about terms because "soft fork" just means that the blocks under the new behaviour will still be accepted by old nodes. Under that definition mining with pre-softfork behaviour wouldn't necessarily work.

Yet the development community prefers a much stricter set of criteria-- perhaps call it "safe soft-forks", where at least no widely used or recent software prior to the fork will inadvertently initiate a fork which is invalid with respect to the new rule. These "safe soft-forks" also have other properties like great care taken to avoid inadvertently depriving users from access to their funds (unless they were doing something absurd like using policy prohibited extension opcodes or version numbers).   Unless it's impossible to avoid, you should expect "safe softforks" to always be used.

Per the question, per consensus rules you can mine bitcoin blocks without the segwit coinbase commitment even with transactions -- just not any segwit ones.  However, all the mining interfaces in current bitcoind mandate that you use segwit to avoid misconfiguration where miners accidentally don't and then miss out on a ton of fees (and cause nuisance delays to users). Bitcoin nodes also don't fetch blocks from non-witness capable peers and so on.

The effort to add the witness commitment is pretty trivial,  probably smaller than the effort of working around all the ways that the rest of the software expects you to have it. Plus the fees you miss by not having transactions are a couple grand per block -- a good 2% of the mining income right now.

[Carlton Banks]

I.e. it means that only additional restrictions are added (in the case of segwit, that the witness must be valid in addition to the old-style signature).  This means that old nodes accept all new blocks,

yep

(but before you said "old nodes will accept new rules", which is not the same, and self-evidently not possible)

but it also means that transactions that would be accepted by old nodes may be invalid according to the new rules.

nope, you're wrong

if nodes that observe a soft-fork's rules rejected blocks containing previously valid transactions, that would fork the chain. which isn't possible in this case, because the segwit soft-fork does not reject any rules that were valid before it's activation, that's the definition of a soft fork

but I don't see much point in arguing this further, tell us which blocks have been rejected by any soft-fork's rule set, that would constitute evidence of your claim

aren't segwit blocks supposed to have a new commitment in the coinbase, which old mining software wouldn't create?

yes.

But the old way of doing it is still valid, because soft-fork

[DaveF]

The effort to add the witness commitment is pretty trivial.

Aside from that part of your statement you are correct.
It's (fairly) easy to get NOMP working.
I did it in less then half an hour over the weekend to get something fixed:
https://www.reddit.com/r/defcoin/comments/dxr3xe/network_outage/

1) A few lines in the conf file for a coin
2) Copy / paste a few apt-get install lines that are listed everywhere.
3) 2 Quick edits of 2 pool configuration files.
4) Beer

I have no idea if the above would work / does work / can be made to work with SegWit.
There are other pools that do, most do take more effort then the 30 minute NOMP setup.

Am I going to find out?: No
Could I find out?: Yes
Why?: Not going to run any real pool and I just don't have the time.
With a quick search (~10 minutes while having my AM coffee) could I find the answer: No, I did not.


-Dave

[Carlton Banks]

There is confusion about terms because "soft fork" just means that the blocks under the new behaviour will still be accepted by old nodes. Under that definition mining with pre-softfork behaviour wouldn't necessarily work.

Yet the development community prefers a much stricter set of criteria-- perhaps call it "safe soft-forks", where at least no widely used or recent software prior to the fork will inadvertently initiate a fork which is invalid with respect to the new rule.

it seems disingenuous to describe the bolded behavior as a soft fork imo. it would be clearer to simply call the "safe" soft-fork plainly soft forks, and to popularize the "new rules can invalidate old rules" behavior as a... "permissive hard fork" is the expression that occurs to me from the top of my head. Maybe it's the wrong expression in some ways, but it's less misleading than the "impermissive" soft fork. If segwit had been implemented in such a way that, for instance, invalidated a common pre-fork consensus parameter, there would have been a least some orphaned blocks on that basis (and more likely an entire chain split, and a very likely failure of the segwit compliant fork). I am aware of no such event happening.

more simply; if a self-described soft fork can potentially produce a hard fork, then we need better words to describe what's really going on

[domob]

I.e. it means that only additional restrictions are added (in the case of segwit, that the witness must be valid in addition to the old-style signature).  This means that old nodes accept all new blocks,

yep

(but before you said "old nodes will accept new rules", which is not the same, and self-evidently not possible)

Why do you say that's not possible?  In the case of segwit (and a soft fork in general), old nodes will accept transactions and blocks following the new rules - in your terms because soft fork.  They will simply see an "anyone can spend" signature.  (This assumes that you do not send them witness data they do not understand, but that is part of the network protocol and not the consensus rules.  You could also discuss a softfork of "reduce block size" to make it simpler to understand.)

but it also means that transactions that would be accepted by old nodes may be invalid according to the new rules.

nope, you're wrong

if nodes that observe a soft-fork's rules rejected blocks containing previously valid transactions, that would fork the chain. which isn't possible in this case, because the segwit soft-fork does not reject any rules that were valid before it's activation, that's the definition of a soft fork

Sorry, but that's just what a soft fork is.  Segwit introduced new rules, namely that in addition to the pre-segwit signature validation, also the witness needs to be valid.  If you take a segwit transaction and remove its witness, it is perfectly valid under the old rules and will be accepted by old nodes.  But it won't be accepted (without the witness) by nodes following segwit rules.  In other words, if you were to build a block of segwit transactions without the witness data, then old nodes would accept the block while new nodes would reject it.  That's exactly what a soft fork is.

The reason why that is useful (and thus why a soft fork is better than a hard fork) is because this allows nodes and wallets running according to the old rules to remain working, since they will accept anything that used to be valid before.  (Just their security might be reduced to SPV level.)  But if you do a soft fork and miners also do not upgrade, then yes, you get a chain fork.  (Which is why there were things like BIP9 signalling required from a majority of nodes.

[Carlton Banks]

ok, instead of fanning the argument out into things everyone agrees with, let's stick to the point I disagreed with:

transactions that would be accepted by old nodes may be invalid according to the new rules.

that didn't happen with segwit, it was implemented using the so-called "safe" soft fork design

you implied it could, and that was wrong

[domob]

transactions that would be accepted by old nodes may be invalid according to the new rules.

that didn't happen with segwit, it was implemented using the so-called "safe" soft fork design

you implied it could, and that was wrong

Actually segwit has cases like this.  As I wrote in my last post, if you take a segwit transaction and strip the witness data off it, then it will be a valid transaction under the old rules but not valid in the new ones (because it is missing the witness).  Adding restrictions like this (in the case of segwit, that the witness is verified in addition to the old-style signature) actually is the only point of a soft fork, and hence why you may call it "enforcing segwit".  gmaxwell's mention of "safe" is very useful in the context of this discussion, but if you re-read his answer precisely, it only states that that means "extra care" is taken.  My quote above, as it stands, is correct as far as I can tell.

Perhaps it is easier to illustrate this with the BIP16 soft fork instead of segwit.  In this case, the validation rules for scripts were modified, so that

OP_HASH160 [20-byte-hash-value] OP_EQUAL

would be validated according to the old rules (i.e. you need to provide a preimage for the published hash), but also the preimage itself must validate as script.  Pre-fork, you could spend a P2SH output by simply knowing the redeem script itself, even if you did not provide valid signatures for it - because that is what the script above says.  The soft fork then introduced the bolded extra rule, so that P2SH would work meaningfully.  But this means that you can produce blocks and transactions that are valid for pre-BIP16 nodes but invalid with BIP16.  There is one pre-BIP16 block, 00000000000002dc756eebf4f49723ed8d30cc28a5f108eb94b1ba88ac4f9c22, which actually has this situation - it is valid pre-BIP16 (and is valid because it was created before the soft fork), but it would not be valid anymore now due to the added restriction.

But maybe we are just misunderstanding each other's meanings.  I think the OP's question related to the use of "old mining pool software" has been answered.

[Carlton Banks]

Actually segwit has cases like this.  As I wrote in my last post, if you take a segwit transaction and strip the witness data off it, then it will be a valid transaction under the old rules but not valid in the new ones (because it is missing the witness).  Adding restrictions like this (in the case of segwit, that the witness is verified in addition to the old-style signature) actually is the only point of a soft fork, and hence why you may call it "enforcing segwit".  gmaxwell's mention of "safe" is very useful in the context of this discussion, but if you re-read his answer precisely, it only states that that means "extra care" is taken.  My quote above, as it stands, is correct as far as I can tell.

ok, well it's difficult then to see how any soft fork can be implemented without an inherent risk of a hard fork. You're basically saying that pre-fork nodes are blind to the new rules, and so one can break the new rules without the pre-fork nodes being aware, that's perfectly sensible.

But you're still wrong. The old nodes are not "accepting the new rules", as you said. They cannot be accepting any new rules, because they are not designed to recognize rules that did not exist before that version of the consensus rules were written, Bitcoin does not (yet) have an algorithm that can see into the future!

What's happening is that they do not understand any new rules, and blocks with transactions adhering to any new rules are interpreted as "no signature required", i.e. anyone can pay. How else, logically, could old nodes be capable of accepting tx's that both observe and violate a new soft-fork? It's because they don't know what the new rules are that they are able to accept both, that's how anyone-can-pay can function as a backwards compatible upgrade mechnanism, because it enables nodes following only the older rules to ignore any and all assessment of adherence to new rules with 1 universal rule: "this transaction is valid, because it has no spending conditions". Saying that's the same thing as accepting new rules (which you did) is not accurate.

You seem to be dancing around the semantics, for reasons that are

1. Not obvious. What is this achieving
2. Not relevant to the OP anyway


I could instigate an attack on the ambiguities of the way you've worded things too, but I'm not going to do it. Why do you think that is?

[domob]

ok, well it's difficult then to see how any soft fork can be implemented without an inherent risk of a hard fork. You're basically saying that pre-fork nodes are blind to the new rules, and so one can break the new rules without the pre-fork nodes being aware, that's perfectly sensible.

Yes indeed, that's what I'm saying.

But you're still wrong. The old nodes are not "accepting the new rules", as you said. They cannot be accepting any new rules, because they are not designed to recognize rules that did not exist before that version of the consensus rules were written, Bitcoin does not (yet) have an algorithm that can see into the future!

Ok, so I see where the confusion comes from - my original post above is probably not worded in the best way.  However, this one is clear IMHO:

transactions that would be accepted by old nodes may be invalid according to the new rules.

But you claim to disagree with that exact quote.  Can you please explain what you think is wrong about this formulation?

[Carlton Banks]

i don't care to really

(i've since then agreed already that it's possible, why are you ignoring that?)


here's what we're supposed to be discussing:

A soft fork means that old code will accept the new rules

this is wrong, by any definition

you've agreed it's wrong. you're not helping anyone in particular, least of all the OP

[gmaxwell]

The definition of a soft-fork is pretty widely spread and firmly established, so I don't think it can really be changed.

Also the safer subset that Bitcoin does in practice is much harder to define precisely.  For one, many softforks have been unsafe with respect to sufficiently old code that no one is using anymore-- particularly, before 'standardness' was introduced, nodes would relay and mine arbitrary scripts including explicit forward extension stuff,  ... a long time ago nodes would also mine transactions with arbitrary version numbers, etc.

Besides, the term softfork stinks because there isn't even necessarily any fork at all. ... particularly in the case of the safer ones.

[domob]

ok, instead of fanning the argument out into things everyone agrees with, let's stick to the point I disagreed with:

transactions that would be accepted by old nodes may be invalid according to the new rules.

that didn't happen with segwit, it was implemented using the so-called "safe" soft fork design

you implied it could, and that was wrong

This post of yours is what I am referring to (here quoted in full).  In here, you imply that the quoted statement here (not the other about "new rules" which I'd say is still not wrong, just formulated in a misleading way) is wrong, or what you "disagreed with".  But if you now agree this statement is correct and you no longer disagree with it, then all is clear to me.  (And if you already agreed to it earlier, I must have overlooked that in your previous post - sorry for that.)

[Carlton Banks]

I now understand better the reason for having a high (95%) signalling threshold for entering soft fork activation periods; hard forks can be provoked with transactions that are crafted to break the rules introduced by the soft fork. So I guess something small was achieved out of all of this to-ing and fro-ing

[gmaxwell]

I now understand better the reason for having a high (95%) signalling threshold for entering soft fork activation periods; hard forks can be provoked with transactions that are crafted to break the rules introduced by the soft fork. So I guess something small was achieved out of all of this to-ing and fro-ing

Yes, well kinda. For most softforks no remotely recent mining software will provoke a hardfork.

I believe that someone to erroneously mine a segwit invalid block with unmodified I think would have required they use software from 2012. ... cleanstack was added later than that, but before cleanstack there were rules to only allow standard transactions.

But they could also do it intentionally and the non-upgraded hashrate would follow them. Having a high hashrate guarantees that any such fork will be short... which protects non-upgraded users and spv wallets from seeing a big reorg. If there won't be a big reorg then there isn't much incentive to do it intentionally.

[Carlton Banks]

so here's an attack (with a significant social engineering angle) I thought up that large miners/pools could attempt. Could it work?


Right now, anyone-can-pay can be used as a wrapper around a transaction that subscribes to any fantasy soft-fork. Regular people with no hashpower cannot try to popularize a fork they like by getting such transactions confirmed, as the network will ignore the fantasy rules, and simply allow whoever picks the money up of the ground to do so. Someone using that strategy is no more than a cough in a stadium full of voices.

But big miners/pools could start a public campaign for their preferred soft-fork, and launch a Bitcoin client that can perform the transactions that observe the new rules. Then they can mine these transactions themselves out of band, to ensure users that other miners/pools won't allow picking the money up off the sidewalk. And of course miners/pools can artificially inflate the numbers of people using their minority soft fork's rules, simply by stuffing blocks with their own transactions using the same rules.

Maybe it's a high risk strategy, but the bigger the miner/pool, the more likely they could pressurize the overall marketplace to fully endorse their softfork, thus taking (some) control of protocol development. How much control they could attain depends alot on the nature of the fork.... not sure, what do we all think about this? willing to take this to a new thread also

[jackg]

This was actually done recently from a vulnerability found on the bitcoin network (wasn't social engineering though) the core devs asked a load of miners to update their software but I think bitmain still held a majority share at the time (40-55%) but if the 95% compliance is necessary this won't be enough and I assume a lot of miners reviewed the code before updating it (or we can at least hope they did). You'd probably also have to make it look like you knew what you were talking about too and probably reference the code adjustments and how they can add and compile them from source.


I reported this thread from your second post in the hope it could be split into a new topic as it'll be an interesting standalone thread.

[Carlton Banks]

This was actually done recently from a vulnerability found on the bitcoin network (wasn't social engineering though) the core devs asked a load of miners to update their software but I think bitmain still held a majority share at the time (40-55%) but if the 95% compliance is necessary this won't be enough and I assume a lot of miners reviewed the code before updating it (or we can at least hope they did).

which vulnerability was that? I don't recall anything that needed a soft fork (or where the miners were proposing changes instead of the devs)

[jackg]

https://bitcointalk.org/index.php?topic=5034070.msg45966212#msg45966212

It was this one.

Stored funds are not at risk, and never were at risk. Even if the bug had been exploited to its full extent, the theoretical damage to stored funds would have been rolled back, exactly as it was in the value overflow incident.

That makes me feel like it was a hard fork but I haven't read the thread since September 2018 so... One things certain: I've been here too long 🤣.

[Carlton Banks]

i've definitely been here too long

no, that's not really what I'm thinking of. I mean the whole sequence of events the other way round


1. big miner/pool thinks of a forking change to consensus
2. miner/pool implements it as a softfork
3. miner/pool releases new Bitcoin client with softfork included
4. miner/pool accepts the supa-dupa new tx's over their website (or maybe bake that into the client)
5. miner/pool begins including transactions in it's blocks, because no other miners are
6. miner/pool can inflate success of the fork, by stuffing blocks with tx's that they made themselves
7. miner/pool hopes everyone will follow the fork, bouyed by the (faked) success

I guess a reason against the strategy would be the same as was discussed upthread: if another miner/pool wanted to take the opportunity to add to the uncertainty of this situation, they could look for a way to force a hard fork right at a critical point of the soft fork's acceptance (i.e. ~50%). Although that would also be a risk in of itself, as there is no direct way to know whether those signalling support are actually running the putsch-client (does this actually imply a positive incentive for miners not to run an upgraded client while signalling readiness, until the readiness signalling level is very high?)