SPV Mining and how to slow it down ... if you care to ...

[tl121]

Payment systems that rely on transaction confirmation suddenly are in a position where they no longer know if their confirmed transactions are confirmed at all.

Transaction confirmation tends to be rather important ...

If >50% of the network is mining on an "invalid fork" then is that fork "invalid"?

It is never certain whether a bitcoin transaction is confirmed or not.  There is always a risk that a transaction can become unconfirmed in a reorganization.  It is up to payment systems to deal with this intrinsic behavior of bitcoin.  The presence of occasional SPV mining might increase the number of blockchain reorganizations, making it desirable to use a more conservative number of confirmations before releasing goods.  But SPV mining also reduces the orphan rate, so the reduced orphan rate lowers the risks associated with small numbers of confirmations.  Note that payment processors can always run a full node and make sure that any blocks in the chain up to their payouts are valid.  If they don't run a full node, then they have to trust any full nodes that their client uses.

If there is a significant risk, IMO it comes where a payment system uses an SPV client to access a "full" node that isn't validating the entire chain.  I'm not sure this extra risk is particularly bad, because it increases the incentives for people to run a full node.

Another way to look at this is that the miners are primarily performing an ordering and publishing process.  Miner validation doesn't really provide that much security when there are hundreds of times more full nodes that validate blocks for their users than nodes that mine a majority of the hash power.  (However, mining validation does serve a spam prevention purpose, otherwise double spends would pollute the block chain with useless data for transactions that will be rejected by every honest full node.)

The case of differences of opinion as to block validity is a separate issue from SPV mining.  It ultimately comes down to what is, or is not, "bitcoin" and how disputes between Alice and Bob get settled when Alice she claims she paid Bob and Bob rejected the payment because their definitions for "bitcoin" differed.

[1715246565]

1715246565

[1715246565]

1715246565

[tl121]

Well Prop by design is flawed - so people don't use it any more.
PPS by design is flawed - but pools still use it ... ... ...

I wonder what people think about block withholding attacks,  particularly the advisability, morality and/or legality thereof.

[kano]

The actual issue is the rules of Bitcoin.
The basic issue is that not everyone everywhere follows the rules that define Bitcoin.

It doesn't really matter what opinions exist about what is and isn't Bitcoin, the issue is simply the rules that define Bitcoin.

I can go make some altcoin scam and have rules that allow double spends and let people steal coins etc ... but that, of course, isn't Bitcoin.

The Bitcoin rules are defined as to what the block header looks like BUT also includes the rules that define the validity of transactions that the block can confirm.
It isn't just the first, it is BOTH

So when a pool is ignoring the second set of rules, by building on block headers that aren't guaranteed to be valid Bitcoin block headers, they are not Bitcoin.
Yes they may produce valid Bitcoin blocks most of the time, but no they are not guaranteed to produce valid Bitcoin blocks all the time, because they are ignoring some of the rules that define what Bitcoin is.

I guess one solution to this would be to fork off the SPV miners into some SPVCoin altcoin and be done with them.

However this is also problematic in adding NEW rules to Bitcoin to ignore data from those who are breaking the Bitcoin rules.

Basically, they need to own up to the fact that they are NOT following the Bitcoin rules and either piss off and make their own SPVScamCoin or fix their mining and ensure it follows the Bitcoin rules.

They seem to want the value of Bitcoin rules applied to their non-Bitcoin rule mining.

In my opinion that's a scam and no one who wants to be a part of Bitcoin should be instead supporting altcoin pools that pretend to be a part of it.

[cypherdoc]

hey, you guys should take down your BIP100 block flag support that you're broadcasting on the blockexplorers. @jgarzik has scrapped it: https://twitter.com/cypherdoc2/status/677184373699969024

[p3yot33at3r]

I guess one solution to this would be to fork off the SPV miners into some SPVCoin altcoin and be done with them.

Probably the best solution I've heard so far. See how quick they change their ways once they realize they will be kicked of the main chain.....

[kano]

hey, you guys should take down your BIP100 block flag support that you're broadcasting on the blockexplorers. @jgarzik has scrapped it: https://twitter.com/cypherdoc2/status/677184373699969024

Though I have no idea what BIP100 has to do with SPV mining ...

Weird I though it was called Bitcoin, not GarzikCoin ................

[sloopy]

Kano, that was a great way way to put it. They are not following the rules of bitcoin when they do not properly validate.

I agree with everyone who commented about people mining at any pool which uses SPV mining, but I also see the Bitcoin publicity crew kissing their ass, like the bitcoin meetup in HK. It sets a very bad example for miners, especially new ones. I wonder how many miners even think of another pool when they receive their miner and set it for antpool. They see another large pool and try f2pool. It takes a while for many new people who would care about SPV, zero transactions, etc to find a pool they agree with.

It goes to show how bad choices for the entire arena are made in plain sight and most do not care, but some do, and many others are beginning to pay attention.

I hear the people mentioning the block withholding attacks, but I feel like that would put me on the F2pool and antpool level and I am not a scummy POS no matter what one particular forum member who is involved with macbook-air tells people I am, heh.

I am also not saying someone else is a "bad" person for performing a block withholding attack. Everyone must decide what their actions are going to be.

To me, it is extremely unique we need to have a conversation regarding the technical specifics of why they choose to continue ignoring the proper way to validate. I understand they want to gain an advantage, but when they are already doing scummy things like dead-pool mining without permission, SPV, zero transaction, doesn't that pretty much size it up for anyone with common sense?
Maybe it is the pure enjoyment of discussing technical aspects. Or, an attempt to define a combative method.

If things were handled correctly the core devs would realize it is important and have already put something in code.
They don't care, they are too busy with their block size debate and I haven't heard any progress on that front, at all. I know there were several proposals, and a whole bunch of talk with XT, BIP 10x, etc but what about a vocalized decision? "Hey, we aren't doing jack about blocksize because we are working on this right now." Grow a set and step up. I became bored reading the mailing list.
As Gavin loves to mention, it is indeed a bunch of "bike-shedding". Don't get me wrong, I am not saying Gavin and Hearn had it right either. At least they made a valid attempt.
Why aren't CK and Kano core Devs? It isn't like there aren't a couple of real losers in that stack who should be replaced. Starting with Luke-JR.
By the way cypherdoc thanks for pointing that irc convo with him out, it is usually the biggest preachers who are also the biggest hypocrites. The whole "Preacher's Daughter is a wild woman" thing is not a stereotype because it isn't true. Personal experience stands strong on that point. Good or bad I loved going to church as a teenager for that reason more than faith.

I have been sending some messages with few responses to people who I think can have the most impact on the problem, without involving the real media. Sometimes it can take a while for people to respond, it has happened before and worked out well. We do not need the bad press which would surely come from involving big media. The story is too big. The media will love writing about the potential Bitcoin Blowup of China versus the rest of the world, and that hurts everyone. I think a big help can be the equivalent of writing or calling your senator here in the US. It takes a mountain of people doing it, or someone with dirt on the right person, but it can help a situation. I've seen it with Veteran Affairs and my father, but it only helped for about a year and the problems came back.

I hate the idea of giving up liberty for protection, and I feel the same way about Bitcoin. I don't want a "big brother" governing every aspect just to make sure a dozen people stop being so freakin greedy they ruin everything for everyone.
At some point though something has to give. The rate of growth of just Antpool and F2pool is monstrous compared to every other pool because of the momentum. Unless someone does some heavy advertising about it, the concept simply does not reach enough people who care in our "little" bitcoin world. Even the youtube videos of bitcoin meetups never mention any names, it is just the facts mam, even if you get to hear those. Most average miners do not understand it. Which is why I push for simple talk but enjoy the technical.

What is enough for the "big" individual miners?
When they see it impact their pocket.

Maybe one of the spreadsheet math geniuses here could put some numbers together showing what happens when 95% of the network is  SPV mining and a fork happens which isn't caught for 24 hours, or what if that fork happened and they decided to stay on that fork?
Bitcoin would drop like a rock.
Or something showing projections on the growth over the next two years if nothing changed in the core code and growth continues at the current trends. Even factor in a media frenzy about the issues we are discussing.
Surely people who recently invested 100k in S7s would want to do something to have an impact if it was presented by someone viewed as more neutral, maybe a non-miner, non pool operator, just someone holding coin since 2011 who doesn't want to see their investment end up in shambles.

Of course, I mentioned this before. I believe the people holding hundreds of thousands of dollars worth of bitcoin or more have already setup escape plans. Besides the fact that even if they lost it all today they still are far from ever being negative. They are already sitting pretty well.
However we do have some people and companies with recent incursions, investments and shareholders who would want answers, ala overstock, the twins, but that is too close to mainstream media, which invites oversight. Governments could even collude to control bitcoin to "help" the system, and that is something we do not need regardless of it being possible. The effort alone is bad publicity.

No, I think we need to start with the people who represent bitcoin at the meetups, the conferences, and especially the ones in Asia.
It is so odd that they openly debate the impact of the great firewall and the people behind it talk about what a disadvantage it causes yet they own the network. To me, again, common sense, and the statement is laughable. There are so many ways they are currently getting around the GFW without including SPV and ZERO Transaction blocks. Maybe this is an area I do not understand but the numbers speak loudly to me.

My main reason for this post is too encourage others to use the brilliant minds I have been witness too so many times here on this forum. I've seen so many creative people here, and I know the common sense, ability to create unbelievable software, the creativity which goes in miner modifications, and simply the thoughts of so many people about any random topic far outweigh those same things I see from people on a daily basis. IT isn't that I am not around some brilliant people, but they do not have the same intellectual need for an outlet of those attributes. Not to mention this is bitcoin, and we love it for it, and what it has already shown is possible. The number of doors one can enter are endless.

I told someone the other day in a PM that they are wasting their abilities. Oh, he has his hands full. He is a fairly large miner, he writes his own custom software, he works a day job, and sleeps a few hours like me. He also does charity work, has a wife and kids, builds other projects all the time, and on, and on, but his skills are so varied he could be doing so much more to actually help Bitcoin take a next step and it be profitable for him. I have no reason to think there is much he couldn't do, but he is not interested in other projects outside of his safe bubble.

I understand that. I spent about twenty years living where I am still spending 10, 12 - 18 or more hours a day, but I did start going outside that environment through my work and doing other things in the community which made some friends and some enemies. I enjoy knowing a small group of people can have a huge positive impact on many without spending a dime.

One last thing I will touch on is people should support the people doing good things whenever possible and you do not have to do it openly you can be covert. If you aren't well off and you are struggling just to make ends meet use that creative genius to do something different. Put a project on hold and make something. You don't have to spend a thousand or even a hundred bucks to show appreciation.  Make something with a soldering iron out of scrap wire and wood. Design a circuit which does something interesting even if it is a play on something else. Research someone and code something you can use to pay someone a compliment.
I have a very small group of guys who have been through some crazy things together where we send each other something out of the blue we know that person would appreciate. Be it a local item they cannot get where they are, a pocketknife, a sign, a pin up picture, Venison, or something they have never tried before, etc. It not only encourages that person more, it damn sure is one hell of a thank you and unbelievably motivating. I am not saying to expect anything, simply to give something. Don't buy someone a gift card for Christmas unless you know that will thrill them, or get the gift card and a little something you had creative input with. That whole pay it forward thing is an amazing concept and has sure made some days that were going very bad given the perspective that those bad things didn't even matter. Yes I practice what I preach, and I am going to keep doing what I can to change SPV mining and mining zero transactions with the strengths I have while honing my weaknesses until they are strengths.

You don't have to spend a bunch of money, but you do have to spend some time.
That goes for selectively paying it forward, and changing the Bitcoin issues we face.

I am very interested in being part of a group of people who are serious about causing a change in bitcoin. No judgement, No Dictator, a common goal of causing a change to SPV mining and zero transactions, using creativity and other means as necessary. I am very open-minded to other goals as well, but I feel these are two extremely high priorities.

Have a great day, evening, morning, or wherever you are, I hope it goes well.  

[cypherdoc]

hey, you guys should take down your BIP100 block flag support that you're broadcasting on the blockexplorers. @jgarzik has scrapped it: https://twitter.com/cypherdoc2/status/677184373699969024

Though I have no idea what BIP100 has to do with SPV mining ...

Weird I though it was called Bitcoin, not GarzikCoin ................

thought you might be flagging 100 support in your blocks.  i know ck is.  that won't be necessary now.

[kano]

hey, you guys should take down your BIP100 block flag support that you're broadcasting on the blockexplorers. @jgarzik has scrapped it: https://twitter.com/cypherdoc2/status/677184373699969024

Though I have no idea what BIP100 has to do with SPV mining ...

Weird I though it was called Bitcoin, not GarzikCoin ................

thought you might be flagging 100 support in your blocks.  i know ck is.  that won't be necessary now.

I want BIP100 - I'll be leaving it there.

[cypherdoc]

hey, you guys should take down your BIP100 block flag support that you're broadcasting on the blockexplorers. @jgarzik has scrapped it: https://twitter.com/cypherdoc2/status/677184373699969024

Though I have no idea what BIP100 has to do with SPV mining ...

Weird I though it was called Bitcoin, not GarzikCoin ................

thought you might be flagging 100 support in your blocks.  i know ck is.  that won't be necessary now.

I want BIP100 - I'll be leaving it there.

why?  when one pool of 20% or more, like f2pool, can veto any block size consensus?

[tl121]

The actual issue is the rules of Bitcoin.
The basic issue is that not everyone everywhere follows the rules that define Bitcoin.

It doesn't really matter what opinions exist about what is and isn't Bitcoin, the issue is simply the rules that define Bitcoin.

I can go make some altcoin scam and have rules that allow double spends and let people steal coins etc ... but that, of course, isn't Bitcoin.

The Bitcoin rules are defined as to what the block header looks like BUT also includes the rules that define the validity of transactions that the block can confirm.
It isn't just the first, it is BOTH

So when a pool is ignoring the second set of rules, by building on block headers that aren't guaranteed to be valid Bitcoin block headers, they are not Bitcoin.
Yes they may produce valid Bitcoin blocks most of the time, but no they are not guaranteed to produce valid Bitcoin blocks all the time, because they are ignoring some of the rules that define what Bitcoin is.

I guess one solution to this would be to fork off the SPV miners into some SPVCoin altcoin and be done with them.

However this is also problematic in adding NEW rules to Bitcoin to ignore data from those who are breaking the Bitcoin rules.

Basically, they need to own up to the fact that they are NOT following the Bitcoin rules and either piss off and make their own SPVScamCoin or fix their mining and ensure it follows the Bitcoin rules.

They seem to want the value of Bitcoin rules applied to their non-Bitcoin rule mining.

In my opinion that's a scam and no one who wants to be a part of Bitcoin should be instead supporting altcoin pools that pretend to be a part of it.

The important issues are:  What are the rules of bitcoin?  How are they defined?  How can they change?  Who can change them? How can the relative merits and demerits be weighed when different stake holders have different interests?   How can nodes breaking these rule be caught, coaxed, or coerced into changing their behavior?  These issues are much more important than any particular technical discussions, because without context all of the technical debates have no context and little value.  Defining the problem is much more important than defining the solution.

[kano]

NEW rules would be problematic.

The current rules are clear.
Bitcoin's value is in those rules.
Someone breaking those rules is not mining bitcoin.

They seem to want the value of Bitcoin rules applied to their non-Bitcoin rule mining.

We know they are breaking the rules, but it's rare that they are punished for doing that.
It only happens when their 'rule breaking' results in an invalid block.

What I am already suggesting people do is 'coax' them into not breaking the rules.

[bodaboda]

Who are the pools who keep doing this and empty blocking?

[loshia]

Who are the pools who keep doing this and empty blocking?

If a pool mines empty block it does not mean that pool is SPV mining for sure
But some pool ops have clearly stated something like: yes we are innovative and because of that we are and will continue with SPV minig. Which probably means something like we are big you are small so go and fuck yourselves
And all those pools have tx and non emty blocks also + support from the community. how pathetic...it is about time all of you who mine there for whatever reason to stop doing so if you truly understand and believe that SPV is bad
No ifs and no excuses just fuckem

[AussieHash]

http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012103.html

[kano]

http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012103.html

Just another name for what they need for side chains.
Sneaky aren't they

[Matias]

So would you mind and tell us which other pools you connected to for "reducing their orphans"? If you post this, we can check their orphan rate - otherwise, I call this BS.

When F2Pool "steals" the hash of B(N) from Kano in advance of the ful B(N), it can start mining an empty block B(N+1) on top of it, right away.  The head start gives F2Pool a greater chance to solve B(N+1), which protects B(N) against orphaning.

If Kano did not let F2pool get the hash of his B(N) in advance, some other miner might succeed in sending an alternative B1(N) to F2pool before Kano sends the full B(N). Then F2Pool would mine his B(N+1) on top or B1(N), rather than B(N); and so B(N) would run an increased risk of being orphaned by B1(N).

Therfore, it is in the interest of both pools to ensure that F2pool (and all other major pools) gets the hash as quicky as possible, and starts mining an empty block on top of it, without waiting to get the full B(N). 

It is in the interest of F2pool (but not of Kano) to download the full B(N) and replace his empty B(N+1) by a full one -- but only because of the tx fees.  If F2pool fails to do that, it is actually slightly better for all the other miners, because they will have a chance to get the tx fees that F2pool failed to get.

Now Tx fees are about 1% of total block reward (25Btc+Tx fees). When block halving occurs, they will be about 2%. Doesn't this discourage mining empty B(N+1) and switch to full B(N) immediately when it is available.

[JorgeStolfi]

Hm, I just came back to this old thread to find that some people still don't get it.  (And some of them resort to insults, on top of that.)

There is a social aspect of mining and bitcoin in general which is not controlled by code. Simply, it is the act of good and bad, the intentions of creating a pool structure which promotes a healthy network even when costing them in other ways.  [ ... ]  I believe you will find  this social aspect or better yet, the court of public opinion can have a significant impact on these pools in the coming months and years.

Sorry, that is exactly what I can't agree with.  It seems that miners have come to constitute a connected social group, much as ranchers in a county or plumbers in a city, and believe that they must have a code of ethics and have common policies and decision --- such as ostracizing peers that don't follow those rules.

But that is bad, terribly bad.  Bitcoin's "Fundamental Assumption" is that mining is distributed among thousands of independent miners that act individually, motivated by their individual gain.  That is, selfish and greedy, not "good" or "honest" in any sense.  If miners start acting as a group, sacrificing some selfish gain in pursuit of group profits (or other goals), then it is impossible to give any guarantees at all, not even probabilistic ones.

Today the "good miners guild" may decide that its members should not do hash-stealing and should not mine empty blocks while waiting for the parent to download -- because they decided that such things are bad for bitcoin. (I don't think so, but let's skip that issue for now.)  

But tomorrow they may decide that the minimum fee should be raised to well above the cost, because of suitable excuses --- and all guild members agree to orphan any blocks that contain low-paying transactions.  Or the guild may decide that transactions from or to a certain wallet should not be processed, because that wallet belongs to the "wrong" side in some war, and the guild does not want trouble with the government that is backing the "right" side. Or the guild may decide to increase the block reward via a soft  fork.  Or...

These last things are typical of what the "bankers guilds" -- formal or informal, national and international -- do all the time; and are percisely the sort of thing that the bitcoin protocol was hoped to avoid: miners colluding to act so as to achieve a commong goal, rather than their individual selfish interests.

That is the the only original feature of bitcoin, that justifies its existence: it is a system of incentives that is supposed to achieve the desired group behavior out of the selfish choices of anonymous, uncoordinated, unregulated and unscrupulous miners.  

And it almost worked!  It fialed, not because of trifles like hash-stealing, but because mining, quite unexpectedly, became concentrated in a handful of big pools and a few dozen big farms.  (In a rarely-quoted post, Satoshi estimated that the number of independent miners would grow to "100'000, maybe less", serving "millions" of clients.) Not only the big miners, but even miners in the 5% range, like Slush and KnC, are already too big to make the Fundamental Assumption believable.

Indeed, objectively, the present concentration of mining means that bitcoin is already dead.  It still works as a payment system, but it requires trusting some third parties -- the 5 big miners.  Than, what is the point?  

(What is worse, the concentration happened because of many unavoidable social and economic factors; the advantage of short propagation delay is only one of the weakest.  I see no hope of this problem being fixed -- unless the price crashes to pennies ber BTC, n which case industrial mining woudl not be worthwhile, whatever the difficulty.)  

The court of public opinion has destroyed the reputations of companies, and sometimes those companies are so big they can withstand that assault and with changes can maintain and even grow beyond those terrible choices.

Big pools and miners identify themselves for marketing reasons, but they don't have to.  In the original design, in fact, it was sort of assumed that  miners would be anonymous, coming and going at random times with no control or coordination.  A miner with 75% of the hashpower, who did not care fro marketing, could easily disguise himself as hundreds of anonymous miners with less than 1% each, with unknown geographic location.

Thus, boycotts and public campaign against transgressors of the "guild ethics", besides wrong in principle, are ineffective.  The attacked company can split and mutate into other pools, and no one will know whether they are independent or owned by the same persons (and maybe located under the same roof).

My advice is that miners forget about "the good of bitcoin", as doing so will only bring them frustration and stress; and just behave as Satoshi expected them to behave.

[kano]

Oh you just woke up after falling asleep in your chair for a few weeks and decided to post some random tripe?

Firstly, hmm I wonder why F2Pool has way fewer empty blocks lately?

Secondly, yeah I know time moves on and you forget what was posted before ...
I'll repeat the comment: Read the topic title!
Most of your post clearly missed what the topic title says.

... and lastly something that happened in the meantime ...

Antpool's empty block count capacity was in the order of the size of the mempool build up recently that was delaying transactions.
So, if you ever have a complaint about slow transactions, I can tell you where to go ... talk to Bitmain
Block capacity is still big enough to cover transactions, except that certain pools are still sending out lots of empty blocks that ignore the transactions that are waiting to be confirmed.
My pool's average txn fees gained over the last 11 weeks, 250 blocks, is 101.2% i.e. 1.2% extra due to transactions.
After the halving that would be 2.4% ... yeah takes a little primary grade maths to work out what that means ... though that's probably above the level of an arm chair economist with a point and click web degree

Meanwhile, Bitcoin is continuing on as usual. It's still working.
Yes I know you want Bitcoin to fail, but no it wont.

[Quickseller]

Antpool's empty block count capacity was in the order of the size of the mempool build up recently that was delaying transactions.
So, if you ever have a complaint about slow transactions, I can tell you where to go ... talk to Bitmain

I don't think you understand how mining works. The blocks that are empty because the pools are still processing which transactions were included in the previous block would not have been found period if the pools had waited to validate which transactions were included in the previous block and to change which transactions were to be included in their next block....