Bitcoin Forum
December 09, 2016, 02:25:00 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 [8] 9 10 11 12 13 »  All
  Print  
Author Topic: Miners that refuse to include transactions are becoming a problem  (Read 15316 times)
guruvan
Hero Member
*****
Offline Offline

Activity: 518

ShastaFarEye Prospectors mazaclub & mazacha.in


View Profile WWW
March 24, 2012, 11:54:06 PM
 #141

And to that end, this is what I dug up researching this today:

Right now the transaction fee address is left "blank" and the block generator fills it out.
Now you would fill it in with the address of the person you are asking to build the block.  
If you're only going to have one person work on building the block, that could take days.  Oh, do you mean send a different variation to each node with the tx fee written to them?

The way it is now, it's whoever builds this gets it.

If we needed to, we could have a BitTorrent-esque tit-for-tat for transaction broadcast.  Relay paying transactions to me, or I won't relay them to you.  It probably won't be an actual problem though.  It only takes one node relaying like it should to cancel out 7 others greedily not relaying.

Some food for thought.

Mine at the Maza Club! with ShastaFarEye Prospectors! Mazacoin PPS & P2pool mining, and more services coming soon!
Maza Means Money! Check yours at the mazacha.in!

Please contact me  on my  OTC registered GPG (A54E87F2) Key's email address or guruvan@shastafareye.net  and encrypt all correspondence.
1481250300
Hero Member
*
Offline Offline

Posts: 1481250300

View Profile Personal Message (Offline)

Ignore
1481250300
Reply with quote  #2

1481250300
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481250300
Hero Member
*
Offline Offline

Posts: 1481250300

View Profile Personal Message (Offline)

Ignore
1481250300
Reply with quote  #2

1481250300
Report to moderator
1481250300
Hero Member
*
Offline Offline

Posts: 1481250300

View Profile Personal Message (Offline)

Ignore
1481250300
Reply with quote  #2

1481250300
Report to moderator
theymos
Administrator
Legendary
*
expert
Offline Offline

Activity: 2506


View Profile
March 25, 2012, 01:25:56 AM
 #142

Then Haplo's clarification is correct?

I think so.

Quote from: westkybitcoins
So it means instead of forcing miners to include transactions, they would instead be forced to access recent blocks and extract data from them.

What's still a little confusing about that on a conceptual level is this: mining ALREADY requires access to recent blockchain data, in the form of a hash of the prior block. Even if it required new code, couldn't the method already being used to distribute that prior-block-hash to all the nodes on a botnet be copied/modified to also distribute the newly-required "proof of verification?"


It's not just recent blocks: they'd need to be able to access random unspent transactions in the chain.

The proof of verification could be distributed, but then at least you know that someone is doing the proper verification and miners aren't blindly building onto invalid blocks.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
March 25, 2012, 03:06:01 AM
 #143

Why would a rational miner want to allow free tx forever?

To encourage users to use the network.

A network's value increases as its userbase increases (and vice versa).


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
March 25, 2012, 03:06:55 AM
 #144

reread what you quoted.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
March 25, 2012, 03:11:53 AM
 #145

Personally, I favor a drastic rule such as

* Build a list of "likely to be in next block" transactions, from memory pool
* Do not relay blocks unless they contain at least 50% of the transactions in the "likely to be" list

This is "drastic" in my estimation because such a rule has notable downsides,

* Increases possibility of short term fork
* Creates de facto requirement that at least 50% of each block are standard transactions (as defined by isStandard)
* and some other minor fallout

However, it is a strong rule that does address the issue at hand, while permitting valid, no-tx-activity zero-transaction blocks to exist.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
March 25, 2012, 03:55:04 AM
 #146

Personally, I favor a drastic rule such as

* Build a list of "likely to be in next block" transactions, from memory pool
* Do not relay blocks unless they contain at least 50% of the transactions in the "likely to be" list

This is "drastic" in my estimation because such a rule has notable downsides,

* Increases possibility of short term fork
* Creates de facto requirement that at least 50% of each block are standard transactions (as defined by isStandard)
* and some other minor fallout

However, it is a strong rule that does address the issue at hand, while permitting valid, no-tx-activity zero-transaction blocks to exist.



So miners are required to include tx that don't meet their fee requirements?  If they don't they risk having the valid blocks not forwarded?
Haplo
Full Member
***
Offline Offline

Activity: 168



View Profile
March 25, 2012, 04:11:10 AM
 #147

Personally, I favor a drastic rule such as

* Build a list of "likely to be in next block" transactions, from memory pool
* Do not relay blocks unless they contain at least 50% of the transactions in the "likely to be" list

This is "drastic" in my estimation because such a rule has notable downsides,

* Increases possibility of short term fork
* Creates de facto requirement that at least 50% of each block are standard transactions (as defined by isStandard)
* and some other minor fallout

However, it is a strong rule that does address the issue at hand, while permitting valid, no-tx-activity zero-transaction blocks to exist.

That's similar to what Gavin and Revalin proposed, the downsides of which have been discussed at length Tongue.

Gmaxwell's proposal should achieve the same effect without the downsides of a threshold-based approach. The main advantage of an approach like yours is that if the bot owner finds a way to include verification without including current tx, he still gets shut out for non-cooperation.

I think if you were to do it that way, a much lower threshold would suffice. AFAIK our mystery miner only includes a single token tx in his blocks, or none. With ~80tx average per block, a 5% threshold would neatly wipe them out. At that rate, miners have little to complain about, and the likelihood of a fork is lower. At 50%, a pool might end up having an otherwise valid block excluded if they happen to charge on the high end of fees. At 5%, a pool charging enough to get excluded would not be providing anything extra to its users (if any).

I'm So Meta, Even This Acronym
Nim
Member
**
Offline Offline

Activity: 67


View Profile
March 25, 2012, 04:28:44 AM
 #148


Going forward (once I have modified bitcoind and tested it) I will be excluding ALL transaction with < 0.01 BTC in fees.  I will also share the patch as open source and encourage other miners to adopt similar minimum fee rules (although each miner could choose a different amount).
What would happen if everyone did this and I send out a transaction tomorrow with no tithe to the system. Where does that transaction go? How would the user have any idea what has happened? How would they know if it still floating around out there, maybe waiting on a generous miner who has a long pro bono publico wait list, and that they don't need to resend it?

What if the fee requiring miners are in the minority but each of the 8 node connections that my client has just happens to have this fee rule? Again, does my transaction get lost in the nether? How would I get my transaction out to the miners who are willing to include my transaction? Would the nodes say "nope, fees too low on this transaction, into the round file it goes..." or would the nodes say "nope, fees too low on this transaction but maybe Bob down the street would be willing to help you out, I'll pass it on"?
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
March 25, 2012, 04:37:40 AM
 #149

What would happen if everyone did this and I send out a transaction tomorrow with no tithe to the system. Where does that transaction go? How would the user have any idea what has happened? How would they know if it still floating around out there, maybe waiting on a generous miner who has a long pro bono publico wait list, and that they don't need to resend it?

The modification I was proposing wouldn't affect relaying so tx would be relayed to every node even my node would continue to relay it.The tx is valid so rejecting it is inappropriate I just don't want to mine it.

As far are mining it into a block.  If every single miner on the planet (and all new miners coming online) had a min fee rule with no exceptions higher than your fee then it would never enter a block.  However that is obviously unrealistic.  If 10% of miners had such a rule then 90% of network would still include it in the next block so your expected confirmation time would simply be 11 minutes vs 10 minutes.  If say 99% of network excluded it from the next block (due to insufficient fee) then your expected confirmation time would be 16 hours.  Obviously both of those numbers are expected confirmation time, the actual time would be subject to variance just like any confirmaiton time is.

Quote
What if the fee requiring miners are in the minority but each of the 8 node connections that my client has just happens to have this fee rule? Again, does my transaction get lost in the nether? How would I get my transaction out to the miners who are willing to include my transaction? Would the nodes say "nope, fees too low on this transaction, into the round file it goes..." or would the nodes say "nope, fees too low on this transaction but maybe Bob down the street would be willing to help you out, I'll pass it on"?

The later.  My proposed change would only affect what tx go into the next block and only for those miners/pools which implement the patch.  Any valid tx would continued to be relayed.
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
March 25, 2012, 04:41:09 AM
 #150

So miners are required to include tx that don't meet their fee requirements?  If they don't they risk having the valid blocks not forwarded?

Yes, you will be orphaned if you set a "fee > 100BTC no exceptions" include policy.  It is not reasonable to demand that the network have no right to question your policy if you institute a bad one: the whole point is to stop people from excluding all txes.  We're trying to find a minimal method that won't affect any reasonable include policy, but it does create a requirement that you might have to do something.  A relay policy with no actual mandate is pointless.  As I said:  Checks and balances.

You're becoming a broken record.  Your point has been heard and is being taken into consideration.

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
March 25, 2012, 04:50:24 AM
 #151

So miners are required to include tx that don't meet their fee requirements?  If they don't they risk having the valid blocks not forwarded?

Yes, you will be orphaned if you set a "fee > 100BTC no exceptions" include policy.  It is not reasonable to demand that the network have no right to question your policy if you institute a bad one: the whole point is to stop people from excluding all txes.  We're trying to find a minimal method that won't affect any reasonable include policy, but it does create a requirement that you might have to do something.  A relay policy with no actual mandate is pointless.  As I said:  Checks and balances.

You're becoming a broken record.  Your point has been heard and is being taken into consideration.

Please don't use strawman to support your argument.

What if fee policy was 0.01 BTC?  Less than 10% of txs in last 20 blocks had a fee of 0.01 or higher.  Hell less than 1/3rd have a fee of any kind.    Thus a requirement to include 50% of txs mandates including free txs also.  Any fee policy no matter how low (even min 1 satoshi) would face exclusion.

So at least be honest with the ramifications. 
Revalin
Hero Member
*****
Offline Offline

Activity: 728


165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g


View Profile
March 25, 2012, 05:04:12 AM
 #152

It's not a strawman.  I'm just emphasizing that some minimum standard is reasonable.  I'm sure we can come up with a method that will work.

Rather than having a minimum bar, why not make it a sliding scale?  For example:  you must include at least 20% of the top-10-percentile-fees (either since the last block or sliding-window) transactions?  As always, -5 or so to accommodate latency etc.  That would give you a lot of latitude to cherry-pick.  In many circumstances (only a couple transactions with high fees available) you would even be able to mine a null block and have it accepted.

Edit:  Or rather than allowing a null block, we could mandate that every block include at least one transaction if there are at least 5 outstanding.  Is it an undue imposition to include one free transaction if there are absolutely none that meet your include policy?  That provides a very strong incentive to include fees, and completely solves the null-miner problem:  Mystery can mine a null block if he wants to - but only in the few seconds after the a previous block that cleared out all waiting transactions, during which a null block is reasonable by any standard.

      War is God's way of teaching Americans geography.  --Ambrose Bierce
Bitcoin is the Devil's way of teaching geeks economics.  --Revalin 165YUuQUWhBz3d27iXKxRiazQnjEtJNG9g
JoelKatz
Legendary
*
Offline Offline

Activity: 1386


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 25, 2012, 05:55:19 AM
 #153

Rather than having a minimum bar, why not make it a sliding scale?  For example:  you must include at least 20% of the top-10-percentile-fees (either since the last block or sliding-window) transactions?  As always, -5 or so to accommodate latency etc.  That would give you a lot of latitude to cherry-pick.  In many circumstances (only a couple transactions with high fees available) you would even be able to mine a null block and have it accepted.
The goal is not to force any particular policy but just to avoid miners being lazy and not gathering any transactions at all. So long as they can't take the shortcut of not including any transactions at all, their self-interest will lead them to include fee-paying transactions.

So the rule could be this simple and this modest: If there are at least four transactions that are not free and have at least the standard fee that were not included in the previous block even though they could have been, discourage any block with just a coinbase transaction.

So you are never punished for not including a free transaction. And you are never punished for not getting a new work unit when you didn't have to anyway because a new block was found. But this will solve the problem of paid transactions backing up if multiple blocks are found by miners who refuse to include any transactions.

This is, IMO, the least onerous rule that would solve the problem.

I am an employee of Ripple.
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
theymos
Administrator
Legendary
*
expert
Offline Offline

Activity: 2506


View Profile
March 25, 2012, 06:18:56 AM
 #154

So the rule could be this simple and this modest: If there are at least four transactions that are not free and have at least the standard fee that were not included in the previous block even though they could have been, discourage any block with just a coinbase transaction.

You also need to discourage the block if it has only transactions you don't know about, since otherwise the miner could make up transactions.

I'm not bothered by this rule from an economic perspective, but I'm not confident in the effectiveness of discouragement (due to bad incentives), and I'm uneasy about relying on transaction propagation in any way, since propagation is imperfect and will get worse with time. I prefer gmaxwell's proposal because it only relies on the block chain.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
Killdozer
Full Member
***
Offline Offline

Activity: 204



View Profile
March 25, 2012, 11:25:45 AM
 #155

Quote
I think if you were to do it that way, a much lower threshold would suffice. AFAIK our mystery miner only includes a single token tx in his blocks, or none. With ~80tx average per block, a 5% threshold would neatly wipe them out. At that rate, miners have little to complain about, and the likelihood of a fork is lower. At 50%, a pool might end up having an otherwise valid block excluded if they happen to charge on the high end of fees. At 5%, a pool charging enough to get excluded would not be providing anything extra to its users (if any).
Just wanted to say that this makes a lot of sense to me.
It still probably needs some detals, like only take into consideration transactions which are older than 1 minute or so (so that they have very high probability of having been globally-relayed) and perhaps not enforcing this rule if the total number of unconfirmed transactions (that are "supposed" to be in the next block) is too little. (This can happen due to a bug, or, what is more realistic, due to 2 blocks being found very short time apart (which can happen, it's all just probabilities)). So for example if there are only 5 transactions that are supposed to be included, don't enforce the rule, because in that case the outcome of the percentage calculation is biased too much by every single transaction (and thus much more prone to errors and can create unnecessary forks).

Timo Y
Legendary
*
Offline Offline

Activity: 938


bitcoin - the aerogel of money


View Profile
March 25, 2012, 12:44:11 PM
 #156

The market purists on this forum should agree that the market is working exactly as it should.

The block award is for securing the network against a double spending attack, not for confirming transactions. Transaction fees are for confirming transactions.

The null miners are essentially specialized miners that are only in the business of security, and not in the business of security+confirmation like most other miners.

I don't think this is a huge problem. Even if the null miners had 50% of hashing power, average users would hardly notice any difference. Even if they had 90%, bitcoin would still work fine. It would just take longer for transactions to confirm. 

This problem is likely to disappear in future because:

a) The block award will drop
b) As bitcoin becomes more popular, there will be more transactions per block, and thus more fees to be collected
c) FPGAs and ASICs will marginalize the hashing power of botnets.
d) Most day-to-day transactions will be between trusted parties who don't care whether a transaction isn't confirmed yet, and who don't even broadcast their transactions most of the time.

Is it worth adding complexity to the bitcoin protocol, with possible unintended consequences, in order to solve a temporary problem?

GPG ID: FA868D77   bitcoin-otc:forever-d
Haplo
Full Member
***
Offline Offline

Activity: 168



View Profile
March 25, 2012, 08:34:27 PM
 #157

The market purists on this forum should agree that the market is working exactly as it should.

The block award is for securing the network against a double spending attack, not for confirming transactions. Transaction fees are for confirming transactions.

The null miners are essentially specialized miners that are only in the business of security, and not in the business of security+confirmation like most other miners.

I don't think this is a huge problem. Even if the null miners had 50% of hashing power, average users would hardly notice any difference. Even if they had 90%, bitcoin would still work fine. It would just take longer for transactions to confirm. 

This problem is likely to disappear in future because:

a) The block award will drop
b) As bitcoin becomes more popular, there will be more transactions per block, and thus more fees to be collected
c) FPGAs and ASICs will marginalize the hashing power of botnets.
d) Most day-to-day transactions will be between trusted parties who don't care whether a transaction isn't confirmed yet, and who don't even broadcast their transactions most of the time.

Is it worth adding complexity to the bitcoin protocol, with possible unintended consequences, in order to solve a temporary problem?

Several reasons.

1: The block reward may drop in half come 2013, however if the value of 1BTC doubles, then nothing has been lost. That leaves him fully secure to continue ignoring tx until at least 2017.

2: He isn't actually contributing to security, since he's only generating blocks off the previous hash without validating anything. That means if an attacker starts throwing down invalid blocks, our friendly botnet might blindly build right on top of them and give them a nice helping hand.


Also, it depends on which method is used. Gmaxwell's proposal would not affect legitimate miners in any way. Any proposal which excludes blocks that are "too empty" would have unintended consequences, but in the event of an attempted 51% empty-block DoS attack, the ability to screen empty or cheap blocks would severely limit the damage they could do.

We've been over this like 50 times =\.

I'm So Meta, Even This Acronym
JoelKatz
Legendary
*
Offline Offline

Activity: 1386


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 25, 2012, 09:15:42 PM
 #158

2: He isn't actually contributing to security, since he's only generating blocks off the previous hash without validating anything. That means if an attacker starts throwing down invalid blocks, our friendly botnet might blindly build right on top of them and give them a nice helping hand.
Not true. All the building on top of an invalid block can't hide or change the fact that it's invalid and so would do no harm whatsoever. It would simply be wasted effort.

These botnets do increase the amount of hashing power an attacker would need to launch something like a 51% attack. They do increase the security of the network. And that's what the block reward is for. If the system is broken, it's because the transaction fee and volume aren't sufficient to justify the expense of including transactions in a block.

I wonder if we can detect these botnet nodes somehow, perhaps by the particular messages they send or don't send, and send them *invalid* blocks. It won't harm normal clients since they'll just ignore them. But the botnet nodes, lacking the block chain, will work on the wrong blocks more often than the right ones. This may do more harm than good in the long term though.

I am an employee of Ripple.
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
benjamindees
Legendary
*
Offline Offline

Activity: 1288


View Profile
March 26, 2012, 03:18:02 AM
 #159

Quote from: jgarzik
* Build a list of "likely to be in next block" transactions, from memory pool

you must include at least 20% of the top-10-percentile-fees (either since the last block or sliding-window) transactions

So what is the downside to this again?

This seems to solve the problem by setting a reasonable limit on miners' ability to extort the network for fees.  We really should consider a somewhat drastic solution.  Even though the mystery miner is likely a botnet, within a year or so we could easily see this same scenario play out with a large FPGA cluster that doesn't want to include transactions for whatever reason.  It isn't a problem that is going to just go away, or be solved any time soon by falling block rewards.  We're already subsidizing miners with 30% inflation;  it might as well go to those that at least process a few transactions.

Civil Liberty Through Complex Mathematics
Jon
Donator
Member
*
Offline Offline

Activity: 98


No Gods; No Masters; Only You


View Profile
March 26, 2012, 03:19:24 AM
 #160

They aren't becoming a problem. They are simply making transaction processing more valuable. When the incentive comes for them to process transactions, will be the day a higher fee is set across the board.

I see no issue. If the problem becomes bad enough, people will talk with their pocketbook.

Although, you could always coerce the miners into giving you free transactions. As for the consequences to Bitcoin as a whole in such a case, I'll leave that to you to figure out.

The Communists say, equal labour entitles man to equal enjoyment. No, equal labour does not entitle you to it, but equal enjoyment alone entitles you to equal enjoyment. Enjoy, then you are entitled to enjoyment. But, if you have laboured and let the enjoyment be taken from you, then – ‘it serves you right.’ If you take the enjoyment, it is your right.
Pages: « 1 2 3 4 5 6 7 [8] 9 10 11 12 13 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!