Idea: new rules for block validation

[ovidiusoft]

if you did read the next line, you would find out! STUPID FUCK! it would cause a block chain split. where half of the network would mine block upon the 'invalid' block. while the rest of the network would toss it away.
unless you understand how stuff works, please dont go posting about it.

I respectfully ask you to control your language. I don't think we grew up together, so a minimum of respect would help.

I understand what a block chain split is. I asked the rhetorical question because you implied that a split is very unlikely in the current state and very much likely in my proposal. I believe you are wrong, and my proposal doesn't make it much more likely to have splits.

[1714997374]

1714997374

[1714997374]

1714997374

[kokjo]

if you did read the next line, you would find out! STUPID FUCK! it would cause a block chain split. where half of the network would mine block upon the 'invalid' block. while the rest of the network would toss it away.
unless you understand how stuff works, please dont go posting about it.

I respectfully ask you to control your language. I don't think we grew up together, so a minimum of respect would help.

I understand what a block chain split is. I asked the rhetorical question because you implied that a split is very unlikely in the current state and very much likely in my proposal. I believe you are wrong, and my proposal doesn't make it much more likely to have splits.

i will not control my language, when a STUPID FUCK like you comes and suggest a stupid change, without even learning how the stuff works now.

the blockchain will recover from the splits.
but you proposal would make many permanent splits in the blockchain, that it you not be able to recover from. every time someone does have a transaction that is not included in a block yet, it would reject it, while the rest of the network(that does not know about the tx), would continue to mine on the block that you consider invalid. and you would not be able to confirm more txs. 

a reorg will happen if the client receives a block with more total-work(sum of all the work done in all previous block), then the current one.
also how do you proof that a transaction took place before the block was mined?
the blockchain's function is to proof that at least some time have passed since something was made.

you are a STUPID FUCK, unless you are shutting your mouth now, and go learn how the stuff works now. then you may come back, and maybe i will talk to you in a more polite language.

[ovidiusoft]

i will not control my language

Thank you for helping me decide whether or not you should make it on my block list. I will also be grateful if you will ignore me too, there's no reason for us to interact anymore.

[kokjo]

i will not control my language

Thank you for helping me decide whether or not you should make it on my block list. I will also be grateful if you will ignore me too, there's no reason for us to interact anymore.

so you do not like people who told you that you fucked up, and your suggest sucks? Great! it is really productive. STUPID FUCK!

[kokjo]

It undermines your entire argument to start screaming obscenities about a proposal.  Not even a formal proposal but more an inquiry.  If you need to be beligerent to get your point across then you don't really have anything useful to say.

... sorry about that, sometimes i just gets too annoyed on stupid people. especially when they can't see the fatal consequences of a change that their think would make the network more efficient or fair.

BTW: before you start ranting I agree this would be a very bad idea, isn't needed, and would have significant unintended consequences.

WHAT? ME RENTING? ARE YOU TELLING ME TO STFU?

It also has exactly a 0% chance of ever being implemented.

yes you are right, i should leave the topic now.

[ovidiusoft]

It undermines your entire argument to start screaming obscenities about a proposal.  Not even a formal proposal but more an inquiry.  If you need to be beligerent to get your point across then you don't really have anything useful to say.

Thank you. I almost started believing that it's not possible anymore to have a debate and disagree without everybody insulting eachother .

[Meni Rosenfeld]

I paid suggested transaction fee. Why would a user care about how large it was (I assume you mean the number of inputs) and how many blocks? Why would a user care about blocks at all?

He shouldn't, and to the extent that he needs to care about it in the current implementation, these are all issues with the usability of the current client.

I did exactly what the Bitcoin client suggested I do. The system then ignored my transaction.

Mostly likely the transaction wasn't included in the block due to some technical error in the way the miners' client chooses transactions. It's not due to malice on the miners' part.

but I say that allowing the miners to ignore transactions indefinitely is the problem that should be fixed.

Miners provide a service for a fee, it doesn't make sense to force them to provide a service to you. Ideally the incentive structure is designed in such a way that the market is efficient and you will find miners offering this service to you at a competitive rate. You're extrapolating from an occasional instance of a technical error, and your proposal does not solve the actual problems (which do exist, and for which a solution will have to be found).

[ovidiusoft]

Miners provide a service for a fee, it doesn't make sense to force them to provide a service to you. Ideally the incentive structure is designed in such a way that the market is efficient and you will find miners offering this service to you at a competitive rate. You're extrapolating from an occasional instance of a technical error, and your proposal does not solve the actual problems (which do exist, and for which a solution will have to be found).

Maybe I didn't express myself correctly. I don't want miners to do something much different than what they do now. Obviously, I don't want a service *for me*. I just want to add a "and don't leave out tx older than x blocks" restriction - which, looking at http://bitcoinstats.org/, is not even a problem, since most transactions are confirmed much faster anyway. For the miners the effect will be minimal. For the end-user though, this restriction brings a much-needed warranty.

You might be right about the technical glitch. I supposed my tx simply tripped some checks in the miners. I assumed it was intentional, but I might be wrong.

I do believe the problem still remains, though - how can we make sure transactions are always confirmed in a predetermined time (max. number of blocks)?

The standard suggestion - "increase fee" - has the following problems:

* it's impossible for a user to determine what the fee should be. Most will use the suggested value, which might or might not be enough. And...

* ...once a transaction is out, there's nothing you can do to speed it up. I suggested that the network should guarantee that it will get into a block, what other solutions can there be?

[Meni Rosenfeld]

* it's impossible for a user to determine what the fee should be. Most will use the suggested value, which might or might not be enough.

The current "suggested fee" feature of the client is a temporary hack. In the future it will receive as input the desired wait time (or use a default value) and give a reasonable estimate based on historical data.

* ...once a transaction is out, there's nothing you can do to speed it up. I suggested that the network should guarantee that it will get into a block, what other solutions can there be?

The protocol supports multiple versions of a transaction. It should be possible to use this to release a new version of the transaction with a higher fee if the old one isn't accepted, and the client can do this automatically.

[ovidiusoft]

The current "suggested fee" feature of the client is a temporary hack. In the future it will receive as input the desired wait time (or use a default value) and give a reasonable estimate based on historical data.

The protocol supports multiple versions of a transaction. It should be possible to use this to release a new version of the transaction with a higher fee if the old one isn't accepted, and the client can do this automatically.

Do you know if these features are currently being worked on? The second one is enough for the moment, I think. One can simply start with 0 or a small fee and keep increasing the fee. I'm not much of a developer, but I'll help test.

If these features would be available and work correctly, I think it will fix my problem. It won't protect against miners that simply don't want to include a specific transaction, but let's assume that won't be the case.

[Meni Rosenfeld]

The current "suggested fee" feature of the client is a temporary hack. In the future it will receive as input the desired wait time (or use a default value) and give a reasonable estimate based on historical data.

The protocol supports multiple versions of a transaction. It should be possible to use this to release a new version of the transaction with a higher fee if the old one isn't accepted, and the client can do this automatically.

Do you know if these features are currently being worked on? The second one is enough for the moment, I think. One can simply start with 0 or a small fee and keep increasing the fee. I'm not much of a developer, but I'll help test.

If these features would be available and work correctly, I think it will fix my problem. It won't protect against miners that simply don't want to include a specific transaction, but let's assume that won't be the case.

I should have mentioned earlier that at this point in time, transaction fees actually have very little to do with the incentive of miners. They exist only to prevent attacking the network by spamming it with useless transactions. So there are hardcoded rules for tx priority which are believed to be enough to prevent spamming. They are tweaked from time to time and there could be glitches which prevent transactions from being included even when they are legitimate.

Features like those I described will only be necessary years from now, though Gavin has implementing an open market for tx fees high on his todo list. I'm not sure what are his plans for this though.

[Gavin Andresen]

Features like those I described will only be necessary years from now, though Gavin has implementing an open market for tx fees high on his todo list. I'm not sure what are his plans for this though.

It is high on my list because I think most miners (and pools) would be happy to include many more free transactions than the current rules allow, and if there is another price spike or somebody rich decides it would be fun to make the block chain a couple of gigabytes bigger it is much easier to react if the fees are not hard-coded.

The rough plan is:

 + Give miners more "knobs" to set fee policy-- let them specify (via command-line switch and maybe bitcoind RPC command) how much (if any) space to set aside in blocks for free transactions, how much to charge per-kilobyte and/or per-ECDSA-signature-validation, and what the priority/size/number-of-signatures thresholds are for considering a transaction for inclusion in the free space.

 + As Meni says, teach clients to look at the recent blockchain history and, for a given transaction, estimate how much of a fee will be required to get it into a block reasonably quickly. Maybe a "createtransaction" RPC call that locks coins for a certain amount of time and returns the how-long-to-confirm estimate along with "commit/aborttransaction" calls....

 + Figure out a reasonable UI for fees. Maybe:  calculate the probability sending the transaction with 0 fee will get into the next, oh, 3 blocks, and if it is greater than, oh, 90% then just send it without a fee.  Otherwise, let the user decide between paying a fee that will get it included (with 90% probability) in the next 3 blocks or letting them know how long it might take if they pay no fee.

Lots of details to be worked out...

[ovidiusoft]

+ Figure out a reasonable UI for fees. Maybe:  calculate the probability sending the transaction with 0 fee will get into the next, oh, 3 blocks, and if it is greater than, oh, 90% then just send it without a fee.  Otherwise, let the user decide between paying a fee that will get it included (with 90% probability) in the next 3 blocks or letting them know how long it might take if they pay no fee.

This sounds to be exactly what is needed to solve my initial problem. If I could make a suggestion here: it's confusing for the user that sometimes a transaction of X BTC costs a certain fee, and sometimes a completely different fee. I understand why, but the regular user won't. I know that it will probably be impossible to keep the fee constant, but it would be nice if the algorithm to calculate it will try to avoid erratic fees.

I can only imagine what a user will think if sending 1 BTC will require 0 fee, 0.05 fee, 0.000002 fee, 0.9 fee and so on

[btc_artist]

Features like those I described will only be necessary years from now, though Gavin has implementing an open market for tx fees high on his todo list. I'm not sure what are his plans for this though.

It is high on my list because I think most miners (and pools) would be happy to include many more free transactions than the current rules allow, and if there is another price spike or somebody rich decides it would be fun to make the block chain a couple of gigabytes bigger it is much easier to react if the fees are not hard-coded.

The rough plan is:

 + Give miners more "knobs" to set fee policy-- let them specify (via command-line switch and maybe bitcoind RPC command) how much (if any) space to set aside in blocks for free transactions, how much to charge per-kilobyte and/or per-ECDSA-signature-validation, and what the priority/size/number-of-signatures thresholds are for considering a transaction for inclusion in the free space.

 + As Meni says, teach clients to look at the recent blockchain history and, for a given transaction, estimate how much of a fee will be required to get it into a block reasonably quickly. Maybe a "createtransaction" RPC call that locks coins for a certain amount of time and returns the how-long-to-confirm estimate along with "commit/aborttransaction" calls....

 + Figure out a reasonable UI for fees. Maybe:  calculate the probability sending the transaction with 0 fee will get into the next, oh, 3 blocks, and if it is greater than, oh, 90% then just send it without a fee.  Otherwise, let the user decide between paying a fee that will get it included (with 90% probability) in the next 3 blocks or letting them know how long it might take if they pay no fee.

Lots of details to be worked out...

+1

The client shouldn't force the user to pay any fee. (The interface to choose a fee and calculate inclusion probability sounds nice)

[DeathAndTaxes]

I can only imagine what a user will think if sending 1 BTC will require 0 fee, 0.05 fee, 0.000002 fee, 0.9 fee and so on

There are never required fees (other than those designed to prevent transaction spam).  So a user can always send 0.  The client is never going to force the user to pay more.

I doubt it would be confusing if user understood transactions can be free but including a fee increases the likelihood it will be included sooner.

Imagine you click "send" and the client advises you "based on prior 48 hour transaction volume a transaction fee of at least 0.1 BTC has a 90% likelihood of being included in the next 3 blocks.  Do you wish to pay a fee for priority processing?  YES/NO".

Trying to lock things down into fixed fees and fixed guaranteed timelines is simply not possible for Bitcoin.  It isn't a credit card and trying to make it one is like trying to put a square peg in a round hole.

Bitcoin transactions are generally low cost however there is no guarantee of execution.  Higher transaction fees increase the likelihood of priority processing.  Eventually the client can make this more "user friendly" and intuitive however be clear the client will NEVER be able to guaratee a confirmation time of x or that a fee of y will ensure a transaction will be in the next z blocks.  That simply is not possible (nor necessary).

[ovidiusoft]

I can only imagine what a user will think if sending 1 BTC will require 0 fee, 0.05 fee, 0.000002 fee, 0.9 fee and so on

There are never required fees (other than those designed to prevent transaction spam).  So a user can always send 0.  The client is never going to force the user to pay more. I doubt it would be confusing if user understood transactions can be free but including a fee increases the likelihood it will be included sooner. Imagine you click "send" and the client advises you "based on prior 48 hour transaction volume a transaction fee of at least 0.1 BTC has a 90% likelihood of being included in the next 3 blocks.  Do you wish to pay a fee for priority processing?  YES/NO".

Yes, yes, I meant it like in your example. I think it would be confusing if "90% chance of being included in the next 3 blocks" will vary wildly. But it's too soon to tell. Just something to keep in mind.

Eventually the client can make this more "user friendly" and intuitive however be clear the client will NEVER be able to guaratee a confirmation time of x or that a fee of y will ensure a transaction will be in the next z blocks.  That simply is not possible (nor necessary).

Well, I guess this is a point where we will not reach a compromise. I believe there should be a guarantee, and it must be enforced by the protocol/network. On the other hand, I don't really care what the technical solution is, if it works.

[Syke]

After spending more than a day frustrated that one of my transactions didn't get through, even though it contained a fee, simply because it was too large

How do you *know* that was the reason it took a long time? Post a link to the transaction so we can try to figure out what really happened.

Here's what might have happened. You started Bitcoin, created your transaction, and sent it to the network while being connected to just 1 other node. That node then shut down for whatever reason before relaying it to any other nodes. Such a transaction won't be confirmed for a very long time because it failed to propagate. Tweaking the fee schedule won't fix such a problem.

[ovidiusoft]

You started Bitcoin, created your transaction, and sent it to the network while being connected to just 1 other node. That node then shut down for whatever reason before relaying it to any other nodes. Such a transaction won't be confirmed for a very long time because it failed to propagate. Tweaking the fee schedule won't fix such a problem.

That wasn't the case. The tx was visible in the network, I checked with at least three sites (blockchain.info, bitcoincharts and another one I don't remember). However, it's possible my original assumption about it being ignored because of the size was wrong, some possible explanations were given by others. The idea that there should be a maximum wait time to confirm for transactions remains.

[grue]

so your transaction didn't get accepted into the next block. either the transaction wasn't propagated in time for the next block, or the block was already full.

for propagation time, it's purely luck and your solution wouldn't help in that case.

as for the block being full, blocks are (almost) never "full", the fees just get exponentially higher as the block gets "fuller". so suck it up, and pay more fees, or be a bit more patient.

How do you know if you should include more fees? go on http://bitcoincharts.com/bitcoin/ and check how many transactions are pending, and adjust your rate accordingly.

Most (95%+) of my "normal" and "high" priority transactions get included in 1 block, and even "low" priority transactions are relatively fast for me. Something tells me that your problem isn't priority related...

tl;dr: pay more fees, QQ less.

[2_Thumbs_Up]

If your transaction doesn't get processed in the time frame you want, you should include a higher fee. That's the point of fees.

I am not trying to alienate miners, but they have, IMHO, too much power regarding which transactions are included and when.

They have no obligation to include a transaction. They are doing the work. You are the one that wants it included, so you are the one that need to convince them.

I would however have liked some feature where all transactions should specify a time frame after which they become invalid. This would solve the problem where a transactions gets stuck in limbo if it doesn't get included in a block soon enough. And it would also possibly allow some neat physical smart card wallets/coins I've been brainstorming about recently.