rpietila Wall Observer - the Quality TA Thread ;)

[rpietila]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

[1714492420]

1714492420

[danielW]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

Create pull request then?  I don't know enough to know exactly how to implement and test it. I do know that in development often things seem deveptively simple and easy from the outside.

[solex]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

You are assuming a functional fees market exists. This is not yet the case. Most miners are simply selling hashing power into pools. Basically, only a handful of pool owners can define X, and they don't want to constantly play with variables. They want to set up an optimized mining operation and let it run.

Bitcoin as a store of value is currently it's main use case (in my eyes) and a takeoff for micro transactions would kill it. I really don't know why there are so many who were thinking that bitcoin has a future for micro-txs although there is a fixed blockchain. (If at all, transaction replacements are the future in that regard)

This isn't about micro-tx. The fee structure change a year ago already made micro-tx non-viable, and push it off-chain. This is good.
The problem is that the block size limit is simply too small for Bitcoin to maintain reasonable growth, and it will hit a bottleneck just trying to process normal business.

[phatsphere]

Why is the floating transaction fee not implemented already?

(iow: why centralization?)

it's already implemented, i've read the code and I'm also actually doing some statistics how it behaves. just because i'm really interested in monitoring it. it's just that implementing it in the dev version of bitcoin core doesn't automatically push it out to everyone. we have to wait for 0.10 first.
also, the bitcoin core code is entirely independent of what the miners are going to do. i.e. it doesn't impose arbitrary rules any more (which is true right now, e.g. hard coded default/minimum fee ...)

i don't know what you mean by the last part in brackets.

[rpietila]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

You are assuming a functional fees market exists. This is not yet the case. Most miners are simply selling hashing power into pools. Basically, only a handful of pool owners can define X, and they don't want to constantly play with variables. They want to set up an optimized mining operation and let it run.

In that case, defining "X" is not needed at all. You just always take the best paying transactions, regardless of how much they pay. ("X" is for artificially restricting supply to extract more fees, and completely unnecessary.)

block size limit is simply too small for Bitcoin to maintain reasonable growth, and it will hit a bottleneck just trying to process normal business.

If it's about 260,000 transactions per day, it is indeed too small. If it can be increased, I favor increasing. And also setting in stone the increasing schedule so that no further vote or community decision is needed ever concerning this matter.

[BTCtrader71]

block size limit is simply too small for Bitcoin to maintain reasonable growth, and it will hit a bottleneck just trying to process normal business.

If it's about 260,000 transactions per day, it is indeed too small. If it can be increased, I favor increasing. And also setting in stone the increasing schedule so that no further vote or community decision is needed ever concerning this matter.

Perhaps a block size limit that periodically self-adjusts based upon a function that takes as input the historical number of transactions per unit time?

[Biodom]

Suppose Monero (or a different alt) were to gain enough traction for it to appear inevitable that it would eventually topple bitcoin. How feasible would it be to fork bitcoin to adopt some or all features of Monero? iow, a merger of the first mover advantage of bitcoin with the technical advantages of Monero. I suppose this question has two parts: 1) What would be the technical feasibility of doing this? and 2) Would the community go along?
 

The chances of such a merger are zero. From a technical perspective the coins are very different, and the economic interests strongly align against this. More importantly such a merger would violate the most basic economic fundamentals of both coins. This means that such a merger would break all trust in the coins and yes could make them both worthless. Both communities would reject such a merger and for very good reasons.

What is more likely to happen here is that Bitcoin would have to fork to deal with the 1 MB Limit. Both coins would live side by side, compete and hopefully learn from each other. First mover is not everything. A very good example is the credit card industry. The first movers were Diner's Club and then American Express. Then came the later entrants including Visa and MasterCard. The first movers are still around but with vastly reduced market share.

The challenge for the Bitcoin is to deal with the 1 MB blocksize limit before the above happens and not become the American Express or even Diner's Club of crypto-currency.

Edit: Bitcoin might even fork in the middle of a boom, like it did the last time. After all it is way easier to get consensus when everyone is getting wealthier.

You guys made me look at Monero (XMR), which trades pretty thinly and recently diving despite one 667 XMR order, which must be either Risto or someone else from around here  
I am thinking of diversifying a small % of my BTC to alts, such as XMR, etc. (NOT LTC)

[DeathAndTaxes]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

That was implemented years ago and has nothing to do with a floating fee.  Miners today can set a min fee amount, a block size, and they order txns by fee and include the top x which will fit.

[rpietila]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

That was implemented years ago and has nothing to do with a floating fee.  Miners today can set a min fee amount, a block size, and they order txns by fee and include the top x which will fit.

Ah, please tell me what is a floating fee then?

[phatsphere]

Ah, please tell me what is a floating fee then?

currently, a client like bitcoin core -- and the floating fees are ONLY about clients sending out TXs, nothing related to miners -- has a fixed minimum fee. other online wallets also have a fixed minimum fee, etc. you can also send out TXs with any other fee, also zero, but it's just a guess out of your butt.

what you want is actually this: higher fee -> quicker inclusion in the next block. lower fee -> maybe wait an hour or two, but it will get included.

solution:
1.  let your client monitor the last 2500 successfully confirmed transactions (ignore what's currently in the mempool or what your client hasn't seen so far) whereas they are grouped by the number of blocks they had to wait until they are confirmed in bins of maximum size 100,
2. calculate for each of them the TX-fee per kB,
3. sort them by this value and put them into 25 bins of size 100 each (and yes, if you don't have 2500 TXs, bins are smaller),
4. select bin number "n" for your desired number of blocks "n" you want wait for inclusion -> calculate the median TX-fee of that bin number "n" and multiply it by the kB your new TX.

For example, this is what my dev client says are the fees to be included in the next "n"-th block:

Code:

$ echo `seq 1 25` | xargs -n 1 bitcoin-cli estimatefee | nl
     1 	0.00045662
     2 	0.00044247
     3 	0.00038910
     4 	0.00038610
     5 	0.00026809
     6 	0.00022831
     7 	0.00019193
     8 	0.00015885
     9 	0.00014461
    10	0.00013614
    11	0.00012547
    12	0.00012315
    13	0.00012241
    14	0.00012195
    15	0.00011851
    16	0.00011778
    17	0.00011764
    18	0.00011475
    19	0.00011282
    20	0.00011040
    21	0.00010928
    22	0.00010787
    23	0.00010548
    24	0.00010405
    25	0.00006574

[dillpicklechips]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

But what about when blocksize is too small that the fees become high lowering BTC's usefulness as a cheap payment system.

[dillpicklechips]

Why is the floating transaction fee not implemented already?

(iow: why centralization?)

Somewhere else by Gavin...

Smart (dynamic, floating) fees for the reference implementation wallet was pulled today:
  https://github.com/bitcoin/bitcoin/pull/4250

... and should appear in version 0.10.

The estimation code only considers transactions that are broadcast on the network, enter the memory pool (so are available to any miner to mine), and then are included in a block. So it is immune to miners putting pay-to-self transactions with artificially high fees in their blocks.

Right now if you use the default fee rules your transactions will take 2-6 blocks to confirm:
  http://bitcoincore.org/smartfee/fee_graph.html

The priority estimation code is even more broken; the reference implementation wallet will send a 56-million-priority transaction with no fee, which is nowhere near enough priority to get confirmed quickly:
  http://bitcoincore.org/smartfee/priority_graph.html

(the smart fee code estimates priority, too).

Release notes from doc/release-notes.md in the source tree:

Transaction fee changes
=======================

This release automatically estimates how high a transaction fee (or how
high a priority) transactions require to be confirmed quickly. The default
settings will create transactions that confirm quickly; see the new
'txconfirmtarget' setting to control the tradeoff between fees and
confirmation times.

Prior releases used hard-coded fees (and priorities), and would
sometimes create transactions that took a very long time to confirm.


New Command Line Options
========================

-txconfirmtarget=n : create transactions that have enough fees (or priority)
so they are likely to confirm within n blocks (default: 1). This setting
is over-ridden by the -paytxfee option.

New RPC methods
===============

Fee/Priority estimation
-----------------------

estimatefee nblocks : Returns approximate fee-per-1,000-bytes needed for
a transaction to be confirmed within nblocks. Returns -1 if not enough
transactions have been observed to compute a good estimate.

estimatepriority nblocks : Returns approximate priority needed for
a zero-fee transaction to confirm within nblocks. Returns -1 if not
enough free transactions have been observed to compute a good
estimate.

Statistics used to estimate fees and priorities are saved in the
data directory in the 'fee_estimates.dat' file just before
program shutdown, and are read in at startup.

[rpietila]

Many things are difficult, but that shouldn't be difficult at all: miners just include transactions that pay the most (if space is limited) or that pay minimum X bits, X being defined by the miner, (if the space is not limited). THAT'S IT!

But what about when blocksize is too small that the fees become high lowering BTC's usefulness as a cheap payment system.

I am not in favor of artificially reducing the blocksize. But if the technology (storage space, propagation speed of blocks etc. similar factors) demands it, then I believe typical transaction fees in the $1-$2 range (50 times higher than now) won't kill it, ie. the Bitcoin's unique characteristics of a secure store-of-value ledger are unaffected even if the transactions were more expensive.

[phatsphere]

But what about when blocksize is too small that the fees become high lowering BTC's usefulness as a cheap payment system.

well, why does it need to be cheap when it is still useful? the tradeoff is this: either the system starts to fall apart by a combination of growing too fast, no incentive for miners and flaky behaviour -- or a well working and tested system, where there is a market for the fee to use it.
i lean on the side of caution, better a well tested system than killing it by a step forward which cannot really be undone.

also: a higher fee for bitcoin just means, that a new market opens up: one for other coins which do what the pissed-off users demand. e.g. really fast transactions of small amounts, etc.

[rpietila]

also: a higher fee for bitcoin just means, that a new market opens up: one for other coins which do what the pissed-off users demand. e.g. really fast transactions of small amounts, etc.

If transacting is prohibitively expensive, then Bitcoin might lose users, and the resulting loss of adoption turn to loss of network effect, vicious circle, etc. So it is not trivial to let the fee shoot up, but we are well on the safe side with sub-$1 fees.

Like I said a week ago, in very few applications, is the total transaction cost less than $1 anyway even if the "handing money" part were free (as is the case with cash).

[greenlion]

I disagree philosophically with the direction Gavin is taking this where floating fees are integrated into the reference client.

The methodology of estimating them is never a non-controversial undertaking, and there are no structural limitations that prevent data and market information determining tx pricing from getting decided outside of the Bitcoin network. All the reference client actually has to do in this respect is create and sign raw transactions through an API that provides all the relevant parameters, including tx fee itself. On the mining end, all the reference client has to do is provide configuration parameters robust enough for pools and solo miners to decide their tx fee policy, or integrate through API to take external data about appropriate pricing in order to select transactions for candidate blocks as desired.

The decision on methodology of pricing transactions is properly offloaded to wallet and mining pool software developers, and the market will decide the correct infrastructure for tx fee price discovery. Not some guy working on an institutional stipend.

[giveBTCpls]

private property as we knowing must basically dissapear. Things will be lended. Say you need whatever device to do whatever task, like record a movie or whatever, you ask for any free cameras and you get one sent, when you are done you send it back and some other person uses it.

Who sends you the camera? Who stores it? Who fixes it when it's broken? If two people want it at the same time, who decides who gets it? What happens to the person who "borrows" it and never sends it back?

Statistics could get extracted in terms of "how many cameras are used and for how much time are they used, and how many of them get broken" and then manufacture under these %'s. These cameras are built by automated robots, which manufacture given the data input that I mentioned earlier. The wasted stuff aka unrepairable will be decomposed and the prime materials used again. In 1000 years the human imput to keep this cycle going will be peanuts.

Who extracts these statistics? Who decides how much in terms of resources gets spent on digital cameras, smartphone cameras, video cameras, other types of cameras, and / or R&D on new types of cameras?

Of course some sort of centralized (as in central or "Main", not closed source) computer having realtime date of worldwide resources. I don't see any other solution. The core of what dictates if something can be manufactured or not is judging by the dictatorship that is nature (nature is only true dictatorship, we must align to it, to whatever resources are available) and also to the well being of everyone. If what you are requesting, whatever that is, compromises the basic needs of a person a continent away, then you deal with it and not get said thing until this equilibrum of resources and global well being + your specific not basic need demand not compromising any of the former is meet.

How do you plan to convince the world population to subjugate themselves to your centralized computer?

They will naturally as they see it's just better. Of course you are free to live outside the civilized world.

[phatsphere]

All the reference client actually has to do in this respect is create and sign raw transactions through an API that provides all the relevant parameters, including tx fee itself.

that's still there

On the mining end, all the reference client has to do

there is no reference client for mining. all major pools have their own thing.

The decision on methodology of pricing transactions is properly offloaded to wallet and mining pool software developers...

the reference client is also a wallet, therefore it is ok to have some estimation function inside. there is also no barrier in implementing your own function to do that!
my guess is, there will be different methods to estimate the fee and the data of the reference client is more like a monitoring tool (that's how I see it currently).
there is also no decision made regarding the protocol itself.

hence i regard what you write a bit like FUD, sorry for that.

[greenlion]

All the reference client actually has to do in this respect is create and sign raw transactions through an API that provides all the relevant parameters, including tx fee itself.

that's still there

On the mining end, all the reference client has to do

there is no reference client for mining. all major pools have their own thing.

The decision on methodology of pricing transactions is properly offloaded to wallet and mining pool software developers...

the reference client is also a wallet, therefore it is ok to have some estimation function inside. there is also no barrier in implementing your own function to do that!
my guess is, there will be different methods to estimate the fee and the data of the reference client is more like a monitoring tool (that's how I see it currently).
there is also no decision made regarding the protocol itself.

hence i regard what you write a bit like FUD, sorry for that.

To address all these points --

1. I never said it wasn't there, I just said that it shouldn't do more than just provide that functionality. The idea of going beyond the basic protocol functionality introduces a gray area between protocol and policy.

2. Mining pool software and solo miners call bitcoind (or at least a custom-compiled version of bitcoind), are you somehow insinuating that miners all use libbitcoin or have all written their own top secret independent implementations?

3. The notion of the reference client being a wallet is being phased out gradually anyway per statements made by Gavin previously.

4. Ideas that you disagree with just because you're mistaken about them do not constitute "FUD".

[BTCtrader71]

private property as we knowing must basically dissapear. Things will be lended. Say you need whatever device to do whatever task, like record a movie or whatever, you ask for any free cameras and you get one sent, when you are done you send it back and some other person uses it.

Who sends you the camera? Who stores it? Who fixes it when it's broken? If two people want it at the same time, who decides who gets it? What happens to the person who "borrows" it and never sends it back?

Statistics could get extracted in terms of "how many cameras are used and for how much time are they used, and how many of them get broken" and then manufacture under these %'s. These cameras are built by automated robots, which manufacture given the data input that I mentioned earlier. The wasted stuff aka unrepairable will be decomposed and the prime materials used again. In 1000 years the human imput to keep this cycle going will be peanuts.

Who extracts these statistics? Who decides how much in terms of resources gets spent on digital cameras, smartphone cameras, video cameras, other types of cameras, and / or R&D on new types of cameras?

Of course some sort of centralized (as in central or "Main", not closed source) computer having realtime date of worldwide resources. I don't see any other solution. The core of what dictates if something can be manufactured or not is judging by the dictatorship that is nature (nature is only true dictatorship, we must align to it, to whatever resources are available) and also to the well being of everyone. If what you are requesting, whatever that is, compromises the basic needs of a person a continent away, then you deal with it and not get said thing until this equilibrum of resources and global well being + your specific not basic need demand not compromising any of the former is meet.

How do you plan to convince the world population to subjugate themselves to your centralized computer?

They will naturally as they see it's just better. Of course you are free to live outside the civilized world.

Does that mean you are going to force people who do not want to be controlled by this computer to move elsewhere? Or can they stay where they are and live by their own rules? Is this a purely voluntary system? Opt-in, opt-out when you want? Abide by the computer's decisions when you want, don't abide by them when you don't? Or will there be an enforcement mechanism? If so, how do you enforce the rules?