Bitcoin Forum
May 06, 2024, 11:59:37 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Q: The difference between txn memory pool and UTXO  (Read 960 times)
1t0ph20 (OP)
Newbie
*
Offline Offline

Activity: 16
Merit: 7


View Profile
March 27, 2017, 08:14:51 PM
 #1

Hi there, new to Bitcoin and just had a clarification question.

When does a transaction leave the UTXO and when does a transaction leave the memory pool?

From my understanding, a validating node receives a transaction and is added to node's mem. pool. The node looks up that the inputs are valid in the UTXO. If input exists in the UTXO, the txn is removed from the UTXO. Once a block arrives at the node (or if the node is a miner and solves a block) and the block is validated, the transaction is removed from the transaction memory pool.

Is this true? I'm just kinda unsure about at what point txns are removed from UTXO vs memory pool.

Thanks!!
1714996777
Hero Member
*
Offline Offline

Posts: 1714996777

View Profile Personal Message (Offline)

Ignore
1714996777
Reply with quote  #2

1714996777
Report to moderator
1714996777
Hero Member
*
Offline Offline

Posts: 1714996777

View Profile Personal Message (Offline)

Ignore
1714996777
Reply with quote  #2

1714996777
Report to moderator
1714996777
Hero Member
*
Offline Offline

Posts: 1714996777

View Profile Personal Message (Offline)

Ignore
1714996777
Reply with quote  #2

1714996777
Report to moderator
The trust scores you see are subjective; they will change depending on who you have in your trust list.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714996777
Hero Member
*
Offline Offline

Posts: 1714996777

View Profile Personal Message (Offline)

Ignore
1714996777
Reply with quote  #2

1714996777
Report to moderator
1714996777
Hero Member
*
Offline Offline

Posts: 1714996777

View Profile Personal Message (Offline)

Ignore
1714996777
Reply with quote  #2

1714996777
Report to moderator
1714996777
Hero Member
*
Offline Offline

Posts: 1714996777

View Profile Personal Message (Offline)

Ignore
1714996777
Reply with quote  #2

1714996777
Report to moderator
AgentofCoin
Legendary
*
Offline Offline

Activity: 1092
Merit: 1001



View Profile
March 27, 2017, 08:25:02 PM
Last edit: March 28, 2017, 04:35:25 AM by AgentofCoin
 #2

Hi there, new to Bitcoin and just had a clarification question.

When does a transaction leave the UTXO and when does a transaction leave the memory pool?

A UTXO is an "unspent transaction output", and is what "bitcoins' are after being confirmed and not moving.
A mempool tx is a "spent transaction, and are "bitcoins" that are still pending for confirmation.

 - UTXOs in theory never leave the UTXO set, but are divided and consolidated over time.

 - Mempool Txs are either dropped when they are confirmed or dropped after 72 hours of no confirmation.
(the 72 hour limit is based on the standard node code, but nodes can choose their own preference).


...
Is this true? I'm just kinda unsure about at what point txns are removed from UTXO vs memory pool.

In theory, UTXOs are never "removed", only mempool txs can be "removed".

The spent output is not removed from the UTXO until it's associated mempool tx has been confirmed.
When the blockchain is updated, the old UTXO is overwritten into either consolidation (added to an
already existing UTXO), division (split into multiple new UTXOs) or a single new UTXO.


EDIT: I was mostly incorrect, see Danny's answer below.

I support a decentralized & unregulatable ledger first, with safe scaling over time.
Request a signed message if you are associating with anyone claiming to be me.
1t0ph20 (OP)
Newbie
*
Offline Offline

Activity: 16
Merit: 7


View Profile
March 27, 2017, 10:40:12 PM
 #3

Good reply.  Thanks!
DannyHamilton
Legendary
*
Offline Offline

Activity: 3388
Merit: 4616



View Profile
March 28, 2017, 12:08:08 AM
Last edit: March 28, 2017, 03:20:39 AM by DannyHamilton
 #4

I'm not sure if AgentofCoin is misunderstanding the process, or if I am just misunderstanding what AgentofCoin is saying however...


Transactions are made up of inputs and outputs.

Inputs are references to outputs of earlier transactions. They meet the requirements that the output was encumbered with, and they supply value to the transaction.

Outputs encumber that value with requirements that must be met for that output to be used as an input to another transaction.

The UTXO is a list of the outputs which a node has not yet seen used as an input.  When an output in the UTXO is seen as an input to a transaction, the node removes that otuput from its UTXO so that any other transaction trying to use it as an input will be identified as invalid.

The mempool is a list of the transactions that a node has heard about from a peer, but which is not in the node's blockchain.  When the transaction shows up in the node's blockchain, it is removed from the node's mempool.  It also may be removed from the mempool to free up space for newer transactions if too much time has passed without being confirmed.
AgentofCoin
Legendary
*
Offline Offline

Activity: 1092
Merit: 1001



View Profile
March 28, 2017, 12:58:44 AM
 #5

I'm not sure if AgentofCoin is misunderstanding the process, or if I am just misunderstanding what AgentofCoin is saying however...
...

I intended to mean what you have stated. My answer was the layman version.
I didn't want to go into the output is an input is an output, depending on its status
at a certain point in time.

I guess when I said "spent transaction", I should have said "spent transaction output"
(which is the input to the "unconfirmed transaction"), and when that gets included into
the blockchain, becomes an unspent output itself. What I mean is, that the input
aspect of the tx is always transient since its existence is only to become an output (or
be rejected). There are spent and unspent outputs, but not spent or unspent inputs, since
inputs only exists within the mempool, which is temporary. At least that is how I see it.

But if there is something in my statement that is contradictory or wrong or I am wrong in
general, I will correct it.

I support a decentralized & unregulatable ledger first, with safe scaling over time.
Request a signed message if you are associating with anyone claiming to be me.
DannyHamilton
Legendary
*
Offline Offline

Activity: 3388
Merit: 4616



View Profile
March 28, 2017, 03:18:30 AM
 #6

I didn't want to go into the output is an input is an output, depending on its status
at a certain point in time.

I understand.  Bitcoin doesn't work the way a lot of people expect it to, and when they are first exposed to the whole input and output concept it can be confusing.


I guess when I said "spent transaction", I should have said "spent transaction output"
(which is the input to the "unconfirmed transaction"), and when that gets included into
the blockchain, becomes an unspent output itself.

Actually, the outputs of an unconfirmed transaction are added to the UTXO as soon as the transaction is seen.  It doesn't wait for the transaction to be "included in the blockchain".  Furthermore, the only way a "spent transaction output" becomes an unspent output is if the transaction that spent it either becomes invalid or is dropped from the mempool to free up space for newer transactions. It doesn't become an unspent output when it gets included in the blockchain. It remains spent.

What I mean is, that the input aspect of the tx is always transient since its existence is only to become an output (or
be rejected).

This is incorrect.  An input doesn't become an output.  It remains an input.  Outputs that haven't been spent yet can be referenced in a future input, but inputs don't get referenced, or removed, or change state.

There are spent and unspent outputs, but not spent or unspent inputs, since
inputs only exists within the mempool, which is temporary.

This is also incorrect.  Inputs exist in transactions.  Transactions are recorded in the blockchain.  The inputs don't disappear when the transaction is added to the blockchain. They remain a part of the transaction itself.
AgentofCoin
Legendary
*
Offline Offline

Activity: 1092
Merit: 1001



View Profile
March 28, 2017, 04:02:02 AM
Last edit: March 28, 2017, 07:04:13 AM by AgentofCoin
 #7

I guess when I said "spent transaction", I should have said "spent transaction output"
(which is the input to the "unconfirmed transaction"), and when that gets included into
the blockchain, becomes an unspent output itself.
Actually, the outputs of an unconfirmed transaction are added to the UTXO as soon as the transaction is seen.  It doesn't wait for the transaction to be "included in the blockchain".  

I guess I just don't understand then.
So you are saying that even before a tx receives a confirmation, it is "added" to the UTXO set.
So if the mempool is flooded with millions of spammy valid txs, but they pay an appropriate fee, in
theory, there would be an increase of millions of new UTXOs within the UTXO set for at least 72 hours?
So the data in the UTXO set are not always valid unspent transactions?

Furthermore, the only way a "spent transaction output" becomes an unspent output is if the transaction that spent it either becomes invalid or is dropped from the mempool to free up space for newer transactions. It doesn't become an unspent output when it gets included in the blockchain. It remains spent.

Don't you mean the prior tx remains spent, and it child is unspent, but is already in the UTXO set?

So, all broadcasted but unconfirmed txs are all already in the UTXO set and thus are only removed
from the UTXO set if the spending tx was invalidated or dropped from the mempool?
The UTXO set assumes all broadcast txs are valid until told otherwise.

What I mean is, that the input aspect of the tx is always transient since its existence is only to become an output (or
be rejected).
This is incorrect.  An input doesn't become an output.  It remains an input.  Outputs that haven't been spent yet can be referenced in a future input, but inputs don't get referenced, or removed, or change state.

What I thought was that: The inputs were the spent outputs from it's parent tx. When an
address has valid inputs that are unmoved, the address is an "unspent output". Thus the UTXO term.
So metaphorically, the input only exists to validate spent outputs that are being transformed into
unspent outputs within a new address for example. Coinbase's would be the only true input in my
thinking, since they "originate" outside of the blockchain. This was my reasoning.

I thought outputs were moving from a state of unspent to spent continuously.
Inputs were just another representation for of an unspent output.


There are spent and unspent outputs, but not spent or unspent inputs, since
inputs only exists within the mempool, which is temporary.
This is also incorrect.  Inputs exist in transactions.  Transactions are recorded in the blockchain.  The inputs don't disappear when the transaction is added to the blockchain. They remain a part of the transaction itself.

My point was that the term "input" was just a place holder to describe an unspent output.
So when I gather multiple inputs to make a new tx with 1 output, I am really gathering
multiple unspent outputs and making a new tx to spend them to another address where the
process could begin again if I controlled that new address.

Do inputs exist within a UTXO?
Are you saying UTXOs are unspent inputs then?

I guess I really do not understand.
This is not my field, so I do not know the basics of computer science that some would use
as a standard foundation. I should probably read about this aspect specifically.


Edit: After a superficial search, I understand why I am incorrect. When I am using the
term "input" I am only referring to the "coins" associated with the prior tx, when in reality an
"input" contains an "outpoint", a "signature script", and a "sequence number". I assumed they
were separate and part of the full tx and not part of the input/output aspects of the tx.
Ultimately, I need to read a well written book on Bitcoin for Novices and really learn everything
from the beginning and properly.

I support a decentralized & unregulatable ledger first, with safe scaling over time.
Request a signed message if you are associating with anyone claiming to be me.
DannyHamilton
Legendary
*
Offline Offline

Activity: 3388
Merit: 4616



View Profile
March 28, 2017, 01:10:37 PM
Last edit: March 28, 2017, 01:32:24 PM by DannyHamilton
 #8

I guess I just don't understand then.
So you are saying that even before a tx receives a confirmation, it is "added" to the UTXO set.

There are no transactions in the UTXO.  Only outputs.

Even before a transation receives a confirmation, the outputs that are referenced by the inputs are REMOVED from the UTXO, and the new outputs are ADDED to the UTXO.

So if the mempool is flooded with millions of spammy valid txs, but they pay an appropriate fee, in
theory, there would be an increase of millions of new UTXOs within the UTXO set for at least 72 hours?

Every one of those transactions also spends outputs that used to be in the UTXO.  Those spent outputs are removed.

So the data in the UTXO set are not always valid unspent transactions?

They are valid unspent outputs.  They can be used in another transaction.

Furthermore, the only way a "spent transaction output" becomes an unspent output is if the transaction that spent it either becomes invalid or is dropped from the mempool to free up space for newer transactions. It doesn't become an unspent output when it gets included in the blockchain. It remains spent.
Don't you mean the prior tx remains spent, and it child is unspent, but is already in the UTXO set?

Transactions don't get spent.  Outputs do.  The spent output remains spent.  The new confirmed transaction has its own outputs. Those outputs were unspent and in the UTXO before the transaction got confirmed. They remain unspent and in the UTXO when the transaction is confirmed.  Confirming the transaction doesn't change that.

So, all broadcasted but unconfirmed txs are all already in the UTXO set and thus are only removed
from the UTXO set if the spending tx was invalidated or dropped from the mempool?

The outputs of those unconfirmed transactions are in the UTXO.  The outputs that are spent by the inputs are removed from the UTXO.

The outputs that are added to the UTXO are removed from the UTXO if the transaction is invalidated OR if the transaction is dropped from the mempool OR if the outputs are spent by another transaction.

The UTXO set assumes all broadcast txs are valid until told otherwise.

Transactions are validated before being added to the UTXO.  If the transaction is invalid, then it won't be added to the mempool or the UTXO.  However, a transaction that was previously valid can become invalid. In that case the now invalid transaction is removed from the mempool and its outputs are removed from the UTXO (and depending on why it became invalid, some of the outputs spent by its inputs may be added back into the UTXO.

What I thought was that: The inputs were the spent outputs from it's parent tx. When an
address has valid inputs that are unmoved, the address is an "unspent output". Thus the UTXO term.
So metaphorically, the input only exists to validate spent outputs that are being transformed into
unspent outputs within a new address for example. Coinbase's would be the only true input in my
thinking, since they "originate" outside of the blockchain. This was my reasoning.

I thought outputs were moving from a state of unspent to spent continuously.
Inputs were just another representation for of an unspent output.

Well, to start with, there are no addresses.  Those are an abstraction that our wallets and block explorers create for us humans to make it easier to talk about the transfer of control over value. They don't exist in transactions or the blockchain.

Next, inputs never "move".  They are listed in a transaction and they remain there.

Finally, coinbase transactions have outputs, but never have inputs.

Here's a good analogy:

==========
Transaction A (Coinbase)
Inputs:
        none
Outputs:
        Value:1250000000   Encumbrance: Signature required from private key associated with public key Z
==========

("Transaction A, first output", added to UTXO)

==========
Transaction B
Inputs:
        Transaction A, first output, signature from valid private key
Outputs:
        Value 1000000000  Encumbrance: Signature required from private key associated with public key Y
        Value 0240000000  Encumbrance: Signature required from private key associated with public key X

==========

("Transaction A, first output", removed from UTXO
 "Transaction B, first output", added to UTXO
 "Transaction B, second output", added to UTXO
)

==========
Transaction C
Inputs:
        Transaction B, first output, signature from valid private key.
Outputs:
        Value 0500000000  Encumbrance: Signature required from private key associated with public key W
        Value 0400000000  Encumbrance: Signature required from private key associated with public key V

==========

("Transaction B, first output", removed from UTXO
 "Transaction C, first output", added to UTXO
 "Transaction C, second output", added to UTXO
)

==========
Invalid transaction D
Inputs:
        Transaction A, first output, signature from valid private key
Outputs:
        Value 0500000000  Encumbrance: Signature required from private key associated with public key U
        Value 0740000000  Encumbrance: Signature required from private key associated with public key T
==========

(Transaction D rejected because Transaction A, first output is not in the UTXO, it was removed by Transaction B)

==========
Transaction E
Inputs:
        Transaction B, second output, valid signature
        Transaction C, first output, valid signature
Outputs:
        Value 0730000000 Encumbrance: Signature required from private key associated with public key S
==========

("Transaction B, second output", removed from UTXO
 "Transaction C, first output", removed to UTXO
 "Transaction E, first output", added to UTXO
)


So at this point the UTXO lists:
  • "Transaction C, second output"
  • "Transaction E, first output"

If transaction E takes too long to confirm and is removed from the mempool then "Transaction E, first output" will be removed from the UTXO list.  "Transaction B, second output" and "Transaction C, first output" will both be added back into the UTXO list.

In that case, the UTXO list would revert to:
  • "Transaction B, second output"
  • "Transaction C, first output"
  • "Transaction C, second output"

Note that the UTXO is just a list.  Things are added to that list, and removed from that list, but that doesn't have any effect on what is in the transaction.  They transaction remains static exactly the way it was created with its own lists of inputs and outputs.

My point was that the term "input" was just a place holder to describe an unspent output.
So when I gather multiple inputs to make a new tx with 1 output, I am really gathering
multiple unspent outputs and making a new tx to spend them to another address where the
process could begin again if I controlled that new address.

The term "input" is not just a place holder. An input is an actual list item in a transaction.  A transaction consists of two lists.  The list of inputs is a list of references to transaction outputs.  It is part of the transaction and it doesn't go away. It is permanently stored in the blockchain when the transaction is confirmed.

Do inputs exist within a UTXO?

No.  Inputs are not listed in the UTXO.  Additionally, outputs are removed from the UTXO when they are referenced by an input.

Are you saying UTXOs are unspent inputs then?

Inputs are never spent or unspent.  Their existence is what causes an output to be removed from the UTXO.

I guess I really do not understand.

Like I said earlier,
"Bitcoin doesn't work the way a lot of people expect it to, and when they are first exposed to the whole input and output concept it can be confusing."

Edit: After a superficial search, I understand why I am incorrect. When I am using the
term "input" I am only referring to the "coins" associated with the prior tx, when in reality an
"input" contains an "outpoint", a "signature script", and a "sequence number". I assumed they
were separate and part of the full tx and not part of the input/output aspects of the tx.

There are no "coins".  There are ONLY inputs and outputs in a ledger.

Ultimately, I need to read a well written book on Bitcoin for Novices and really learn everything
from the beginning and properly.

I'm happy to discuss it with you.  The first step is to unlearn what you think you know and then to understand that from a technical standpoint:

"coins" are an abstraction for us humans. They do not actually exist at transaction and blockchain level.

"addresses" are an abstraction for us humans. They do not actually exist at transaction and blockchain level.

A transaction is really just 2 lists. There isn't much more to it. The first list is called "inputs", and is a list of outputs of other transactions along with signatures that satisfy the encumbrance of those outputs.  The sum of the values of those outputs is the total value available for the transaction to use in its own outputs. The second list is called "outputs" and is a list of new unspent outputs each encumbered with a requirement (typically a signature requirement). The sum of the values of those outputs is not allowed to exceed the total value supplied by the inputs.
paul gatt
Hero Member
*****
Offline Offline

Activity: 896
Merit: 500


View Profile
March 28, 2017, 01:24:58 PM
 #9

I guess I just don't understand then.
So you are saying that even before a tx receives a confirmation, it is "added" to the UTXO set.

There are no transactions in the UTXO.  Only outputs.

Even before a transation receives a confirmation, the outputs that are referenced by the inputs are REMOVED from the UTXO, and the new outputs are ADDED to the UTXO.

So if the mempool is flooded with millions of spammy valid txs, but they pay an appropriate fee, in
theory, there would be an increase of millions of new UTXOs within the UTXO set for at least 72 hours?

Every one of those transactions also spends outputs that used to be in the UTXO.  Those spent outputs are removed.

So the data in the UTXO set are not always valid unspent transactions?

They are valid unspent outputs.  They can be used in another transaction.

Furthermore, the only way a "spent transaction output" becomes an unspent output is if the transaction that spent it either becomes invalid or is dropped from the mempool to free up space for newer transactions. It doesn't become an unspent output when it gets included in the blockchain. It remains spent.
Don't you mean the prior tx remains spent, and it child is unspent, but is already in the UTXO set?

Transactions don't get spent.  Outputs do.  The spent output remains spent.  The new confirmed transaction has its own outputs. Those outputs were unspent and in the UTXO before the transaction got confirmed. They remain unspent and in the UTXO when the transaction is confirmed.  Confirming the transaction doesn't change that.

So, all broadcasted but unconfirmed txs are all already in the UTXO set and thus are only removed
from the UTXO set if the spending tx was invalidated or dropped from the mempool?

The outputs of those unconfirmed transactions are in the UTXO.  The outputs that are spent by the inputs are removed from the UTXO.

The outputs that are added to the UTXO are removed from the UTXO if the transaction is invalidated OR if the transaction is dropped from the mempool OR if the outputs are spent by another transaction.

The UTXO set assumes all broadcast txs are valid until told otherwise.

Transactions are validated before being added to the UTXO.  If the transaction is invalid, then it won't be added to the mempool or the UTXO.  However, a transaction that was previously valid can become invalid. In that case the now invalid transaction is removed from the mempool and its outputs are removed from the UTXO (and depending on why it became invalid, some of the outputs spent by its inputs may be added back into the UTXO.

What I thought was that: The inputs were the spent outputs from it's parent tx. When an
address has valid inputs that are unmoved, the address is an "unspent output". Thus the UTXO term.
So metaphorically, the input only exists to validate spent outputs that are being transformed into
unspent outputs within a new address for example. Coinbase's would be the only true input in my
thinking, since they "originate" outside of the blockchain. This was my reasoning.

I thought outputs were moving from a state of unspent to spent continuously.
Inputs were just another representation for of an unspent output.

Well, to start with, there are no addresses.  Those are an abstraction that our wallets and block explorers create for us humans to make it easier to talk about the transfer of control over value. They don't exist in transactions or the blockchain.

Next, inputs never "move".  They are listed in a transaction and they remain there.

Finally, coinbase transactions have outputs, but never have inputs.

Here's a good analogy:

Transaction A (Coinbase)
Inputs:
        none
Outputs:
        Value:1250000000   Encumbrance: Signature required from private key associated with public key Z

("Transaction A, first output", added to UTXO)

Transaction B
Inputs:
        Transaction A, first output, signature from valid private key
Outputs:
        Value 1000000000  Encumbrance: Signature required from private key associated with public key Y
        Value 0240000000  Encumbrance: Signature required from private key associated with public key X

("Transaction A, first output", removed from UTXO
 "Transaction B, first output", added to UTXO
 "Transaction B, second output", added to UTXO)

Transaction C
Inputs:
        Transaction B, first output, signature from valid private key.
Outputs:
        Value 0500000000  Encumbrance: Signature required from private key associated with public key W
        Value 0400000000  Encumbrance: Signature required from private key associated with public key V

("Transaction B, first output", removed from UTXO
 "Transaction C, first output", added to UTXO
 "Transaction C, second output", added to UTXO)

Invalid transaction D
Inputs:
        Transaction A, first output, signature from valid private key
Outputs:
        Value 0500000000  Encumbrance: Signature required from private key associated with public key U
        Value 0740000000  Encumbrance: Signature required from private key associated with public key T

(Transaction D rejected because Transaction A, first output is not in the UTXO)

Transaction E
Inputs:
        Transaction B, second output, valid signature
        Transaction C, first output, valid signature
Outputs:
        Value 0730000000 Encumbrance: Signature required from private key associated with public key S

("Transaction B, second output", removed from UTXO
 "Transaction C, first output", removed to UTXO
 "Transaction E, first output", added to UTXO)

So at this point the UTXO lists:
  • "Transaction C, second output"
  • "Transaction E, first output"

If transaction E takes too long to confirm and is removed from the mempool then "Transaction E, first output" will be removed from the UTXO list.  "Transaction B, second output" and "Transaction C, first output" will both be added back into the UTXO list.

In that case, the UTXO list would revert to:
  • "Transaction B, second output"
  • "Transaction C, first output"
  • "Transaction C, second output"

Note that the UTXO is just a list.  Things are added to that list, and removed from that list, but that doesn't have any effect on what is in the transaction.  They transaction remains static exactly the way it was created with its own lists of inputs and outputs.

My point was that the term "input" was just a place holder to describe an unspent output.
So when I gather multiple inputs to make a new tx with 1 output, I am really gathering
multiple unspent outputs and making a new tx to spend them to another address where the
process could begin again if I controlled that new address.

The term "input" is not just a place holder. An input is an actual list item in a transaction.  A transaction consists of two lists.  The list of inputs is a list of references to transaction outputs.  It is part of the transaction and it doesn't go away. It is permanently stored in the blockchain when the transaction is confirmed.

Do inputs exist within a UTXO?

No.  Inputs are not listed in the UTXO.  Additionally, outputs are removed from the UTXO when they are referenced by an input.

Are you saying UTXOs are unspent inputs then?

Inputs are never spent or unspent.  Their existence is what causes an output to be removed from the UTXO.

I guess I really do not understand.

Like I said earlier,
"Bitcoin doesn't work the way a lot of people expect it to, and when they are first exposed to the whole input and output concept it can be confusing."

Edit: After a superficial search, I understand why I am incorrect. When I am using the
term "input" I am only referring to the "coins" associated with the prior tx, when in reality an
"input" contains an "outpoint", a "signature script", and a "sequence number". I assumed they
were separate and part of the full tx and not part of the input/output aspects of the tx.

There are not "coins".  There are ONLY inputs and outputs in a ledger.

Ultimately, I need to read a well written book on Bitcoin for Novices and really learn everything
from the beginning and properly.

I'm happy to discuss it with you.  The first step is to unlern what you think you know and understand that from a technical standpoint:

"coins" are an abstraction for us humans. They do not actually exist at transaction and blockchain level.

"addresses" are an abstraction for us humans. They do not actually exist at transaction and blockchain level.

A transaction is really just 2 lists. There isn't much more to it. The first list is called "inputs", and is a list of outputs of other transactions along with signatures that satisfy the encumbrance of those outputs.  The sum of the values of those outputs is the total value available for the transaction to use in its own outputs. The second list is called "outputs" and is a list of new unspent outputs each encumbered with a requirement (typically a signature requirement). The sum of the values of those outputs is not allowed to exceed the total value supplied by the inputs.

I am really impressed with you, you are a member legend, and the information you provide is really helpful, I feel like you. You give the answer to all the above comments, I feel very surprised. This forum needs more than members like you. The information is really useful, I thank you. The currency and address is actually an abstract, the true nature of which is a sequence of interrelated and marked data that does not really exist. There are inputs and there is always output, that is a transaction, they have a close relationship and have equal value. No difference is allowed. That's what I can understand from your information.
DannyHamilton
Legendary
*
Offline Offline

Activity: 3388
Merit: 4616



View Profile
March 28, 2017, 01:38:51 PM
 #10

- snip -
There are inputs and there is always output, that is a transaction, they have a close relationship and have equal value. No difference is allowed. That's what I can understand from your information.

This is not true.

The sum of the values associated with the outputs is not allowed to be greater than the sum of the value provided by the inputs.

However, the sum of the values associated with the outputs IS allowed to be less than the sum of the value provided by the inputs.  When this is so, the extra value is a "transaction fee" which the miner can include in the output of their coinbase transaction when they include the transaction in their block.

At the block level, the sum of all the outputs of all the transactions in the block is not allowed to be greater than the current block subsidy plus the value provided by all the inputs of all the transactions in the block. (It is allowed to be less than, but why would a miner pay themselves less than they are allowed to?).
1t0ph20 (OP)
Newbie
*
Offline Offline

Activity: 16
Merit: 7


View Profile
March 28, 2017, 04:22:30 PM
 #11

I truly appreciate the reply's and overall discussion. As mentioned, I'm new to Bitcoin (only been researching for about 2 months) but what I've come to realize every day is there's more and more misconceptions of how things actually work.

You can come across a concept at a high level and it seems fine, but then when you get down to the nuts and bolts the details are harder to track down. But that is why it is important to have these kinds of discussion to break this flow of misconstrued understanding that has probably been built upon something we read somewhere and presumed to be true.



AgentofCoin
Legendary
*
Offline Offline

Activity: 1092
Merit: 1001



View Profile
March 28, 2017, 07:14:39 PM
 #12

I guess I just don't understand then.
So you are saying that even before a tx receives a confirmation, it is "added" to the UTXO set.
There are no transactions in the UTXO.  Only outputs.
Even before a transation receives a confirmation, the outputs that are referenced by the inputs are REMOVED from the UTXO, and the new outputs are ADDED to the UTXO.

So if the mempool is flooded with millions of spammy valid txs, but they pay an appropriate fee, in
theory, there would be an increase of millions of new UTXOs within the UTXO set for at least 72 hours?
Every one of those transactions also spends outputs that used to be in the UTXO.  Those spent outputs are removed.

So the data in the UTXO set are not always valid unspent transactions?
They are valid unspent outputs.  They can be used in another transaction.

Furthermore, the only way a "spent transaction output" becomes an unspent output is if the transaction that spent it either becomes invalid or is dropped from the mempool to free up space for newer transactions. It doesn't become an unspent output when it gets included in the blockchain. It remains spent.
Don't you mean the prior tx remains spent, and it child is unspent, but is already in the UTXO set?
Transactions don't get spent.  Outputs do.  The spent output remains spent.  The new confirmed transaction has its own outputs. Those outputs were unspent and in the UTXO before the transaction got confirmed. They remain unspent and in the UTXO when the transaction is confirmed.  Confirming the transaction doesn't change that.

So, all broadcasted but unconfirmed txs are all already in the UTXO set and thus are only removed
from the UTXO set if the spending tx was invalidated or dropped from the mempool?
The outputs of those unconfirmed transactions are in the UTXO.  The outputs that are spent by the inputs are removed from the UTXO.
The outputs that are added to the UTXO are removed from the UTXO if the transaction is invalidated OR if the transaction is dropped from the mempool OR if the outputs are spent by another transaction.

The UTXO set assumes all broadcast txs are valid until told otherwise.
Transactions are validated before being added to the UTXO.  If the transaction is invalid, then it won't be added to the mempool or the UTXO.  However, a transaction that was previously valid can become invalid. In that case the now invalid transaction is removed from the mempool and its outputs are removed from the UTXO (and depending on why it became invalid, some of the outputs spent by its inputs may be added back into the UTXO.

What I thought was that: The inputs were the spent outputs from it's parent tx. When an
address has valid inputs that are unmoved, the address is an "unspent output". Thus the UTXO term.
So metaphorically, the input only exists to validate spent outputs that are being transformed into
unspent outputs within a new address for example. Coinbase's would be the only true input in my
thinking, since they "originate" outside of the blockchain. This was my reasoning.

I thought outputs were moving from a state of unspent to spent continuously.
Inputs were just another representation for of an unspent output.
Well, to start with, there are no addresses.  Those are an abstraction that our wallets and block explorers create for us humans to make it easier to talk about the transfer of control over value. They don't exist in transactions or the blockchain.
Next, inputs never "move".  They are listed in a transaction and they remain there.
Finally, coinbase transactions have outputs, but never have inputs.

Yes, I understand now.
My issue was that not only was I overall incorrect in my understanding, but also that my
termonology and vocabulary was preventing me from articulating my intention to you. For
example, which you state, "coins" and "addresses" are abstractions and do not exist within
the code for functioning, but are representations for convenience and use. The problem is
that I am aware that they do not exist, but since I am a laymen with all of this in general,
when I use "address" and "coin" it is due to me not knowing the proper terms, so I refer
to the abstraction so you can determine what I am intending to say. But ultimately, that
leads to increased confusion and misunderstandings. Majority of the time, the proper term
I should have been using was "input", and at other times, "output".


Here's a good analogy:

==========
Transaction A (Coinbase)
Inputs:
        none
Outputs:
        Value:1250000000   Encumbrance: Signature required from private key associated with public key Z
==========

("Transaction A, first output", added to UTXO)

==========
Transaction B
Inputs:
        Transaction A, first output, signature from valid private key
Outputs:
        Value 1000000000  Encumbrance: Signature required from private key associated with public key Y
        Value 0240000000  Encumbrance: Signature required from private key associated with public key X

==========

("Transaction A, first output", removed from UTXO
 "Transaction B, first output", added to UTXO
 "Transaction B, second output", added to UTXO
)

==========
Transaction C
Inputs:
        Transaction B, first output, signature from valid private key.
Outputs:
        Value 0500000000  Encumbrance: Signature required from private key associated with public key W
        Value 0400000000  Encumbrance: Signature required from private key associated with public key V

==========

("Transaction B, first output", removed from UTXO
 "Transaction C, first output", added to UTXO
 "Transaction C, second output", added to UTXO
)

==========
Invalid transaction D
Inputs:
        Transaction A, first output, signature from valid private key
Outputs:
        Value 0500000000  Encumbrance: Signature required from private key associated with public key U
        Value 0740000000  Encumbrance: Signature required from private key associated with public key T
==========

(Transaction D rejected because Transaction A, first output is not in the UTXO, it was removed by Transaction B)

==========
Transaction E
Inputs:
        Transaction B, second output, valid signature
        Transaction C, first output, valid signature
Outputs:
        Value 0730000000 Encumbrance: Signature required from private key associated with public key S
==========

("Transaction B, second output", removed from UTXO
 "Transaction C, first output", removed to UTXO
 "Transaction E, first output", added to UTXO
)


So at this point the UTXO lists:
  • "Transaction C, second output"
  • "Transaction E, first output"

If transaction E takes too long to confirm and is removed from the mempool then "Transaction E, first output" will be removed from the UTXO list.  "Transaction B, second output" and "Transaction C, first output" will both be added back into the UTXO list.

In that case, the UTXO list would revert to:
  • "Transaction B, second output"
  • "Transaction C, first output"
  • "Transaction C, second output"

Note that the UTXO is just a list.  Things are added to that list, and removed from that list, but that doesn't have any effect on what is in the transaction.  They transaction remains static exactly the way it was created with its own lists of inputs and outputs.

This visual representation is great.
If you didn't create that, I would still be reading all your words very slowly over and
over attempting to understanding the procedure properly.

I can see all the different parts, their individual importance, and how it reacts to
certain events or non-events.


My point was that the term "input" was just a place holder to describe an unspent output.
So when I gather multiple inputs to make a new tx with 1 output, I am really gathering
multiple unspent outputs and making a new tx to spend them to another address where the
process could begin again if I controlled that new address.
The term "input" is not just a place holder. An input is an actual list item in a transaction.  A transaction consists of two lists.  The list of inputs is a list of references to transaction outputs.  It is part of the transaction and it doesn't go away. It is permanently stored in the blockchain when the transaction is confirmed.

Do inputs exist within a UTXO?
No.  Inputs are not listed in the UTXO.  Additionally, outputs are removed from the UTXO when they are referenced by an input.

Are you saying UTXOs are unspent inputs then?
Inputs are never spent or unspent.  Their existence is what causes an output to be removed from the UTXO.

I guess I really do not understand.
Like I said earlier,
"Bitcoin doesn't work the way a lot of people expect it to, and when they are first exposed to the whole input and output concept it can be confusing."

I understand now. My original understanding was very wrong.


Edit: After a superficial search, I understand why I am incorrect. When I am using the
term "input" I am only referring to the "coins" associated with the prior tx, when in reality an
"input" contains an "outpoint", a "signature script", and a "sequence number". I assumed they
were separate and part of the full tx and not part of the input/output aspects of the tx.
There are no "coins".  There are ONLY inputs and outputs in a ledger.

I always knew that certain terms the community used didn't actually exist within the code,
but I did not understand that it essentially breaks down to the input/output specifically. I always
viewed the input/output only as the ledger's accounting for "this is where the coins originated"
and "this is where the coins are going". Its very wrong, on different levels.

Ultimately, I need to read a well written book on Bitcoin for Novices and really learn everything
from the beginning and properly.
I'm happy to discuss it with you.  The first step is to unlearn what you think you know and then to understand that from a technical standpoint:
"coins" are an abstraction for us humans. They do not actually exist at transaction and blockchain level.
"addresses" are an abstraction for us humans. They do not actually exist at transaction and blockchain level.

A transaction is really just 2 lists. There isn't much more to it. The first list is called "inputs", and is a list of outputs of other transactions along with signatures that satisfy the encumbrance of those outputs.  The sum of the values of those outputs is the total value available for the transaction to use in its own outputs. The second list is called "outputs" and is a list of new unspent outputs each encumbered with a requirement (typically a signature requirement). The sum of the values of those outputs is not allowed to exceed the total value supplied by the inputs.

I fully understand now.
Thank you for taking the time to correct and explain this.

I still need to get a good book about the basics of Bitcoin and really learn about everything else properly.
My misunderstanding centered from assuming input/output was just the ledger entries' status.
Which is either due to my lack of computer programming/science or just my incorrect assumption.

It is probably time for me to start reading books by experts who write for novices.

I support a decentralized & unregulatable ledger first, with safe scaling over time.
Request a signed message if you are associating with anyone claiming to be me.
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!