Are we stress testing again?

[Argwai96]

well it looks like we are back to the normal volume, no more people crying about paying higher fees or their confirmations taking more then 4 hours https://blockchain.info/charts/n-transactions i think most of us where able to handle the situation.

Cheers back to normal.

[achow101]

What exactly do you mean by sending node? Do you mean, when I shut down my client the transcation could be lost?
When I use mycelium, I just activate my internet connection for sending the transaction and deactivate it immediately after that, but I still had transaction that where unconfirmed for some hours and got confirmed in the end.

Theoretically your transaction could be lost. Some nodes will rebroadcast transactions, but the only one that reliably does so is the node that created and sent the transaction initially. It will rebroadcast every 30 minutes AFAIK. However, the transaction will stay in the mempools of other nodes for a while before it is either confirmed or dropped.

[Mikestang]

Where is this officially documented?  Provide a link please.  I find it concerning that a transaction sent could be "forgotten" by the network for any reason, that seems like a huge problem.

It is documented in the source code.

Thanks to you and other who replied with this.  Unfortunately I do not speak code, but I have read a couple books and the white paper and such and never saw this mentioned.  Makes me kind of nervous, but I guess nothing to worry about as long as the tx backlog is small.

[2112]

Thanks to you and other who replied with this.  Unfortunately I do not speak code, but I have read a couple books and the white paper and such and never saw this mentioned.  Makes me kind of nervous, but I guess nothing to worry about as long as the tx backlog is small.

It is opposite to "nothing to worry". It is one of the most troublesome features of Bitcoin, it whitewashes any attempts to fraud, double-spend, abuse malleability, etc.

Any real accountant or other financial professional will tell you that "when in doubt, look through the 'errors and omissions' file/account, that's where the truth is". Bitcoin is the opposite, by design it forgets about any attempt of abuse. If you ever wonder "why fraudsters love Bitcoin?", this is the partial answer as to "why?".

I don't remember the specifics, but in the traditional financial system the attempts to "kite a check" or "overdraw" are stored longer than the regular transactions, just because their value as the indicator of the trustworthiness is much better than the numbers of the uncontested transactions. It is something like '10 years' in the 'errors&omissions' versus '1-2-3 years' in the 'regular' file.

Edit:

Speaking in the Bitcoin terms:

The (financial) value of analyzing 'mempool' is significantly higher than analyzing 'blockchain (as stored on the disk for the "longest chain"'.

[sidhujag]

Thanks to you and other who replied with this.  Unfortunately I do not speak code, but I have read a couple books and the white paper and such and never saw this mentioned.  Makes me kind of nervous, but I guess nothing to worry about as long as the tx backlog is small.

It is opposite to "nothing to worry". It is one of the most troublesome features of Bitcoin, it whitewashes any attempts to fraud, double-spend, abuse malleability, etc.

Any real accountant or other financial professional will tell you that "when in doubt, look through the 'errors and omissions' file/account, that's where the truth is". Bitcoin is the opposite, by design it forgets about any attempt of abuse. If you ever wonder "why fraudsters love Bitcoin?", this is the partial answer as to "why?".

I don't remember the specifics, but in the traditional financial system the attempts to "kite a check" or "overdraw" are stored longer than the regular transactions, just because their value as the indicator of the trustworthiness is much better than the numbers of the uncontested transactions. It is something like '10 years' in the 'errors&omissions' versus '1-2-3 years' in the 'regular' file.

Edit:

Speaking in the Bitcoin terms:

The (financial) value of analyzing 'mempool' is significantly higher than analyzing 'blockchain (as stored on the disk for the "longest chain"'.

What do you want it to do? hold tx's in mempool for longer time increasing memory requirements exponentially across the network? The real value of using the system is for VALID transactions. Ofcourse people are free (not literally) to try to fraudulently create bad tx's but that is the beauty of the consensus protocol that doesn't allow these to be entering the network and creates the value of what we call Bitcoin.

In software speak, I suggest we follow KISS which is what it is. A clear seperation of concerns and not worry about holding metadata or increasing tx lengths because of another small use-case of analytically drawing conclusions from that dataset that does not strongly correlate with the true intention of Bitcoin.

[DooMAD]

Thanks to you and other who replied with this.  Unfortunately I do not speak code, but I have read a couple books and the white paper and such and never saw this mentioned.  Makes me kind of nervous, but I guess nothing to worry about as long as the tx backlog is small.

It is opposite to "nothing to worry". It is one of the most troublesome features of Bitcoin, it whitewashes any attempts to fraud, double-spend, abuse malleability, etc.

Any real accountant or other financial professional will tell you that "when in doubt, look through the 'errors and omissions' file/account, that's where the truth is". Bitcoin is the opposite, by design it forgets about any attempt of abuse. If you ever wonder "why fraudsters love Bitcoin?", this is the partial answer as to "why?".

I don't remember the specifics, but in the traditional financial system the attempts to "kite a check" or "overdraw" are stored longer than the regular transactions, just because their value as the indicator of the trustworthiness is much better than the numbers of the uncontested transactions. It is something like '10 years' in the 'errors&omissions' versus '1-2-3 years' in the 'regular' file.

Edit:

Speaking in the Bitcoin terms:

The (financial) value of analyzing 'mempool' is significantly higher than analyzing 'blockchain (as stored on the disk for the "longest chain"'.

What do you want it to do? hold tx's in mempool for longer time increasing memory requirements exponentially across the network? The real value of using the system is for VALID transactions. Ofcourse people are free (not literally) to try to fraudulently create bad tx's but that is the beauty of the consensus protocol that doesn't allow these to be entering the network and creates the value of what we call Bitcoin.

In software speak, I suggest we follow KISS which is what it is. A clear seperation of concerns and not worry about holding metadata or increasing tx lengths because of another small use-case of analytically drawing conclusions from that dataset that does not strongly correlate with the true intention of Bitcoin.

Indeed.  There's a balance to be struck in any such risk/reward scenario.  I can't see how the cost required would outweigh the relatively small benefit of having an extended record of the mempool.  And if we did keep a record of anything that doesn't make it into the blockchain (i.e. errors and omissions) , how do you then make that record decentralised and tamper-proof?  We'd have to have a new blockchain that records what doesn't go into the original blockchain, heh.

[2112]

What do you want it to do? hold tx's in mempool for longer time increasing memory requirements exponentially across the network? The real value of using the system is for VALID transactions. Ofcourse people are free (not literally) to try to fraudulently create bad tx's but that is the beauty of the consensus protocol that doesn't allow these to be entering the network and creates the value of what we call Bitcoin.

In software speak, I suggest we follow KISS which is what it is. A clear seperation of concerns and not worry about holding metadata or increasing tx lengths because of another small use-case of analytically drawing conclusions from that dataset that does not strongly correlate with the true intention of Bitcoin.

Indeed.  There's a balance to be struck in any such risk/reward scenario.  I can't see how the cost required would outweigh the relatively small benefit of having an extended record of the mempool.  And if we did keep a record of anything that doesn't make it into the blockchain (i.e. errors and omissions) , how do you then make that record decentralised and tamper-proof?  We'd have to have a new blockchain that records what doesn't go into the original blockchain, heh.

I kinda knew it that my post is going to attract some code monkeys and their attempts to burn the strawmen. I'm just going to quote them for the future reference.

Anyway, for non code monkeys, the longer explanation is as follows:

There's absolutely no point of storing the unconfirmed transactions in RAM. As any competent software engineer will explain you it is sufficient to store a cache (or map, in CS terms) of transaction hash to single bit, meaning "transaction was seen and it is known to be 'valid' or 'invalid'". The 'valid' transactions will ultimately get promoted to the permanent blockchain, the 'invalid' transactions are mostly of the forensic interest to investigate errors and attempted frauds. The transaction not in cache are neither 'valid' nor 'invalid', they just need the full verification run.

Obviously, some of those code monkeys are in the employ of the fraudsters who explicitly rely on their fraud attempts to get no permanent record on the blockchain. That's why I quoted them above, we'll see what fraud they will try to cook up.

[sidhujag]

What do you want it to do? hold tx's in mempool for longer time increasing memory requirements exponentially across the network? The real value of using the system is for VALID transactions. Ofcourse people are free (not literally) to try to fraudulently create bad tx's but that is the beauty of the consensus protocol that doesn't allow these to be entering the network and creates the value of what we call Bitcoin.

In software speak, I suggest we follow KISS which is what it is. A clear seperation of concerns and not worry about holding metadata or increasing tx lengths because of another small use-case of analytically drawing conclusions from that dataset that does not strongly correlate with the true intention of Bitcoin.

Indeed.  There's a balance to be struck in any such risk/reward scenario.  I can't see how the cost required would outweigh the relatively small benefit of having an extended record of the mempool.  And if we did keep a record of anything that doesn't make it into the blockchain (i.e. errors and omissions) , how do you then make that record decentralised and tamper-proof?  We'd have to have a new blockchain that records what doesn't go into the original blockchain, heh.

I kinda knew it that my post is going to attract some code monkeys and their attempts to burn the strawmen. I'm just going to quote them for the future reference.

Anyway, for non code monkeys, the longer explanation is as follows:

There's absolutely no point of storing the unconfirmed transactions in RAM. As any competent software engineer will explain you it is sufficient to store a cache (or map, in CS terms) of transaction hash to single bit, meaning "transaction was seen and it is known to be 'valid' or 'invalid'". The 'valid' transactions will ultimately get promoted to the permanent blockchain, the 'invalid' transactions are mostly of the forensic interest to investigate errors and attempted frauds. The transaction not in cache are neither 'valid' nor 'invalid', they just need the full verification run.

Obviously, some of those code monkeys are in the employ of the fraudsters who explicitly rely on their fraud attempts to get no permanent record on the blockchain. That's why I quoted them above, we'll see what fraud they will try to cook up.

Great lets exponentially increase bandwidth+storage requirements at 0 cost because fraud attempts knowingly will pay 0 fees and won't get in to a block but will be stored on the harddrives of their victims causing the whole network to shutdown in just a few days.

You don't know what you are talking about. Move along.

[Syke]

Speaking in the Bitcoin terms:

The (financial) value of analyzing 'mempool' is significantly higher than analyzing 'blockchain (as stored on the disk for the "longest chain"'.

The great thing about bitcoin is all that information is public. If you think fraud, double-spend, abuse malleability, etc. are important, you're quite welcome to write a node that analyzes those issues. Sounds like an excellent project.

[turvarya]

What do you want it to do? hold tx's in mempool for longer time increasing memory requirements exponentially across the network? The real value of using the system is for VALID transactions. Ofcourse people are free (not literally) to try to fraudulently create bad tx's but that is the beauty of the consensus protocol that doesn't allow these to be entering the network and creates the value of what we call Bitcoin.

In software speak, I suggest we follow KISS which is what it is. A clear seperation of concerns and not worry about holding metadata or increasing tx lengths because of another small use-case of analytically drawing conclusions from that dataset that does not strongly correlate with the true intention of Bitcoin.

Indeed.  There's a balance to be struck in any such risk/reward scenario.  I can't see how the cost required would outweigh the relatively small benefit of having an extended record of the mempool.  And if we did keep a record of anything that doesn't make it into the blockchain (i.e. errors and omissions) , how do you then make that record decentralised and tamper-proof?  We'd have to have a new blockchain that records what doesn't go into the original blockchain, heh.

I kinda knew it that my post is going to attract some code monkeys and their attempts to burn the strawmen. I'm just going to quote them for the future reference.

Anyway, for non code monkeys, the longer explanation is as follows:

There's absolutely no point of storing the unconfirmed transactions in RAM. As any competent software engineer will explain you it is sufficient to store a cache (or map, in CS terms) of transaction hash to single bit, meaning "transaction was seen and it is known to be 'valid' or 'invalid'". The 'valid' transactions will ultimately get promoted to the permanent blockchain, the 'invalid' transactions are mostly of the forensic interest to investigate errors and attempted frauds. The transaction not in cache are neither 'valid' nor 'invalid', they just need the full verification run.

Obviously, some of those code monkeys are in the employ of the fraudsters who explicitly rely on their fraud attempts to get no permanent record on the blockchain. That's why I quoted them above, we'll see what fraud they will try to cook up.

Great lets exponentially increase bandwidth+storage requirements at 0 cost because fraud attempts knowingly will pay 0 fees and won't get in to a block but will be stored on the harddrives of their victims causing the whole network to shutdown in just a few days.

You don't know what you are talking about. Move along.

I guess not everybody would have to store such transactions, if there is a bureau that wants this data, they can store it them self.
I also don't see the concern about tempered records. You can't make a fake "address A tried to send money to address B"-transaction, since it still would have to be signed with the private key of address A.

[canth]

I kinda knew it that my post is going to attract some code monkeys and their attempts to burn the strawmen. I'm just going to quote them for the future reference.

Anyway, for non code monkeys, the longer explanation is as follows:

There's absolutely no point of storing the unconfirmed transactions in RAM. As any competent software engineer will explain you it is sufficient to store a cache (or map, in CS terms) of transaction hash to single bit, meaning "transaction was seen and it is known to be 'valid' or 'invalid'". The 'valid' transactions will ultimately get promoted to the permanent blockchain, the 'invalid' transactions are mostly of the forensic interest to investigate errors and attempted frauds. The transaction not in cache are neither 'valid' nor 'invalid', they just need the full verification run.

Obviously, some of those code monkeys are in the employ of the fraudsters who explicitly rely on their fraud attempts to get no permanent record on the blockchain. That's why I quoted them above, we'll see what fraud they will try to cook up.

There is a point in storing unconfirmed transactions in RAM. As a miner, nodes are holding those transactions in memory as the basis for forming the next block. Once the block is mined it needs to be broadcast - if the data is on disk then it adds extra latency to retrieve the data from disk before broadcasting it to the network. Yes, SSDs are very fast but for a miner there's no reason to introduce any latency in a system where speed is relevant and contributes to profitability.

[SebastianJu]

Interesting.  So it is potentially possible that BitPay never gets that transaction, and I receive what I purchased for free.

Yes, but the merchant should never ship their products without having at least 1 confirmation.

Though thats not how bitpay works. They only wait until they see the transaction and they have huge turnovers. I wonder two what will happen. They will have huge amounts of coins that dont reach at the moment.

[malykii]

Interesting.  So it is potentially possible that BitPay never gets that transaction, and I receive what I purchased for free.

Yes, but the merchant should never ship their products without having at least 1 confirmation.

Though thats not how bitpay works. They only wait until they see the transaction and they have huge turnovers. I wonder two what will happen. They will have huge amounts of coins that dont reach at the moment.

My transactions still have no confirmations, and nothing has been communicated to me from the merchant or BitPay.  I am going on almost 36 hours since the initial transaction happened. 

[DooMAD]

What do you want it to do? hold tx's in mempool for longer time increasing memory requirements exponentially across the network? The real value of using the system is for VALID transactions. Ofcourse people are free (not literally) to try to fraudulently create bad tx's but that is the beauty of the consensus protocol that doesn't allow these to be entering the network and creates the value of what we call Bitcoin.

In software speak, I suggest we follow KISS which is what it is. A clear seperation of concerns and not worry about holding metadata or increasing tx lengths because of another small use-case of analytically drawing conclusions from that dataset that does not strongly correlate with the true intention of Bitcoin.

Indeed.  There's a balance to be struck in any such risk/reward scenario.  I can't see how the cost required would outweigh the relatively small benefit of having an extended record of the mempool.  And if we did keep a record of anything that doesn't make it into the blockchain (i.e. errors and omissions) , how do you then make that record decentralised and tamper-proof?  We'd have to have a new blockchain that records what doesn't go into the original blockchain, heh.

I kinda knew it that my post is going to attract some code monkeys and their attempts to burn the strawmen. I'm just going to quote them for the future reference.

Anyway, for non code monkeys, the longer explanation is as follows:

There's absolutely no point of storing the unconfirmed transactions in RAM. As any competent software engineer will explain you it is sufficient to store a cache (or map, in CS terms) of transaction hash to single bit, meaning "transaction was seen and it is known to be 'valid' or 'invalid'". The 'valid' transactions will ultimately get promoted to the permanent blockchain, the 'invalid' transactions are mostly of the forensic interest to investigate errors and attempted frauds. The transaction not in cache are neither 'valid' nor 'invalid', they just need the full verification run.

Obviously, some of those code monkeys are in the employ of the fraudsters who explicitly rely on their fraud attempts to get no permanent record on the blockchain. That's why I quoted them above, we'll see what fraud they will try to cook up.

Wow.  Jumping to conclusions much?

a) Not a coder.  Can just about cope with basic HTML and CSS.  
b) Saying "obviously" doesn't make your baseless (and frankly insane) accusations sound any more credible.  
c) If you want people to take you seriously, this isn't how you go about it.  

I'm sorry that someone on the internet disagreed with you because they thought your idea didn't sound practical.  Try not to take it personally.

[Meuh6879]

i have emit (this day !) a transaction with 0,0001 BTC of fees, no problem ... no delay.

[scarsbergholden]

i have emit (this day !) a transaction with 0,0001 BTC of fees, no problem ... no delay.

I have also been able to send a transaction with auto fees with no problem today, most likely we will be in this normal volume of transactions for the comings days since we don't know when the next stress test will begin.

[Meuh6879]

well ... in no stress test pass, the normal fees will be 0,00001 BTC instead of 0,0001 BTC. 
but if it's for the network and is function ... (and strong protection against bankster), i pay for this !

[RoadTrain]

well ... in no stress test pass, the normal fees will be 0,00001 BTC instead of 0,0001 BTC. 
but if it's for the network and is function ... (and strong protection against bankster), i pay for this !

Are you sure about that? Because the defaut fee per kB has been 0,0001 BTC for a long time even before the stress test.

[Meuh6879]

yes, i'm sure ... every month, i create a new wallet from an other wallet to store in cold storage (life economies).

2014 = 0,0001 BTC, so i have include 5 transfer in the amount of each wallet (to not have 99999 numbers).  
2015 = happy, fees pass to 0,00001 (i can execute 50 transfer without 9999 numbers).  
2015 stress test = return to 0,0001 BTC fees, not a big deal but ... it's new.  

coin destroyed on each wallet (40-70 days).

[LFC_Bitcoin]

Unconfirmed transactions still at over 11,000 so this spam fest is continuing then. I really think it's time for miners to reject all transactions carrying no fee. This rubbish will stop then.