Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: 0xB10C on May 03, 2020, 02:26:54 PM



Title: Writeups of my Mempool Observations
Post by: 0xB10C on May 03, 2020, 02:26:54 PM
I've recently started to write up some of my observations I had on my Bitcoin Transaction Monitor project (https://mempool.observer/monitor (https://mempool.observer/monitor)).
I write about activity on the network I find interesting and worth sharing.

The first one is about fees-sniping and why transactions are time-locked to the current block height.
I expected something like this when plotting transactions by their arrival time and locktime:

https://b10c.me/data/mo/mo1-locktime-stairs/expected-pattern.png

However, it turned out that some of the transactions I observed were even time-locked to the next block.
These should have normally not been relayed to me.

The full write up can be found here:

The stair-pattern in time-locked Bitcoin transactions - The off-by-one-error that covered another off-by-one-error up - https://b10c.me/mempool-observations/1-locktime-stairs/ (https://b10c.me/mempool-observations/1-locktime-stairs/)

Will post more here if there is interest.


Title: Re: Write ups of my Mempool Observations
Post by: gmaxwell on May 04, 2020, 10:56:39 AM
Nice article! but you messed up: your site called on users to request things--- so I went and opened a half dozen issues. :)


Title: Re: Write ups of my Mempool Observations
Post by: 0xB10C on May 04, 2020, 08:42:48 PM
Nice article! but you messed up: your site called on users to request things--- so I went and opened a half dozen issues. :)

Thanks gmaxwell! I enjoyed reading your ideas. I can't promise that they'll end up as interactive charts somewhere. However, I already had plans to write about some of these, which would at least include a static plot.


Title: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: 0xB10C on May 04, 2020, 09:10:53 PM
I've published a new Mempool Observation today.

It's about the daily withdrawal broadcast of BitMEX. They broadcast multiple megabytes in around two minutes every day.

This has an effect on the whole network: The feerate estimator's estimates, and, as a reaction, the observed average feerate spikes. The minimum feerate for block inclusion is higher for a few hours and the time-to-confirmation sees an increase.

All because their users deposit directly into 3-of-4 P2SH multisig (redeem scripts with four uncompressed public keys, it's 2020...). That's more than 500 bytes per input. I discuss a few ways of reducing the transaction count and size. Most promising seems to use Schnoor + Taproot combined with output batching. With that it could even be possible for them to keep their '3BMex' vanity addresses by iterating a leaf of the merkle tree that's used as tweak until they get a Bech32 vanity they like. (Not saying that's a good idea: the privacy and fungibility benefits from Taproot would be gone.)

Full article is here:

The daily BitMEX broadcast at 13:08 UTC - Using a fingerprint to reason about a footprint - https://b10c.me/mempool-observations/2-bitmex-broadcast-13-utc/

Planning to write up my observations on seasonality next but haven't started yet. Seasonality seems to overlap with what gmaxwell mentioned in one of this issues (https://github.com/0xB10C/memo/issues/66).


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: goatpig on May 05, 2020, 07:47:41 AM
redeem scripts with four uncompressed public keys, it's 2020...

I'm guessing that's part of the vanity prefix and not something they're willing to let go unless offered a better solution.

Quote
With that it could even be possible for them to keep their '3BMex' vanity addresses by iterating a leaf of the merkle tree that's used as tweak until they get a Bech32 vanity they like. (Not saying that's a good idea: the privacy and fungibility benefits from Taproot would be gone.)

Clearly you do not care about privacy and fungibility when you are using a vanity prefix. Merklerizing their script candidates is indeed the proper solution.

Quote
combined with output batching

That's debatable. Obviously they sign all their withdrawals in a single daily trip to the cold storage, and we can infer they then broadcast the whole lot in one go. If the idea is to reduce impact on fee estimation and confirmation time, a trickle of transaction throughout the day would achieve this better than batching outputs, which is a bandaid to blasting the mempool with all your withdrawals in one giant broadcast.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: gmaxwell on May 05, 2020, 02:36:42 PM
I'm guessing that's part of the vanity prefix and not something they're willing to let go unless offered a better solution.
There is no requirement to use uncompressed keys to use vanity addresses-- in fact, vanity addresses can be generated slightly faster with compressed keys.

If their prefix is just '3BMex' that's pretty short and easily found in any case.

Quote
throughout the day would achieve this better than batching outputs, which is a bandaid to blasting the mempool with all your withdrawals in one giant broadcast.
These things are not mutually exclusive, batching into a modest number of outputs per txn could halve the load they place on the network-- and that's true regardless of when they emit the transactions. (doubly so if someone created a two-sided version of the branch and bound search in Bitcoin core that can often result in changeless transactions when run with large wallets)

However, an even more block space-efficient alternative would be to use a non-multisig wallet for user deposits, which are periodically consolidated into a multisig wallet.
Meh. I'm not sure that's great advice. With the normal way that Bitcoin us used, you don't have any idea how much a user will deposit. So, sure, it would be reasonable to send *small* deposits to a less secure wallet, and preferentially use those payments... but then you might also have a user depositing 2000 BTC in one go and then have those funds stolen from the hot system before your sweep moves them.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: goatpig on May 05, 2020, 03:05:57 PM
I'm guessing that's part of the vanity prefix and not something they're willing to let go unless offered a better solution.
There is no requirement to use uncompressed keys to use vanity addresses-- in fact, vanity addresses can be generated slightly faster with compressed keys.

If their prefix is just '3BMex' that's pretty short and easily found in any case.

I was saying this more in the context of "we already have them, why bother?". Of course compressing the keys would make more sense, but how motivated would they be to at least gradually replace their current "fleet" of addresses (which obviously isn't trivial to generate)? I'm guessing there is some form of key ceremony they're not so willing to either modify nor run through (I assume they're not generating these keys deterministically). It would be significantly easier with a script merklerisation scheme where you can separate the vanity search from the public key generation.

I'm trying to play devil's advocate here.

Quote
These things are not mutually exclusive, batching into a modest number of outputs per txn could halve the load they place on the network-- and that's true regardless of when they emit the transactions. (doubly so if someone created a two-sided version of the branch and bound search in Bitcoin core that can often result in changeless transactions when run with large wallets)

Most certainly the optimal solution would lay at some intersection of batching and broadcast delays. Broadcast delays would take them 5 minutes to implement. Batching? Who knows.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: gmaxwell on May 05, 2020, 05:52:15 PM
Most certainly the optimal solution would lay at some intersection of batching and broadcast delays. Broadcast delays would take them 5 minutes to implement. Batching? Who knows.
I don't believe that delaying announcing transactions they just constructed would provide any value to the network. It's better to have them out there and provide people with more complete information about them.

Generally the big barrier to deploying batching is delaying the payment (and learning its txid) in the first place.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: goatpig on May 07, 2020, 07:40:15 AM
I don't believe that delaying announcing transactions they just constructed would provide any value to the network. It's better to have them out there and provide people with more complete information about them.

I would assume they're motivated to cut down on their own tx fees. I'm not going to expect any market actors to act for the network's benefit at the behest of their own.

Granted they could still bloat the mempool in a single batch broadcast with low fees and RBF later where necessary. Delays aren't the only way to cut down on fees.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: gmaxwell on May 07, 2020, 07:56:37 AM
I would assume they're motivated to cut down on their own tx fees.
One challenge with that kind of argument is that their tx fees are probably a rounding error relative to the overall cashflow of their operation, probably making it a low-to-no-priority for them.

In the bygone days when long distance service actually cost companies a lot of money (at least in absolute terms, if not relative to their operation) there were companies that would come in and go through your bills and optimize your service (e.g. by switching carriers, installing dedicated lines to offices you constantly called, getting contracts for special rates to certain locations or bulk discounts, recommending usage changes-- etc.) in exchange for 50% of the savings for the next year.  If they save nothing, you pay nothing.

It was a good business for both the contractor who could make a boatload off their specialized expertise and the company who got costs reduced in an area that wasn't their focus.

I could imagine the same thing existing for bitcoin-- but the security considerations would probably make it a harder sell.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: goatpig on May 07, 2020, 08:50:52 AM
One challenge with that kind of argument is that their tx fees are probably a rounding error relative to the overall cashflow of their operation, probably making it a low-to-no-priority for them.

This is true more often than not in Bitcoin's history, however I heard Coinbase left a sizable part of their revenue to miners in 2017 over their simplistic withdrawal strategy, when 1000 sat/B fees were "common". That was the result of the price runoff from the previous halving *hint*. Better be prepared now, just my 2 cents.

Quote
and the company who got costs reduced in an area that wasn't their focus.

A bit contradictory when one of the main benefit of Bitcoin is bypassing 3rd party custodians. One thing is for sure, the lack of fee pressure will suppress this side of R&D.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: 0xB10C on May 07, 2020, 11:44:19 AM
I would assume they're motivated to cut down on their own tx fees.

BitMEX does not pay the transaction fees. I guess that's part of the reason they still use their tech from 2014 for withdrawals. Quoting from my article:

Quote
The miner fees for withdrawals are not paid by BitMex. They are deducted from the withdrawing users. The transaction feerate does not depend on the input count nor the transaction size. Users choose a miner fee when withdrawing. However, they do not know how many inputs the final withdrawal transaction will spend and thus can not reason about the transaction size and required feerate. All observed transaction fees are a multiple of 10 000 satoshi, which is the minimum step size the withdrawal frontend allows. 10 000 satoshi is the smallest, and most commonly (44%) observed withdrawal fee. It is followed by 100 000 satoshi (30%), 20 000 satoshi (17.5%), and 50 000 satoshi (3%).

Edit to clarify: I think they could be consolidating the user deposits, but might not even be doing that. I haven't done any research there.


I could imagine the same thing existing for bitcoin-- but the security considerations would probably make it a harder sell.

I thought about creating a service similar to what you describe. Reasons I didn't are that exchanges/other services often don't pay the transaction fees themselves, I don't have enough exchange-security experience to not mess anything up (at least I'm not confident and insured enough) and I'd be (at least partially) competing with the Bitcoin Optech guys.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: harding on May 13, 2020, 12:40:27 PM
I thought about creating a service similar to what you describe. Reasons I didn't are that [...]  I'd be (at least partially) competing with the Bitcoin Optech guys.

Please, please, please don't hesitate to compete with us!  Although Optech is technically a for-profit company, we operate like a non-profit, have only modest revenue, and almost all of us are contributing for free (or through salaries paid for by other organizations, like Chaincode or Square Crypto).  We occasionally write about fee management because we think it's important, but there's plenty of other important things we could be doing with our time (both as individuals and as a team).

Early on, we made a decision to focus on producing educational material and holding workshops rather than offering direct consulting---so you (or anyone else) working directly with companies to optimize their spending would be a complimentary service, not a competitive one.


Title: Re: Writeups of my Mempool Observations
Post by: gmaxwell on May 20, 2020, 06:51:51 PM
Here is some likely nonsense which could use a bit of debunking, if someone feels like doing some analysis: https://www.reddit.com/r/Bitcoin/comments/gngglt/blockchaincom_says_they_account_for_a_third_of/


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: 0xB10C on May 21, 2020, 10:43:59 AM
Here is some likely nonsense which could use a bit of debunking, if someone feels like doing some analysis: https://www.reddit.com/r/Bitcoin/comments/gngglt/blockchaincom_says_they_account_for_a_third_of/

Had that on my todo list and moved it up a bit yesterday. However, I don't think their claim of '30% of all transactions' is to far off.

Both their low and high feerate estimatimates as well as what properties their online wallet transactions could and certainly don't have is known. For example:
  • Only spending P2PKH outputs (thus not spending SegWit nor multisig)
  • Not signaling explicit RBF
  • Not setting a locktime
  • BIP-69 compliant (IIRC parts of their wallet are on GitHub, don't have the link handy right now)
  • Not more than two outputs are created (one payment and maybe one change output)

Applying these filters on my site currently draws 33.50% of the 30.000 loaded transactions. A majority of these follow either the low or high feerate estimates. Obviously, there are outliners. https://mempool.observer/monitor/?filter=RT00JmM9MCZkPTImYj0wJmE9MCZ2aT0xJms9MiZnaT0yJnY9MiZuPTImZz0wJnI9MCZzPTAmdD0wJnE9MCZtPTAmcD0wJm89MCZ6PTAmUj0w

https://i.imgur.com/byosQpq.png

I might start writing this up over the weekend, if I have the time.


Title: Re: Writeups of my Mempool Observations
Post by: 0xB10C on May 21, 2020, 10:50:58 AM
Interestingly, currently 4% of all transactions with the above properties are one input - one output P2PKH transactions just above the low and high estimates. https://mempool.observer/monitor/?filter=RT00JmM9MCZkPTImYj0wJmE9MCZ2aT0xJms9MiZnaT0xJnY9MiZuPTImZz0yJnI9MCZzPTAmdD0wJnE9MCZtPTAmcD0wJm89MCZ6PTAmUj0wJmZpPTEmZj0y

https://i.imgur.com/WeHKgVw.png

Wonder what those are...

 


Title: Re: Writeups of my Mempool Observations
Post by: Wind_FURY on May 21, 2020, 11:13:58 AM
Interestingly, currently 4% of all transactions with the above properties are one input - one output P2PKH transactions just above the low and high estimates. https://mempool.observer/monitor/?filter=RT00JmM9MCZkPTImYj0wJmE9MCZ2aT0xJms9MiZnaT0xJnY9MiZuPTImZz0yJnI9MCZzPTAmdD0wJnE9MCZtPTAmcD0wJm89MCZ6PTAmUj0wJmZpPTEmZj0y

https://i.imgur.com/WeHKgVw.png

Wonder what those are...


Those are probably HODLed coins being moved. Are the transactions from Legacy addresses to Segwit addresses? Or have the coins not been moved for years, then moved? That's an indicator of SODL. Hahaha.


Title: Re: New Mempool Observation: The daily BitMEX broadcast at 13:08 UTC
Post by: gmaxwell on May 21, 2020, 03:01:14 PM
Both their low and high feerate estimatimates as well as what properties their online wallet transactions could and certainly don't have is known. For example:
  • Only spending P2PKH outputs (thus not spending SegWit nor multisig)
  • Not signaling explicit RBF
  • Not setting a locktime
  • BIP-69 compliant (IIRC parts of their wallet are on GitHub, don't have the link handy right now)
  • Not more than two outputs are created (one payment and maybe one change output)

Maybe also "full length signatures" -- not doing the nonce grinding to reduce the signature size the bitcoin core does? Though that won't filter much except on txn that have many inputs and BIP-69 will probably already eliminate bitcoin core for those.

If there are two outputs, at least one of them will be P2PKH.

One other thing to consider is that there is likely a lot of fee sensitive variance in transaction origins. I wouldn't be too surprised if e.g. exchanges stopped aggregating behaviour during high fee periods, increasing the share of transactions from end user wallets, particularly ones that were more fee-blind. Working off mempool data is a little limiting for that reason.


Title: Following the Blockchain.com feerate recommendations
Post by: 0xB10C on July 13, 2020, 12:02:51 PM
I finally came around to finish the Blockchaincom wallet write-up.

The Blockchaincom wallets follow the feerate recommendations of the Blockchaincom API closely (or over/underpay by a fixed percentage due to bugs in the wallets). This allows to identify a majority of the Blockchaincom transactions by combining the fingerprint and the feerate recommendation. I still believe the reported network share of one-third of all transactions is reasonably accurate.

The full article is here:

Following the Blockchain.com feerate recommendations - Wallet fingerprinting nearly a third of all Bitcoin transactions - https://b10c.me/mempool-observations/3-blockchaincom-recommendations/ (https://b10c.me/mempool-observations/3-blockchaincom-recommendations/)

Feedback, ideas, and potential future topics to add to my already long list are welcome. However, I'll be busy with other things in the coming weeks.


Title: Re: Writeups of my Mempool Observations
Post by: DougM on July 14, 2020, 01:37:45 PM
Thanks for sharing..nice collection of articles on your website!  Most were over my newbie head still, but I enjoyed your timeline https://b10c.me/projects/bitcoin-dev-history/ 


Title: Re: Writeups of my Mempool Observations
Post by: TheArchaeologist on July 14, 2020, 06:50:36 PM
Thanks for sharing. Enjoyed this very thorough write-up :)

Keep up the good work :)


Title: Re: Following the Blockchain.com feerate recommendations
Post by: TheArchaeologist on July 15, 2020, 06:22:07 AM
I finally came around to finish the Blockchaincom wallet write-up.

The Blockchaincom wallets follow the feerate recommendations of the Blockchaincom API closely (or over/underpay by a fixed percentage due to bugs in the wallets). This allows to identify a majority of the Blockchaincom transactions by combining the fingerprint and the feerate recommendation. I still believe the reported network share of one-third of all transactions is reasonably accurate.

The full article is here:

Following the Blockchain.com feerate recommendations - Wallet fingerprinting nearly a third of all Bitcoin transactions - https://b10c.me/mempool-observations/3-blockchaincom-recommendations/ (https://b10c.me/mempool-observations/3-blockchaincom-recommendations/)

Feedback, ideas, and potential future topics to add to my already long list are welcome. However, I'll be busy with other things in the coming weeks.
As said in your article the ability to deduct the wallet used for a specific transaction is a data leak. However I also think that if a third of the transactions is using the same Blockchain.com feerate API also centralization is an issue. As user of this wallet you have to trust the estimated fee to be correct. If Blokchain.com or a third party would manipulate the outcome of this API than almost a third of all on-chain transactions could be tampered with.


Title: Re: Following the Blockchain.com feerate recommendations
Post by: 0xB10C on July 15, 2020, 11:08:00 AM
If Blokchain.com or a third party would manipulate the outcome of this API than almost a third of all on-chain transactions could be tampered with.

That's an interesting thought...
In theory, somebody controlling the feerate recommendations could collude with multiple miners and share the additional fee revenue. Thousands of transactions paying a higher feerate would affect other feerate estimators, for example, Bitcoin Core's estimator as well. This would increase the potential revenue even more.   


Title: Re: Following the Blockchain.com feerate recommendations
Post by: gmaxwell on July 15, 2020, 11:02:19 PM
As said in your article the ability to deduct the wallet used for a specific transaction is a data leak. However I also think that if a third of the transactions is using the same Blockchain.com feerate API also centralization is an issue. As user of this wallet you have to trust the estimated fee to be correct. If Blokchain.com or a third party would manipulate the outcome of this API than almost a third of all on-chain transactions could be tampered with.
I don't think it's *that* concerning. If they set the feerate to zero the txn won't confirm.. but they could also just take their service down.

If they set the feerate very high-- well in most cases they could just substitute the JS/app or guess the user's usually-weak password and simply take their coins directly.

Thousands of transactions paying a higher feerate would affect other feerate estimators, for example, Bitcoin Core's estimator as well. This would increase the potential revenue even more.    
Not by much: only to the extent that they increased the fee of transactions that weren't making it into blocks.

E.g. Say you add 1MW/block of transactions paying 0.001 BTC / WU in fees -- this would have the same effect on Core's estimator as if you added 1MW/block of txn paying 0.1 BTC/ WU in fees.  All that matters is that they were high enough to push things out... it's the feerate of the things that were actually delayed (and which were later confirmed) which drive the estimator.

It appears to me that they already bid pretty high too, enough that increasing their high fee would have no effect on what bitcoin-qt pays, and increasing their low fee would do no more than switching their users all from low to high would.