[INFO - DISCUSSION] Compact Block Filter (BIP158)

[cygan]

with 'compact block filters' another topic follows today, for which i would like to create and open this thread. as usual, i would also like to upload the corresponding slides and present them to you.
this proposal with the tag (bip158) was created on 2017-05-24 and if i am not mistaken only implemented in november 2019 in the Bitcoin core 0.19 version
bip158 was the replacement for the 'bloom filters' (bip37), which were disabled in the same bc 0.19 release.

This BIP describes a structure for compact filters on block data, for use in the BIP 157 light client protocol[1]. The filter construction proposed is an alternative to Bloom filters, as used in BIP 37, that minimizes filter size by using Golomb-Rice coding for compression. This document specifies one initial filter type based on this construction that enables basic wallets and applications with more advanced smart contracts.

https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki



https://twitter.com/BTCillustrated

[1714771224]

1714771224

[1714771224]

1714771224

[Kruw]

Good topic.  Although I recommend everyone run their own fully validating node, filtered nodes are at least the right way to do a light client.  Privacy can be improved even further by using a different Tor identity to download each block from the P2P network so serving nodes cannot build a list of blocks a particular entity is interested in (even if that list includes false positives).

[BlackHatCoiner]

Great work cygan. You should make a thread wherein you list every single of these animated slides.

Compact block filters is the private way to go for the average user. It neither takes much time to synchronize. But, I have a question: do we have any data suggesting that the full node can de-anonymize the user the more the blocks they request? Or perhaps there is lack of privacy protection when the user's transaction is included in a block with very few transactions? I presume that for the former, if you request more blocks, the full node could analyze transactions that share blockchain connection.

[cygan]

✂️
And while we're at it, does anyone know which full node software support BIP 158 (aside from Bitcoin Core, Bitcoin knots, bcoin and btcd)?

unfortunately i can't find a listing of wallets/full node software where bip158 might be integrated either

Great work cygan. You should make a thread wherein you list every single of these animated slides.
✂️

thank you
i'm already doing that by opening a separate thread for each topic and then uploading the slides

[pooya87]

✂️
And while we're at it, does anyone know which full node software support BIP 158 (aside from Bitcoin Core, Bitcoin knots, bcoin and btcd)?

unfortunately i can't find a listing of wallets/full node software where bip158 might be integrated either

Proposals that remain in the "draft" status and are incomplete such as this BIP are usually not implemented by any of the Bitcoin softwares out there.

By the way BIP158 should not be confused with the completed, accepted and implemented BIP-152 which is compact block format and filters used by full nodes when they relay new blocks to each other. Your title should include the words "for Light Clients" to avoid that confusion.

[NotATether]

Why can't we have a variant of compact block filters but only filter the transactions that are associated a given address?

I'm pretty sure the initial calculation will be quite resource-intensive, but then you'd be able to save lots of time when importing addresses inside a wallet or calling things like scantxoutset, instead of having to check each transaction inputs/outputs and fetch raw transactions and so on.

[pooya87]

Which is weird since Bitcoin Core along with few other software already implement it. I wonder whether developer simply forget or don't bother to bump the status.

Are you sure that you are not talking about BIP152 instead of 158? Otherwise can you point to the part of the code in core because I can't find it.

[NotATether]

Which is weird since Bitcoin Core along with few other software already implement it. I wonder whether developer simply forget or don't bother to bump the status.

Are you sure that you are not talking about BIP152 instead of 158? Otherwise can you point to the part of the code in core because I can't find it.

It's probably here: https://github.com/bitcoin/bitcoin/pull/14121

Some context:

Basic BIP158 support merged in Bitcoin Core: with the merge of a PR by Jim Posen into Bitcoin Core’s master development branch, users can now enable a new blockfilterindex configuration option (defaults to off) that will generate a BIP158 compact block filter for each block on the chain plus its corresponding filter header needed for BIP157 support.1 This will operate in the background while the program otherwise continues functioning normally, taking about one to three hours on most computers. The user can then retrieve the filter for a specific block using the new getblockfilter RPC. Filters for the entire block chain currently use about 4 gigabytes.

[satscraper]

I would refer to the second slide from part 1 of OP's set. One could get it context in wrong way by thinking that after BIP 158 light clients do not utilize Bloom filters. But, AFAIK, so far, only Wasabi takes advantage of  compact block filters when working in turbosynch mode. Correct me if I'm wrong and there are other light clients that use such BIP 158 technique.