An API for returning blocks and transaction with illegal content filtered

[NotATether]

Hi guys, this is a draft which I am going to try to create for the wider Bitcoin community, in particular for anybody who needs to get blockchain data quickly but doesn't want to expose themselves to any potential legal ramifications for transmitting various forms of illegal content that has been embedded on the blockchain.

Right now the definition is very vague, and I am not a lawyer so I'm only going to filter things that are widely accepted to be illegal. (like CP, malware etc.)

The offending content will simply be replaced with a null object, and the RPC request/response stays exactly the same otherwise. This means nobody has to rewrite their software.

There is no proof of concept yet, just this tweet I sent to crypto twitter a few minutes ago

SHA256 checksums of the stack or the witness data will be stored on a centralized server where apps can opt-in to receiving filtered txn data to shield them from legal risks of serving illegal content embedded on the blockchain.

This is something I want to do. Don't hate. Watch.

@adam3us A separate frontend can be made to sanitize Bitcoin Core RPC responses. Specifically, it will replace illegal media with a null object.

This will be designed to be used by all bitcoin applications except for verification software, in order to mitigate legal risks.

I recognize that this development will be quite unpopular with many people, and also that I'm aware that nobody is currently prosecuting node runners or application developers (yet!), however it is going to be quite an interesting research for me to undertake, and I'm also going to post the finished product to the Bitcoin mailing list when I'm done.

[LoyceV]

I read it, and I read it again. I can't see why anyone would want this. When syncing Bitcoin Core, you sync everything. When using an API, I assume you're looking for specific parts, and the parts you're looking for shouldn't be altered. So I see absolutely no use for this.

I'm only going to filter things that are widely accepted to be illegal. (like CP, malware etc.)

Allow me to turn this around: so you're building a database of CP, malware and more, basically highlighting the bad parts inside the blockchain. That sounds much worse than having them lost in large amounts of data.

[ABCbits]

Personally i don't see the point. Even if you use RPC call such as decoderawtransaction, the arbitrary data isn't human readable/viewable. It still requires additional effort from developer to decode and show it properly.

I'm only going to filter things that are widely accepted to be illegal. (like CP, malware etc.)

Allow me to turn this around: so you're building a database of CP, malware and more, basically highlighting the bad parts inside the blockchain. That sounds much worse than having them lost in large amounts of data.

I would say its index/long list to content that deemed illegal.

[Donneski]

I think the main risk here is not just altering data but the precedent it sets. Once you create an API that edits blockchain responses, you’ve basically introduced a middle layer of truth that people have to trust. Bitcoin’s whole foundation is that the data is verifiable and final, no one edits it for you.

If you're creating a service to sanitize for legal reasons, that's fair but it should be clear to the end user that they’re interacting with a filtered view. That way, you keep the trust model intact without creating confusion or accidental reliance on a modified dataset.

[Ferib]

I'm only going to filter things that are widely accepted to be illegal. (like CP, malware etc.)

Allow me to turn this around: so you're building a database of CP, malware and more, basically highlighting the bad parts inside the blockchain. That sounds much worse than having them lost in large amounts of data.

Very interesting, I don't know why but there is a similar post up regarding OP_RETURN about filtering similar data before block propagation (https://bitcointalk.org/index.php?topic=5559355.msg65815592#msg65815592)

Same conclusion there, building a database of "bad" data is just impossible imo. Most social media platforms or data centers have a similar issues with moderating user generated content and I don't think any of those giants have it figured out.

I don't think there is a global consensus on what data is right or wrong, and I think Bitcoin's database should NOT be censored.

[NotATether]

Allow me to turn this around: so you're building a database of CP, malware and more, basically highlighting the bad parts inside the blockchain. That sounds much worse than having them lost in large amounts of data.

I can definitely see this causing a Streisand effect if this goes wrong, yeah. Right now we at least do not know where in the blockchain this data is being stored, if at all.

But a counterargument is that I fear that CEXes and large institutional companies might create their own solution involving a whitelist of transactions and blocks, and apply the same KYC scrutiny that we are seeing now with addresses.

To put it simply, they might invent "taint" for raw transactions and blocks, and then make it really hard to use their services unless you have coins from "clean transactions/blocks".

If this happens it will make the whole of Bitcoin even less pseudonymous.

So this work is to hopefully create a blueprint for avoiding that fate. My version does not censor any transactions, because they can be obtained elsewhere due to the decentralized nature of Bitcoin.

Personally i don't see the point. Even if you use RPC call such as decoderawtransaction, the arbitrary data isn't human readable/viewable. It still requires additional effort from developer to decode and show it properly.

Which would make the developer liable, but there aren't strong legal protections for node runners yet.

I would say its index/long list to content that deemed illegal.

It's going to be a hash table of SHA256 checksums, exactly which part of the transactions or blocks are going to be hashed I haven't figured it out yet.

I think the main risk here is not just altering data but the precedent it sets. Once you create an API that edits blockchain responses, you’ve basically introduced a middle layer of truth that people have to trust. Bitcoin’s whole foundation is that the data is verifiable and final, no one edits it for you.

If you're creating a service to sanitize for legal reasons, that's fair but it should be clear to the end user that they’re interacting with a filtered view. That way, you keep the trust model intact without creating confusion or accidental reliance on a modified dataset.

I specifically said that it's not suitable for use for verifying blockchain data i.e. nodes.

The API specifications will be exactly the same as Bitcoin Core JSON-RPC.

I don't think there is a global consensus on what data is right or wrong, and I think Bitcoin's database should NOT be censored.

I am not censoring blockchain data, I am merely filtering it. This project depends on unfiltered nodes so there will always be many of these kinds of regular nodes running.

[Ferib]

I don't think there is a global consensus on what data is right or wrong, and I think Bitcoin's database should NOT be censored.

I am not censoring blockchain data, I am merely filtering it. This project depends on unfiltered nodes so there will always be many of these kinds of regular nodes running.

So if I understand correctly, you just want to apply a "filter" between the Bitcoin Network and a public interface you host (e.g. mempool.space or some other public app exposed to the blockchain) to moderate "unwanted" transactions... which you define yourself?

I can kinda see your reasoning, I believe a similar issue is known in public exposing IPFS nodes, as they also need to follow legal bullshit (like DMCA takedown requests).

[odolvlobo]

I guess it could be beneficial for companies that don't want their employees accessing certain data. And a person using an app that decodes and displays inscriptions may want to avoid certain ones.

I think you should go for it and see what happens.

I am not censoring blockchain data, I am merely filtering it. This project depends on unfiltered nodes so there will always be many of these kinds of regular nodes running.

But, you are censoring block chain data. If someone asks for it, you may block it. That is the very definition of censorship.

[LoyceV]

To put it simply, they might invent "taint" for raw transactions and blocks

I think you just invented this! I'm not worried a CEX may block funds that were confirmed in block 666 because of whatever reason they come up with. They themselves can't know which block their withdrawals are going to get confirmed in.

[ABCbits]

I would say its index/long list to content that deemed illegal.

It's going to be a hash table of SHA256 checksums, exactly which part of the transactions or blocks are going to be hashed I haven't figured it out yet.

You could start with,
1. Data on OP_RETURN output.
2. Data pushed with certain opcodes (e.g. OP_PUSHDATA).
3. Pubkey on P2MS output. You'll want to read SRC-20 specification.

[NotATether]

I am not censoring blockchain data, I am merely filtering it. This project depends on unfiltered nodes so there will always be many of these kinds of regular nodes running.

But, you are censoring block chain data. If someone asks for it, you may block it. That is the very definition of censorship.

I don't see it that way.

If a government censors sites on the internet (let's say they also censor proxies, VPNs and Tor for the purpose of this discussion), then all ISPs in the country are blocking access to the site. You do not have any way of accessing that site again.

In Bitcoin, the vast majority of providers will still provide unfettered access to all data through their APIs. This is just a small project that prevents you from requesting certain types of data. I can't (and for the record, will not) try to get other people to block said transaction data; that would undermine Bitcoin anyway. So given the circumstances, how is this new API going to be blocking people from getting said data from elsewhere?

I think you just invented this! I'm not worried a CEX may block funds that were confirmed in block 666 because of whatever reason they come up with. They themselves can't know which block their withdrawals are going to get confirmed in.

I invented quite a few things, but to gain mass adoption, they need to be marketed. And I'm a quite bad marketer.

[d5000]

I would develop the tool into another direction, because I second the worries about an "illegal data index".

Instead of manually filtering data you know or suspect that are illegal, you could filter out all "data" transactions. That would not only include all OP_RETURN outputs, but also Ordinals-style Taproot hacks and known protocols for "fake public keys", like Bitcoin Stamps.

Regarding the last point, it seems to be the most difficult one, as you may "catch" some valid P2MS, P2PK etc. transactions without data if your model is not precise enough. But according other posts about this topic there are ways to distinguish about 50% of the fake public keys (those that correspond to valid ECDSA data points) from "real" ones. The rest could be filtered out by using explorers like the Stampchain explorer.

This could later be enhanced into a tool which almost replaces a full node and could be used for alternative clients to sync the blockchain only with financially relevant data, ignoring data txes. To maintain block integrity, you could include the merkle tree of all transactions, including the where you have filtered out "data" outputs and inputs. I think for a real full node replacement an additional proof that the "data" in/outputs would lead to the transaction and block hashes would be necessary, but I guess this should be possible in some way.

This is of course related to the idea I've mentioned in other threads: to make NFTs disappear from nodes in the long run, and thus reduce the economic incentives for NFTs already today.

[ertil]

you could filter out all "data" transactions

Which means filtering literally all transactions. There are already settings for that, you can just receive new blocks, and stop relaying any transactions at all.

you may "catch" some valid P2MS, P2PK etc. transactions without data if your model is not precise enough

Not "may", but "will", because people can store data in their private keys, and use weak signatures, to allow going from public to private key (it would have spamming efficiency of around 1/3 in the worst case, which means wasting 3 MB of block space, to put 1 MB of any payload on-chain; and only non-spammers will process these 3 MB, because spammers can compute everything out of this 1 MB in deterministic way). And it works across all chains, including Grin and Monero.

Also, in case of Monero, it can be used to deanonymize some traffic, if you interact with other users. Then, some user could mix some UTXOs with your coins, and you can break someone else's privacy, if for example 19 out of 20 keys are weak. Then, it is quite clear, who sent what, and where, because all weak keys can be easily detected, and stripped, so the real payment can be revealed to everyone.

The rest could be filtered out by using explorers like the Stampchain explorer.

That's the main problem of spammers: they don't care about "private inscriptions", which are cheaper, and which are as hard to censor, as the regular payment is. Because if users would legitimately want to use Bitcoin to commit to some JPEGs, then they could do so in a better way, than by pushing easily-to-detect, non-consensus data. A single public key can commit to gigabytes, if users would want to. And Ordinals are possible even in a system, where only P2PK is allowed.

This could later be enhanced into a tool which almost replaces a full node and could be used for alternative clients to sync the blockchain only with financially relevant data, ignoring data txes.

In general, you don't have to process anyone's transactions, except your own. As long as you can get external proof, that the chain is valid, and validate it properly, you can just sync the block headers, verify your own transactions, and if you have a proof, that the whole chain is correct, then your transactions also are, as long as they are just included in it.

[NotATether]

I would develop the tool into another direction, because I second the worries about an "illegal data index".

Instead of manually filtering data you know or suspect that are illegal, you could filter out all "data" transactions. That would not only include all OP_RETURN outputs, but also Ordinals-style Taproot hacks and known protocols for "fake public keys", like Bitcoin Stamps.

Regarding the last point, it seems to be the most difficult one, as you may "catch" some valid P2MS, P2PK etc. transactions without data if your model is not precise enough. But according other posts about this topic there are ways to distinguish about 50% of the fake public keys (those that correspond to valid ECDSA data points) from "real" ones. The rest could be filtered out by using explorers like the Stampchain explorer.

This could later be enhanced into a tool which almost replaces a full node and could be used for alternative clients to sync the blockchain only with financially relevant data, ignoring data txes. To maintain block integrity, you could include the merkle tree of all transactions, including the where you have filtered out "data" outputs and inputs. I think for a real full node replacement an additional proof that the "data" in/outputs would lead to the transaction and block hashes would be necessary, but I guess this should be possible in some way.

This is of course related to the idea I've mentioned in other threads: to make NFTs disappear from nodes in the long run, and thus reduce the economic incentives for NFTs already today.

Well, since searching for harmful content inside transactions is going to be manual and time-consuming - and one thing I have not had the luxury of having these days is time! - I am starting to lean towards making a simple filter for automatically excluding OP_RETURN, ordinals and other jpegs.

Of course, it's not going to be 100% accurate this way.

I think it is too complex to identify fake public keys with good accuracy. Besides, these are way too small to carry illegal stuff anyway.

Also I might make it a transparent service - you can send an RPC url to call in HTTP, and my proxy will filter that's output. Then I won't have to worry about node DDoS.

[d5000]

Well, since searching for harmful content inside transactions is going to be manual and time-consuming - and one thing I have not had the luxury of having these days is time! - I am starting to lean towards making a simple filter for automatically excluding OP_RETURN, ordinals and other jpegs.

Good decision.

I think it is too complex to identify fake public keys with good accuracy. Besides, these are way too small to carry illegal stuff anyway.

No, a single "fake public key" transaction can include dozens of outputs, so you can definitely include a photo -- not in extreme high resolution but enough to be potentially problematic.

According to Stampchain's FAQ the "classic stamp" protocol based on OP_MULTISIG can carry up to 7 kB, but a newer format called "OLGA" can store up to 64 kB of data per transaction.

I think the easiest way would be to simply identify the transactions with the Stampchain explorer, or to filter out all those txes following this protocol (besides of the fake public keys they use also a small OP_RETURN string so they are quite easy to identify). Of course other fake public key formats may be less well documented and lack such tools.

[NotATether]

No, a single "fake public key" transaction can include dozens of outputs, so you can definitely include a photo -- not in extreme high resolution but enough to be potentially problematic.

According to Stampchain's FAQ the "classic stamp" protocol based on OP_MULTISIG can carry up to 7 kB, but a newer format called "OLGA" can store up to 64 kB of data per transaction.

I was under the impression that fake public key outputs were just "bitcoin eater" address (Base58 with slightly intelligible english). Is that incorrect?

It sounds like there are a lot more ways to embed data than I thought.

I haven't looked at the Stampchain website, but I'm guessing the data part comes from exposing the output script as part of being spent elsewhere?

[joker_josue]

@NotATether
I consider you someone with good ideas and interesting solutions for the Bitcoin ecosystem. But... I don't think this will be one of them.

I'm starting to get the general feeling that people are worrying too much about content/information that 99% of Bitcoin users don't even know exists, don't even know where it is, and those who know it exists don't even remember where it is. _{(Yes... the percentage value is hypothetical)}

Creating a database with this content does the opposite of what the system itself does indirectly - it leads to oblivion. If the database lists this content, even hiding or omitting those considered illegal, it makes life easier for people to find this content. Imagine someone browsing this database and finding a block with "null" content. This person will be curious and want to know what's actually in that block, which will lead them to search for that content.

This is one of those things you can't erase from the blockchain. But we can let the blockchain's complexity simply make this content fall into oblivion, making it difficult to find. With this comic, you're not making it difficult to find, but rather making it easier.

[alani123]

I read it, and I read it again. I can't see why anyone would want this. When syncing Bitcoin Core, you sync everything. When using an API, I assume you're looking for specific parts, and the parts you're looking for shouldn't be altered. So I see absolutely no use for this.

When bitcoin core 30 is implemented (it's already scheduled to) the OP_RETURN limit being turned off will result to unlimited amounts of data being added in a block and utilizing the witness discount.

Ordinals become obsolete. There will be no more need for stitching together 40 byte inputs through third party tools and external implementations. This means that anyone storing the Blockchain in full could potential have entire images or files of whatever has been uploaded. And yes it's no lie that this can result in harmful files which many parties would be interested to filter.

As if everyone hadn't been warned before this being implementation on core 30. This makes bitcoin prone to sabotage and nodes even more centralised.

[d5000]

I was under the impression that fake public key outputs were just "bitcoin eater" address (Base58 with slightly intelligible english). Is that incorrect?

Yes, that's incorrect. "fake public keys" protocols generate hex strings from random data with the length of a Bitcoin public key. They do not even have to be "correct" in the sense that they "could" be spent. There is no limitation to base58 (Stampchain for example uses base64). You can encode everything in these protocols.

Simplest form is to encode the data with the old P2PK format where the public key is directly stored on the blockchain without hashing it. You can then store 32 bytes per output and transactions can have hundreds of outputs. That has however overhead of a few bytes per output. The classic Stampchain protocol for example uses OP_MULTISIG, i.e. P2MS, where you can store multiple (fake) public keys in a single output reducing the overhead.

I've looked a bit into the OLGA protocol description and it seems to use P2WSH and a more efficient encoding than Base64. See the documentation here.

I haven't looked at the Stampchain website, but I'm guessing the data part comes from exposing the output script as part of being spent elsewhere?

As far as I understand it there's no need to spend it (like it occurred with Ordinals) and it can't also be spent (because the key is "fake"). The data is stored directly in the outputs. Quite simple actually ...

[NotATether]

@NotATether
I consider you someone with good ideas and interesting solutions for the Bitcoin ecosystem. But... I don't think this will be one of them.

I'm starting to get the general feeling that people are worrying too much about content/information that 99% of Bitcoin users don't even know exists, don't even know where it is, and those who know it exists don't even remember where it is. _{(Yes... the percentage value is hypothetical)}

Creating a database with this content does the opposite of what the system itself does indirectly - it leads to oblivion. If the database lists this content, even hiding or omitting those considered illegal, it makes life easier for people to find this content. Imagine someone browsing this database and finding a block with "null" content. This person will be curious and want to know what's actually in that block, which will lead them to search for that content.

This is one of those things you can't erase from the blockchain. But we can let the blockchain's complexity simply make this content fall into oblivion, making it difficult to find. With this comic, you're not making it difficult to find, but rather making it easier.

Perhaps existing solutions can be leveraged.

BIP 157 light clients already exclude the spam from their filters by the very nature that they cannot be spent, so the more people who create BIP 157 clients and associated wallet software that removes such unspendable outputs automatically, the less of a problem unlimited OP_RETURN will be.

There are already some light clients out there such as Wasabi and any LN wallet that uses Neutrino, so all we'd have to do is make the SPV method popular again.

Spam will never be filtered from full nodes which means the storage will always be there but network traffic will decrease slightly with this method. SPV should be the default mode of accessing blockchain data.

When bitcoin core 30 is implemented (it's already scheduled to) the OP_RETURN limit being turned off will result to unlimited amounts of data being added in a block and utilizing the witness discount.

Ordinals become obsolete. There will be no more need for stitching together 40 byte inputs through third party tools and external implementations. This means that anyone storing the Blockchain in full could potential have entire images or files of whatever has been uploaded. And yes it's no lie that this can result in harmful files which many parties would be interested to filter.

As if everyone hadn't been warned before this being implementation on core 30. This makes bitcoin prone to sabotage and nodes even more centralised.

This means that the market value of Ordinals, Runes, BRC-20 and other hacks nosedive to zero.