Searching for OP_return data

[fillippone]

I want to search for data in OP_Returns.

I want to look for text in OP_Return data and see in which transaction is a text like "Chancellor" or "fillippone" or any other text in the OP_Return data.

I know some websites were allowing that, like preturnio.com or blockchair.com or others, but for various reasons (offline, API only), I haven't been able to get use of them.


I have a Windows-based Bitcoin node as well (my Raspberry node is currently offline).
Is there any step-by-step guide for this?

Thanks!

[satscraper]

Try opreturn.net, probably it would help you as it may deliver searching  results from the content of OP_Return on a few relevant  networks including bitcoin. I have made a few searches with positive outcomes.

BTW,  you may use this site to save your  OP_Return message on Dogecoin blockchain for free.

[fillippone]

Try opreturn.net, probably it would help you as it may deliver searching  results from the content of OP_Return on a few relevant  networks including bitcoin.

Thanks for pointing me to this website that actually never popped out in various research I have done so far.
Actually, searching  for OP_returns in the Bitcoin Blockchain seems disabled.
Can you explain, like I'm 5, how to search the word "brink" on the Bitcoin blockchain using I-return.net?

[satscraper]

Try opreturn.net, probably it would help you as it may deliver searching  results from the content of OP_Return on a few relevant  networks including bitcoin.

Thanks for pointing me to this website that actually never popped out in various research I have done so far.
Actually, searching  for OP_returns in the Bitcoin Blockchain seems disabled.
Can you explain, like I'm 5, how to search the word "brink" on the Bitcoin blockchain using I-return.net?

Oh. it seems that the connection to btc is disabled at the moment, which is very strange for me. This site is among my bookmarks and last time i have used it ^{(around half a year ago)} this connection was active. For the search particular word you had to input it into search field on the front page but right now  such action results in the following message:  

Probably, some temporary problems with their node, who knows, should check it in few days.

[ABCbits]

I know some websites were allowing that, like preturnio.com or blockchair.com or others, but for various reasons (offline, API only), I haven't been able to get use of them.

preturnio was a great website, which also index Ordinal data. But it seems to be discounted due to high operational cost and lack of interest.

I have a Windows-based Bitcoin node as well (my Raspberry node is currently offline).
Is there any step-by-step guide for this?

I've read and try few lightweight self-hosted block explorer, but none of them index OP_RETURN. At best, you could try using unpopular GitHub project such as https://github.com/sripwoud/bitcoin-opreturn-indexer which seems to index OP_RETURN data, where you can query it using hex data.

[paid2]

It's sad that Preturnio is now offline :  Project: Preturnio - a full text search engine for Bitcoin

It was a nice tool, especially to search for OP_RETURN data.

[Ambatman]

I want to search for data in OP_Returns.

I want to look for text in OP_Return data and see in which transaction is a text like "Chancellor" or "fillippone" or any other text in the OP_Return data.

I know some websites were allowing that, like preturnio.com or blockchair.com or others, but for various reasons (offline, API only), I haven't been able to get use of them.


I have a Windows-based Bitcoin node as well (my Raspberry node is currently offline).
Is there any step-by-step guide for this?

Thanks!

Not quite easy to find a site that assist in finding specific text in an OP return without API.
Your best bet would be using your node but I guess there's no manual for it.
The chancellor,  was the message on the Genesis block by satoshi
https://btc.com/btc/block/0

The Times 03/Jan/2009 Chancellor on brink of second bailout for banks

[nc50lc]

I want to look for text in OP_Return data and see in which transaction is a text like "Chancellor" or "fillippone" or any other text in the OP_Return data.

I know one site that shows every plain text in each of blkxxxx.dat files but it'll be a manual process (checking each blk file then CTRL+F to find a string).
Here's the site: https://bitcoinstrings.com/

I have a Windows-based Bitcoin node as well (my Raspberry node is currently offline).
Is there any step-by-step guide for this?

Not with Bitcoin Core alone since it has no RPC command that can parse OP_Return data into plain text.
The closest is getrawtransaction true or decodetransaction to check for "OP_Return" outputs and convert hex into text. But that's not what you need.

[amaclin1]

It is not very difficult to write a parser of blk*.dat files

Code:

#include "Bitcoin.h"

BlockChain::BlockChain ( QObject* parent ) : QFile ( parent ), blkFile ( START_BLOCK )
{
  QTimer::singleShot ( 0, this, SLOT ( start ( ) ) );
}

void BlockChain::start ( )
{
  setFileName ( blkFileName ( blkFile++ ) );
  if ( ( END_BLOCK >= 0 ) && ( blkFile == END_BLOCK ) )
  {
    _trace ( QString ( "done [%1]" ).arg ( fileName ( ) ) );
    Chainer ( ).block ( QByteArray ( ), blkFile - 1 );
    deleteLater ( );
  }
  else if ( !open ( QIODevice::ReadOnly ) )
  {
    _trace ( QString ( "cant open [%1]" ).arg ( fileName ( ) ) );
    Chainer ( ).block ( QByteArray ( ), blkFile - 1 );
    deleteLater ( );
  }
  else
  {
    _trace ( QString ( "processing [%1] defu=%2/%3 uxto=%4 sum=%5" ).arg ( fileName ( ) )
                 .arg ( DefUnknown ( ).economy ( ) ).arg ( DefUnknown ( ).size ( ) )
                 .arg ( ScriptResolver ( ).size ( ) )
                 .arg ( Util::getAmount ( MyHash ( ).getUxToAmount ( ) ) ) );
    QTimer::singleShot ( 0, this, SLOT ( next ( ) ) );
  }
}

void BlockChain::next ( )
{
  if ( pos ( ) < size ( ) )
  {
    quint32 magic;
    quint32 sz ( read ( (char*)&magic, 4 ) );
    while ( !magic && pos ( ) < size ( ) - 4 )
      read ( (char*)&magic, 4 );
    xassert ( ( ( magic == MAGIC_ID ) || !magic ) && ( sz == 4 ) )
    if ( magic )
    {
      read ( (char*)&sz, 4 );
      Chainer ( ).block ( read ( sz & 0x07FFFFFFFuLL ), blkFile - 1 );
      QTimer::singleShot ( 0, this, SLOT ( next ( ) ) );
      return;
    }
  }
  close ( );
  QTimer::singleShot ( 0, this, SLOT ( start ( ) ) );
}

const QString BlockChain::blkFileName ( const int i ) const
{
  return
    ( i < 10 ) ? QString ( DATA_ROOT "\\blocks\\blk0000%1.dat" ).arg ( i ) :
    ( i < 100 ) ? QString ( DATA_ROOT "\\blocks\\blk000%1.dat" ).arg ( i ) :
    ( i < 1000 ) ? QString ( DATA_ROOT "\\blocks\\blk00%1.dat" ).arg ( i ) :
    QString ( DATA_ROOT "\\blocks\\blk0%1.dat" ).arg ( i );
}

[fillippone]

It is not very difficult to write a parser of blk*.dat files
<...>

I am not very sure on how I shall use it.
I have to run my own node, but then?
Where shall I run that code, and how shall I input the text I want to look for?
As I said, I need clear instructions. I am not the most technical guy, to say the least.

[amaclin1]

I am not very sure on how I shall use it.

This is just a demonstration that parsing blockchain code is simple.
It is not difficult to write your own parser.
I think this is better, than looking for existing solutions

[satscraper]

I am not very sure on how I shall use it.

This is just a demonstration that parsing blockchain code is simple.
It is not difficult to write your own parser.
I think this is better, than looking for existing solutions

Yeah, it is capable to parse the  blocks rather than the relevant transactions ^{(holding OP_return data)} withing each block. Thus it will not help OP ^{(who, avows oneself  to be not tech-savvy guy)} to accomplish his task. Definitely it could be adapted but it requires both some time and skills.   You have the skills rather than time, I understand this.

[mcdouglasx]

I want to search for data in OP_Returns.

I want to look for text in OP_Return data and see in which transaction is a text like "Chancellor" or "fillippone" or any other text in the OP_Return data.

I know some websites were allowing that, like preturnio.com or blockchair.com or others, but for various reasons (offline, API only), I haven't been able to get use of them.


I have a Windows-based Bitcoin node as well (my Raspberry node is currently offline).
Is there any step-by-step guide for this?

Thanks!

This script is to search for specific texts within blockchain blocks, it uses the blockchain API but you can adapt it to your own node for speed, it stores the tx where matches are found.

update: Case Insensitivity: To make the search case-insensitive, you can convert both the text and the value to lowercase before comparing them.

Code:

import requests
import binascii
import json
import logging

logging.basicConfig(level=logging.INFO)

def is_hex(s):
    if isinstance(s, str):
        try:
            int(s, 16)
            return True
        except ValueError:
            return False
    return False

def decode_hex(value):
    try:
        return binascii.unhexlify(value).decode('latin-1', errors='ignore')
    except (binascii.Error, ValueError):
        return value

def search_text_in_block(texts, start_block=0):
    latest_block_url = "https://blockchain.info/latestblock"
    try:
        latest_block_response = requests.get(latest_block_url)
        latest_block_response.raise_for_status()
        latest_block = latest_block_response.json()
        latest_block_height = latest_block['height']
    except requests.RequestException as e:
        logging.error(f"Error fetching latest block: {e}")
        return

    for block_height in range(start_block, latest_block_height + 1):
        logging.info(f"Processing block: {block_height}")
        block_url = f"https://blockchain.info/block-height/{block_height}?format=json"
        try:
            block_response = requests.get(block_url)
            block_response.raise_for_status()
            block_data = block_response.json()
        except requests.RequestException as e:
            logging.error(f"Error fetching block {block_height}: {e}")
            continue

        for block in block_data['blocks']:
            for tx in block['tx']:
                match_found = False
                for key, value in tx.items():
                    if isinstance(value, str):
                        for text in texts:
                            if text.lower() in value.lower():
                                logging.info(f"Found '{text}' in transaction {tx['hash']} in field '{key}' (raw): {value}")
                                match_found = True
                            if is_hex(value) and key != 'hash':
                                decoded_data = decode_hex(value)
                                if text.lower() in decoded_data.lower():
                                    logging.info(f"Found '{text}' in transaction {tx['hash']} in field '{key}' (decoded): {decoded_data}")
                                    match_found = True
                    elif isinstance(value, list):
                        for item in value:
                            if isinstance(item, dict):
                                for sub_key, sub_value in item.items():
                                    if isinstance(sub_value, str):
                                        for text in texts:
                                            if text.lower() in sub_value.lower():
                                                logging.info(f"Found '{text}' in transaction {tx['hash']} in field '{sub_key}' (raw): {sub_value}")
                                                match_found = True
                                            if is_hex(sub_value) and sub_key != 'hash':
                                                decoded_data = decode_hex(sub_value)
                                                if text.lower() in decoded_data.lower():
                                                    logging.info(f"Found '{text}' in transaction {tx['hash']} in field '{sub_key}' (decoded): {decoded_data}")
                                                    match_found = True
                if match_found:
                    decoded_tx = {k: (decode_hex(v) if is_hex(v) and k != 'hash' else v) for k, v in tx.items()}
                    with open(f"{tx['hash']}_decoded.json", "w") as f:
                        json.dump(decoded_tx, f, indent=4)
                    logging.info(f"Transaction {tx['hash']} saved to {tx['hash']}_decoded.json")

search_text_in_block(["Chancellor", "bitcoin", "btc", "free"], start_block=0)

[pooya87]

Use RPC commands like getrawtransaction to fetch transactions, then parse the transaction hex to find OP_Return outputs.

To use getrawtransaction you need to know txids and provide each of them to the command to fetch them. Also by default it is returning txs from mempool not blocks.

Additionally, since according to statoshi.info there has been 1,081,370,642 (1 billion) transactions. Fetching and parsing the transactions to get OP_RETURNs if they existed is a tremendous amount of work, specially if you want to do it through RPC commands.
Reading the blk*.dat files is the most suitable method.

[DaCryptoRaccoon]

With some mods this would do what your looking for.

https://github.com/markrussinovich/btcgraffiti