PHP - Bidcoind check for new transactions.

[OsNaP]

Hi Guys,

My question is about the "bitcoind listtransactions" method.
I want to have a scalable way of getting all the new "receive" transactions.
This means the only method available would be "bitcoind listtransactions".
Because only that method lets me define a "count" and "from" parameter.

> bitcoind help listtransactions
listtransactions [account] [count=10] [from=0]

So even if i have 100.000 transactions, i could loop through a 100 at a time.
But I don't want to loop over all 100.000 each time i need to check for new transactions.
Only the new transactions are relevant.
This is my main problem.

Ofc, I could stop the loop if I encounter any old transactions.
But this would only be possible if all new transactions would always be added on top of the list.
This wouldnt work if transactions can be added in de middle of the list.

[1715388766]

1715388766

[1715388766]

1715388766

[1715388766]

1715388766

[OsNaP]

Doesn't anyone know if new transactions are added to the list at random places.
Or are they always added on top.

[theymos]

Doesn't anyone know if new transactions are added to the list at random places.
Or are they always added on top.

Transactions aren't reordered within a single wallet, and new transactions are always added to the top, though transactions could be reordered if you move keys between wallets or use -zapwallettxes.

I want to have a scalable way of getting all the new "receive" transactions.

listtransactions loops through every transaction in your wallet, so it isn't very fast or scalable. It's better to use one of these:
- getreceivedbyaccount or getreceivedbyaddress
- -walletnotify + gettransaction
- listsinceblock

Keep in mind that the chain can reorg, which can cause confirmed transactions to have a different number of confirmations or even become invalid. You need to handle this, especially if you use walletnotify or listsinceblock. I know that they're limited and a bit confusing, but I usually recommend bitcoind accounts for small- to medium-size sites because it's very easy to do things correctly with them.

[OsNaP]

listtransactions loops through every transaction in your wallet, so it isn't very fast or scalable.

I could make the "listtransactions" loop scalable by using the "[count=10] [from=0]" parameters.
Right now I loop 50 entries at a time.
And if I see a transaction thats already in my database i stop the loop.

This will make my approach scalable.
But it will only work if all new incoming transactions are added to the top of the list.

Because if any new transaction is added before a known transaction, my approach will skip that transaction.

[theymos]

I could make the "listtransactions" loop scalable by using the "[count=10] [from=0]" parameters.

listtransactions loops through all wallet transactions (internally, in the implementation) no matter what arguments you give it.

[OsNaP]

Ha ok,
So lets say i'll use listsinceblock.
Will new transactions always be added in higher blocks.
Or could there be blocks added before a older block.

Basically my question is, will I loss transactions using "listsinceblock".

[theymos]

New transactions will either be in new blocks or there will be a reorg and the block hash you give to listsinceblock will be outside of the main chain, causing listsinceblock to dump all of your transactions.

[OsNaP]

New transactions will either be in new blocks or there will be a reorg and the block hash you give to listsinceblock will be outside of the main chain, causing listsinceblock to dump all of your transactions.

But dumping all transactions is a problem right?
Like memory.
Is there a way to batch the transactions?
We will be having a lot of transactions.

[theymos]

You could check that the block is in the main chain by using getblock and ensuring that the number of confirmations is non-zero. You need to do something to handle reorgs, though; probably something like bitcoind's CBlockLocator.

walletnotify is also good.

[OsNaP]

So if a block has 0 confirmations its labeled an orphan block.
And listsinceblock will return all my transactions.

I just came up with some concept code.
It fixes the memory problem.
But my concept will only work if reorgs stops happening after a certain amount of confirmations (7).
Could you maybe verify.

Code:

min = 1
max = 7

lastBlockHash = database.lastBlockHash
blockData = bitcoind.getblock(lastBlockHash)

if(blockData.confirmations < min)
{
    lastBlockHash = bitcoind.getblockhash(max)
}

response = bitcoind.listsinceblock(lastBlockHash)

foreach(response.transactions)
{
    ...
}

[01BTC10]

I wonder if this snippet could help:

Code:

#!/usr/bin/env php

/* depends on: 	*/
/* php5-curl 	*/
/* https://github.com/aceat64/EasyBitcoin-PHP 	*/

<?php
require_once 'include/easybitcoin.php';

$min_confirm = 6;
$bitcoinrpc_user = 'bitcoinrpc';
$bitcoinrpc_pass = 'bitcoinrpc_pass';
$bitcoinrpc_host = 'localhost';
$bitcoinrpc_port = '8332'; 

if ($argc != 2 || strlen($argv[1]) != 64)
  {
    echo "usage:$argv[0] tx\n";
    die();
  }

$tx = $argv[1];
$file = "/tmp/".md5($tx);
$lock = fopen($file, "w+");

if (flock($lock, LOCK_EX | LOCK_NB))
  {
    fwrite($lock, $tx);
  }
else
  {
    echo "Process already running for same tx, exiting\n";
    die();
  }

$bitcoin = new Bitcoin($bitcoinrpc_user, $bitcoinrpc_pass, $bitcoinrpc_host, $bitcoinrpc_port);
$bitcoin->gettransaction($tx);

if ($bitcoin->status != 200)
  {
    echo "Error: $bitcoin->error\n";
    die();
  }

if ($bitcoin->response['result']['details'][0]['category'] === "receive")
  {
    while (1)
      {
        $bitcoin->gettransaction($tx);
        $address =$bitcoin->response['result']['details'][0]['address'];
        $amount  =$bitcoin->response['result']['amount'];
        $confirm =$bitcoin->response['result']['confirmations'];

        echo "$address -> $amount confirmations:$confirm\n";

        if ($confirm >= $min_confirm)
          {
            echo "$min_confirm or more confirmations for $tx\n";
            $payment = array("address" => $address, "tx" => $tx, "amount" => $amount);

            /****************************************** Process tx stored in $payment here ******************************************/
            die();
          }

        sleep(30); // Check number of confirmations every 30sec.
      }
  }

?>

The infinite loop has to be modified so it won't loop to infinity when getting a tx from a yet to be orphaned block.

Then you only need to add this line to bitcoin.conf to trigger the script for each new tx that is affecting the wallet:

Code:

walletnotify=/path_to/script.php %s

[OsNaP]

Hi 01BTC10,

Srry, but thats not really helpful.
I wouldn't really recommend using walletnotify.
Because I can't afford to loss or skip any transactions.

Let's say something goes wrong.
Maybe your website is down for some hours.
Bitcoind will still call walletnotify, but your script can't process the transactions.

I would prefer using listsinceblock.
Because you'll get all transactions since your last synced up.