Secure Block Compacting (Updated)

[mcdouglasx]

Strolling through this fascinating thread, I came up with this idea. While I admit I’m not an expert in blockchain, I thought it might be interesting to share it with you, hoping to exchange ideas or simply hear your thoughts.



An optimized approach to enhance Bitcoin’s block efficiency by consolidating redundant data while maintaining compatibility with all transaction types.

Why Optimize Bitcoin Blocks?

Bitcoin’s blockchain grows consistently, presenting scalability challenges. Each block contains redundant data, such as multiple individual signatures, increasing storage requirements and slowing down propagation. This proposal introduces a method to reduce block size by consolidating signatures into a single aggregated signature. The approach improves efficiency without compromising security or transparency and ensures compatibility with transactions that do not explicitly include public keys (PubKey), preserving decentralized operation and independent validation.



Detailed Steps
Step 1: Receiving Transactions

Users:

Submit standard transactions that include:

• PubKey: Public key as proof of ownership.
• Signature: Transaction authorization (can be ECDSA or Schnorr).
• txid: Unique transaction identifier.

Key Benefit: No changes are required in user behavior or tools. Existing wallets and systems remain fully compatible.

Mempool:

Collects and stores validated transactions, ready for inclusion in blocks.

Relevance: Preserves Bitcoin’s current infrastructure.


Step 2: Generation of Optimized Preliminary Block

a. Calculating Ephemeral Public Keys (EPK)

Each transaction generates a unique ephemeral public key:



H: Represents the SHA-256 hash function applied to the concatenated data (PubKey and txid).

G: A fixed elliptic curve generator point ensuring cryptographic validity.


What if there’s no PubKey? For transactions that do not include PubKey (such as P2SH, OP_RETURN, or SegWit), alternative data is used to generate EPKs:




RedeemScript: For P2SH transactions, the redeem script (once revealed) serves as a unique identifier.

WitnessData: For SegWit transactions, witness data is combined with txid.

Purpose: This ensures that all transactions, even those that do not explicitly expose PubKey, can participate in the aggregation process without disruption.


b. Aggregating Ephemeral Keys

Miners aggregate all EPK𝑖 values:



Why aggregate? Aggregation reduces block size, providing a compact representation of all transactions.


c. Creating a Single Schnorr Signature

A single Schnorr signature is calculated using:

Nonce Aggregation: Public nonces Ri from each transaction are combined:




Signature Consolidation: The final s component includes contributions from all transactions:



Message: A representation of the block’s data (e.g., Merkle root).



d. Compact Block Structure

The optimized block contains:

• Header: Metadata like the hash of the previous block, nonce and the Single Schnorr Signature.
• Transactions: Includes the necessary data (such as PubKey, txid, and others).
• Merkle Root: Ensures the integrity of the transactions.

Step 3: Mining and Propagation

Mining Process: The miner performs PoW on the compact block, which propagates faster due to its reduced size.

Node Verification: Each node validates the block by:

• Recomputing EPKagg from the included data.
• Verifying that the Schnorr signature (R,s) matches the aggregated key.
• Verifying that the Pow match.

The aggregated signature is part of the PoW, making any attempt to tamper with transactions or include fake transactions both costly and unfeasible.





Step 4: Audit Period (100 Confirmations)

Audit: During 100 confirmations, Nodes download individual signatures on demand (via BIP 152) and verify:

• That the individual signatures are valid.
• That the single Schnorr signature in the preliminary block was derived correctly.
• That the PoW is valid.

-Nodes that have seen the transactions before their inclusion in the block can retain them in their mempool during the audit.


Step 5: Transition to Immutable Compact Block

After 100 confirmations:

• Individual signatures are discarded.
• Only the compact block is retained, significantly reducing storage requirements.

Step 6: User Transparency and Verification

Verification Process: Users can independently verify that their transaction was included by:

Recalculating EPK



where H is SHA-256 applied to concatenated data.

Validating that EPKi is included in EPKagg via Merkle proofs.

Public Keys are retained in the compact block to allow for decentralized validation.

Why Retain Public Keys in the Compact Block?

• Transparency: Users can verify their participation even after the Individual signatures are discarded.
• Decentralization: Nodes and users can perform independent validation without relying on external systems.

Key Advantages

• Full Compatibility: Works with all transaction types, including those without explicit PubKey.
• Increased Scalability: Compact blocks allow more transactions per block.
• Improved Efficiency: Faster propagation and reduced storage requirements.
• Transparency: Users can verify their transactions independently.
• Enhanced Security: The audit period detects any manipulation before the compact block becomes definitive.

Cost of an Attack

A malicious miner would need to:

• Spend energy to mine a block with a false agg_sig (costly PoW).
• Wait 100 blocks for the audit to detect and invalidate it.
• Lose the reward and face penalties.

Outcome: It is not viable.

[ABCbits]

I only skimmed your idea, but here's my thought.

• Temporary Traditional Block:

-This block maintains Bitcoin’s conventional structure and includes all individual transaction signatures. It is temporarily stored for 100 confirmations, allowing for a full audit to validate the data within the compact block.

Bitcoin node could have choice between storing traditional block temporarily or permanently.

• Optimized Preliminary Block:

-This block eliminates redundant data and replaces it with a single compact Schnorr signature, significantly reducing block size. Its design improves scalability and enables faster network propagation.

Why use two blocks? The traditional block ensures transparency and initial validation, while the optimized block provides efficiency and reduces long-term storage requirements. This dual approach ensures security during the transition to a compact structure.

I have doubt block propagation could be faster since miner initially need to braodcast both traditional and optimized block. Even if a node decide not to accept or verify traditional block, i'm not sure whether optimized block can take advantage of compact block relay[1].

[1] https://bitcoincore.org/en/2016/06/07/compact-blocks-faq/

[mcdouglasx]

Bitcoin node could have choice between storing traditional block temporarily or permanently.

I suppose this could be an option, although if the goal is to reduce costs, only the preliminary block should always be stored.

I have doubt block propagation could be faster since miner initially need to braodcast both traditional and optimized block. Even if a node decide not to accept or verify traditional block, i'm not sure whether optimized block can take advantage of compact block relay.
https://bitcoincore.org/en/2016/06/07/compact-blocks-faq/

You're right, a possible solution could be for nodes to attempt reconstructing the traditional block exclusively using the txids included in the preliminary block. However, they would inevitably face challenges due to differences between mempools. This would complicate ensuring that all the necessary data for the individual signatures is available. Perhaps requesting the missing data on demand seems like a viable solution, but it would also come with its challenges. It would need to be reconsidered and refined further.

edit:

I have updated the idea based on your suggestion, thank you!