Bitcoin Forum
May 24, 2024, 06:34:22 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
   Home   Help Search Login Register More  
Pages: [1] 2 3 »  All
Author Topic: Satoshi Client Operation: Overview  (Read 64593 times)
bitrick (OP)
Offline Offline

Activity: 64
Merit: 140

View Profile
September 06, 2011, 03:38:09 AM
Last edit: September 23, 2011, 12:49:48 PM by Gavin Andresen
Merited by ABCbits (22)

Satoshi's Original Bitcoin Client - An Operational View


I thought my client was taking too long to download the block chain and it
did not appear to operate smoothly. I thought I could do something to decrease
the block download time. So I downloaded the code and dug in.  Ultimately,
I failed to find the silver bullet to eliminate the long download delays
(big suprise!). But I did manage to penetrate the C++ code and figure
out how things worked for the most part.

So, I decided to write down my understanding of the code from an operational
perspective, to spare those who are not fluent in C++ from having to wade
through the code, which is quite dense and bit of a chore to pick apart,
when they really just want to know "how it works".

My focus was initially on the block download process, but I decided to
go ahead and cover all the major operational aspects I could (before losing
interest Wink. I do think I found some areas for improvement, but that is not
the point of these articles. I will try to make it clear when I am stating
the facts versus when I am writing commentary.

I intend these articles to go into the Wiki at some point but I also
thought it would be useful to open topics in the forum in order to
allow for review in case I made a mistake or missed something big,
and for reference.


This series of articles will focus on how the Satoshi bitcoin client
program operates, and less so on the protocol details and the rules
for processing blocks and transactions.

Satoshi's bitcoin client is a C++ program, so be sure to look for code in
both the .cpp and the .h header files. Also, the program is multithreaded.
This leads to some complexity and the use of certain code patterns to deal
with concurrency that may be unfamiliar to many programmers. Also, the
code is aggresive in the use of C++ constructs, so it will help to be
fluent with map, multmap, set, string, vector, iostream, and templates.

For information on how the bitcoin protocol works, see:
    The original Satoshi whitepaper:
    The articles on the Wiki:
    With special mention of the protocol specification:
    And the protocol rules:

-- Operations --

The client is oriented around several major operations, including:

    Initialization and Startup
        Upon startup, the client performs various initilization routines
        including starting multiple threads to handle concurrent operations.

    Node Discovery
        The client uses various techniques find out about other bitcoin
        nodes that may exist.

    Node Connectivity
        The client initiates and maintains connections to other nodes.

    Sockets and Messages
        The client processes messages from other nodes and sends
        messages to other nodes using socket connections.
    Block Exchange
        Nodes advertise their inventory of blocks to each other and
        exchange blocks to build block chains.

    Transaction Exchange
        Nodes exchange and relay transactions with each other.
        The client associates transactions with bitcoin addresses in the
        local wallet.

    Wallet Services
        The client can create transactions using the local wallet.
        The client associates transactions with bitcoin addresses in the
        local wallet. The client provides a service for managing
        the local wallet.

    RPC Interface
        The client offers an JSON-RPC interface over HTTP over sockets
        to perform various operational functions and to manage the local

    User Interface
        The user interface code is scheduled to be superseded by bitcoin-qt.
        Therefore, it is not covered in further detail.

See their individual articles for more detail on each of these operations.

-- fClient Mode --

It is worth noting that there is code in the client to allow it to
operate in a mode where it only downloads block headers.
The implementation is intended to be used as a lightweight client mode which
can operate without verifying and storing all blocks and transactions.

This is controlled by the fClient variable in the code which is currently
hard coded to false.  This is currently not considered to be finished code.

This mode is known as fClient mode and the phrase Simplified Payment
Verification (or SPV) mode has also been used to describe a lightweight
client approach.

-- Main Thread Level Functions --


-- Significant Classes By File --

    CNode:  handes one socket connection




    CAlert : CUnsignedAlert

    CWallet : CKeyStore
    CWalletTx : CMerkleTx



    CCriticalSection: used for thread contention

Search on "Satoshi Client Operation" for more articles in this series.

Transaction Exchange :
Block Exchange :
Sockets and Messages :
Node Connectivity :
Node Discovery :
Initialization and Thread Startup :
Sr. Member
Offline Offline

Activity: 266
Merit: 254

View Profile
September 06, 2011, 08:04:35 AM

I'm only part way through 2 of yr articles so far but I just wanna say thanks for an awesome job.  This sort of documentation is exactly what new devs need to get across the client quickly.

Are you planning to publish this series on a site somewhere?  It would be great to have it all in one place where people can bookmark it and find it easily for reference.  It's going to get buried eventually in the forum.

PoolServerJ Home Page - High performance java mining pool engine

Quote from: Matthew N. Wright
Stop wasting the internet.
Pieter Wuille
Offline Offline

Activity: 1072
Merit: 1174

View Profile WWW
September 06, 2011, 08:18:40 AM
Merited by ABCbits (1)

Thank you very much for this. I haven't had the time to go over everything in detail, but from what I've seen, it is correct and well researched. Please make sure this ends up on the wiki - people wanting to contribute need this kind of information.

I do Bitcoin stuff.
Alex Zee
Offline Offline

Activity: 112
Merit: 10

View Profile WWW
September 06, 2011, 09:12:14 AM

I didn't yet read all of this, but it seems like a terrific job! Thanks for doing this!

I suggest putting it somewhere, like a wiki, and adding some images and formatting for better presentation.

BTC Monitor - systray price ticker - new Ripple forum
Sr. Member
Offline Offline

Activity: 310
Merit: 253

View Profile
September 06, 2011, 10:50:09 AM

I haven't read all of this yet either, but it looks like an amazing piece of badly needed documentation. Thank you so much.
Offline Offline

Activity: 1428
Merit: 1093

Core Armory Developer

View Profile WWW
September 06, 2011, 09:02:52 PM
Merited by ABCbits (1)

Bitrick, this is very good information.  It is exactly the kind of information I wanted, as someone who wants to get involved in BTC development but doesn't have the advanced C++ software skills needed to comprehend the Satoshi client code.  I look forward to using this as a secondary reference in my work, and helping iron out the details.

I would recommend you put links to each of the other posts at the bottom of this "Overview" post, and then one of the admins make this thread sticky.  I think people would expect to see this kind of information when they come to the "Development & Technical Discussion" forum,  and right now this is the most concise yet seemingly-complete version of this information I've seen so far.

Nice work!

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
Offline Offline

Activity: 1176
Merit: 1255

May Bitcoin be touched by his Noodly Appendage

View Profile
September 06, 2011, 10:19:14 PM

I don't have the time to read or use these info but they look really awesome -> wiki!

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
Offline Offline

Activity: 103
Merit: 10

View Profile
September 06, 2011, 10:52:00 PM

Excellent stuff!

Are you satoshi?  Roll Eyes

Help Me Help You Donations:
Gavin Andresen
Offline Offline

Activity: 1652
Merit: 2217

Chief Scientist

View Profile WWW
September 06, 2011, 11:25:47 PM
Merited by ABCbits (1)

Very nice bitrick!

By the way, this patch speeds up initial download quite a lot: do these, which have already been pulled into git head:

How often do you get the chance to work on a potentially world-changing project?
bitrick (OP)
Offline Offline

Activity: 64
Merit: 140

View Profile
September 07, 2011, 12:04:39 AM

Thanks Gavin!
The first patch is the most promising. I actually tested turning off VerifySignature last week and calculated around double the block processing rate, when CPU limited. I was hoping for more, but every bit helps.

The last two patches I have less hope for. I know 0.3.23 can disconnect inappropriately, but in my testing any such disconnects should only result in a "brief" stall (brief == a few minutes of a multi-hour process). Note that the client should not disconnect a connection until all incoming and outgoing buffers are drained, so even if there is a disconnect, all the blocks queued prior to that can still get processed, so there should not be totally unproductive ongoing disconnections. (Or did I misunderstand the nature of the 0.3.23 problem?) I did not see a lot of 0.3.23 disconnect delays in my testing but maybe I just got lucky.
Thanks everyone for the nice comments. To reiterate: I do plan on putting this information into the wiki for future reference (and thanks for the suggestion Eto).

p.s. I am not Satoshi.
Sr. Member
Offline Offline

Activity: 322
Merit: 251

FirstBits: 168Bc

View Profile
October 15, 2011, 03:26:49 PM

Thanks Bitrick.

Greenlandic tupilak. Hand carved, traditional cursed bone figures. Sorry, polar bear, walrus and human remains not available for export.
Sr. Member
Offline Offline

Activity: 280
Merit: 250

Firstbits: 12pqwk

View Profile
October 17, 2011, 05:19:43 PM

these are quiet educational, but it will take me days to digest as a newbie programmer.

You can not roll a BitCoin, but you can rollback some. Cheesy
Roll me back: 1NxMkvbYn8o7kKCWPsnWR4FDvH7L9TJqGG
Hero Member
Offline Offline

Activity: 2030
Merit: 501

★★ Play Plinko or Invest!

View Profile WWW
November 03, 2011, 02:15:54 PM

Great job! Grin

[15.00000000 BTC]







Daily Quests
Offline Offline

Activity: 3066
Merit: 1147

The revolution will be monetized!

View Profile
November 03, 2011, 03:29:11 PM

Thanks bitrick! I have not seen this information elsewhere.

The gospel according to Satoshi -
Free bitcoin in ? - Stay tuned for this years Bitcoin hunt!
Hero Member
Offline Offline

Activity: 714
Merit: 500

View Profile
November 04, 2011, 08:03:24 AM
Last edit: November 04, 2011, 08:21:43 AM by finway


Instresting, abandon downloading blockchains from .323 client speed the downloading up.

Offline Offline

Activity: 72
Merit: 10

View Profile
November 08, 2011, 02:10:51 AM

nice job!
Full Member
Offline Offline

Activity: 154
Merit: 101


View Profile WWW
November 29, 2011, 04:25:42 PM


LTC: LMS7SqZJnqzxo76iDSEua33WCyYZdjaQoE
Full Member
Offline Offline

Activity: 238
Merit: 100


View Profile WWW
April 05, 2012, 10:32:52 AM

I find it impressive you were able to synthesize a summary from such a complex set of code.
can into space
Offline Offline

Activity: 3472
Merit: 1721

View Profile
May 16, 2012, 09:14:58 PM

This is some interesting info for me, post your address so I can send a few bitcents ;-)

Signature space available for rent.
Offline Offline

Activity: 980
Merit: 1003

I'm not just any shaman, I'm a Sha256man

View Profile
August 14, 2012, 02:38:23 AM

This reminds me of my Pseudo Bitcoin Client, were the idea is to develop a bitcoin client from scratch in each language to better help others what is happening in the Bitcoin client in their native (programming) lanauge and as well as documentation in Pseudo Code to allow others that only know English(but have a solid understanding of hashing functions) to understand fundamentally whats going on in the official Bitcoin client.

So far I just have Generating Bitcoin addresses in java code.
I would love to expand on it with different language however I don't know how the Bitcoin client works well enough to do it my self, anyways here is the link
Pages: [1] 2 3 »  All
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!