How to understand the Bitcoin Core source code?

[lhrbeijing]

I am a graduate student. My major is software engineering and the direction is blockchain.
I have basic C++ and OO skills, and some computer related knowledge.

But I would like to read the Bitcoin source code roughly. is it possible?
If I want to know how such a huge Bitcoin Core code was designed, what sources can I refer to?

How did the code work as a team when it was written? (Well, just curious.)
If one of the cpp files has thousands of lines of code, was it written by one person? How to ensure that it is bug-free?

Thank you for any reply.

[PawGo]

What exactly you are interested in? Do you want to analyze the code or the way how team worked?
The code is available under https://github.com/bitcoin/bitcoin but I am not sure how old versions are there, probably - if you look for the very early releases - you should look somewhere else.
Look at that topic : https://bitcointalk.org/index.php?topic=68121.0, you will find there link to the very first release (and discussion).

[NotATether]

You should start with the Bitcoin software as it was written in 2011 ie. one of the early versions of Core, then slowly work your way through the pull requests and merges as C++11/14/17 stuff was introduced. The whole thing is 200K lines of code, and it wasn't all written in one shot.

[lhrbeijing]

What exactly you are interested in? Do you want to analyze the code or the way how team worked?
The code is available under https://github.com/bitcoin/bitcoin but I am not sure how old versions are there, probably - if you look for the very early releases - you should look somewhere else.
Look at that topic : https://bitcointalk.org/index.php?topic=68121.0, you will find there link to the very first release (and discussion).

Analyze the code, but not just compile and run.
I pulled the latest released Bitcoin source code in VS Code. Well, I actually read a little bit, such as pow.cpp, chainparams.cpp, block.h, validation.h, txmempool.h.
But I'm always jumping from file to file while looking at some detail. Maybe I should understand by module? Such as P2P, consensus mechanism, mathematics and cryptography, transaction memory pool, transaction verification, block verification, wallet, etc.

[PawGo]

But I'm always jumping from file to file while looking at some detail. Maybe I should understand by module? Such as P2P, consensus mechanism, mathematics and cryptography, transaction memory pool, transaction verification, block verification, wallet, etc.

Yes, maybe it is a good idea. Try to understand which features are present and how they are implemented. Is there any particular subject you are interested in?
Remember that many features were added "recently" and they did not exists in the early versions, so maybe you should start with more basic code, not the latest release.

[lhrbeijing]

You should start with the Bitcoin software as it was written in 2011 ie. one of the early versions of Core, then slowly work your way through the pull requests and merges as C++11/14/17 stuff was introduced. The whole thing is 200K lines of code, and it wasn't all written in one shot.

You remind me that it can be difficult to directly understand the latest released code.
But it will be better understood if I look at earlier versions.
By the way, do I need to read books like how to read source code?

Yes, maybe it is a good idea. Try to understand which features are present and how they are implemented. Is there any particular subject you are interested in?
Remember that many features were added "recently" and they did not exists in the early versions, so maybe you should start with more basic code, not the latest release.

Is it certain that no one is proficient in all modules?
Maybe I'm not ready to delve into a certain part of Bitcoin. I'm probably just wondering how each part is implemented without putting too much emphasis on performance or the code being concise enough.

[ABCbits]

But it will be better understood if I look at earlier versions.

If you really really want to look at earlier version, you might want to read series of thread "Satoshi Client Operation". IIRC it's applicable for version 0.3.X.

Overview : https://bitcointalk.org/index.php?topic=41718.0
Transaction Exchange : https://bitcointalk.org/index.php?topic=41730.0
Block Exchange : https://bitcointalk.org/index.php?topic=41729.0
Sockets and Messages : https://bitcointalk.org/index.php?topic=41727.0
Node Connectivity : https://bitcointalk.org/index.php?topic=41726.0
Node Discovery : https://bitcointalk.org/index.php?topic=41722.0
Initialization and Thread Startup : https://bitcointalk.org/index.php?topic=41719.0

But one of Bitcoin Core contributor say it won't help you understand latest Bitcoin Core source code.

Hello all, i've been a developer for the past couple of years and i would like to jump onto the blockchain coding paradigms, i've downloaded the first satoshi client's code (0.1) and there's this overview https://bitcointalk.org/index.php?topic=41718.40 that i would like to follow but there's a couple of missing files (i presume) mainly, i can't find the init.cpp file,

It doesn't exist in 0.1.0. A lot of stuff has changed since then, and it really is not worth your time to read through the original source code. So much has changed that whatever you learn is not applicable to the latest source code. You should instead try to learn and understand Bitcoin Core's latest source code. Keep in mind that the project has grown significantly since 0.1.0 and is much more complex.

[NotATether]

You should start with the Bitcoin software as it was written in 2011 ie. one of the early versions of Core, then slowly work your way through the pull requests and merges as C++11/14/17 stuff was introduced. The whole thing is 200K lines of code, and it wasn't all written in one shot.

You remind me that it can be difficult to directly understand the latest released code.
But it will be better understood if I look at earlier versions.
By the way, do I need to read books like how to read source code?

Since Bitcoin Core has documentation written with Doxygen, I recommend browsing the source docs comments and class/function annotations online at https://doxygen.bitcoincore.org/ before you read the source code directly.

Also, familiarize yourself with C++11, then 14, then 17, as they are used quite heavily in Core. I myself am a C++11 dev, so I found all the C++17 stuff hard to understand when I was submitting my first pull request to Core.

[lhrbeijing]

If you really really want to look at earlier version, you might want to read series of thread "Satoshi Client Operation". IIRC it's applicable for version 0.3.X.
But one of Bitcoin Core contributor say it won't help you understand latest Bitcoin Core source code.

Well, perhaps my ultimate goal is how to better understand the latest source code.
But I don't expect to understand it all. It is enough to understand the basic operations in common modules.
(My current level is unlikely to understand why it is designed this way.)

Since Bitcoin Core has documentation written with Doxygen, I recommend browsing the source docs comments and class/function annotations online at https://doxygen.bitcoincore.org/ before you read the source code directly.

For this site, am I typing the class/function/variable that I want to get annotated into in the top right search box?

[jackg]

Since Bitcoin Core has documentation written with Doxygen, I recommend browsing the source docs comments and class/function annotations online at https://doxygen.bitcoincore.org/ before you read the source code directly.

For this site, am I typing the class/function/variable that I want to get annotated into in the top right search box?

There's a good chance an IDE can produce a similar output that's a lot clearer to search through for specific variable references/assignments than that website (even though it's helpful for visualising the layout of the code). An IDE won't be able to provide some of the same descriptors too though, like bech32 (which is searchable but easier to spot in the docs than it is in the code).

[lhrbeijing]

There's a good chance an IDE can produce a similar output that's a lot clearer to search through for specific variable references/assignments than that website (even though it's helpful for visualising the layout of the code). An IDE won't be able to provide some of the same descriptors too though, like bech32 (which is searchable but easier to spot in the docs than it is in the code).

Alright.
I would use the search box in VS Code and go to the relevant definition, declaration or reference, etc.
The outline that comes with VS Code is also okay. But maybe not as good as some plugins.

Although the project is huge, the naming and comments in the code are really nice.
It's just that there are always some confusing statements for me.

[NotATether]

Since Bitcoin Core has documentation written with Doxygen, I recommend browsing the source docs comments and class/function annotations online at https://doxygen.bitcoincore.org/ before you read the source code directly.

For this site, am I typing the class/function/variable that I want to get annotated into in the top right search box?

Yes, that's how it works. Although local variables are not annotated, I believe only the global variables are. But everything else should have annotations. It's much easier this way than searching the src/ folder for a class, because Core has a crazy code layout that combines includes and sources in the same folder.

[BlackHatCoiner]

If you want to read the source code, but you've never done this before for such huge project, I recommend you begin with v0.1, which is based on Satoshi's moderate skills.

If you want to read more about the technical parts of Bitcoin, you should check Doxygen, as said by NotATether. For any questions, use stackexchange or just this forum.

If you don't want to dive into the hard stuff, begin with the greatly formulated "Mastering Bitcoin" by Andreas Antonopoulos.

[lhrbeijing]

Also, familiarize yourself with C++11, then 14, then 17, as they are used quite heavily in Core. I myself am a C++11 dev, so I found all the C++17 stuff hard to understand when I was submitting my first pull request to Core.

Yes, I feel it. Some of the confusion stems from the latest usage of C++. At the same time I don't know what the advantages are.

While I'm not yet aware of the latest BIPs and PRs, I have a vague feeling that the Bitcoin community has introduced many new features to improve performance and security. Perhaps that's another reason I can't understand.

If you want to read the source code, but you've never done this before for such huge project, I recommend you begin with v0.1, which is based on Satoshi's moderate skills.

If you want to read more about the technical parts of Bitcoin, you should check Doxygen, as said by NotATether. For any questions, use stackexchange or just this forum.

Thanks for your advice.
I went through that book, Mastering Bitcoin, and it does a good job. I think I should go a little deeper than this. But I really don't have a lot of engineering practice.
I'm somewhat interested in Bitcoin scaling, but still working on a theoretical level.


There are some similar questions and answers in StackExchange.

How to understand Bitcoin source code?

Where to find help understanding Bitcoin's source code in C++?

[n0nce]

I would like to read the Bitcoin source code roughly. is it possible?

Of course it is. Latest version is found at https://github.com/bitcoin/bitcoin; earlier versions are mirrored on different sites and linked in this thread already.

If I want to know how such a huge Bitcoin Core code was designed, what sources can I refer to?

Well, it wasn't huge in the beginning; and I'm not sure satoshi ever shared much about how his 'design process' looked like.

How did the code work as a team when it was written? (Well, just curious.)

There was no team at first; satoshi wrote it and published it to SourceForge.

If one of the cpp files has thousands of lines of code, was it written by one person?

Again, those files didn't always have thousands of lines of code. The codebase evolved over more than a decade. There is a list of 879 GitHub contributors (this only includes the GitHub repo 'era' since roughly 2010 / v0.3). https://github.com/bitcoin/bitcoin/graphs/contributors

How to ensure that it is bug-free?

Like in any other software. Tests, bug reports and fixes.

If you want to read the source code, but you've never done this before for such huge project, I recommend you begin with v0.1, which is based on Satoshi's moderate skills.

(just quoting one of the various recommendations to read the very-early code)

Of course, the first releases were smaller; less total code to go through, but they were also less structured and modularized than what we have now. Just wanted to mention this, in case OP runs into struggles with the readability of the 'satoshi releases'.

[NotATether]

If one of the cpp files has thousands of lines of code, was it written by one person?

Again, those files didn't always have thousands of lines of code. The codebase evolved over more than a decade. There is a list of 879 GitHub contributors (this only includes the GitHub repo 'era' since roughly 2010 / v0.3). https://github.com/bitcoin/bitcoin/graphs/contributors

Go to any source code file on Github, select a single line in the gutter (hold Shift key for multi-line selections), click the three dots button that appears on the left and then click "view git blame", it will show you who wrote that particular line(s) of code.

[0745dk]

hi I have the same problem as you...
can we have a communication?
(we are both from china)
you can send me a message if you agree with me