When building Bitcoin Core from source, there are some platform-dependant instructions to follow.

To learn how to build for your platform, visit the Bitcoin Core bitcoin/doc directory, and read the file named "build-*.md", where "*" is the name of your platform. For windows this is "build-windows.md", for macOS this is "build-osx.md" and for most linux distributions this is "build-unix.md".

There is also a guide by Jon Atack about how to compile and test Bitcoin Core.

It can be interesting to hear stories of how current contributors entered the space to hear about the approach they took and things they found useful, but also about any pitfalls they identified along their way.

amitiuttarwar, jonatack and jimmysong have kindly documented their experiences for others to read about and learn from.

• Amiti Uttarwar — Onboarding to Bitcoin Core

• Jon Atack — On Reviewing, and Helping Those Who Do It

• Jimmy Song — A Gentle Introduction to Bitcoin Core Development

Bitcoin Core uses Doxygen to generate developer documentation automatically from its annotated C++ codebase. Developers can access documentation of the current release of Bitcoin Core online at doxygen.bitcoincore.org, or alternatively can generate documentation for their current head using make docs (see Generating Documentation for more info).

Bitcoin Core uses a GitHub-based workflow for development. The primary function of GitHub in the workflow is to discuss patches and connect them with review comments.

Whilst some other prominent projects, e.g. the Linux kernel use email for soliciting feedback and review, Bitcoin Core has used GitHub for many years. Initially Satoshi distributed the code through private emails and hosting source archives at bitcoin.org, and later by hosting on SourceForge (which used SVN but did not at that time have a pull request system like GitHub). The earliest reviewers submitted changes using patches either through email exchange with Satoshi, or by posting them on the bitcoin forum.

In August 2009, the source code was moved to GitHub by Sirius and development has remained there and used the GitHub workflows ever since.

• What stops a hacker hijacking the Bitcoin Core website and hosting malicious binaries?

  • How about malicious binaries hosted by linux package managers?

• Where can you go for help if Bitcoin Core doesn’t build on your machine?

• Before you create a pull request to the main bitcoin core repo, what checks should you do locally?

  • Are there any additional checks you can think of which are only run in the bitcoin core repo (and not your fork)?

• Each submit a PR on a team member’s fork of Bitcoin Core (not the main repo)

• Review a different team member’s PR

• Submit your review of the PR as a GitHub comment on the PR

Olivia Lovenmark and Amiti Uttarwar describe in their blog post Developing Bitcoin how changes to bitcoin follow the pathway from proposal to being merged into the software, and finally adopted by users.

The Bitcoin Core project itself contains two documents of particular interest to contributors:

1. CONTRIBUTING.md — How to get started contributing to the project.

2. developer-notes.md — Development guidelines, coding style etc.

lsilva01 has written a deep technical dive into the architecture of Bitcoin Core as part of the bitcoin core onboarding documentation in Bitcoin Architecture.

Once you’ve gained some insight into the architecture of the program itself you can learn further details about which code files implement which functionality using the document Bitcoin Core regions.

James O’Beirne has recorded 3 videos which go into detail on how the codebase is laid out, how the build system works, what devtools there are, as well as what the primary function of many of the files are:

1. Architectural tour of Bitcoin Core (part 1 of 3)

2. Architectural tour of Bitcoin Core (part 2 of 3)

3. Architectural tour of Bitcoin Core (part 3 of 3)

Signet is both a tool that allows developers to create their own networks for testing interactions between different Bitcoin software and the name of the most popular of these testing networks. Signet was codified in BIP325.

To connect to the "main" Signet network, simply start bitcoind with the signet flag, e.g. bitcoind -signet. Don’t forget to also pass the signet flag to bitcoin-cli if using it to control bitcoind, e.g. bitcoin-cli -signet. Instructions on how to setup your own Signet network can be found in the Bitcoin Core Signet README.md. The Bitcoin wiki Signet page provides additional background on Signet.

Bitcoin uses Bitcoin Improvement Proposals, or BIPs, as a design document for introducing new features or behaviour into bitcoin. Bitcoin magazine describes what a BIP is in their article What Is A Bitcoin Improvement Proposal (BIP), specifically highlighting how BIPs are not necessarily binding documents required to achieve consensus.

The BIPs are hosted on GitHub and include BIP2 which self-describes the BIP process in more detail. Of particular interest might be the sections BIP Types and BIP Workflow.

What are the best ways to get started with Bitcoin Core development? As mentioned earlier, one of the roles most in demand from the project is that of code review, and in fact this is also one of the best ways of getting familiarised with the codebase too! Reviewing a few PRs, and importantly submitting your review to GitHub on the PR can be really valuable. This Google Code Health blog post gives some good advice on how to go about code review and getting past "feeling that you’re not as smart as the programmer who wrote the change". If you’re going to ask some questions as part of review, try and keep questions respectful.

Aside from review, there are 3 main avenues which might lead you to submitting your own pull request to the repository:

1. Finding a good first issue, as tagged in the issue tracker

2. Fixing a bug (you’ve found yourself?)

3. Adding a new feature (that you want for yourself?)

Of these three, I’d highly recommend choosing a good first issue from an area of the codebase that seems interesting to you. The reason is that these have been somewhat implicitly "concept ACKed" by other contributors as "something that is likely worth working on".

Hopefully now you have an idea of roughly what your PR is going to do; often this is the hardest part to getting started! If you don’t have a bugfix or new feature in mind, and you’re struggling to find a good first issue which looks suitable for you, don’t panic. Instead keep reviewing other developers' PRs to continue improving your understanding of the process (and the codebase), while you watch the issue tracker for something which you like the look of.

Now that you’ve decided what to work on it’s time to take a look at the current behaviour of that part of the code and perhaps more importantly, try to understand why this was originally implemented in this way. This process of code "archaeology" will prove invaluable in the future when you are trying to learn about other parts of the codebase on your own.

When considering changing code it can be helpful to try and first understand the rationale behind why it was implemented that way originally, if possible. One of the best ways to do this is by using a combination of git tools — git blame, git log -S, and less commonly git log -G — and the discussions on GitHub.

• Read lsilva01’s 1.0 Bitcoin Architecture. Particularly sections:

  • Executables

  • Regions (and all sub-sections)

TODO: Add questions on current architecture of Core

Review of the design of Bitcoin Core from Overview and Architecture will naturally lead to a region of the project titled "consensus/" which one might conclude contains all the logic for maintaining consensus. However this is not entirely the case…​

Aspects of consensus-enforcement code can be found across the Bitcoin Core codebase in a number of regions and files, including notably:

• validation.{h|cpp}

• consensus/

• policy/

• script verification

Why is such a critical function split up between many files, and how do they all interact? Part of the answer can be learned from sdaftuar’s Stack Exchange answer to the question "What is the difference between policy and consensus when it comes to a Bitcoin Core node validating scripts?"

The answer teaches us that policy checks are a superset of validation checks, that is to say that a transaction that passes policy checks has implicitly passed consensus checks too. Nodes perform policy-level checks on all transactions they learn about before adding them to their local mempool. Many of the policy checks contained in policy are called from inside validation, in the context of adding a new transaction to the mempool.

Pieter Wuille disclosed the possibility of a consensus failure related to signature verification when using OpenSSL. The issue was that OpenSSL was accepting multiple signature serialization formats (for the same transaction) as valid. This meant that a transaction’s ID (txid) could be changed, because the signature contributes to the txid hash.

There were a few main cases to consider:

1. first party malleation: signature length descriptor is extended to 5 bytes

2. third party malleation: signatures are "slightly" tweaked (or padded)

3. third party malleation: negating the S value of the ECDSA signature

In the length descriptor case there is a higher risk of causing a consensus-related chainsplit. The first party (the sender) can create a valid (normal length) signature, but which uses a 5 byte length descriptor meaning that it might not be accepted by OpenSSL on all platforms.

In the second case, of signature tweaking or padding, there is a lesser risk of causing a consensus-related chainsplit. However the ability of third parties to tamper with valid transactions may open up off-chain attacks related to Bitcoin services or layers (e.g. Lightning) in the event that they are relying on txids to track transactions.

It is interesting to consider the order of the steps taken to fix this potential vulnerability:

1. First the default policy in Bitcoin Core was altered (via isStandard()) to prevent the software from relaying or accepting into the mempool transactions with non-DER signature encodings.
   This was carried out in PR #2520.

2. Following the policy change, the strict encoding rules were later enforced by consensus in PR #5713.

Do you think this approach — first altering policy, followed later by consensus — made sense for implementing the changes needed to fix this consensus vulnerability? In what circumstances might it not make sense? Having OpenSSL as a consensus-critical dependency to the project was ultimately fixed in PR #6954 which switched to using libsecp256k1 for signature verification.

Historically Bitcoin Core used Berkeley DB (BDB) for transaction and block indices. In 2013 a migration to LevelDB for these indices was included with Bitcoin Core v0.8. What developers at the time could not foresee is that nodes that were still using BDB for these indices (all pre 0.8 nodes), were silently consensus-bound by a relatively obscure BDB-specific database lock counter…​

BDB required a configuration setting for the total number of locks available to your database. Bitcoin Core was also interpreting failure to grab the required number of locks as the block being invalid — a consensus failure. This combination caused some BDB-using nodes to mark blocks created by LevelDB-using nodes as invalid and caused a consensus split. BIP 50 provides further explanation on this incident.

Note that that database code is not found in, or even in close proximity to, the /src/consensus region of the codebase.

We can follow most of the journey of a transaction through Bitcoin Core by following glozow’s notes on transaction Validation and submission to the mempool. glozow details what different types of checks are run on a new transaction before it’s accepted into the nodes local mempool — consensus vs policy, script vs non-script, contextual vs context-free.

glozow continues with sections on P2P transaction relay, orphans and mining, but more relevant to consensus is the following section, Block Validation, which describes the consensus checks performed on newly-learned blocks, specifically:

Since v0.8, Bitcoin Core nodes have used a UTXO set rather than blockchain lookups to represent state and validate transactions. To fully validate new blocks nodes only need to consult their UTXO set and knowledge of the current consensus rules. Since consensus rules depend on block height and time (both of which can decrease during a reorg), they are recalculated for each block prior to validation.

Regardless of whether or not transactions have already been previously validated and accepted to the mempool, nodes check block-wide consensus rules (e.g. total sigop cost, duplicate transactions, timestamps, witness commitments block subsidy amount) and transaction-wide consensus rules (e.g. availability of inputs, locktimes, and input scripts) for each block.

Script checking is parallelized in block validation. Block transactions are checked in order (and coins set updated which allows for dependencies within the block), but input script checks are parallelizable. They are added to a work queue delegated to a set of threads while the main validation thread is working on other things. While failures should be rare - creating a valid proof of work for an invalid block is quite expensive - any consensus failure on a transaction invalidates the entire block, so no state changes are saved until these threads successfully complete.

If the node already validated a transaction before it was included in a block, no consensus rules have changed, and the script cache has not evicted this transaction’s entry, it doesn’t need to run script checks again - it just uses the script cache!

The section on script verification also highlights how the script interpreter is called from at least 3 distinct sites within the codebase:

• when the node receives a new transaction.

• when the node wants to broadcast a new transaction.

• when receiving a new block

Having considered both transactions that were already known about (in the mempool), and any new transactions that were first learned about in the block itself (as part of block validation), we now understand both ways a transaction can be deemed consensus-valid.

TODO: Reorgs, undo data, DisconnectBlock

Bitcoin nodes should ultimately converge in consensus on the most-work chain. Being able to track and monitor multiple chain (tips) concurrently is a key requirement for this to take place. There are a number of different states which the client must be able to handle:

1. A single, most-work chain being followed

2. Stale blocks learned about but not used

3. Full reorganisation from one chain tip to another

BlockManager is tasked with maintaining a tree of all blocks learned about, along with their total work so that the most-work chain can be quickly determined.

CChainState is responsible for updating our local view of the best tip, including reading and writing blocks to disk, and updating the UTXO set. A single BlockManager is shared between all instances of CChainState.

ChainstateManager is tasked with managing multiple CChainStates. Currently just a "regular" IBD chainstate and an optional snapshot chainstate, which might in the future be used as part of the assumeUTXO project.

When a new block is learned about (from src/net_processing.cpp) it will call into ChainstateManagers ProcessNewBlockHeaders method to validate it.

1. What is the difference between contextual and context-free validation checks?

   Contextual checks require some knowledge of the current "state", e.g. ChainState, chain tip or UTXO set.

   Context-free checks only require the information required in the transaction itself.

   See {glozow-tx-mempool-validation}[glozow-tx-mempool-validation] for more info.

2. What are some examples of each?

   context-free:

   1. tx.isCoinbase()

   2. 0 ≤ tx_value ≤ MAX_MONEY

   3. tx not overweight

   contextual: check inputs are available

3. In which function(s) do UTXO-related validity checks happen?

   ConnectBlock()

4. What type of validation checks are CheckBlockHeader() and CheckBlock() performing?

   context-free

5. Which class is in charge of managing the current blockchain?

   ChainstateManager()

6. Which class is in charge of managing the UTXO set?

   CCoinsViews()

7. Which functions are called when a longer chain is found that we need to re-org onto?

   TODO

8. Are there any areas of the codebase where the same consensus or validation checks are performed twice?

   Again see glozows notes for examples

9. Why does CheckInputsFromMempoolAndCache exist?

   To prevent us from re-checking the scripts of transactions already in our mempool during consensus validation on learning about a new block

10. Which function(s) are in charge of validating the merkle root of a block?

    BlockMerkleRoot() and BlockWitnessMerkleRoot() construct a vector of merkle leaves, which is then passed to ComputeMerkleRoot() for calculation.

11. Can you find any evidence (e.g. PRs) which have been made in an effort to modularize consensus code?

    A few examples: #10279, #20158

12. What is the function of BlockManager()?

    It manages the current most-work chaintip and pruning of unneeded blocks (*.blk) and associated undo (*.rev) files

13. What stops a malicious node from sending multiple invalid headers to try and use up a nodes' disk space? (hint: these might be stored in BlockManager.m_failed_blocks)

    Even invalid headers would need a valid proof of work which would be too costly to construct for a spammer

14. Which functions are responsible for writing consensus-valid blocks to disk?

    TODO: answer

15. Are there any other components to Bitcoin Core which, similarly to the block storage database, are not themselves performing validation but can still be consensus-critical?

    Not sure myself, sounds like an interesting question though!

16. In which module (and class) is signature verification handled?

    src/script/interpreter.cpp#BaseSignatureChecker

17. Which function is used to calculate the Merkle root of a block, and from where is it called?

    src/consensus/merkle.cpp#ComputeMerkleRoot is used to compute the merkle root.

    It is called from src/chainparams.cpp#CreateGenesisBlock, src/miner.cpp#IncrementExtraNonce & src/miner.cpp#RegenerateCommitments and from src/validation.cpp#CheckBlock to validate incoming blocks.

18. Practical question on Merkle root calculation

    TODO, add exercise

The outline of the mechanism at work is that a node relaying a transaction can slightly modify the signature in a way which is still acceptable to the underlying OpenSSL module. Once the signature has been changed, the transaction ID (hash) will also change. If the modified transaction is then included in a block, before the original, the effect is that the sender will still see the outgoing transaction as "unconfirmed" in their wallet. The sender wallet should however also see the accepted (modified) outgoing transaction, so their balance will be calculated correctly, only a "stuck doublespend" will pollute their wallet. The receiver will not perceive anything unordinary, unless they were tracking the incoming payment using the txid as given to them by the sender.

1. Wallets are stored on disk as databases, either using Berkeley Database (BDB) or sqlite format.

2. These wallets can be one of two types, "legacy" or "descriptor".

3. Wallets do not have to store the private keys associated with the addresses and public keys they are monitoring.

• Bitcoin core onboarding - wallet/ describes the main functions of a wallet, along with some of the differences between legacy and descriptor wallets.

The wallet component is initialised via the WalletInitInterface class as specified in src/walletinitinterface.h. The member functions are marked as virtual in the WalletInitInterface definition, indicating that they are going to be overridden later by a derived class.

Both walletinit.cpp and dummywallet.cpp include derived classes which override the member functions of WalletInitInterface, depending on whether the wallet is being compiled in or not.

The primary src/Makefile.am describes which of these modules is chosen to override: if ./configure has been run with the wallet feature enabled (default), then wallet/init.cpp is added to the sources, otherwise (./configure --disable-wallet) dummywallet.cpp is added.

src/walletinitinterface.h declares the global g_wallet_init_interface which will handle the configured WalletInitInterface.

The wallet interface is created when the Construct() method is called on the g_wallet_init_interface object by AppInitInterfaces() in init.cpp. Construct takes a reference to a NodeContext as argument, and then checks that the wallet has not been disabled by a runtime argument before calling interfaces::MakeWalletClient() on the node. This initialises a new WalletClientImpl object which is then added to the node object, both to the general list of node.chain_clients (wallet processes or other clients which want chain information from the node) in addition to being assigned as the unique node.wallet_client role, which specifies the particular node.chain_client that should be used to load or create wallets.

The NodeContext struct is defined as the following:

…​contains references to chain state and connection state.

…​used by init, rpc, and test code to pass object references around without needing to declare the same variables and parameters repeatedly, or to use globals…​ The struct isn’t intended to have any member functions. It should just be a collection of references that can be used without pulling in unwanted dependencies or functionality.

Wallets can optionally be loaded as part of main program startup (i.e. from src/init.cpp). Any wallets loaded during the life cycle of the main program are also unloaded as part of program shutdown.

Grepping the src/wallet directory for locks, conventionally of the form cs_*, yields 501 matches. For comparison the entire remainder of the codebase excluding src/wallet/* yields 925 matches. Many of these matches are asserts and declarations, however this still illustrates that the wallet code is highly reliant on locks to perform atomic operations.

As we can see wallet component startup and shutdown is largely driven from outside the wallet codebase from src/init.cpp.

Once the wallet component is started and any wallets supplied via argument have been verified and loaded, wallet functionality ceases to be called from init.cpp and instead is controlled using external programs in a number of ways. The wallet can be controlled using bitcoin-cli, the bitcoin-qt GUI or the stand-alone bitcoin-wallet tool.

Both bitcoind and bitcoin-qt run a (JSON) RPC server which is ready to service, amongst other things, commands to interact with wallets. The command line tool bitcoin-cli will allow interaction of any RPC server started by either bitcoin or bitcoin-qt.

If using the bitcoin-qt GUI itself then communication with the wallet is done directly via qt’s WalletModel interface.

Commands which can be used to control the wallet via RPC are listed in rpcwallet.cpp.

The CWallet object is the fundamental wallet representation inside Bitcoin Core. CWallet stores transactions and balances and has the ability to create new transactions. CWallet also contains references to the chain interface for the wallet along with storing wallet metadata such as nWalletVersion, wallet flags, wallet name and address book.

Each wallet contains one or more ScriptPubKeyManagers, who are in control of storing the scriptPubkeys managed by that wallet.

A CWallet in the general sense therefore becomes "a collection of ScriptPubKeyManagers", which are each managing an address type. In the current implementation, this means that a default (descriptor) wallet consists of 6 ScriptPubKeyManagers, one for each of combination of {legacy | p2sh | bech32} for {receive | change} addresses.

Compare this to the equivalent descriptor wallet code fragment which sets up an SPKM for each output type:

Script pubkey managers are stored inside CWallet in a map according to output type:

Prior to c729afd0 the equivalent SPKM functionality (fetching new addresses and signing transactions) was contained within CWallet itself, now being split out for better maintainability and upgradability brought by modularisation as per the wallet box class structure changes. The ultimate effect of this is that the CWallet object itself no longer handles keys and addresses.

The change to a CWallet made up of (multiple) {Descriptor|Legacy}ScriptPubKeyMan's is also sometimes referred to as the "Wallet Box" model, where each SPKM is thought of as a distinct (black?) "box" within the wallet, which can be called upon to perform new address generation and signing functions.

In order to construct a transaction the wallet will validate the outputs, before selecting some coins to use in the transaction. This involves multiple steps and we can follow an outline of the process by walking through the sendtoaddress RPC command, which returns by calling SendMoney(), shown below:

After initialisation SendMoney() will call wallet.CreateTransaction() (CWallet::CreateTransaction()) followed by wallet.CommitTransaction() if successful. If we follow wallet.CreateTransaction() we see that this is a public wrapper function which in its turn calls private member function CWallet::CreateTransactionInternal().

Work on the multiwallet project means that Bitcoin Core can now handle dynamic loading and unloading of multiple wallets while running.

TODO

TODO

Much of the meat of the recently soft-forked changes (e.g. Taproot) reside not inside consensus code, but rather require improvements to the wallet.

• When adding new wallet features which will be included in the GUI, it can be good practice to first implement them as RPC commands because it’s easier to create good test coverage for them.

• Advanced transaction signature operations (e.g. signature aggregation, sighash flags) happen in the wallet code.

bitcoin-qt, which is the GUI version of the node software, is built automatically when the build dependencies are met. Required packages can be found in the build instructions in src/doc/build-*.md as appropriate for your platform. If you have the required packages installed but do not wish to build the bitcoin-qt then you must run ./configure with the option --with-gui=no.

We can see how the Qt directory is related to the rest of the codebase from its directory dependency graph:

Developers would ideally like to reduce these dependencies even further.

There is useful documentation for developers looking to contribute to the Qt side of the codebase found at Developer Notes for Qt Code.

The loading point for the GUI is src/qt/main.cpp. main() calls GuiMain() from src/qt/bitcoin.cpp, passing along any program arguments with it. GuiMain starts by calling SetupEnvironment() which amongst other things, configures the runtime locale and charset.

Next an empty NodeContext is setup, which is then populated into a fully-fledged node interface via being passed to interfaces::MakeNode(), which returns an interfaces::Node. Recall that in Wallet component initialisation we also saw the wallet utilising a NodeContext as part of its WalletInitInterface. In both cases the NodeContext is being used to pass chain and network references around without needing to create globals.

After some QT setup, command-line and application arguments are parsed. What follows can be outlined from the code comments:

3. Application identification

4. Initialization of translations, so that intro dialog is in user’s language

5. Now that settings and translations are available, ask user for data directory

6. Determine availability of data directory and parse bitcoin.conf

7. Determine network (and switch to network specific options)

8. URI IPC sending

9. Main GUI initialization

After configuration the GUI is initialised. Here the Node object created earlier is passed to app.SetNode() before a window is created and the application executed.

The bulk of the Qt GUI classes are defined in src/qt/bitcoingui.{h|cpp}.

Since writing this documentation focus has been directed towards re-writing the Qt code leveraging the Qt QML framework. This will allow developers to create visually-superior, and easier to write and reason-about GUI code, whilst also lowering the barriers to entry for potential new developers who want to be able to focus on GUI code.

The recommendation therefore is to familiarise yourself with Qt QML and review the current codebase for the latest developments. You can follow along with the latest QML work in the specific bitcoin-core/qml-gui repo.

The Bitcoin design guide provides some guidance on common pitfalls that Bitcoin GUI designers should look out for when designing apps (like bitcoin-qt).