libbitcoin

[genjix]

libbitcoin: Product of Calafou. Endorsed by Enric Duran.

Website, http://libbitcoin.dyne.org
IRC, Freenode, #darkwallet
Forum, https://forum.unsystem.net

libbitcoin is a community of developers building the open source library, tools and implementation necesary for a free, independent and vibrant Bitcoin. In this way we are helping to build a better future.

libbitcoin believes in the revoltionary promise of Satoshi's original protocol.

The libbitcoin development project aims to create an extendable, scalable and configurable architecture, along with useful software. Making Bitcoin super-pluggable, highly configurable and easy to interact with.

Our Values:

• Privacy: Bitcoin should always remain as private as possible for its users.
• Scalability: Bitcoin built today with the future in mind.
• Integrity: No individual or group should have enough power over the network to compromise its original aims.

Here is the git repo,
https://github.com/spesmilo/libbitcoin

By having a bitcoin library, it enables everybody to build apps really quickly using a common stable codebase. This project aims to create that architecture to be extendable, scalable and configurable in the future.

    rewrite bitcoin, make it super-pluggable, very easy to do and hack everything at every level, and very configurable

Documentation: http://libbitcoin.dyne.org/doc/index.html

Slides: http://libbitcoin.dyne.org/libbitcoin.pdf

The Zen of libbitcoin
Readability over speed.
Beauty over convenience.
Simplicity over complexity.
Architected, not hacked.
Flat, not nested.
Explicit, not implicit.
Errors should be loud.
Never is better than right now.
Now is better than never.
Be flexible and configurable.
Build houses from bricks, software from modules.

libbitcoin is licensed as Lesser AGPL:
- When used in the p2p network, you only need to provide changes on demand (LGPL).
- When used on a webserver, you must proactively provide sourcecode for any changes you have made to libbitcoin.
- Applications can link against libbitcoin, and they do not need to release their changes publically.
Thanks to Aaron Williamson of the SFLC and Richard Stallman of the FSF for helping draft up this license.

Mailing list: https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin

Why the GPL?

http://arstechnica.com/gadgets/2013/10/googles-iron-grip-on-android-controlling-open-source-by-any-means-necessary/

http://hintjens.com/blog:68

[1715386374]

1715386374

[1715386374]

1715386374

[genjix]

This is the inner core which consists of a bunch of subsystems co-operating over clearly defined interfaces.



As an example one such message could command the block_chain_manager to create a transaction. It then sends the new transaction to the network component to actualise it.



The inner core is very difficult to use so we abstract it's usage to a more usable outer core. Think of the inner core like X-windows and outer core an easy to use GUI toolkit like Qt (by analogy).



Into the core we can switch in and out different components. In this example we choose to replace a MySQL serialiser/de-serialiser with an SQLite one. Maybe there could also be in-memory serialisers (never store to disk) or whatever.

A serialiser takes in a data object and then stores it. The type of component decides where + how it looks when stored. When reading the object it de-serialises in reverse.

The whole point is not to just dump binary blobs into MySQL rows but to put them there in a readable format so an admin can dissect them in case of problems. We also will only create a MySQL serialiser for the time being unless it's possible to use standard SQL and simultaneously support MySQL, PostgresSQL, SQLite.



Above the diagram for the outer core indicated an exposed C api with auto-generated Swig bindings.

We try to keep the core very simple and focused; no mining, no accounts, no GUI, no RPC server. These things can all be implemented in Python. Already genjix has Spesmilo as a Qt GUI using bitcoind that we can easily use since it has a special interface that can be re-implemented for whatever Bitcoin implementation. Accounts can be provided as a Python module or special plugin in C or C++.



As another example, by being able to switch the networking component, we can replace it with a dummy networking component that is used for debugging.



Or use one networking component for multiple instances of Bitcoin or other currencies.

We try to split everything up as much as possible to help developers long term. It makes it easier for new contributors to get started. That's the free-software way: lots of components and clear separation to allow maintainers to work on their piece of turf.

Callbacks

A good plugin+component system is a must for this project. The core will probably have to be thread-safe if we use threads for networking. Otherwise an event system could be a bottleneck since we get the worst of both worlds.

If we make a block-chain verifier subsystem, is there an object to making it one-shot (verify once or every X blocks the block chain in one go) or does it need to be after every block is downloaded?

In any case, it would be nicer to be able to hook the verifier system to a generic callback as opposed to calling it manually in the code.

boost::signals2 seems like a good choice to be considered, although it has to be seen whether there is any bottleneck. Also any callback system would have to be exposed in Python somehow

Random notes:

    * Our main target is large scale deployments. Bitcoin Enterprise (utilising our libbitcoin). The Red Hat of Bitcoin, not the Ubuntu.
    * Use boost::property_map?
    * boost::asio is a good choice for networking.
    * Externally we expose a C api for people that cannot program C++ but use C++ internally. A C ABI is also nice since it doesn't get mangled by the compiler.
    * Status during initialisation for debugging/showing progress in GUI client.

[error]

Nice work! You'll also have a C++ API, right?

[genjix]

A document describing the rationale behind the database structure.



Axioms

Starting with these basic rules, we design our database.

    * Blocks are mostly sequential.
    * Blocks contain a list of transactions.
    * Transactions contain a list of inputs + outputs.
    * Blocks can be free floating (orphan blocks).
    * Transactions can be free floating (not yet in a block).
    * There is one sequence of events that we accept as the truth at any one time.

From big to small:

    * Blocks
    * Transactions
    * Inputs
    * Outputs
    * Scripts
    * Operations

Block-chain

The block-chain is a series of blocks chained on in a linear sequence. The block-chain is an append only database of all transactions in the network.

What's accepted as true, is the chain with the highest difficulty. Difficulty is computed using a function from the 'bits' field of the block. Given an entire chain, we can calculate the difficulty using:

Code:

    SELECT SUM(COMPUTE_DIFFICULTY(bits)) FROM our_chain;

Occasionally this chain of events may be conflicting when the chain splits in half because there are two next rival blocks trying to append to the current chain. Always, the chain with the highest cumulative difficulty is preferred over the others.

It can happen that more blocks build on a chain that previously had lower difficulty and so take over the main chain. This is termed a block-chain re-organisation. But as more blocks build off the main chain, that becomes exponentially less likely until eventually it becomes a near impossibility. Branches off the main block-chain can be pruned once they reach a certain depth that it's near unlikely they will ever become the main branch again.

We can represent our blocks using this schema:

Code:

blocks:
    block_id
    block_hash
    depth       -- block number in the chain
    span_left
    span_right
    ... other fields

Code:

block_id 	depth 	span_left 	span_right
0 	0 	0 	3
1 	1 	0 	3
2 	2 	0 	2
3 	2 	3 	3
4 	3 	3 	3
5 	3 	0 	2
6 	4 	3 	3
7 	4 	0 	0
8 	5 	0 	0
9 	4 	1 	2
10 	5 	1 	2
11 	6 	1 	2
12 	6 	0 	0
13 	7 	0 	0
14 	8 	0 	0
15 	7 	1 	1
16 	7 	2 	2

In the diagram above blocks are represented by black dots. The block_id is a unique identifier for each new block inserted into the database. Each of the dots above would have a unique block_id.

Depth is the vertical number on the left, and is synonymous with a block's block-number.

Span left and right indicates the range of chains that a block exists in. A block with a span_left of 0, and a span_right of 3, would exist in chains 0, 1, 2 and 3. Using a span, we can easily select a particular chain:

Code:

    SELECT * FROM blocks WHERE span_left=0 AND span_right=0;

Or select the common point between a series of chains:

Code:

    SELECT MAX(depth) FROM blocks WHERE span_left>=0 AND span_right<=2;

Once span_right starts to exceed a value (say 10) and the depth since the chain forked exceeds a certain value, then that chain can be pruned:

Code:

    DELETE FROM blocks WHERE span_left=10 AND span_right=10;

The total span range of all blocks should equal all the number of stored chains. If we have a span of [0, 10] then there should be 11 spans from 0 to 10. A table exists to keep track of all the chains.

Code:

blockchains:
    chain_id
    max_depth
    last_block_id
    total_difficulty

Transaction

Nothing unusual happens here, except that transactions can be included more than once in multiple chains. A seperate table exists to track the parents for a transaction.

Code:

transaction_parents:
    transaction_id
    block_id
    index_in_block

transactions:
    transaction_id
    transaction_hash
    ... other fields

outputs:
    parent_id       -- transaction parent
    index_in_parent
    output_type     -- normal, generated, other
    address
    ... other fields

inputs:
    parent_id
    index_in_parent
    previous_output_id
    ... other fields

Script

A script consists of a serious of opcodes and optional data.

Code:

script_id_type = SEQUENCE

operations:
    operation_id
    script_id
    code
    data

A typical script might appear like:

Code:

operation_id 	script_id 	code 	data
132 	45 	dup 	NULL
133 	45 	hash160 	NULL
134 	45 	special 	89 ab cd ef ab ba ab ba ab ba ab ba ab ba ab ba ab ba ab ba
135 	45 	equalverify 	NULL
136 	45 	checksig 	NULL

To fetch the script:

Code:

    SELECT * FROM operations WHERE script_id=45 ORDER BY operation_id ASC;

Broadcasted inventories

This will be stored in memory for now as there are potential attack vectors to serialising inv items. An attacker could cause many disk read/writes by spamming a host potentially. More thought needs to go into this later.

Main block-chain

A convenience table is provided for the end user to read the block-chain transaction history. It contains the currently accepted version of the truth. It provides a way for the user to quickly query transactions and see funds in addresses.



The inputs and outputs are joined in a view. All the outputs which are final and have not been spent, would have no equivalently joined on input.

[genjix]

Nice work! You'll also have a C++ API, right?

Yeah the C++ API is the default, but I want to provide something for C users too later. And also Ruby, Perl, Python, Java .etc through SWIG bindings.

[wumpus]

I think it is a nice design overall, but

A script consists of a serious of opcodes and optional data.

Code:

script_id_type = SEQUENCE

operations:
    operation_id
    script_id
    code
    data

A typical script might appear like:

Code:

operation_id 	script_id 	code 	data
132 	45 	dup 	NULL
133 	45 	hash160 	NULL
134 	45 	special 	89 ab cd ef ab ba ab ba ab ba ab ba ab ba ab ba ab ba ab ba
135 	45 	equalverify 	NULL
136 	45 	checksig 	NULL

To fetch the script:

Code:

    SELECT * FROM operations WHERE script_id=45 ORDER BY operation_id ASC;

I wonder here, what is the advantage to decomposing the scripts into separate database rows, instead of storing it as a binary blob?

I mean, the most common operation is "fetch the entire script and execute the opcodes". By decomposing the data structure this deep this becomes a more expensive query.

[jrmithdobbs]

I wonder here, what is the advantage to decomposing the scripts into separate database rows, instead of storing it as a binary blob?

I mean, the most common operation is "fetch the entire script and execute the opcodes". By decomposing the data structure this deep this becomes a more expensive query.

For a node, maybe. What about for data processing? Say you just want to quickly find all scripts with the same destination address. The performance difference will be negligible for the amount of flexibility gained. Returning data like this is what RDBMS' are optimized for.

[wumpus]

For a node, maybe. What about for data processing? Say you just want to quickly find all scripts with the same destination address.

Yes there are cases in which you want to search in a script, but they are not very common. Usually you want to optimize for the most common case first.

The performance difference will be negligible for the amount of flexibility gained. Returning data like this is what RDBMS' are optimized for.

Maybe; I wouldn't be so quick to call it 'negligible', as I do know that for the current client, database accesses are by far the biggest source of performance issues.

Anyway, don't do too much premature optimisation, it might not be useful yet to discuss in this phase.

[LightRider]

Good effort, thank you so much for doing this work.

[genjix]

Maybe there is some merit to storing the script as:

"dup hash160 [1a 4b ... data ... ff 65] equalverify checksig"

And parsing them in and out.

The primary reason I stored them the way I do is because that's how they're represented internally:

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/include/bitcoin/script.hpp

[grondilu]

(registering to this thread)

[Mike Hearn]

Could you summarize what this library is capable of and what remains to be done? I took a quick look but it seems to be mostly outlines so far.

[genjix]

ATM it's a very basic library. You can construct transactions and send them over the network. Also you can download blocks/txs and partially verify scripts.

What's missing mainly is the transaction abstraction engine.

TODO:
phase 2:
- OP_CHECKSIG
phase 3:
- EC keypair generating
- High level construction of transactions.
- session class (session->connect();  session->send_funds("1djkj3jk3kj3k", 1000000000); session->get_balance(); .etc)
phase 4:
- Lastly would be to make a gui, but I'm hoping others will build them as this library becomes more mature.
- Fully complete scripting engine that isn't disabled.

As you can see, ATM it's 0.1alpha release.

Progress can be followed at, https://bitcoinconsultancy.com/wiki/index.php/Libbitcoin#Roadmap

[jackjack]

Monitoring this awesome thread!

A pic has been deleted: http://forum.bitcoin.org/index.php?topic=30646.msg384841#msg384841

[davout]

Monitoring this awesome thread!

+1

[ThomasV]

this is fantastic. thank you!!!

[vector76]

Great work, I just have a question about the span.  I don't understand how it works, because if you fork the block chain, won't all the existing span numbers have to be incremented?

I've been trying to implement a sqlite backend (so far just dump to sqlite which is mostly working), and it seems to be helpful to distinguish between dynamic data and data which is immutable such as the successor relationship between blocks, transaction containment within a block, etc.

Whether a block is in the best chain and whether a txout has been spent in the best chain is dynamic, so I have to implement queries that more or less correspond to ConnectBlock and DisconnectBlock.  I would love to be able to eliminate it using your span mechanism or something similar but I'm not understanding how it works.

[genjix]

Great work, I just have a question about the span.  I don't understand how it works, because if you fork the block chain, won't all the existing span numbers have to be incremented?

I've been trying to implement a sqlite backend (so far just dump to sqlite which is mostly working), and it seems to be helpful to distinguish between dynamic data and data which is immutable such as the successor relationship between blocks, transaction containment within a block, etc.

Whether a block is in the best chain and whether a txout has been spent in the best chain is dynamic, so I have to implement queries that more or less correspond to ConnectBlock and DisconnectBlock.  I would love to be able to eliminate it using your span mechanism or something similar but I'm not understanding how it works.

I'm still working on ProcessBlock (verifying the blocks) and everything below it (ConnectBlock/DisconnectBlock). You can see an example of how the span works in src/storage/postgresql_storage.cpp:reorganize_block_chain()... That function will be moved internally and within verify_block under include/bitcoin/verify.hpp

Indeed you're correct that all the spans of the parents would have to be incremented if you fork the block chain. This is OK since it can be done in a simple single UPDATE, inserting a new block is an infrequent operation (doesn't have to be as fast as SELECTs) and it's not a big deal if adding a new block isn't immediate.

Far more important is the select speed. If you run the poller and postgres benchmark:

Code:

$ ./bin/examples/poller dbname dbuser dbpass localhost:8333
... wait to download all the blocks
$ make psql
$ time ./bin/tests/psql
... run benchmark to fetch 100 blocks

With my current setup it takes 10 secs to fetch 100 blocks or 0.1 secs for 1 block. That means 4 hours to fetch the entire block-chain of 140k blocks. Using a rule of thumb that verifying the block-chain will be 10 times as slow, that might mean 40 hours to verify all 140k blocks. ATM I plan to just bundle the first 120k pre-verified blocks in the code, then later see about optimising this.

But I am running it over a VPS on rackspace, so that also could be a reason for the speed too.

It should be possible for you to easily modify the current storage engine I'm using to enable sqllite... The library supports multiple backends. See the constructor in src/storage/postgresql_storage.cpp:postgresql_storage() ... The sql_ object is being explicitly initialised with postgresql parameters. I don't mind making this a generic sql_storage backend if you're able to get it to work

The current way of joining the blocks using reorganize_block_chain() will be removed and done as part of the block-chain verification- that's the best way I see currently to do it... But I am doing more research of the current bitcoin sourcecode to make sure I understand everything fully. The current bitcoin is very inefficient/obtuse and there's a whole bunch of ways to do things quicker or nicer.

[vector76]

Ok that makes sense.  I am a bit concerned that even though the update is a single statement, it must touch a lot of rows, but it should be infrequent enough to not be a problem.

My next question is regarding the very common operation of determining whether an output has been spent.  Each output could be spent in a single transaction that belongs to multiple blocks (in separate forks) or by multiple transactions (also in separate forks).

So it seems that to determine if an output has been spent in a particular chain, you will have to query for all the inputs that spend the output, find (JOIN) the transactions that contain those inputs, find (JOIN) the blocks that contain those transactions, and see if any of those blocks have a span range that contains the chain in question.  I know databases are supposed to be good at this kind of thing but it sure sounds like a lot of work.  On the other hand, most of the transactions will not have a double-spend, so all the joining shouldn't make a difference in the most common case where there are no inputs that spend a txout.  All the joining might slow it down a bit when the blockchain forks, but a few extra indexed lookups shouldn't be that big a deal I guess.

I'm thinking I'll switch to your schema with the spans, because it bugs me how my version has dynamic fields marking whether a block is part of the 'best chain' and whether a txout has been spent in the current best chain.  It is a real headache to add a block to a fork other than the current best.

[genjix]

Ok that makes sense.  I am a bit concerned that even though the update is a single statement, it must touch a lot of rows, but it should be infrequent enough to not be a problem.

My next question is regarding the very common operation of determining whether an output has been spent.  Each output could be spent in a single transaction that belongs to multiple blocks (in separate forks) or by multiple transactions (also in separate forks).

So it seems that to determine if an output has been spent in a particular chain, you will have to query for all the inputs that spend the output, find (JOIN) the transactions that contain those inputs, find (JOIN) the blocks that contain those transactions, and see if any of those blocks have a span range that contains the chain in question.  I know databases are supposed to be good at this kind of thing but it sure sounds like a lot of work.  On the other hand, most of the transactions will not have a double-spend, so all the joining shouldn't make a difference in the most common case where there are no inputs that spend a txout.  All the joining might slow it down a bit when the blockchain forks, but a few extra indexed lookups shouldn't be that big a deal I guess.

I'm thinking I'll switch to your schema with the spans, because it bugs me how my version has dynamic fields marking whether a block is part of the 'best chain' and whether a txout has been spent in the current best chain.  It is a real headache to add a block to a fork other than the current best.

I plan to add a flag indicating whether an output is_final (and the parent transaction is_final - if all child outputs all have is_final set) and an output_address and for transaction a transaction_type (standard, generated and other) fields.

Also will be another table with chain_id, cumulative_difficulty... and cumulative difficulty in blocks.

For now still working on the block chain verification specifics (which algorithm is best and whether to organize then verify, pre-verify then organize then verify, or verify then link and organize). There are various pros and cons for all the different ways.

[w1R903]

This is great work.  I'm looking forward to trying out the python API.  Thank you.

[genjix]

Thanks.

I'm disconnecting from the forums for a few days to focus. I can be contacted using my email on the front of bitcoin.org

[zwierzak]

And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

[Matt Corallo]

And what about noSQL databases?

NoSQL is entirely a buzzword and has no meaning whatsoever.  Asking if they will support NoSQL is like asking if they will support about 30 different databases with completely different structures and APIs.

Everyone says that they are faster and more scalable than SQL.

SQL isnt the problem here, its ACID.  If you throw ACID out the window, there are a ton of things you can do to increase performance, which is the reason some of the NoSQL solutions can benchmark so well.  If you are doing something with financial software, not using ACID is about as stupid as you can get (for the actual financial stuff, maybe for the blockchain where you have a bit more leeway and are only updating once every ~10 minutes anyway...)

Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

I can't speak for xelister, but I dont think supporting JSON matters here as its a whole new lib anyway so whether or not the original bitcoin client supports JSON or not is fairly irrelevant.

[Xephan]

And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

BAD idea. NoSQL is only faster because it doesn't do all that ACID checking that transaction-safe RDBMS does. While it's perfectly fine for data that doesn't need to be consistent i.e. nobody's going to particularly care if half the world saw your twit #156483 and the other half only get updated to #156482 for the next 2 minutes. But for financial transaction, having inconsistent data and no referential integrity is malpractice and asking outright to be scammed.

[vector76]

And what about noSQL databases? Everyone says that they are faster and more scalable than SQL. Moreover MongoDB have interface in JSON, so it fits with default bitcoind interface.

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

[Matt Corallo]

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

The type of NoSQL the poster was talking about here appears to be the non-ACID type of which BDB is not one of...

[vector76]

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

The type of NoSQL the poster was talking about here appears to be the non-ACID type of which BDB is not one of...

Writing blocks to a flat file is not very ACID, and there is no attempt at something analogous to BEGIN TRANSACTION/COMMIT when updating the block chain or indices, so a failure part-way through a reorg is very likely to leave the database in an inconsistent state.  Granted, both of these problems are outside of BDB and do not reflect on the strength or weakness of BDB.

I would say that bitcoin is effectively using a NoSQL "style" by not taking advantage of consistency guarantees, or of a relational model, both of which I think it would benefit from.

[Matt Corallo]

Writing blocks to a flat file is not very ACID, and there is no attempt at something analogous to BEGIN TRANSACTION/COMMIT when updating the block chain or indices, so a failure part-way through a reorg is very likely to leave the database in an inconsistent state.  Granted, both of these problems are outside of BDB and do not reflect on the strength or weakness of BDB.

I would say that bitcoin is effectively using a NoSQL "style" by not taking advantage of consistency guarantees, or of a relational model, both of which I think it would benefit from.

Well if the BDB transaction fails, there will be no pointers to the extra crap at the end of the flat file and subsequent blocks will be added later, so all that would happen would be a bit of extra space would be wasted (in theory).  Not quite ACID but not quite leaving the database in an "inconsistent" state.
Anyway, I think weve overtaken this thread enough...

[genjix]

No, don't worry. The conversation is informative. I have no problems if people want to talk about tangential topics here

I uploaded a postgresql dump of the database: https://bitcointalk.org/index.php?topic=38246.0

[David M]

Love your work genjix.

Without downloading the postgresql dump, can you please post the DDL for your schema here or provide just a dump of the schema.

I am very interested in creating a full data model (with ERD's, key discovery, propositions etc...) for Bitcoin using the Relational Model.
After that is created, I would like to help create a transition model for existing accounting/monetary based software packages to incorporate Bitcoins....

I am willing to provide schema's for all major DBMS packages (Mysql, SQL Server, Oracle, DB2 and Postgress)

I am a DBA/modeller with 25 years experience and quite frankly do not have the time ATM to read and collate the schema from the raw code.

[mizerydearia]

Hey, genjix here, posting under someone else's account

It's in the repo, https://gitorious.org/libbitcoin/libbitcoin/blobs/master/bitcoin.sql

[John Tobey]

Monitoring this thread as I build a g++ capable of building libbitcoin.

Meditating on "block-chain verifier subsystem."

[genjix]

Image on how branch deletion works:

[genjix]

Big milestone reached! libbitcoin is the one of the first alternative implementations of the bitcoin protocol (along with node-bitcoin-p2p) to do full blockchain validation! Exciting milestone. It's taken me around 6 hours to download and validate the blockchain.

Here's a sample of validation times at 130k blocks in milliseconds:

112	22	97	316	0
38	407	1795	375
116	51	116	358
81	481	49	325
87	107	18	9
579	56	65	7
2329	98	25	103
8	4	291	155
147	140	185	55
87	15	31	67
12	47	28	16
12	34	92	0

Also I wrote a development statement for libbitcoin (also in the OP).

The Zen of libbitcoin
Readability over speed.
Beauty over convenience.
Simplicity over complexity.
Architected, not hacked.
Flat, not nested.
Explicit, not implicit.
Errors should be loud.
Never is better than right now.
Now is better than never.
Be flexible and configurable.
Build houses from bricks, software from modules.
Stability is god.

Random bitcoin trivia:

First transaction is in block 170

First standard transaction in block 728

The first difficulty change was at block 32256

Blocks 91842 and 91812, and, 91880 and 91722 contain the same duplicate coinbase transaction. The miners can only use that transaction once.
http://blockexplorer.com/block/00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec
http://blockexplorer.com/block/00000000000743f190a18c5577a3c2d2a1f610ae9601ac046a38084ccb7cd721

2016 (amount of blocks for difficulty re-calculation) corresponds to exactly 2 weeks of 10 minute ideal blocks. 210k (amount of blocks for drop in mining reward) only roughly corresponds to 4 years.

There was a forced fork in the blockchain a while back around 710k blocks when someone DoS bitcoin's scripting system using an excessive number of OP_CHECKSIGs. That's why IsStandard() exists today.

Block 142312 was the first block to have a non-standard coinbase. I think this was one of luke-jr's blocks possibly since it follows his pattern of instant spend.

block 546 spends a tx within the same block: 6b0f8a73a56c04b519f1883e8aafda643ba61a30bd1439969df21bea5f4e27e2

[fellowtraveler]

Congratulations! And great work.


We're getting closer to a high-level interface?

[d33tah]

The Zen sounds pretty much like Python's.

[mizerydearia]

The Zen sounds pretty much like Python's.

Code:

The Zen of Python                                                      The Zen of libbitcoin

Beautiful is better than ugly.                                         Readability over speed.
Explicit is better than implicit.                                      Beauty over convenience.
Simple is better than complex.                                         Simplicity over complexity.
Complex is better than complicated.                                    Architected, not hacked.
Flat is better than nested.                                            Flat, not nested.
Sparse is better than dense.                                           Explicit, not implicit.
Readability counts.                                                    Errors should be loud.
Special cases aren't special enough to break the rules.                Never is better than right now.
Although practicality beats purity.                                    Now is better than never.
Errors should never pass silently.                                     Be flexible and configurable.
Unless explicitly silenced.                                            Build houses from bricks, software from modules. 
In the face of ambiguity, refuse the temptation to guess.              Stability is god.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Yeah, that's why I said "pretty much".

I'm just providing immediate access to both to compare them more easily...

[d33tah]

Yeah, that's why I said "pretty much".

[Bitcoin Oz]

Good work genjix. Congrats.

[d33tah]

I'm just providing immediate access to both to compare them more easily...

Ok then. BTW, it's pretty hard to spot "@down" edits IMHO. I noticed this one by mistake.

Anyway, congratulations genjix, I believe this can really speed up Bitcoin clients/tools development!

[jtimon]

Thank you. It seems much more readable than the original.

[log0s]

libbitcoin is the first alternative implementation of the bitcoin protocol to do full blockchain validation!

(Well, the first publicly available one.)

Congratulations!

[slush]

(registering to this thread)

[netrin]

+3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628

[genjix]

Small update:

Thanks to phantomcircuit to DB is much faster and more efficient. I've put up database dumps of the blockchain.

[mizerydearia]

Small update:

Thanks to phantomcircuit to DB is much faster and more efficient. I've put up database dumps of the blockchain.

You should additionally provide hashes for the file, at least md5

[netrin]

Hi Genjix, are you considering implementing the firstbits algorithm in libbitcoin?

[genjix]

Hi Genjix, are you considering implementing the firstbits algorithm in libbitcoin?

Oh! That is clever!

I must certainly will.

Tip: bitcoin addresses always begin with 1, so maybe use Kk5kFb instead.

[netrin]

Glad to hear it! Dropping the '1' prefix has been discussed since the summer with good arguments on either side, however, the '1' prefix distinguishes bitcoin from most altcoin addresses.

[genjix]

Glad to hear it! Dropping the '1' prefix has been discussed since the summer with good arguments on either side, however, the '1' prefix distinguishes bitcoin from most altcoin addresses.

I'm not sure that's a good idea.

I never get confused between a fax number and a telephone number. You usually state what an identifier means- it's far easier to remember.

Bitcoin: 1kk5k

vs

Bitcoin: kk5kfb

The 1 is redundant and not needed. We don't need a checksum to make sure it's a bitcoin address as that is unlikely to happen.

reduce the chance that a typo will lead to a mistaken payment.

There are some good single character checksums such as the Luhn algorithm.

[netrin]

I suggested just as you, including a suffix or prefix checksum. Perhaps you could carry on the discussion this with FreeMoney and SgtSpike who were responsible for the original implementation and maybe Puik who implemented firstbits in the http://blockchain.info service.

I'm not sure where else it's implemented, but I hope it spreads. My primary concern is to get firstbits implemented as deeply into bitcoin as possible. It's far less cumbersome dealing with 5-7 case-insensitive characters than 30 char base58 strings. Since an address is only a hash of the public key, firstbits could become a first/second class citizen of the blockchain and perhaps reduce the size a wee bit. But most importantly, it would make bitcoin more accessible. The '1' prefix is a minor, though interesting, detail.

How would you represent the genesis address?

[genjix]

How would you represent the genesis address?

a1zp1e

Also my vote goes to having it not fixed at any length. You can for instance say a1zp1e or a1zp1ep5q or a1 (as long as there's no conflicting matches- in which case it fails).

[slush]

My primary concern is to get firstbits implemented as deeply into bitcoin as possible.

OT: Agree, I like firstbits idea a lot. I already implemented firstbits lookup on my pool.

[netrin]

How would you represent the genesis address?

a1zp1e

Ha ha. Ok. It's difficult arguing against oneself. I really have no substantial objection to dropping the '1' prefix, I only tried to recall the arguments against. How about this from the neo-diablo advocating committee: Your suggestion would cause an unresolvable backward compatibility issue for many addresses beginning with '11'. For example, in block 134 is the address 13pS5Wui... (original firstbits 13p) and then in block 662 is address 113pDtuG... (original firstbits 113, though you would accept 13p). Under your regime, we could not support an OPTIONAL '1' prefix without extending the minimal length of numerous extant firstbits. Please think of the children.

EDIT: A possible work around exception reminds me of British telephone numbers. If the address begins with '11' then the minimal firstbits would be the firstbits for which dropping the '1' prefix would be unambiguous with all previous firstbits with and without the '1' prefix. All subsequent firstbits would still have to compare against all previous firstbits with and without the '1' prefix. While the '11*' are rare, those subsequent addresses which would newly conflict ('1*' ) could* be numerous.

I wonder how many addresses are actually effected. In order to preserve the beautiful simplicity of the original algorithm, I believe the '1' prefix can not be optional. Either always with or always without. And because of that (if true)*, I advocate always with a '1' prefix.

* Without a list of all addresses and their extant firstbits, I can't attempt to calculate the 'risk' of an optional prefix. Though I was surprised that the first instance I discovered of "13pdt..." in my very incomplete list was only recently (block 134795) and no instance of '113ps..." (again in my incomplete address list).

[genjix]

ok so I just commited an example app to get the balance of a bitcoin address:

Code:

#include <iostream>

#include <bitcoin/types.hpp>
#include <bitcoin/kernel.hpp>
#include <bitcoin/storage/postgresql_storage.hpp>
#include <bitcoin/util/logger.hpp>

using namespace libbitcoin;

void display_balance(const std::error_code& ec, uint64_t value)
{
    if (ec)
    {
        log_fatal() << ec.message();
        return;
    }
    log_info() << "Balance: " << value;
}

int main()
{
    kernel_ptr app(new kernel());
    storage_ptr app_storage(new postgresql_storage(
        app, "bitcoin", "genjix", ""));
    app->register_storage(app_storage);

    data_chunk address = 
        bytes_from_pretty("12 ab 8d c5 88 ca 9d 57 87 dd "
                          "e7 eb 29 56 9d a6 3c 3a 23 8c");
    app_storage->fetch_balance(address, display_balance);

    std::cin.get();
    return 0;
}

Does anyone here know how I could do something like:

Code:

SELECT sum(value) 
FROM outputs 
WHERE script LIKE decode('76a91412ab8dc588ca9', 'hex') + '%';

Where script is a bytea object in order for me to support firstbits?

That way I could add a fetch_balance_partial(...) method to support this.

[netrin]

I assume firstbits would need to be calculated and injected into a table linking to its first block occurrence and full public key. The 33 char base58 address could be dropped entirely from the blockchain and calculated on the fly.

[jimbobway]

Does anyone here know how I could do something like:

Code:

SELECT sum(value) 
FROM outputs 
WHERE script LIKE decode('76a91412ab8dc588ca9', 'hex') + '%';

Where script is a bytea object in order for me to support firstbits?

That way I could add a fetch_balance_partial(...) method to support this.

Maybe create another column of type text that is the text version of the bytea 'script' data.  Then you can use LIKE since each data type is of type  text.

You could use a stored procedure to do the conversion but I think it will take too long.

[John Tobey]

Does anyone here know how I could do something like:

Code:

SELECT sum(value) 
FROM outputs 
WHERE script LIKE decode('76a91412ab8dc588ca9', 'hex') + '%';

Where script is a bytea object in order for me to support firstbits?

I think bytea supports BETWEEN, so this could be (UNTESTED) "script BETWEEN decode('76a91412ab8dc588ca9', 'hex') AND decode('76a91412ab8dc588caa', 'hex')".  Abe does something like this:

Code:

                lo = abe.store.binin_hex(q + '0' * (40 - len(q)))
                hi = abe.store.binin_hex(q + 'f' * (40 - len(q)))

That way I could add a fetch_balance_partial(...) method to support this.

Keep in mind that firstbits is case-insensitive.  1a matches 1a* and 1A*.  You seem willing to implement a new, firstbits-like algorithm, and that's okay, but if it doesn't agree with firstbits.com, you shouldn't call it "firstbits".

[genjix]

Ahhh true, I forgot about the case insensitivity.

Thanks for the advice.

[genjix]

Command line utilities for messing around with private keys and creating your own transactions:

https://bitcointalk.org/index.php?topic=50721.msg604667#msg604667

[genjix]

Ubuntu package available for libbitcoin:

https://launchpad.net/~zgenjix/+archive/libbitcoin

Add these 2 lines to the end of your /etc/apt/sources.list

Code:

deb http://ppa.launchpad.net/zgenjix/libbitcoin/ubuntu oneiric main
deb-src http://ppa.launchpad.net/zgenjix/libbitcoin/ubuntu oneiric main

Code:

$ wget -q "http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x4F8AE60DB3FC740E" -O- | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install libbitcoin-dev

[genjix]

Gentoo ebuild

Kamil Domański has made a gentoo ebuild.

Code:

layman -a bitcoin
emerge libbitcoin

[fivebells]

Minor nit: The OP_CHECKSIG wiki page mentions libbitcoin and

Code:

make ec-key && ./bin/tests/ec-key

 I had to use the development-makefile to get this to start compiling.  Also, it's failing as follows:

Code:

lap% make -f development-makefile ec-key
g++ -std=c++0x -Wall -pedantic -pthread -Wextra -fstack-protector-all -ggdb -fPIC -Iinclude/ -Iusr/include/ -c -o obj/ec-key.o tests/ec-key.cpp
tests/ec-key.cpp: In function ‘bool op_checksig_test()’:
tests/ec-key.cpp:23:221: warning: missing braces around initializer for ‘std::array<unsigned char, 32ul>::value_type [32] {aka unsigned char [32]}’ [-Wmissing-braces]
tests/ec-key.cpp: In function ‘void privkey_tests()’:
tests/ec-key.cpp:114:44: warning: missing braces around initializer for ‘std::array<unsigned char, 32ul>::value_type [32] {aka unsigned char [32]}’ [-Wmissing-braces]
Assembler messages:
Fatal error: can't create obj/ec-key.o: No such file or directory
make: *** [obj/ec-key.o] Error 1

  "mkdir obj" corrects this.  (This is with the apt-get source libbitcoin source.)

[slush]

Ubuntu package available for libbitcoin:

https://launchpad.net/~zgenjix/+archive/libbitcoin

Error: Page not found. It should be probably "genjix" instead of "zgenjix"

[bitclown]

The config script now checks for bdb 5.1, but 5.1 is not included in the list of versions checked at m4/ax_berkeley_db_cxx.m4:60.

The Gentoo ebuild should pull it in a slot:

Code:

DEPEND="sys-libs/db:5.1 ..."

Oh, and ridiculous nice work by the way!

[genjix]

Thanks both. We'll correct those things.

I've made experimental alpha bindings for libbitcoin. Caveat emptor.

Program which connects to a bitcoin node and downloads the first 500 block hashes.

Code:

from bitcoin import *
import time

net = network()
chandle = None

def receive_inv(inv_packet):
    print inv_packet
    for inventory in inv_packet.invs:
        print inventory.type, inventory.hash

def handle_send_getblocks(ec):
    if ec:
        # do something
        print ec

def create_getblocks_message():
    packet = getblocks()
    gen_block = genesis_block()
    print 'c'
    packet.locator_start_hashes.append(hash_block_header(gen_block))
    packet.hash_stop.nullify()
    return packet

def handle_handshake(ec, host_id):
    if ec:
        # do something
        print ec
        return
    chandle = host_id
    net.subscribe_inv(chandle, receive_inv)
    net.send_getblocks(chandle, create_getblocks_message(), handle_send_getblocks)

if __name__ == '__main__':
    handshake_connect(net, "localhost", 8333, handle_handshake)
    time.sleep(2)

Another hacked up example just to test everything:

Code:

from bitcoin import *
import time
i = net_addr()
i.ip_addr = [1, 2, 3]
print i.ip_addr
print inv_type.block
h = hash_digest([1, 2, 3])
h[12] = 110
print h[12]
vers = version()
vers.version = 100
vers.sub_version_num = "abc"
print vers
# from block 170
pubkey = data_chunk([0x04, 0x11, 0xdb, 0x93, 0xe1, 0xdc, 0xdb, 0x8a, 0x01, 0x6b, 0x49, 0x84, 0x0f, 0x8c, 0x53, 0xbc, 0x1e, 0xb6, 0x8a, 0x38, 0x2e, 0x97, 0xb1, 0x48, 0x2e, 0xca, 0xd7, 0xb1, 0x48, 0xa6, 0x90, 0x9a, 0x5c, 0xb2, 0xe0, 0xea, 0xdd, 0xfb, 0x84, 0xcc, 0xf9, 0x74, 0x44, 0x64, 0xf8, 0x2e, 0x16, 0x0b, 0xfa, 0x9b, 0x8b, 0x64, 0xf9, 0xd4, 0xc0, 0x3f, 0x99, 0x9b, 0x86, 0x43, 0xf6, 0x56, 0xb4, 0x12, 0xa3])
print len(pubkey)
print pubkey
print pubkey_to_address(pubkey)
print genesis_block().merkle
print 'Only', max_money(), 'bitcoins will ever exist'
print original_dialect()
s = script()
o = operation()
o.code = opcode.special
s.push_operation(o)
o.code = opcode.nop
o.data = data_chunk([0xde, 0xad, 0xbe, 0xef])
s.push_operation(o)
o = operation()
o.code = opcode.hash160
s.push_operation(o)
print s
tx = transaction()
print tx

n = network()

def foo(ec, hostid):
    if ec:
        print ec
    print 'connected', hostid

handshake_connect(n, "localhost", 8333, foo)

time.sleep(10)

I'm not too knowledgeable on making bindings, so consider this an experiment.

https://gitorious.org/libbitcoin/python-bitcoin

[slush]

genjix, thanks for great work. I have been waiting for python binding and now I have finally something to play with :-).

[genjix]

That's cool But beware that they're a total experimental hack since I don't know what I'm doing when it comes to bindings.

If you need to ask me questions, I'm on Freenode IRC in #bitcoinconsultancy as genjix

[kdomanski]

The config script now checks for bdb 5.1, but 5.1 is not included in the list of versions checked at m4/ax_berkeley_db_cxx.m4:60.

The Gentoo ebuild should pull it in a slot:

Code:

DEPEND="sys-libs/db:5.1 ..."

Thanks, I fixed the ebuild. Also, storage backends are now enabled and disabled by use flags.

[genjix]

What is a good name for the blockchain + acting functions? Currently it's called 'storage'

ledger
blockchain
backend
...

There are other components for 'wallet' and 'node_list'. This only concerns blocks, block locators and transactions in the block chain.

I'm leaning towards backend.

[netrin]

What is 'acting'? I prefer to see personal/private data clearly distinguished from public data by name.

[genjix]

What is 'acting'? I prefer to see personal/private data clearly distinguished from public data by name.

acting = functions that transform or perform operations on the data (public member functions of the interface)

[jtimon]

I like "blockchain" more than "backend". But I don't know exactly what functions are there so...

[slush]

Any idea? It's fresh git clone.

Code:

$ autoconf 
configure.ac:5: error: possibly undefined macro: AM_INIT_AUTOMAKE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:18: error: possibly undefined macro: AM_CONDITIONAL
configure.ac:33: error: possibly undefined macro: AC_MSG_ERROR

[genjix]

ok storage is now called blockchain.

Also, on http://libbitcoin.org/ is a video about the design philosophy behind libbitcoin. I make one mistake in the video:

Code:

f2(); f1(); f3();

Will be called in the order 2, 1, 3, but not be allowed to run at the same time. Not sure why I tripped up there on that- guess it was being on the spot kind of thing

@slush, try autoreconf -i

[btc_artist]

Is this still licensed under the AGPL?

[slush]

@slush, try autoreconf -i

Code:

$ autoreconf -i
Can't exec "libtoolize": Adresář nebo soubor neexistuje at /usr/bin/autoreconf line 196.
Use of uninitialized value in pattern match (m//) at /usr/bin/autoreconf line 196.
src/Makefile.am:1: Libtool library used but `LIBTOOL' is undefined
src/Makefile.am:1:   The usual way to define `LIBTOOL' is to add `AC_PROG_LIBTOOL'
src/Makefile.am:1:   to `configure.ac' and run `aclocal' and `autoconf' again.
src/Makefile.am:1:   If `AC_PROG_LIBTOOL' is in `configure.ac', make sure
src/Makefile.am:1:   its definition is in aclocal's search path.
autoreconf: automake failed with exit status: 1

[slush]

OK, I just made clean git clone. Autoconf failed again, autoconf -i did something, then autoconf passed. I have another error in ./configure now:

Code:

$ ./configure 
configure: error: cannot find install-sh, install.sh, or shtool in build-aux "."/build-aux

I'm little confused. Am I doing anything wrong? My system is pretty standard Ubuntu 11.10.

[genjix]

OK, I just made clean git clone. Autoconf failed again, autoconf -i did something, then autoconf passed. I have another error in ./configure now:

Code:

$ ./configure 
configure: error: cannot find install-sh, install.sh, or shtool in build-aux "."/build-aux

I'm little confused. Am I doing anything wrong? My system is pretty standard Ubuntu 11.10.

Try this:

$ sudo apt-get install build-essential autoconf libtool libdb++-dev protobuf pkg-config libboost-all-dev
$ autoreconf -i

I'm not on the forums too often.

Feel free to poke me on Freenode. I'm either in #bitcoinconsultancy or I'm lurking. Or you can use my email: genjix@riseup.net

Then for fun, you can run the test cases as:

$ make -f development-makefile net bdb-test
$ ./bin/tests/nettest
$ mkdir database
$ ./bin/tests/bdb-test

... and so on (look under tests/)

[kdomanski]

Be advised, libbitcoin 0.1.0 alpha 1 is now tagged in the repository.

Gentoo ebuild for this version is available in the "bitcoin" overlay.
Binary packages are not available at this time and will possibly arrive with soon-to-be-tagged alpha 2.


Also, subvertx 0.1.0 is tagged.
See this announcement for more info.

[ahtremblay]

The project is very good, however I believe the license of this project is, frankly, ridiculous. This library has a use that is similar to openssl, curl in that it can form the basis of a wide range of bitcoin applications. Limiting this library to be used  in open source projects only really kills a ton of project including mine. I really believe you should reconsider the license and release it as LGPL.

[netrin]

While I too would like to see the code released as L-*, frankly your 'ridiculous' comment is extreme. Do you have any intension to compensate Genjix? Will you pledge to contribute improvements upstream? The GPL guarantees that you will... or you won't.

[genjix]

We'll see about the license afterwards. It's always easier to switch to weaker copyleft than vice versa. For now I want to build.

Post about new organize algorithm below.

[genjix]

A document describing the functioning of the organize algorithm.

New blocks when they first come in are added to the orphans pool. The
pool does not order the blocks. To trace a chain of blocks from an endpoint
you have to keep looking up the previous block as you go backwards.



Assuming we have a valid genesis block, then this pool would produce
2 chains like this:



We loop through every block in the orphan's pool, setting each block to
processed each loop. This stops us re-processing a block when a new block is
added again to the pool, re-triggering the organize algorithm.

The blockchain is only ever stored as one main chain. In the blockchain
database is only stored the currently accepted history. Blocks added to the
blockchain are taken out of the orphan's pool, and returned back to the pool
when removed from the blockchain.

If we have the chain:

Code:

0 -> 1 -> 2a -> 3a

Then the remaining blocks in the orphan pool are 2b, 3b, 4b. Block 4b comes
into the pool and will trigger a re-organize. When we run the organize
algorithm, we find out that the chain 2b, 3b, 4b has greater difficulty.

• Trace backwards through the pool until we cannot cycle backwards anymore. We call this the orphan-root.
• Organize will first find the fork point within the blockchain to be at block 1 using the orphan-root's previous block hash.
• The difficulty in the remaining blocks of the main chain and the chain in the orphan pool are calculated. If the main chain algorithm's difficulty is bigger than the orphan's, then the algorithm stops and the main chain is untouched.
• The new blocks in the orphan pool will replace the ones in the main chain. The end blocks from the main chain are popped off and returned back to the orphan pool. The blocks from the orphan pool are added to the main chain as the new chain.
• Reorganization has finished.

Code:

0 -> 1 -> 2b -> 3b -> 4b

[genjix]

Max of 80 characters. 4 space indents.

Variables, functions, class-names, enums, typedefs are lowercase. Macros are uppercase. Private class member variables are suffixed with an _

Each line should be indented by either 1 or 2 levels on the next line, and 1 line for each successive indent.

If you're going more than 3 levels deep with each new line of code, then generally you are doing something wrong and code should be siphoned off into functions. Always prefer: flat over nested, dumb over clever, slow+clear over fast+obtuse.

Functions should never be defined in headers. Always in the implementation unless it's required to be in every translation unit (i.e template or constexpr functions).

Format for header file:

Code:

#ifndef LIBBITCOIN_FOO_H
#define LIBBITCOIN_FOO_H

// derived from project includes (i.e blaa.hpp)

// system includes:
// STD headers first
// boost headers second
// other headers last

// project bitcoin includes like <bitcoin/messages.hpp>

// local bitcoin includes "helper_foo.hpp"

// forward declarations
// class Db;

namespace libbitcoin {

// using ...

// typedefs

class foo
 : public blaa
{
public:
  // typedefs

  // functions logically grouped into blocks

  // variables

private:
  // same as for public block
};

} // libbitcoin

#endif

Format for cpp file:

Code:

#include <bitcoin/foo.hpp>

// same format as header files

namespace libbitcon {

foo::foo()
 : blaa(xyz)
{
}

} // libbitcoin

Every { occupies a newline except for namespaces.

Spaces between all operators: x = 5 * 8 + 4 * 2 ... If the order is confusing then use brackets: x = (5 * + (4 * 2). Each : ; (in for loops) or , has a space after them.

All flow control should always happen on new lines.

Code:

// Bad
if (blaa) do_foo();

// Good
if (blaa)
    do_foo();

// Bad
something(); another_thing(); yadayada();

// Good
something(); 
another_thing(); 
yadayada();

It's nice if function names read like a sentence when used: write_to_stream(data_packet); or say_hello_to(my_friend);

Accepted short-hands

These are for variables and not for type names which should never use abbreviations.

tx = transaction

blk = block

ec = error_code

it = iterator

[netrin]

Is there any length after which a blockchain is 'permanent' in your code? Is it technically possible for a reorg after ten or one hundred blocks since the orphan-root?

[genjix]

Is there any length after which a blockchain is 'permanent' in your code? Is it technically possible for a reorg after ten or one hundred blocks since the orphan-root?

Yes, theoretically.

No, practically, because of the checkpoints in the block validation code.

[netrin]

That was yes AND no, both to the second question, right?

[genjix]

That was yes AND no, both to the second question, right?

No :p There are checkpoints in bitcoin as an added security measure. If they didn't exist then there would be no limits to reverse the chain. However it isn't a concern. The longest re-organisation ever was 6 blocks deep.

[netrin]

Are those checkpoints in the public block chain or in your implementation? perhaps you could briefly describe those check points?

These?
https://github.com/bitcoin/bitcoin/pull/534

https://en.bitcoin.it/wiki/Vocabulary

Checkpoint Lockin
    Every once in a while, a recent block hash is hardcoded into Bitcoin. This prevents pretty much any possible attack from affecting transactions made up to this point. No matter what happens (except perhaps if SHA-256 is broken), these transactions will survive. Satoshi announced the feature here(broken link) and it was discussed to death here (broken link).

[genjix]

Yes. It isn't a requirement to be compliant with the protocol, but it is a little extra added security which is nice.

[kjj]

The project is very good, however I believe the license of this project is, frankly, ridiculous. This library has a use that is similar to openssl, curl in that it can form the basis of a wide range of bitcoin applications. Limiting this library to be used  in open source projects only really kills a ton of project including mine. I really believe you should reconsider the license and release it as LGPL.

Spoken like a true Open Source-er!

Free Software and Open Source are different things, with different goals.  My apologies if you understand the difference, but your post strongly suggests that you do not.

And no, the license doesn't kill your project (or any other, really).  You just can't use this software for it unless you comply with the terms of the license.  You still have the option to do all of the work yourself, or pay someone to do it for you.  You even have the option of negotiating the purchase of a special license from Genjix if you want.

And no, I have no idea what Genjix's stance is on Free Software vs. Open Source.  I'm personally a big fan of Free Software, so I hope he is too, thus I hope that he retains the AGPL license.  But as the creator of the software, the decision on license terms is entirely up to him, so if he feels that this is a commodity library, destined to be one among many, then GPL, LGPL, or something else will be entirely appropriate.

[genjix]

The question here is not GPL or no GPL, the question is either:

AGPL or lesser AGPL.

I've been working with the Aaron Williamson of the SFLC and Richard Stallman of the FSF to fix a bug in the AGPL for bitcoin.

Then I will weigh up the various arguments for adding or not adding a lesser clause. It is whichever will promote development of the library the most. *GPL licenses mean you can either contribute sourcecode or $$$ to benefit the library (by buying proprietary licenses).

Right now as it stands, this library still needs to be completed so these arguments are a time waster

The bug I'm referring to is that the AGPL requires not only that you give the sourcecode on demand, but that you're proactive in providing the sourcecode visibly. This is problematic because if you make a small modification while developing or building on a compilation based system (like gentoo) then the act of connecting to the p2p network means you have to provide source. There is no means to do this in the bitcoin sourcecode and the idea itself is problematic; every build must store the sourcecode in a bundle which is served using custom bitcoin protocol extensions.

From my understanding, modifying the AGPL for them is complicated and so far the last proposal was:

> If the covered work has no means of communicating an offer to                                                                                                          
> provide Corresponding Source to the users interacting with it                                                                                                          
> remotely over a computer network, then you may comply with this                                                                                                        
> requirement by making the Corresponding Source for your version                                                                                                        
> available for anyone to copy, free of charge and under the terms of                                                                                                    
> this License, through a publicly available network server or other                                                                                                    
> readily accessible means.

This means that you are able to provide the source code through some other means (such as sending it via email if requested) and it solves the issue.

------

In an ideal world, copyrights, patents, trademarks and censorship wouldn't exist. Proprietary software would not be able to support itself against rampant piracy. And I would put my sourcecode out in the open for anyone to use as they like.

However, we don't live in that ideal world. And I will use what little power I have in the form of the GPL to fight this asymmetric warfare. The balance is totally tilted away from our favour, so why wouldn't you use the only protection afforded to you as a developer.

Licenses like the GPL hack the law, twisting it to our will. It is the best example of subversion.

When people talk about GPL or MIT licenses being free, they're talking about different kinds of freedom as they're essentially rivalrous freedoms. Being free does not always mean no rules necessarily:

According to Assange, "It’s not correct to put me in any one philosophical or economic camp, because I’ve learned from many. But one is American libertarianism, market libertarianism. So as far as markets are concerned I’m a libertarian, but I have enough expertise in politics and history to understand that a free market ends up as monopoly unless you force them to be free."

And this coming from a person with anarchist tendencies.

[genjix]

I tell you what though, short of writing my own berkeley db wrapper there is no decent way to abstract this libary. Their 'C++ API' is essentially the same as their C one except you don't call DB->put(mydb, ...) but instead do mydb->put(...). Nice. Not sure why they bothered.

[netrin]

Perhaps we can fork a thread on licensing.

Lesser or not, I think the Affero licenses fit nicely within the bitcoin ecosystem. It would be excellent if the bitcoin community limited (or combined) the licenses used by its various projects. The Satoshi client uses MIT (3-clause BSD), but later projects seem to be focusing on the GPL family.

Is the community better served by sharing code or supporting proprietary businesses?

[genjix]

heh didn't notice before, but block 546 spends a tx within the same block:
http://blockexplorer.com/tx/6b0f8a73a56c04b519f1883e8aafda643ba61a30bd1439969df21bea5f4e27e2
check input 0

[btc_artist]

Is the community better served by sharing code or supporting proprietary businesses?

As much as I like *free* software, I think we need to support proprietary businesses if Bitcoin is to go mainstream.  And I don't see that conflicting with my ideals or the Bitcoin ideals.  I think an LGPL license would be ideal for libbitcoin.  Proprietary programs can use/link it, but if they make any improvements to libbitcoin itself, they must be made available to pull back into libbitcoin.  I think it's a perfect compromise between a GPL license and an MIT/BSD style license.

[ahtremblay]

I will not contribute any code to this project unless the license is changed to LGPL.

[Mark Oates]

As much as I like *free* software, I think we need to support proprietary businesses if Bitcoin is to go mainstream.

I agree with this 100%

I'm also thrilled that libbitcoin is here.  In my opinion, its a milestone.

Has anyone tried compiling on MSVC?

[maaku]

I, too, would use and contribute to this project if it were released under a more permissive license (BSD preferably, LGPL if you must). As it stands I cannot even look at this code due to the possibility of legal risk to my company.

[genjix]

Today we enter a new era with all core components done. But this is not the end. This is the beginning!

Everything is still far from finished:

https://bitcoinconsultancy.com/wiki/index.php?title=Libbitcoin

Services:

- blockchain
-- bdb_blockchain (berkeley db)
-- postgresql_blockchain
- network
- transaction memory pool

Other:

- scripting system
- full validation of blocks and unconfirmed transactions
- all the usual utilities like base58, encryption, hashing

Todo (minor things):

- wallet
- node discovery

Headers:

include/bitcoin/error.hpp
include/bitcoin/address.hpp
include/bitcoin/network/channel.hpp
include/bitcoin/network/discovery.hpp
include/bitcoin/network/handshake.hpp
include/bitcoin/network/shared_const_buffer.hpp
include/bitcoin/network/network.hpp
include/bitcoin/block.hpp
include/bitcoin/data_helpers.hpp
include/bitcoin/exporter.hpp
include/bitcoin/validate.hpp
include/bitcoin/constants.hpp
include/bitcoin/transaction_pool.hpp
include/bitcoin/types.hpp
include/bitcoin/messages.hpp
include/bitcoin/blockchain/postgresql_blockchain.hpp
include/bitcoin/blockchain/organizer.hpp
include/bitcoin/blockchain/bdb_blockchain.hpp
include/bitcoin/blockchain/blockchain.hpp
include/bitcoin/utility/threads.hpp
include/bitcoin/utility/elliptic_curve_key.hpp
include/bitcoin/utility/assert.hpp
include/bitcoin/utility/ripemd.hpp
include/bitcoin/utility/logger.hpp
include/bitcoin/utility/sha256.hpp
include/bitcoin/utility/base58.hpp
include/bitcoin/utility/big_number.hpp
include/bitcoin/utility/serializer.hpp
include/bitcoin/utility/clock.hpp
include/bitcoin/script.hpp
include/bitcoin/transaction.hpp

The benchmark I use for how the API is progressing is the classical poller application which has become much more intuitive and simple yet flexible and powerful since the early days:

Code:

#include <bitcoin/bitcoin.hpp>
using namespace libbitcoin;

using std::placeholders::_1;
using std::placeholders::_2;

class pollapp
  : public threaded_service,
    public std::enable_shared_from_this<pollapp>
{
public:
    pollapp();

    void start(std::string hostname, unsigned int port);

private:
    void handle_connect(const std::error_code& ec, channel_ptr node);
    void initial_ask_blocks(const std::error_code& ec,
        const message::block_locator& loc);

    void recv_inv(const std::error_code& ec,
        const message::inventory& packet);
    void recv_blk(const std::error_code& ec,
        const message::block& blk);

    void handle_store(const std::error_code& ec, block_info info,
        const hash_digest& block_hash);
    void ask_blocks(const std::error_code& ec,
        const message::block_locator& loc, const hash_digest& hash_stop);

    network_ptr network_;
    handshake_ptr handshake_;
    blockchain_ptr chain_;
    channel_ptr node_;
};

typedef std::shared_ptr<pollapp> pollapp_ptr;

pollapp::pollapp()
{
    network_ = std::make_shared<network>();
    handshake_ = std::make_shared<handshake>();
    chain_ = std::make_shared<bdb_blockchain>("database/");
}

void pollapp::start(std::string hostname, unsigned int port)
{
    handshake_->connect(network_, "localhost", 8333,
        std::bind(&pollapp::handle_connect, shared_from_this(), _1, _2));
}

void pollapp::handle_connect(const std::error_code& ec, channel_ptr node)
{
    if (ec)
    {
        log_fatal() << ec.message();
        return;
    }
    node_ = node;
    chain_->fetch_block_locator(
        std::bind(&pollapp::initial_ask_blocks, shared_from_this(), _1, _2));
}

void handle_send_packet(const std::error_code& ec)
{
    if (ec)
        log_error() << ec.message();
}

void pollapp::initial_ask_blocks(const std::error_code& ec,
    const message::block_locator& loc)
{
    if (ec)
    {
        log_fatal() << ec.message();
        return;
    }
    node_->subscribe_inventory(
        std::bind(&pollapp::recv_inv, shared_from_this(), _1, _2));
    node_->subscribe_block(
        std::bind(&pollapp::recv_blk, shared_from_this(), _1, _2));
    ask_blocks(ec, loc, null_hash);
}

void pollapp::recv_inv(const std::error_code& ec,
    const message::inventory& packet)
{
    if (ec)
    {
        log_fatal() << ec.message();
        return;
    }
    message::get_data getdata;
    for (const message::inventory_vector& ivv: packet.inventories)
    {
        if (ivv.type != message::inventory_type::block)
            continue;
        getdata.inventories.push_back(ivv);
    }
    node_->send(getdata, handle_send_packet);
    // Re-subscribe
    node_->subscribe_inventory(
        std::bind(&pollapp::recv_inv, shared_from_this(), _1, _2));
}

void pollapp::recv_blk(const std::error_code& ec,
    const message::block& blk)
{
    if (ec)
    {
        log_fatal() << ec.message();
        return;
    }
    chain_->store(blk,
        std::bind(&pollapp::handle_store, shared_from_this(),
            _1, _2, hash_block_header(blk)));
    // Re-subscribe
    node_->subscribe_block(
        std::bind(&pollapp::recv_blk, shared_from_this(), _1, _2));
}

void pollapp::handle_store(const std::error_code& ec, block_info info,
    const hash_digest& block_hash)
{
    if (ec)
    {
        log_fatal() << ec.message();
        return;
    }
    switch (info.status)
    {
        case block_status::orphan:
            chain_->fetch_block_locator(
                std::bind(&pollapp::ask_blocks, shared_from_this(),
                    _1, _2, block_hash));
            break;

        case block_status::rejected:
            log_error() << "Rejected block " << pretty_hex(block_hash);
            break;

        case block_status::confirmed:
            log_debug() << "block #" << info.depth;
            break;
    }
}

void pollapp::ask_blocks(const std::error_code& ec,
    const message::block_locator& loc, const hash_digest& hash_stop)
{
    if (ec)
    {
        log_fatal() << ec.message();
        return;
    }
    message::get_blocks packet;
    packet.locator_start_hashes = loc;
    packet.hash_stop = hash_stop;
    node_->send(packet, std::bind(&handle_send_packet, _1));
}

int main(int argc, const char** argv)
{
    bdb_blockchain::setup("database/");

    pollapp_ptr app = std::make_shared<pollapp>();
    app->start("localhost", 8333);
    // Wait for CTRL-D
    while (true)
    {
        char n;
        std::cin >> n;
        if (std::cin.eof())
            break;
    }
    return 0;
}

[fellowtraveler]

Can you please post a code sample of how to perform multi-sign transactions on the blockchain using your library.

This is a necessary piece for me.

[genjix]

Can you please post a code sample of how to perform multi-sign transactions on the blockchain using your library.

This is a necessary piece for me.

Multi-signature transactions are restricted by IsStandard() right now, so you won't be able to use them until 15th Feb. They will be available then.

https://en.bitcoin.it/wiki/BIP_0016

On February 1, 2012, the block-chain will be examined to determine the number of blocks supporting pay-to-script-hash for the previous 7 days. If 550 or more contain "/P2SH/" in their coinbase, then all blocks with timestamps after 15 Feb 2012, 00:00:00 GMT shall have their pay-to-script-hash transactions fully validated. Approximately 1,000 blocks are created in a week; 550 should, therefore, be approximately 55% of the network supporting the new feature.

If a majority of hashing power does not support the new validation rules, then rollout will be postponed (or rejected if it becomes clear that a majority will never be achieved).

[fellowtraveler]

1. I'm sure there are mining groups now who would accept certain more exotic transaction types.

2. Even if multi-sign isn't accepted yet, I still need to get started on my code, so I still need to see a code sample using your library (preferably.) It will be months before this stuff is fully ready,  IMO.

3. Once low-trust servers are popping up to take advantage of such code, I can almost guarantee you there will be a sea change in the Bitcoin community as various miners, exchanges, et al jump on board.

Need sample code!


------------------------

 

[genjix]

OK, I'm getting ready to finalise a 1.0 release by the end of this month (Feb).

License will now be Lesser AGPL. You can link and use libbitcoin in your closed source project, but the changes you make to libbitcoin must be public. I will give explicit permission through email to people 'myusername'@riseup.net (signed with my GPG key) if you want to be sure.

Will be heavily documenting and writing tutorials soon.

On the development front:
- I have written a blockchain fuzzer which I will use to stress test the blockchain algorithm.
- Node discovery needs to be completed (should be easy- the tricky part is designing a good API).
- A high level synchronous interface for easy programming and integration of bitcoin in your project. This might not make it to 1.0 release. fellowtraveler, this is what you're asking for.

Right now the project is still rough around the edges, but it is rapidly approaching production capability in a rough way. We will have to smooth out some things while using it and the API may change a lot early on. I will also be making a test client as a proof of concept in order to perfect the API better.

libbitcoin requires advanced C++ knowledge (for now). It is heavily built around asynchronous interfaces and the library is thread safe. An understanding of thread contexts is required but I'll document this in the tutorials.

[btc_artist]

License will now be Lesser AGPL.

You mean the Lesser GPL (LGPL) license, right? http://www.gnu.org/licenses/lgpl.html  To my (very limited) knowledge, a Lesser AGPL license doesn't exist.

[Red Emerald]

Keep up the great work.

Reading through this is much easier than trying to read through the Satoshi client although there aren't too many comments (yet (hopefully)).

[genjix]

Red Emerald:
As the codebase matures I'll start adding a ton of comments. The thing I hate more than uncommented code is out of date comments.

License will now be Lesser AGPL.

You mean the Lesser GPL (LGPL) license, right? http://www.gnu.org/licenses/lgpl.html  To my (very limited) knowledge, a Lesser AGPL license doesn't exist.

I'm collaborating with the SFLC/FSF to sort that out. They resolved a bunch of licensing issues for me already.

First tutorial (of many) is written. Just a high level overview. Next one should have some code, and then we'll go back and break everything down as we go along.
[libbitcoin] Overview
libbitcoin is a bitcoin library targeted towards high end use. The library places a heavy focus around asychronicity. The enables a big scope for future scalability as each component has its own thread pool. By increasing the number of threads for that component the library is able to scale outwards across CPU cores. This will be vital in the future as the demands of the bitcoin network grow.

I tried to explain the design principles behind libbitcoin and why things are done how they are.

[btc_artist]

Good deal.

[LightRider]

Well done! Looking forward to a richer bitcoin ecosystem.

[fellowtraveler]

License will now be Lesser AGPL.

You mean the Lesser GPL (LGPL) license, right? http://www.gnu.org/licenses/lgpl.html  To my (very limited) knowledge, a Lesser AGPL license doesn't exist.

Here it is:  http://mo.morsi.org/blog/node/270

[genjix]

Python bindings + tutorials are going to be released later this week:

Code:

import bitcoin
import time

class Application:

    def __init__(self):
        self.stopped = False
        self.net = bitcoin.network()
        self.channel = None

    def start(self):
        self.net.connect("localhost", 8333, self.handle_connect)

    def stop(self):
        self.stopped = True

    def is_stopped(self):
        return self.stopped

    def create_version_message(self):
        vers = bitcoin.version()
        vers.version = 60000
        vers.services = 1
        vers.address_me.servies = 1
        vers.address_me.ip = \
            [0, 0, 0, 0, 0, 0, 0, 0, 
             0, 0, 255, 255, 127, 0, 0, 1]
        vers.address_me.port = 8333
        vers.address_you.services = 1
        vers.address_you.ip = \
            [0, 0, 0, 0, 0, 0, 0, 0, 
             0, 0, 255, 255, 127, 0, 0, 1]
        vers.address_you.port = 8333
        vers.user_agent = "/libbitcoin:0.4/example:1/";
        vers.start_height = 0
        vers.nonce = 42
        return vers

    # First we send our version message then the node will reply back
    def handle_connect(self, ec, channel):
        # check the error_code
        if ec:
            print 'Could not connect:', ec
            self.stop()
            return
        self.channel = channel
        version_message = self.create_version_message()
        self.channel.send_version(version_message, self.handle_send)
        self.channel.subscribe_version(self.read_version_reply)

    def handle_send(self, ec):
        if ec:
            print 'Problem sending:', ec
            self.stop()

    def read_version_reply(self, ec, vers):
        if ec:
            print 'Problem in reply:', ec
            self.stop()
            return
        # Display the version message back
        print vers.address_me.ip
        self.stop()

if __name__ == "__main__":
    app = Application()
    app.start()
    while not app.is_stopped():
        time.sleep(0.1)

Some more stuff (crappy/hackish testing files):

Code:

import bitcoin


d = bitcoin.data_chunk("001212")
print d
h = bitcoin.hash_digest("000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f")
print h
if h == "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f":
    print 'Yes'
print len(h)

tx = bitcoin.transaction()
print bitcoin.hash_transaction(tx)

netaddr = bitcoin.network_address()
print netaddr.ip

s = bitcoin.script()
o = bitcoin.operation()
o.code = bitcoin.opcode.special
s.push_operation(o)
o.code = bitcoin.opcode.nop
o.data = bitcoin.data_chunk("deadbeef")
s.push_operation(o)
o = bitcoin.operation()
o.code = bitcoin.opcode.hash160
s.push_operation(o)
print s
print s.operations()
print s.type()

Code:

import bitcoin

raw_tx_repr = "010000000187493c4c15c76df9f69ddd7aeb7ffbcddad4b7a979210f19602282d5b9862581000000008a47304402202d9e9f75be9c8a4c4304931b032e1de83fd2c6af2c1154a3d2b885efd5c3bfda02201184139215fb74499eae9c71ae86354c41b4d20b95a6b1fffcb8f1c5f051288101410497d11f5c33adb7c3fed0adc637358279de04f72851b7b93fb4a8655613729047c7e2908966551b5fb7f6899f6c3dd358b57eb20a61b2c9909aa106eac6310f9fffffffff0140420f00000000001976a91407e761706c63b36e5a328fab1d94e9397f40704d88b000000000"
raw_tx = bitcoin.data_chunk(raw_tx_repr)
print raw_tx
print len(raw_tx)

ex = bitcoin.satoshi_exporter()
tx = ex.load_transaction(raw_tx)
print "txhash", bitcoin.hash_transaction(tx)
print tx
print ex.save_transaction(tx)
print len(ex.save_transaction(tx))
assert str(ex.save_transaction(tx)) == raw_tx_repr

Code:

import bitcoin

ec = bitcoin.elliptic_curve_key()
print ec.new_key_pair()
privdat = ec.private_key()
print privdat

ec1 = bitcoin.elliptic_curve_key()
ec1.set_private_key(privdat)
assert str(ec1.private_key()) == str(privdat)

h = bitcoin.hash_digest("f003f0c1193019db2497a675fd05d9f2edddf9b67c59e677c48d3dbd4ed5f00b")
print h
sig = ec1.sign(h)
print ec.verify(h, sig)

[genjix]

libbitcoin is licensed as Lesser AGPL:
- When used in the p2p network, you only need to provide changes on demand (LGPL).
- When used on a webserver, you must proactively provide sourcecode for any changes you have made to libbitcoin.
- Applications can link against libbitcoin, and they do not need to release their changes publically.
Thanks to Aaron Williamson of the SFLC and Richard Stallman of the FSF for helping draft up this license.

[genjix]

[libbitcoin] First steps
This tutorial will create a simple program in Python using libbitcoin’s Python bindings to read the version information of a remote bitcoin node. We will need to connect to the bitcoin node and send it a version packet. The other bitcoin node should respond back with their own version packet which we can examine.

[Dusty]

Is there a (even not so) simple way to use this library to dump the raw bytes of a transaction or even a full block?

I.e: something like example #3 of the post above, but in reverse: give the program the hash of a tx so he can get it from the network or the db, and dump it to stdout.

Thanks for any help

[Red Emerald]

So I'm running debian 6.0.4 (squeeze) and trying to get this installed for a small project.

I'm just going to install it on an ubuntu system instead, but figured I would post my error log anyway.

http://pastie.org/private/3jnuenhclxaf4f90wpbyfw

[genjix]

So I'm running debian 6.0.4 (squeeze) and trying to get this installed for a small project.

I'm just going to install it on an ubuntu system instead, but figured I would post my error log anyway.

http://pastie.org/private/3jnuenhclxaf4f90wpbyfw

https://bitcoinconsultancy.com/wiki/Build_libbitcoin

Your g++ is outdated.

@znort, try google-protobuf or something. And dont use the boost headers in ubuntu... they are missing async_connect function. Use 1.48 from boostr website as per instructions above.

@Dusty, sure. Use the network component to connect to a bitcoin node, then fashion a get_data packet. Send it to the node and subscribe to either transactions or blocks. Then once you received the block and/or tx, create an exporter object to serialise them and print the bytes to the screen. Hopefully I can get this next tutorial out soon, and it will clarify some points for you. Just been very busy this week with exchange stuff :/ Constantly stressed for time

[Red Emerald]

So I'm running debian 6.0.4 (squeeze) and trying to get this installed for a small project.

I'm just going to install it on an ubuntu system instead, but figured I would post my error log anyway.

http://pastie.org/private/3jnuenhclxaf4f90wpbyfw

https://bitcoinconsultancy.com/wiki/Build_libbitcoin

Your g++ is outdated.

Yeah I noticed your requirements have higher requirements for libboost and g++.  I needed to upgrade my ubuntu server anyways, so I'm building it on there now. Installing boost is sure taking a while.

[Red Emerald]

Add these 2 lines to the end of your /etc/apt/sources.list

Code:

deb http://ppa.launchpad.net/zgenjix/libbitcoin/ubuntu oneiric main
deb-src http://ppa.launchpad.net/zgenjix/libbitcoin/ubuntu oneiric main

I think this should be "http://ppa.launchpad.net/genjix" without a "z"

I also added it to "/etc/apt/sources.list.d/libbitcoin.list" instead of "/etc/apt/sources.list"

Now lets see if I can get the python bindings built

Code:

[user@silo python-bitcoin]$ make
mkdir -p bitcoin
g++ -fPIC -Wall -ansi `pkg-config --cflags libbitcoin` -I/usr/include/python2.7  -c main.cpp -o main.o
In file included from /home/user/usr/include/bitcoin/exporter.hpp:7:0,
                 from /home/user/usr/include/bitcoin/network/channel.hpp:17,
                 from /home/user/usr/include/bitcoin/bitcoin.hpp:8,
                 from main.cpp:4:
/home/user/usr/include/bitcoin/constants.hpp:37:75: warning: missing braces around initializer for ‘std::array<unsigned char, 32ul>::value_type [32] {aka unsigned char [32]}’ [-Wmissing-braces]
/home/user/usr/include/bitcoin/constants.hpp:39:62: warning: missing braces around initializer for ‘std::array<unsigned char, 20ul>::value_type [20] {aka unsigned char [20]}’ [-Wmissing-braces]
main.cpp: In member function ‘void handshake_wrapper::start(boost::python::api::object)’:
main.cpp:288:61: error: no matching function for call to ‘libbitcoin::handshake::start(pyfunction<const std::error_code&>)’
main.cpp:288:61: note: candidate is:
/home/user/usr/include/bitcoin/network/handshake.hpp:31:10: note: void libbitcoin::handshake::start(libbitcoin::channel_ptr, libbitcoin::handshake::handshake_handler)
/home/user/usr/include/bitcoin/network/handshake.hpp:31:10: note:   candidate expects 2 arguments, 1 provided
main.cpp: In member function ‘void handshake_wrapper::ready(channel_wrapper, boost::python::api::object)’:
main.cpp:299:14: error: ‘class libbitcoin::handshake’ has no member named ‘ready’
/home/user/usr/include/boost/system/error_code.hpp: At global scope:
/home/user/usr/include/boost/system/error_code.hpp:214:35: warning: ‘boost::system::posix_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/system/error_code.hpp:215:35: warning: ‘boost::system::errno_ecat’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/system/error_code.hpp:216:35: warning: ‘boost::system::native_ecat’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:244:45: warning: ‘boost::asio::error::system_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:246:45: warning: ‘boost::asio::error::netdb_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:248:45: warning: ‘boost::asio::error::addrinfo_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:250:45: warning: ‘boost::asio::error::misc_category’ defined but not used [-Wunused-variable]
make: *** [default] Error 1

[genjix]

@znort, try google-protobuf or something. And dont use the boost headers in ubuntu... they are missing async_connect function. Use 1.48 from boostr website as per instructions above.

In case anyone is actually interested, the package is actually called "libprotobuf-dev"

These requirements makes libbitcoin fairly unusable for a large number of developers:
installing the very latest and greatest g++ and boost are:

    A) a fairly annoying undertaking.
    B) not always an option, in particular for production systems

Also, note that the --with-boost= flag to configure seems to have no effect whatsoever:
the Makefile generated keeps on using the system boost headers instead of the ones in
the boost directory specified on the configure cmd line.

This precludes having multiple installs of boost (which is required for a whole swath of
other things to keep functionning).

The only way I got the damn thing to compile was to force a -I to point in the right place
in the CXX and CC env. vars prior to running configure, not exactly a clean or particularly
obvious way to get there.

Bottom line: libbitcoin seems nice, but it's a PITA to compile and adoption will thereby suffer.

https://bitcoinconsultancy.com/wiki/Build_libbitcoin

Follow that. I'm using boost built to a local directory.

The library is still early. These dependencies will all become standard and part of Linux distributions within a few months. It's a worthwhile gamble since they add significant quality to the library.

Add these 2 lines to the end of your /etc/apt/sources.list

Code:

deb http://ppa.launchpad.net/zgenjix/libbitcoin/ubuntu oneiric main
deb-src http://ppa.launchpad.net/zgenjix/libbitcoin/ubuntu oneiric main

I think this should be "http://ppa.launchpad.net/genjix" without a "z"

I also added it to "/etc/apt/sources.list.d/libbitcoin.list" instead of "/etc/apt/sources.list"

Now lets see if I can get the python bindings built

Code:

[user@silo python-bitcoin]$ make
mkdir -p bitcoin
g++ -fPIC -Wall -ansi `pkg-config --cflags libbitcoin` -I/usr/include/python2.7  -c main.cpp -o main.o
In file included from /home/user/usr/include/bitcoin/exporter.hpp:7:0,
                 from /home/user/usr/include/bitcoin/network/channel.hpp:17,
                 from /home/user/usr/include/bitcoin/bitcoin.hpp:8,
                 from main.cpp:4:
/home/user/usr/include/bitcoin/constants.hpp:37:75: warning: missing braces around initializer for ‘std::array<unsigned char, 32ul>::value_type [32] {aka unsigned char [32]}’ [-Wmissing-braces]
/home/user/usr/include/bitcoin/constants.hpp:39:62: warning: missing braces around initializer for ‘std::array<unsigned char, 20ul>::value_type [20] {aka unsigned char [20]}’ [-Wmissing-braces]
main.cpp: In member function ‘void handshake_wrapper::start(boost::python::api::object)’:
main.cpp:288:61: error: no matching function for call to ‘libbitcoin::handshake::start(pyfunction<const std::error_code&>)’
main.cpp:288:61: note: candidate is:
/home/user/usr/include/bitcoin/network/handshake.hpp:31:10: note: void libbitcoin::handshake::start(libbitcoin::channel_ptr, libbitcoin::handshake::handshake_handler)
/home/user/usr/include/bitcoin/network/handshake.hpp:31:10: note:   candidate expects 2 arguments, 1 provided
main.cpp: In member function ‘void handshake_wrapper::ready(channel_wrapper, boost::python::api::object)’:
main.cpp:299:14: error: ‘class libbitcoin::handshake’ has no member named ‘ready’
/home/user/usr/include/boost/system/error_code.hpp: At global scope:
/home/user/usr/include/boost/system/error_code.hpp:214:35: warning: ‘boost::system::posix_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/system/error_code.hpp:215:35: warning: ‘boost::system::errno_ecat’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/system/error_code.hpp:216:35: warning: ‘boost::system::native_ecat’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:244:45: warning: ‘boost::asio::error::system_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:246:45: warning: ‘boost::asio::error::netdb_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:248:45: warning: ‘boost::asio::error::addrinfo_category’ defined but not used [-Wunused-variable]
/home/user/usr/include/boost/asio/error.hpp:250:45: warning: ‘boost::asio::error::misc_category’ defined but not used [-Wunused-variable]
make: *** [default] Error 1

I would stay away from the packages for the time being. Once I make a 1.0 release this month then I will update them.

If anyone manages to get it to build, then build instructions would be much appreciated. I wrote these:
https://bitcoinconsultancy.com/wiki/Build_libbitcoin
off the top of my head. I don't have a clean environment around to test, and I'm also very stressed for time what with the exchange, bitcoin media, libbitcoin and bitcoin conference requirements. I'm trying to devote 80% of my time on this library so it means I have to be really efficient with my time.

Also there's a new tutorial I will post below!

[genjix]

[libbitcoin] First 500 blocks
libbitcoin is a toolkit based library and can be leveraged with different design patterns depending on the task or application. Previously we examined using libbitcoin using an active object pattern. In this example we will use an imperative style program to connect to a bitcoin node and download the first 500 blocks.

[Red Emerald]

I would stay away from the packages for the time being. Once I make a 1.0 release this month then I will update them.

If anyone manages to get it to build, then build instructions would be much appreciated. I wrote these:
https://bitcoinconsultancy.com/wiki/Build_libbitcoin

I installed from the package first, and when that didn't work I used the guide at bitcoinconsultancy.  While that works, I still can't get the python bindings built. I'm flying blind though since you don't have any docs for the python module.

[genjix]

are you using the latest repos from http://gitorious.org/libbitcoin/ ?

If so can you paste the error here (if different from above). It looks like you're using a libbitcoin from a few days ago but latest python bindings.

[Red Emerald]

are you using the latest repos from http://gitorious.org/libbitcoin/ ?

If so can you paste the error here (if different from above). It looks like you're using a libbitcoin from a few days ago but latest python bindings.

Updating to the newest git head for libbitcoin seems to have fixed the last error. I don't know how I ended up with versions that didn't match considering I literally installed the system fresh last night and checked out the projects one after the other, but w/e.

Weeee!

Code:

(env)[user@silo onion-coin]$ python -i
Python 2.7.2+ (default, Oct  4 2011, 20:06:09) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import bitcoin
>>> dir(bitcoin)
['_1', '_2', '_3', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '_bitcoin', 'acceptor', 'address', 'address_to_short_hash', 'bdb_blockchain', 'bind', 'block', 'block_info', 'block_locator', 'block_locator_indices', 'block_reward', 'block_status', 'block_value', 'block_work', 'blockchain', 'bytes_from_pretty', 'channel', 'coin_price', 'coinbase_maturity', 'coinbase_script', 'data_chunk', 'decode_base58', 'elliptic_curve_key', 'encode_base58', 'error', 'error_code', 'exporter', 'generate_merkle_root', 'generate_ripemd_hash', 'generate_sha256_checksum', 'generate_sha256_hash', 'genesis_block', 'get_address', 'get_blocks', 'get_data', 'handshake', 'hash_block_header', 'hash_digest', 'hash_digest_from_pretty', 'hash_transaction', 'header', 'indices_list', 'input_point', 'inventory', 'inventory_list', 'inventory_type', 'inventory_vector', 'is_coinbase', 'magic_value', 'max_bits', 'max_money', 'max_target', 'network', 'network_address', 'network_address_list', 'null_hash', 'null_short_hash', 'opcode', 'opcode_to_string', 'operation', 'operation_stack', 'output_point', 'output_point_list', 'parse_script', 'payment_type', 'ping', 'previous_output_is_null', 'public_key_to_address', 'readjustment_interval', 'reward_interval', 'satoshi_exporter', 'save_script', 'script', 'setup_bdb_blockchain', 'short_hash', 'short_hash_from_pretty', 'short_hash_wrapper', 'sighash', 'string_to_opcode', 'target_spacing', 'target_timespan', 'total_output_value', 'transaction', 'transaction_input', 'transaction_input_list', 'transaction_list', 'transaction_output', 'transaction_output_list', 'verack', 'version']

So I'm trying to use your tutorials, but they aren't working for me.  I'm not running bitcoind locally, so I think that might be why.

Both tutorial.py and first500.py give this:

Code:

$ python tutorial.py 
s: version (112 bytes)
Bad header received.

For tutorial.py, I changed self.net.connect to my system running bitcoind and also I tried changing vers.address_you.ip.

For first500.py, I changed hs.connect to my system, but it also failed.


I am trying to build a lightweight tool for querying nodes to get their version.  Right now, one of my systems has all of the tor hidden services added as nodes, but many of those services are offline and it takes forever trying to connect to all of them.  I was hoping to use PyTorCtl and libbitcoin to map all of the addresses and then return versions for all of the nodes that are up.  I could then drop this list of nodes that I know to be online and running current versions into my bitcoin.conf.  From there it wouldn't be too much work to make it into a bot that updates the wiki page.

It seems to connect, but the handshake is failing.

[genjix]

Yes, that's very weird. It should be fine connecting to localhost. Have you got an IP address I can connect to your bitcoind? It could be that you're running an old version of bitcoin (the version packet has changed since older versions). If that's the case then trying the latest should work.

I'm on Skype as zgenjix or Freenode IRC in #bitcoinconsultancy as genjix

Ring ring, live support. How can I help you sir?

[genjix]

client with a single private key, http://gitorious.org/libbitcoin/libbitcoin/blobs/master/tests/crap/main.cpp
(bad hackjob code for testing)

[shackra]

same problem here with GCC4.7

[genjix]

same problem here with GCC4.7

GCC 4.7 has a bug in it. http://stackoverflow.com/questions/4940259/c11-lambdas-require-capturing-this-to-call-static-member-function

I need to get around to installing that compiler and adding a workaround for that. GCC 4.6 is fine.

No idea why Linux distributions like Arch are including an unstable compiler!!

[grondilu]

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

Has Satoshi ever explained his choice for BerkeleyDB instead of a SQL-based relationnal db?

[Dusty]

Berkeley DB, as it's being used in the existing bitcoin client, is a perfect example of why not to use NoSQL.

Could you please elaborate...?

[genjix]

OK, seems like Ubuntu Quantal uses g++ 4.7 (release date is October) and many distributions are phasing in the new toolchain. I think it's because g++ 4.7 implements the vast majority of the new C++ standard so it's being fast tracked in.

I'm upgrading to a system with 4.7 so hopefully I should be get it working with the new compiler now.

[genjix]

Fixed for g++ 4.7

Related: http://gcc.gnu.org/ml/gcc-help/2012-02/msg00201.html

[genjix]

- serialisation has changed. more intuitive and uses native c++ features to support extendability for saving and loading of bitcoin messages.
- async_service now provides access to underlying io_service ptr which is now the requirement for services.

[da2ce7]

I'm putting a bounty down for the creation of a QT client based upon libbitcoin.  (must be judged to be 'beta' by me).

10btc.

[genjix]

I'm putting a bounty down for the creation of a QT client based upon libbitcoin.  (must be judged to be 'beta' by me).

10btc.

Example quick text-interface client for anyone who wants to have a go at trying to make a Qt version: https://gitorious.org/libbitcoin/libbitcoin/blobs/master/tests/crap/main.cpp

[genjix]

The problem is that all the Linux distributions are moving towards g++ 4.7 which implements the C++11 standard fully and were complaining about compilation problems.

It turns out that g++ 4.6 implements the C++11 incorrectly which causes an incompatibility for me (that I had to workaround using macros). I eventually said whatever and decided to fully transition to 4.7 as it will become the norm in the new Ubuntu which is being released in ~2 months time.

More info:
http://gcc.gnu.org/projects/cxx0x.html
http://stackoverflow.com/questions/7214623/rule-for-lambda-capture-variable
https://llvm.org/viewvc/llvm-project/libcxx/trunk/include/system_error?view=markup&pathrev=153194 (incorrect noexcept specifier in 4.6)

I plan to make Ubuntu builds again when the new version (Quantal) is released.

[Andreas Schildbach]

Ubuntu 12.10 is still 4 months away. Also, many people (like me) will stay on 12.04 because this is a release with long term support.

[Red Emerald]

Ubuntu 12.10 is still 4 months away. Also, many people (like me) will stay on 12.04 because this is a release with long term support.

+1

I plan on running LTS for a while.  If that means libbitcoin is going to be a PITA, that is too bad.

[genjix]

Ubuntu 12.10 is still 4 months away. Also, many people (like me) will stay on 12.04 because this is a release with long term support.

+1

I plan on running LTS for a while.  If that means libbitcoin is going to be a PITA, that is too bad.

Hmmm OK. I'll setup a new VPS to get compatibility with the LTS again. I'm making some simplifications, optimisations and cleanup of the interface for the next release.

[genjix]

OK, compatibility with LTS:

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/INSTALL

See the section labelled 'Ubuntu Instructions'. You will need the following line to install the dependencies:

sudo apt-get install build-essential autoconf libtool libboost1.48-all-dev libdb++-dev libprotobuf-dev pkg-config libcurl4-openssl-dev

Then simply run the autoreconf -i && configure && make && sudo make install step(s)

[Red Emerald]

OK, compatibility with LTS:

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/INSTALL

See the section labelled 'Ubuntu Instructions'. You will need the following line to install the dependencies:

sudo apt-get install build-essential autoconf libtool libboost1.48-all-dev libdb++-dev libprotobuf-dev pkg-config libcurl4-openssl-dev

Then simply run the autoreconf -i && configure && make && sudo make install step(s)

That was fast.  I'm going through the instructions now.

EDIT: I was able to build libbitcoin without problem. However, I am getting errors with the python bindings

Code:

$ git clone git://gitorious.org/libbitcoin/python-bitcoin.git
$ cd python-bitcoin
$ make
mkdir -p bitcoin
g++ -fPIC -Wall -ansi `pkg-config --cflags libbitcoin` -I/usr/include/python2.7  -c main.cpp -o main.o
main.cpp: In member function ‘void handshake_wrapper::connect(network_wrapper, const string&, uint16_t, boost::python::api::object)’:
main.cpp:301:14: error: ‘class libbitcoin::handshake’ has no member named ‘connect’
main.cpp: At global scope:
main.cpp:612:39: error: expected ‘)’ before ‘ex’
main.cpp:713:5: error: ‘exporter_ptr’ in namespace ‘bc’ does not name a type
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_header(const libbitcoin::message::header&) const’:
main.cpp:619:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_version(const libbitcoin::message::version&) const’:
main.cpp:623:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_verack(const libbitcoin::message::verack&) const’:
main.cpp:627:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_address(const libbitcoin::message::address&) const’:
main.cpp:631:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_get_address(const libbitcoin::message::get_address&) const’:
main.cpp:635:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_inventory(const libbitcoin::message::inventory&) const’:
main.cpp:639:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_get_data(const libbitcoin::message::get_data&) const’:
main.cpp:643:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_get_blocks(const libbitcoin::message::get_blocks&) const’:
main.cpp:647:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_transaction(const libbitcoin::message::transaction&) const’:
main.cpp:651:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_block(const libbitcoin::message::block&) const’:
main.cpp:655:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_ping(const libbitcoin::message::ping&) const’:
main.cpp:659:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::header exporter_wrapper::load_header(const data_chunk&) const’:
main.cpp:664:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::version exporter_wrapper::load_version(const data_chunk&) const’:
main.cpp:668:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::verack exporter_wrapper::load_verack(const data_chunk&) const’:
main.cpp:672:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::address exporter_wrapper::load_address(const data_chunk&) const’:
main.cpp:676:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::get_address exporter_wrapper::load_get_address(const data_chunk&) const’:
main.cpp:680:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::inventory exporter_wrapper::load_inventory(const data_chunk&) const’:
main.cpp:684:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::get_data exporter_wrapper::load_get_data(const data_chunk&) const’:
main.cpp:688:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::get_blocks exporter_wrapper::load_get_blocks(const data_chunk&) const’:
main.cpp:692:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::transaction exporter_wrapper::load_transaction(const data_chunk&) const’:
main.cpp:697:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::block exporter_wrapper::load_block(const data_chunk&) const’:
main.cpp:701:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘libbitcoin::message::ping exporter_wrapper::load_ping(const data_chunk&) const’:
main.cpp:705:16: error: ‘ex_’ was not declared in this scope
main.cpp: In member function ‘bool exporter_wrapper::verify_header(const libbitcoin::message::header&) const’:
main.cpp:710:16: error: ‘ex_’ was not declared in this scope
main.cpp: In function ‘exporter_wrapper create_satoshi_exporter()’:
main.cpp:718:46: error: ‘satoshi_exporter’ is not a member of ‘bc’
main.cpp:718:46: error: ‘satoshi_exporter’ is not a member of ‘bc’
main.cpp:718:68: error: no matching function for call to ‘make_shared()’
main.cpp:718:68: note: candidate is:
/usr/include/c++/4.6/bits/shared_ptr.h:547:5: note: template<class _Tp, class ... _Args> std::shared_ptr<_Tp1> std::make_shared(_Args&& ...)
main.cpp: In member function ‘void blockchain_wrapper::fetch_block_locator(boost::python::api::object)’:
main.cpp:775:17: error: ‘class libbitcoin::blockchain’ has no member named ‘fetch_block_locator’
main.cpp: In function ‘void fetch_block_by_depth(blockchain_wrapper, size_t, boost::python::api::object)’:
main.cpp:845:53: error: no matching function for call to ‘fetch_block(libbitcoin::blockchain_ptr, size_t&, pyfunction<const std::error_code&, const libbitcoin::message::block&>)’
main.cpp:845:53: note: candidates are:
/usr/local/include/bitcoin/blockchain/blockchain.hpp:305:6: note: void libbitcoin::fetch_block(libbitcoin::blockchain&, size_t, libbitcoin::blockchain_fetch_handler_block)
/usr/local/include/bitcoin/blockchain/blockchain.hpp:305:6: note:   no known conversion for argument 1 from ‘libbitcoin::blockchain_ptr {aka std::shared_ptr<libbitcoin::blockchain>}’ to ‘libbitcoin::blockchain&’
/usr/local/include/bitcoin/blockchain/blockchain.hpp:323:6: note: void libbitcoin::fetch_block(libbitcoin::blockchain&, const hash_digest&, libbitcoin::blockchain_fetch_handler_block)
/usr/local/include/bitcoin/blockchain/blockchain.hpp:323:6: note:   no known conversion for argument 1 from ‘libbitcoin::blockchain_ptr {aka std::shared_ptr<libbitcoin::blockchain>}’ to ‘libbitcoin::blockchain&’
main.cpp: In function ‘void fetch_block_by_hash(blockchain_wrapper, const hash_digest&, boost::python::api::object)’:
main.cpp:853:53: error: no matching function for call to ‘fetch_block(libbitcoin::blockchain_ptr, const hash_digest&, pyfunction<const std::error_code&, const libbitcoin::message::block&>)’
main.cpp:853:53: note: candidates are:
/usr/local/include/bitcoin/blockchain/blockchain.hpp:305:6: note: void libbitcoin::fetch_block(libbitcoin::blockchain&, size_t, libbitcoin::blockchain_fetch_handler_block)
/usr/local/include/bitcoin/blockchain/blockchain.hpp:305:6: note:   no known conversion for argument 1 from ‘libbitcoin::blockchain_ptr {aka std::shared_ptr<libbitcoin::blockchain>}’ to ‘libbitcoin::blockchain&’
/usr/local/include/bitcoin/blockchain/blockchain.hpp:323:6: note: void libbitcoin::fetch_block(libbitcoin::blockchain&, const hash_digest&, libbitcoin::blockchain_fetch_handler_block)
/usr/local/include/bitcoin/blockchain/blockchain.hpp:323:6: note:   no known conversion for argument 1 from ‘libbitcoin::blockchain_ptr {aka std::shared_ptr<libbitcoin::blockchain>}’ to ‘libbitcoin::blockchain&’
main.cpp: In function ‘void fetch_block_locator(blockchain_wrapper, boost::python::api::object)’:
main.cpp:861:61: error: invalid initialization of non-const reference of type ‘libbitcoin::blockchain&’ from an rvalue of type ‘libbitcoin::blockchain_ptr {aka std::shared_ptr<libbitcoin::blockchain>}’
/usr/local/include/bitcoin/blockchain/blockchain.hpp:341:6: error: in passing argument 1 of ‘void libbitcoin::fetch_block_locator(libbitcoin::blockchain&, libbitcoin::blockchain_fetch_handler_block_locator)’
main.cpp: In function ‘blockchain_wrapper create_bdb_blockchain(async_service_wrapper, const string&, boost::python::api::object)’:
main.cpp:874:9: error: ‘create’ is not a member of ‘libbitcoin::bdb_blockchain’
main.cpp: In constructor ‘transaction_pool_wrapper::transaction_pool_wrapper(async_service_wrapper, blockchain_wrapper)’:
main.cpp:921:17: error: ‘create’ is not a member of ‘libbitcoin::transaction_pool’
main.cpp: In constructor ‘session_wrapper::session_wrapper(async_service_wrapper, hosts_wrapper, handshake_wrapper, network_wrapper, protocol_wrapper, blockchain_wrapper, poller_wrapper, transaction_pool_wrapper)’:
main.cpp:969:28: error: uninitialized reference member in ‘struct libbitcoin::session_params’
/usr/local/include/bitcoin/session.hpp:18:16: note: ‘libbitcoin::session_params::handshake_’ should be initialized
main.cpp:969:28: error: uninitialized reference member in ‘struct libbitcoin::session_params’
/usr/local/include/bitcoin/session.hpp:19:15: note: ‘libbitcoin::session_params::protocol_’ should be initialized
main.cpp:969:28: error: uninitialized reference member in ‘struct libbitcoin::session_params’
/usr/local/include/bitcoin/session.hpp:20:17: note: ‘libbitcoin::session_params::blockchain_’ should be initialized
main.cpp:969:28: error: uninitialized reference member in ‘struct libbitcoin::session_params’
/usr/local/include/bitcoin/session.hpp:21:13: note: ‘libbitcoin::session_params::poller_’ should be initialized
main.cpp:969:28: error: uninitialized reference member in ‘struct libbitcoin::session_params’
/usr/local/include/bitcoin/session.hpp:22:23: note: ‘libbitcoin::session_params::transaction_pool_’ should be initialized
main.cpp:970:11: error: ‘struct libbitcoin::session_params’ has no member named ‘hosts_’
main.cpp:971:29: error: no match for ‘operator=’ in ‘p.libbitcoin::session_params::handshake_ = handshake_wrapper::hs()()’
main.cpp:971:29: note: candidate is:
/usr/local/include/bitcoin/network/handshake.hpp:32:10: note: void libbitcoin::handshake::operator=(const libbitcoin::handshake&) <deleted>
/usr/local/include/bitcoin/network/handshake.hpp:32:10: note:   no known conversion for argument 1 from ‘libbitcoin::handshake_ptr {aka std::shared_ptr<libbitcoin::handshake>}’ to ‘const libbitcoin::handshake&’
main.cpp:972:11: error: ‘struct libbitcoin::session_params’ has no member named ‘network_’
main.cpp:973:31: error: no match for ‘operator=’ in ‘p.libbitcoin::session_params::protocol_ = protocol_wrapper::prot()()’
main.cpp:973:31: note: candidate is:
/usr/local/include/bitcoin/network/protocol.hpp:28:10: note: void libbitcoin::protocol::operator=(const libbitcoin::protocol&) <deleted>
/usr/local/include/bitcoin/network/protocol.hpp:28:10: note:   no known conversion for argument 1 from ‘libbitcoin::protocol_ptr {aka std::shared_ptr<libbitcoin::protocol>}’ to ‘const libbitcoin::protocol&’
main.cpp:974:26: error: no match for ‘operator=’ in ‘p.libbitcoin::session_params::poller_ = poller_wrapper::p()()’
main.cpp:974:26: note: candidate is:
/usr/local/include/bitcoin/poller.hpp:9:7: note: libbitcoin::poller& libbitcoin::poller::operator=(const libbitcoin::poller&) <deleted>
/usr/local/include/bitcoin/poller.hpp:9:7: note:   no known conversion for argument 1 from ‘libbitcoin::poller_ptr {aka std::shared_ptr<libbitcoin::poller>}’ to ‘const libbitcoin::poller&’
main.cpp:975:37: error: no match for ‘operator=’ in ‘p.libbitcoin::session_params::blockchain_ = blockchain_wrapper::chain()()’
main.cpp:975:37: note: candidates are:
/usr/local/include/bitcoin/blockchain/blockchain.hpp:18:7: note: libbitcoin::blockchain& libbitcoin::blockchain::operator=(const libbitcoin::blockchain&)
/usr/local/include/bitcoin/blockchain/blockchain.hpp:18:7: note:   no known conversion for argument 1 from ‘libbitcoin::blockchain_ptr {aka std::shared_ptr<libbitcoin::blockchain>}’ to ‘const libbitcoin::blockchain&’
/usr/local/include/bitcoin/blockchain/blockchain.hpp:18:7: note: libbitcoin::blockchain& libbitcoin::blockchain::operator=(libbitcoin::blockchain&&)
/usr/local/include/bitcoin/blockchain/blockchain.hpp:18:7: note:   no known conversion for argument 1 from ‘libbitcoin::blockchain_ptr {aka std::shared_ptr<libbitcoin::blockchain>}’ to ‘libbitcoin::blockchain&&’
main.cpp:976:36: error: no match for ‘operator=’ in ‘p.libbitcoin::session_params::transaction_pool_ = transaction_pool_wrapper::p()()’
main.cpp:976:36: note: candidate is:
/usr/local/include/bitcoin/transaction_pool.hpp:65:10: note: void libbitcoin::transaction_pool::operator=(const libbitcoin::transaction_pool&) <deleted>
/usr/local/include/bitcoin/transaction_pool.hpp:65:10: note:   no known conversion for argument 1 from ‘libbitcoin::transaction_pool_ptr {aka std::shared_ptr<libbitcoin::transaction_pool>}’ to ‘const libbitcoin::transaction_pool&’
main.cpp: In function ‘void init_module__bitcoin()’:
main.cpp:1188:36: error: no matching function for call to ‘boost::python::class_<libbitcoin::message::transaction>::def(const char [9], <unresolved overloaded function type>)’
main.cpp:1188:36: note: candidates are:
/usr/include/boost/python/class.hpp:224:11: note: template<class Derived> boost::python::class_<T, X1, X2, X3>::self& boost::python::class_::def(const boost::python::def_visitor<Derived>&) [with Derived = Derived, W = libbitcoin::message::transaction, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified, boost::python::class_<T, X1, X2, X3>::self = boost::python::class_<libbitcoin::message::transaction>]
/usr/include/boost/python/class.hpp:234:11: note: template<class F> boost::python::class_<T, X1, X2, X3>::self& boost::python::class_::def(const char*, F) [with F = F, W = libbitcoin::message::transaction, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified, boost::python::class_<T, X1, X2, X3>::self = boost::python::class_<libbitcoin::message::transaction>]
/usr/include/boost/python/class.hpp:243:11: note: template<class A1, class A2> boost::python::class_<T, X1, X2, X3>::self& boost::python::class_::def(const char*, A1, const A2&) [with A1 = A1, A2 = A2, W = libbitcoin::message::transaction, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified, boost::python::class_<T, X1, X2, X3>::self = boost::python::class_<libbitcoin::message::transaction>]
/usr/include/boost/python/class.hpp:250:11: note: template<class Fn, class A1, class A2> boost::python::class_<T, X1, X2, X3>::self& boost::python::class_::def(const char*, Fn, const A1&, const A2&) [with Fn = Fn, A1 = A1, A2 = A2, W = libbitcoin::message::transaction, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified, boost::python::class_<T, X1, X2, X3>::self = boost::python::class_<libbitcoin::message::transaction>]
/usr/include/boost/python/class.hpp:266:11: note: template<class Fn, class A1, class A2, class A3> boost::python::class_<T, X1, X2, X3>::self& boost::python::class_::def(const char*, Fn, const A1&, const A2&, const A3&) [with Fn = Fn, A1 = A1, A2 = A2, A3 = A3, W = libbitcoin::message::transaction, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified, boost::python::class_<T, X1, X2, X3>::self = boost::python::class_<libbitcoin::message::transaction>]
main.cpp:1298:27: error: ‘pretty’ is not a member of ‘libbitcoin::script’
main.cpp:1321:35: error: ‘channel_stopped’ is not a member of ‘libbitcoin::error’
In file included from /usr/include/c++/4.6/bits/shared_ptr.h:52:0,
                 from /usr/include/c++/4.6/memory:86,
                 from /usr/include/boost/function/function_base.hpp:16,
                 from /usr/include/boost/function/detail/prologue.hpp:17,
                 from /usr/include/boost/function/function_template.hpp:13,
                 from /usr/include/boost/function/detail/maybe_include.hpp:13,
                 from /usr/include/boost/function/function0.hpp:11,
                 from /usr/include/boost/python/errors.hpp:13,
                 from /usr/include/boost/python/handle.hpp:11,
                 from /usr/include/boost/python/args_fwd.hpp:10,
                 from /usr/include/boost/python/args.hpp:10,
                 from /usr/include/boost/python.hpp:11,
                 from main.cpp:1:
/usr/include/c++/4.6/bits/shared_ptr_base.h: In constructor ‘std::_Sp_counted_ptr_inplace<_Tp, _Alloc, _Lp>::_Sp_counted_ptr_inplace(_Alloc, _Args&& ...) [with _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::hosts>, std::shared_ptr<libbitcoin::handshake>, std::shared_ptr<libbitcoin::network>}, _Tp = libbitcoin::protocol, _Alloc = std::allocator<libbitcoin::protocol>, __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]’:
/usr/include/c++/4.6/bits/shared_ptr_base.h:518:8:   instantiated from ‘std::__shared_count<_Lp>::__shared_count(std::_Sp_make_shared_tag, _Tp*, const _Alloc&, _Args&& ...) [with _Tp = libbitcoin::protocol, _Alloc = std::allocator<libbitcoin::protocol>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::hosts>, std::shared_ptr<libbitcoin::handshake>, std::shared_ptr<libbitcoin::network>}, __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]’
/usr/include/c++/4.6/bits/shared_ptr_base.h:987:35:   instantiated from ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_make_shared_tag, const _Alloc&, _Args&& ...) [with _Alloc = std::allocator<libbitcoin::protocol>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::hosts>, std::shared_ptr<libbitcoin::handshake>, std::shared_ptr<libbitcoin::network>}, _Tp = libbitcoin::protocol, __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]’
/usr/include/c++/4.6/bits/shared_ptr.h:317:64:   instantiated from ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_make_shared_tag, const _Alloc&, _Args&& ...) [with _Alloc = std::allocator<libbitcoin::protocol>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::hosts>, std::shared_ptr<libbitcoin::handshake>, std::shared_ptr<libbitcoin::network>}, _Tp = libbitcoin::protocol]’
/usr/include/c++/4.6/bits/shared_ptr.h:535:39:   instantiated from ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = libbitcoin::protocol, _Alloc = std::allocator<libbitcoin::protocol>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::hosts>, std::shared_ptr<libbitcoin::handshake>, std::shared_ptr<libbitcoin::network>}]’
/usr/include/c++/4.6/bits/shared_ptr.h:551:42:   instantiated from ‘std::shared_ptr<_Tp1> std::make_shared(_Args&& ...) [with _Tp = libbitcoin::protocol, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::hosts>, std::shared_ptr<libbitcoin::handshake>, std::shared_ptr<libbitcoin::network>}]’
main.cpp:407:39:   instantiated from here
/usr/include/c++/4.6/bits/shared_ptr_base.h:410:4: error: no matching function for call to ‘libbitcoin::protocol::protocol(libbitcoin::async_service&, std::shared_ptr<libbitcoin::hosts>, std::shared_ptr<libbitcoin::handshake>, std::shared_ptr<libbitcoin::network>)’
/usr/include/c++/4.6/bits/shared_ptr_base.h:410:4: note: candidate is:
/usr/local/include/bitcoin/network/protocol.hpp:24:5: note: libbitcoin::protocol::protocol(libbitcoin::async_service&, libbitcoin::hosts&, libbitcoin::handshake&, libbitcoin::network&)
/usr/local/include/bitcoin/network/protocol.hpp:24:5: note:   no known conversion for argument 2 from ‘std::shared_ptr<libbitcoin::hosts>’ to ‘libbitcoin::hosts&’
/usr/include/c++/4.6/bits/shared_ptr_base.h: In constructor ‘std::_Sp_counted_ptr_inplace<_Tp, _Alloc, _Lp>::_Sp_counted_ptr_inplace(_Alloc, _Args&& ...) [with _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::blockchain>}, _Tp = libbitcoin::poller, _Alloc = std::allocator<libbitcoin::poller>, __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]’:
/usr/include/c++/4.6/bits/shared_ptr_base.h:518:8:   instantiated from ‘std::__shared_count<_Lp>::__shared_count(std::_Sp_make_shared_tag, _Tp*, const _Alloc&, _Args&& ...) [with _Tp = libbitcoin::poller, _Alloc = std::allocator<libbitcoin::poller>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::blockchain>}, __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]’
/usr/include/c++/4.6/bits/shared_ptr_base.h:987:35:   instantiated from ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_make_shared_tag, const _Alloc&, _Args&& ...) [with _Alloc = std::allocator<libbitcoin::poller>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::blockchain>}, _Tp = libbitcoin::poller, __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]’
/usr/include/c++/4.6/bits/shared_ptr.h:317:64:   instantiated from ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_make_shared_tag, const _Alloc&, _Args&& ...) [with _Alloc = std::allocator<libbitcoin::poller>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::blockchain>}, _Tp = libbitcoin::poller]’
/usr/include/c++/4.6/bits/shared_ptr.h:535:39:   instantiated from ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = libbitcoin::poller, _Alloc = std::allocator<libbitcoin::poller>, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::blockchain>}]’
/usr/include/c++/4.6/bits/shared_ptr.h:551:42:   instantiated from ‘std::shared_ptr<_Tp1> std::make_shared(_Args&& ...) [with _Tp = libbitcoin::poller, _Args = {libbitcoin::async_service&, std::shared_ptr<libbitcoin::blockchain>}]’
main.cpp:894:67:   instantiated from here
/usr/include/c++/4.6/bits/shared_ptr_base.h:410:4: error: no matching function for call to ‘libbitcoin::poller::poller(libbitcoin::async_service&, std::shared_ptr<libbitcoin::blockchain>)’
/usr/include/c++/4.6/bits/shared_ptr_base.h:410:4: note: candidates are:
/usr/local/include/bitcoin/poller.hpp:12:5: note: libbitcoin::poller::poller(libbitcoin::async_service&, libbitcoin::blockchain&)
/usr/local/include/bitcoin/poller.hpp:12:5: note:   no known conversion for argument 2 from ‘std::shared_ptr<libbitcoin::blockchain>’ to ‘libbitcoin::blockchain&’
/usr/local/include/bitcoin/poller.hpp:9:7: note: constexpr libbitcoin::poller::poller(const libbitcoin::poller&)
/usr/local/include/bitcoin/poller.hpp:9:7: note:   candidate expects 1 argument, 2 provided
/usr/local/include/bitcoin/poller.hpp:9:7: note: constexpr libbitcoin::poller::poller(libbitcoin::poller&&)
/usr/local/include/bitcoin/poller.hpp:9:7: note:   candidate expects 1 argument, 2 provided
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_header(const libbitcoin::message::header&) const’:
main.cpp:620:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_version(const libbitcoin::message::version&) const’:
main.cpp:624:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_verack(const libbitcoin::message::verack&) const’:
main.cpp:628:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_address(const libbitcoin::message::address&) const’:
main.cpp:632:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_get_address(const libbitcoin::message::get_address&) const’:
main.cpp:636:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_inventory(const libbitcoin::message::inventory&) const’:
main.cpp:640:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_get_data(const libbitcoin::message::get_data&) const’:
main.cpp:644:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_get_blocks(const libbitcoin::message::get_blocks&) const’:
main.cpp:648:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_transaction(const libbitcoin::message::transaction&) const’:
main.cpp:652:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_block(const libbitcoin::message::block&) const’:
main.cpp:656:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::data_chunk exporter_wrapper::save_ping(const libbitcoin::message::ping&) const’:
main.cpp:660:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::header exporter_wrapper::load_header(const data_chunk&) const’:
main.cpp:665:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::version exporter_wrapper::load_version(const data_chunk&) const’:
main.cpp:669:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::verack exporter_wrapper::load_verack(const data_chunk&) const’:
main.cpp:673:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::address exporter_wrapper::load_address(const data_chunk&) const’:
main.cpp:677:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::get_address exporter_wrapper::load_get_address(const data_chunk&) const’:
main.cpp:681:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::inventory exporter_wrapper::load_inventory(const data_chunk&) const’:
main.cpp:685:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::get_data exporter_wrapper::load_get_data(const data_chunk&) const’:
main.cpp:689:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::get_blocks exporter_wrapper::load_get_blocks(const data_chunk&) const’:
main.cpp:693:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::transaction exporter_wrapper::load_transaction(const data_chunk&) const’:
main.cpp:698:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::block exporter_wrapper::load_block(const data_chunk&) const’:
main.cpp:702:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘libbitcoin::message::ping exporter_wrapper::load_ping(const data_chunk&) const’:
main.cpp:706:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In member function ‘bool exporter_wrapper::verify_header(const libbitcoin::message::header&) const’:
main.cpp:711:5: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In function ‘blockchain_wrapper create_bdb_blockchain(async_service_wrapper, const string&, boost::python::api::object)’:
main.cpp:877:1: warning: control reaches end of non-void function [-Wreturn-type]
main.cpp: In function ‘exporter_wrapper create_satoshi_exporter()’:
main.cpp:719:1: warning: control reaches end of non-void function [-Wreturn-type]
make: *** [default] Error 1

Any ideas?

[genjix]

Yes, the Python bindings need to be updated since the libbitcoin API changed. I'll get onto that soon as I need to get the Electrum server running again. I plan to rebuild the backend in C++ though.

[Red Emerald]

Yes, the Python bindings need to be updated since the libbitcoin API changed. I'll get onto that soon as I need to get the Electrum server running again. I plan to rebuild the backend in C++ though.

I was building libbitcoin just for electrum

Do you know what the most recent commit is that works with the python bindings by chance?  If not, it wil give me an excuse to try out git-bisect.  Do you think that commit be stable enough, or should I just wait for you to write the updated bindings?

[genjix]

Yes, the Python bindings need to be updated since the libbitcoin API changed. I'll get onto that soon as I need to get the Electrum server running again. I plan to rebuild the backend in C++ though.

I was building libbitcoin just for electrum

Do you know what the most recent commit is that works with the python bindings by chance?  If not, it wil give me an excuse to try out git-bisect.  Do you think that commit be stable enough, or should I just wait for you to write the updated bindings?

Best to wait. The Electrum server I made was a prototype. It had lots of hardcoded paths and such isms:

https://gitorious.org/electrum/server/blobs/master/backends/libbitcoin/memory_buffer.hpp (see 2nd include)

[Phinnaeus Gage]

I was surprised to learn that Patrick owns libbitcoin.org.

Domain Name:LIBBITCOIN.ORG
Created On:01-Jun-2011 18:56:20 UTC
Last Updated On:02-Jun-2012 01:27:02 UTC
Expiration Date:01-Jun-2013 18:56:20 UTC
Sponsoring Registrar:Name.com, LLC (R1288-LROR)
Status:CLIENT TRANSFER PROHIBITED
Registrant ID:nec06g69j6059q1w
Registrant Name:Phantom Circuit
Registrant Organization:PhantomCircuit
Registrant Street1:153 Bailey Rd
Registrant Street2:
Registrant Street3:
Registrant City:Fly Creek
Registrant State/Province:NY
Registrant Postal Code:13337
Registrant Country:US
Registrant Phone:+1.6075474139
Registrant Phone Ext.:
Registrant FAX:
Registrant FAX Ext.:
Registrant Email:phantomcircuit@yahoo.com

[jimbobway]

Is libbitcoin going to die?  Ultraprune?  Receipts?  LevelDB?  Anything in the pipeline to keep up with bitcoind?

[kdomanski]

libbitcoin is ATM being developed by one man - genjix. If you want it to succeed - contribute with your own code!

[genjix]

At the moment I'm working on blockchain stuff. You can see the commit log to check my progress:

https://github.com/spesmilo/libbitcoin/commits/master

Mainly I'm concerned with blockchain interface servers right now:

https://github.com/genjix/query-layer

[genjix]

Lots of new things coming. I'm at a basic milestone where I'm content with the overall internal architecture of the library now.

https://github.com/spesmilo/libbitcoin/commits/master

Next steps is Python bindings, and a blockchain query API. The Python bindings were done before but were experimental. We had some issues with asynchronous calls and the GIL. I'm thinking to simply turn all async calls into sync blocking ones for Python anyway (keep it simple).

The query layer is something really nice:
https://github.com/genjix/query-layer
(see the tutorial.py)
You can move around and examine it using logical Python constructs. I want to expand upon this and also give access to new blocks and verified transactions in the memory pool. Then create an Electrum server protocol implementation chained to this.

Also subvertx (command line tools for Bitcoin using libbitcoin) is a bit out of date and not actively developed. I would like to take that to something serious too but it needs time (if someone wants to volunteer, this is a cool project) to firstly create how the tools should look and then implement them. I can help someone with that if they know C++.
https://github.com/genjix/subvertx
https://bitcointalk.org/index.php?topic=50721.0

Finally I'm working on an Android wallet. It's not using libbitcoin currently, but eventually that will be needed in the future.

There's some other various small tools I've made too. One is a fast-monitor tool which sits in the Bitcoin network and looks at transaction outputs in new blocks and unconfirmed transactions for an address so it can tell a website of a new deposit. The tool doesn't validate the new blocks or transactions and is only meant to be a lightweight thing. I don't see it having too many uses except where a website has TONS of Bitcoin addresses and can't afford to poll the balance for all of them or something.

[marcus_of_augustus]

he's back 

[fellowtraveler]

What about tools for multi-sig ?

[pilotniq]

Hello,

I downloaded and built libbitcoin from github. I'm on an Ubuntu 12.10 machine.

When I run the fullnode example, I get the following error:

~/src/bitcoin/libbitcoin/examples$ ./fullnode
terminate called after throwing an instance of 'boost::interprocess::interprocess_exception'
  what():  No such file or directory
Aborted

The backtrace doesn't say much:

(gdb) bt
#0  0xb7fdd424 in __kernel_vsyscall ()
#1  0xb6e9d1df in __GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#2  0xb6ea0825 in __GI_abort () at abort.c:91
#3  0xb70838ad in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#4  0xb70814f3 in ?? () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#5  0xb708152f in std::terminate() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#6  0xb70dd5ac in ?? () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#7  0xb746ed4c in start_thread (arg=0xb48b5b40) at pthread_create.c:308
#8  0xb6f5ddde in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130

Is this a libboost issue? gcc version? Something else?

Thanks for any help!

[pilotniq]

When I run the fullnode example, I get the following error:

~/src/bitcoin/libbitcoin/examples$ ./fullnode
terminate called after throwing an instance of 'boost::interprocess::interprocess_exception'
  what():  No such file or directory
Aborted

Ok, after some stepping, I realized I needed to create a 'database' directory in the same directory as the 'fullnode' application for a leveldb database.

Can't say the error message was very illuminating.

I'll continue exploring.

I'm now getting errors like:

ERROR: Error storing memory pool transaction 5c4c562af2d4667db80f7832138acaf66e2cfa41bbbb3f2ee07b86f80069a98d: Object does not exist

Is this because the blockchain isn't populated?

[genjix]

Correct.

Also you need to run:

$ ./initchain database/

to initialize the blockchain.

Also if you want to full blockchain then it's available at: http://46.4.92.107/database.tar.bz2

[genjix]

We now have experimental Python bindings thanks to Pablo Martin (caedes)!

https://gitorious.org/libbitcoin-bindings/libbitcoin-bindings

Type "make compile" to compile them.

Full node example:

https://gitorious.org/libbitcoin-bindings/libbitcoin-bindings/blobs/master/tests/fullnode.py

[fellowtraveler]

genjix, glad to see you are still working on this library.

Can it be used for sending multi-sig transactions?  Then verifying it was sent? Then for performing votes to get the funds back out?
Can you post a code example?

[genjix]

Yes it can.

You create a transaction as you would with an output sending to a script hash address. You broadcast it to the network. To verify it was sent, you might listen to see if nodes respond back with an inventory and if not resend the transaction within a certain time. Then to release the funds, you'd all need to sign a transaction according to its rules.

[fellowtraveler]

is there any sample code demonstrating this? (Just to save me some time.)

I can promise I will put it to good use.

[genjix]

is there any sample code demonstrating this? (Just to save me some time.)

I can promise I will put it to good use.

not for multisig. there is sample code for normal transactions: https://github.com/genjix/subvertx/blob/master/src/mktx.cpp

let me know if you have questions: amir@unsystem.net

[genjix]

fullnode example using the python bindings:

https://gitorious.org/libbitcoin-bindings/libbitcoin-bindings/blobs/master/tests/fullnode.py

also if you expose libbitcoin through an RPC interface like Apache Thrift, and publish new confirmed blocks and new validated unconfirmed txs using ZeroMQ, then you get a nice piece of server middleware:

https://github.com/genjix/qlayer/tree/master/src

I've made this before in C++: https://github.com/genjix/query but I think it'd better be served being done in Python.

Also if you have an bunch of dumb slaves with their own blockchain, and they just synchronise with the master (see https://github.com/genjix/slave ) then it's a really fast way to serve the blockchain to tons of people. Since everything runs in its own process too, it's resilient and stable (parts can go offline and everything still functions). On the front you have a load balancer selecting which slave to distribute requests to.

We are porting this to a Python libbitcoin and Twisted (or maybe gevent) architecture. Not sure if we'll stick with Thrift/ZeroMQ.

Code:

import sys

from bitcoin import threadpool, hosts, handshake, network, protocol, session
from bitcoin import leveldb_blockchain, poller, transaction_pool, create_session_params
from bitcoin import hash_transaction
import time

def print_block(block):
    time.ctime(block.timestamp), block.merkle.encode('hex')

class fullnode(object):
    def __init__(self):
        self._net_pool = threadpool(1)
        self._disk_pool = threadpool(1)
        self._mem_pool = threadpool(1)
        self._hosts = hosts(self._net_pool)
        self._handshake = handshake(self._net_pool)
        self._network = network(self._net_pool)
        self._protocol = protocol(self._net_pool,
                                  self._hosts,
                                  self._handshake,
                                  self._network)
        self._chain = leveldb_blockchain(self._disk_pool)
        self._poller = poller(self._mem_pool, self._chain)
        self._txpool = transaction_pool(self._mem_pool, self._chain)
        pars = create_session_params(self._handshake,
				     self._protocol,
				     self._chain,
				     self._poller,
				     self._txpool)
        self._session = session(self._net_pool, pars)
        print "[fullnode] ok"

    def start(self):
        self._protocol.subscribe_channel(self.monitor_tx)
        self._chain.start('database', self.on_chain_start)
        self._chain.subscribe_reorganize(self.on_reorganize)
        self._txpool.start()
        self._session.start(self.on_session_start)
        print "[fullnode.start] ok"

    def stop(self):
        self._session.stop(self.on_session_stop)
        self._net_pool.stop()
        self._disk_pool.stop()
        self._mem_pool.stop()
        self._net_pool.join()
        self._disk_pool.join()
        self._mem_pool.join()
        self._chain.stop()
        print "[fullnode.stop] ok"

    def on_chain_start(self, ec):
        print "[fullnode.chain] started", ec

    def on_session_stop(self, ec):
        print "[fullnode.session] stopped", ec

    def on_session_start(self, ec):
        print "[fullnode.session] started", ec
        if ec:
            self.stop()
            sys.exit(1)

    def on_reorganize(self, ec, height, arrivals, replaced):
        print '[fullnode.reorganize]', height, str(ec), len(arrivals), len(replaced)
        if len(arrivals):
            print ' arrival', print_block(arrivals[0])
        if len(list2):
            print ' replaced', print_block(arrivals[1])
        self._chain.subscribe_reorganize(self.on_reorganize)

    def monitor_tx(self, node):
        print "(fullnode.tx)", node
        node.subscribe_transaction(lambda ec, tx: self.recv_tx(node, tx, ec))
        self._protocol.subscribe_channel(self.monitor_tx)

    def handle_confirm(self, ec):
        print "(fullnode.store) confirm", ec

    def recv_tx(self, node, tx, ec):
        print "(fullnode.recv_tx)", ec, tx
        if ec:
            print "error", ec
            return
        print ' *', len(tx.inputs), len(tx.outputs)
        self._txpool.store(tx, self.handle_confirm, lambda _ec, u: self.new_unconfirm_valid_tx(node, tx, u, _ec))
        node.subscribe_transaction(lambda _ec, _tx: self.recv_tx(node, _tx, _ec))

    def new_unconfirm_valid_tx(self, node, tx, unconfirmed, ec):
        print "(fullnode.valid_tx)", ec, tx, unconfirmed
        tx_hash = hash_transaction(tx)
        if ec:
            print "Error", ec
        else:
            print "Accepted transaction"
            print unconfirmed.__class__
            if not unconfirmed.empty():
               print "Unconfirmed"
               for idx in unconfirmed: 
                   print ' ', idx
            print tx_hash


if __name__ == '__main__':
    app = fullnode()
    app.start()
    raw_input()
    app.stop()
 

[pilotniq]

I'm thinking about making a Bitcoin client that deletes blocks from its blockchain database if they are say more than about 6 blocks deeper than the latest block AND all of its transactions have been spent. I figure this way I can still validate new blocks, ensure that transactions to my addresses are valid, and save lots of disk space.

Can I do this with libbitcoin, but without modifying its source? How hard would it be?

In your initial posts of this thread, you mention MySQL, but I see no trace of MySQL in the latest source I downloaded from Github, just leveldb and bdb. What's up with that?

You also mention the blockchain database as 'append only', does that apply that I can't delete from it?

Thanks for any tips!

[genjix]

Yeah you can do that if you rebuild the database. That means creating a new database (see examples/initchain.cpp), then importing the old blocks into the new one. However block headers are only 80 bytes so you should keep those. In Bitcoin, this is what pruning outputs means (libbitcoin by default doesn't do that). I personally believe that disk space is not a big deal (current unmodified chain is 15 Gb). I'm more worried about fetch speed (not writes) and scalability.

I had a PostgreSQL database in the beginning (hence the bitcoin.sql file in the repo) but eventually moved to bdb. The postgres backend became unmaintained, the API changed a lot and it fell out of development before I dropped it entirely. SQL is too slow for blockchains.

There is a method to delete from the blockchain directly, but I haven't exposed that. It's used for blockchain reorgs. You can use that same code, but make sure you stop the blockchain object before modifying the database. You can then reopen it after:

Code:

leveldb_blockchain* chain = new leveldb_blockchain(pool);
chain->start(dbpath, blockchain_started);
// ... do stuff
// stop/join threadpools using the blockchain.
chain->stop();
delete chain;
// open database, delete whatever you want.
leveldb_blockchain* chain = new leveldb_blockchain(pool);
chain->start(dbpath, blockchain_started);
// continue from where you left off...

something like that.

see the method called "end_slice"

https://github.com/spesmilo/libbitcoin/blob/master/src/blockchain/leveldb/leveldb_chain_keeper.cpp

It's deleting a bunch of blocks from the end of the blockchain.

Also `bool leveldb_chain_keeper::clear_transaction_data(leveldb_transaction_batch& batch, const transaction_type& remove_tx)' (same file) might interest you too. It's deleting specific transactions from the database.

[pilotniq]

I personally believe that disk space is not a big deal (current unmodified chain is 15 Gb). I'm more worried about fetch speed (not writes) and scalability.

I'm thinking about building a device for secure Bitcoin storage based on the Raspberry Pi. It doesn't have to be very fast, but if I want to support a full blockchain for a number of years into the future, the storage would get prohibitive (already is). That's why I'm interested in a pruned blockchain.

I'd much rather use (and possibly contribute to) an existing library than writing my own stuff.

SQL is too slow for blockchains.

Could you elaborate on that? What are the operations that require high speed of database access? I am considering using sqlite for a pruned blockchain, do you think that would not work?

Could I write my own implementation of blockchain which automatically pruned itself without rebuilding, and stored the database (blockchain, transactions, addresses maybe) in its own database? Would it create problems for the rest of libbitcoin?

Yeah you can do that if you rebuild the database.
That means creating a new database (see examples/initchain.cpp), then importing the old blocks into the new one.

Sounds expensive.

I wish there were a bit more comments in the code to explain the concepts, like slices, what the chain keeper, chain organizer do, etc.

There is a method to delete from the blockchain directly, but I haven't exposed that. It's used for blockchain reorgs. You can use that same code, but make sure you stop the blockchain object before modifying the database. You can then reopen it after:

Code:

leveldb_blockchain* chain = new leveldb_blockchain(pool);
chain->start(dbpath, blockchain_started);
// ... do stuff
// stop/join threadpools using the blockchain.
chain->stop();
delete chain;
// open database, delete whatever you want.
leveldb_blockchain* chain = new leveldb_blockchain(pool);
chain->start(dbpath, blockchain_started);
// continue from where you left off...

something like that.

see the method called "end_slice"

https://github.com/spesmilo/libbitcoin/blob/master/src/blockchain/leveldb/leveldb_chain_keeper.cpp

It's deleting a bunch of blocks from the end of the blockchain.

Thanks for the pointers.

Why does the database need to be deleted and then opened in order to delete data from it?

Also `bool leveldb_chain_keeper::clear_transaction_data(leveldb_transaction_batch& batch, const transaction_type& remove_tx)' (same file) might interest you too. It's deleting specific transactions from the database.

Where can I find a description of what's in the database? I'm used to SQL databases with tables, but I understand leveldb only stores key-value pairs? What objects are in the database? Is there for instance a way to find unspent transactions?

[genjix]

I think the Raspberry Pi uses the ARM processor. If you manage to get libbitcoin compiled and running on there, I'll happily add you a remove(tx_hash) method.

SQL is too slow when validating the blocks. Loading the entire blockchain into SQL will take a very long time. However there's a ready made schema there for you if you like.

libbitcoin uses a blockchain interface with different backends. As long as you implement the interface, you should be fine. If you're not too worried about validating blocks, then it should be quite easy (a week(end) of work).

I should comment that part. There's lots of comments in the API, but less so in the deeper parts of the internal machinery. I will get around to that.

You need to close the database handle before writing to it from an external context. That's dangerous otherwise. There should be a single environment open with LevelDB, and you don't want other async writes to conflict while you're doing stuff to the DB.

Finding unspent outputs means doing a linear scan through every output and seeing which ones do not have a corresponding spend entry. The comments for what the tables (called databases in LevelDB) do is in leveldb_blockchain.hpp

[genjix]

Experimental C wrapper: https://github.com/genjix/libbitcoin-c-wrapper/blob/master/fullnode.c

[RoadTrain]

Hello genjix!
I'm curious if your library has already implemented BIP 0037?
I'd like to experiment with this feature.

[genjix]

Lots of new things coming. I'm at a basic milestone where I'm content with the overall internal architecture of the library now.

https://github.com/spesmilo/libbitcoin/commits/master

Next steps is Python bindings, and a blockchain query API. The Python bindings were done before but were experimental. We had some issues with asynchronous calls and the GIL. I'm thinking to simply turn all async calls into sync blocking ones for Python anyway (keep it simple).

The query layer is something really nice:
https://github.com/genjix/query-layer
(see the tutorial.py)
You can move around and examine it using logical Python constructs. I want to expand upon this and also give access to new blocks and verified transactions in the memory pool. Then create an Electrum server protocol implementation chained to this.

Also subvertx (command line tools for Bitcoin using libbitcoin) is a bit out of date and not actively developed. I would like to take that to something serious too but it needs time (if someone wants to volunteer, this is a cool project) to firstly create how the tools should look and then implement them. I can help someone with that if they know C++.
https://github.com/genjix/subvertx
https://bitcointalk.org/index.php?topic=50721.0

Finally I'm working on an Android wallet. It's not using libbitcoin currently, but eventually that will be needed in the future.

There's some other various small tools I've made too. One is a fast-monitor tool which sits in the Bitcoin network and looks at transaction outputs in new blocks and unconfirmed transactions for an address so it can tell a website of a new deposit. The tool doesn't validate the new blocks or transactions and is only meant to be a lightweight thing. I don't see it having too many uses except where a website has TONS of Bitcoin addresses and can't afford to poll the balance for all of them or something.

Most of these goals have been reached.

query API: https://github.com/genjix/obelisk

subvertx (now sx): https://sx.dyne.org/

Next stage is I need a final class for watching an address and giving credit/debit updates, and adding multisig to sx tools.

We're moving into wallet development soon, and other exciting projects to push the boundaries of Bitcoin development.

[Arto]

Just wanted to point out here that only one of the Individual Seat candidates for the ongoing Bitcoin Foundation elections has mentioned libbitcoin so far. That was Joerg Platzer, in the Bitcoin Foundation Candidate Debate hosted by Let's Talk Bitcoin the other day. Find the 7-minute recording of Joerg's Q&A answers at https://soundcloud.com/mindtomatter/bf-debate-joerg-platzer

[2weiX]

Just wanted to point out here that only one of the Individual Seat candidates for the ongoing Bitcoin Foundation elections has mentioned libbitcoin so far. That was Joerg Platzer, in the Bitcoin Foundation Candidate Debate hosted by Let's Talk Bitcoin the other day. Find the 7-minute recording of Joerg's Q&A answers at https://soundcloud.com/mindtomatter/bf-debate-joerg-platzer

+inf


Joerg Platzer needs to be on the board of the Bitcoin Foundation (yes, if you're a member, you can vote for him!)... to show these guys how this is an actual factual thing that happens ON THE STREETS.

Nothing for 'sters and guys in suits.


"This guy is in finance..."

Well THIS guy is ON THE STREET actually WORKING with Bitcoin.

/spamrant

[genjix]

libbitcoin now has a blockchain reorg testing tool!

https://github.com/spesmilo/libbitcoin/tree/master/test/fakechain

Run the bash script run.sh

You can customise the test by editing step.hpp

You can test anything by customising step.hpp. The default test now tries to create 3 chains, handle reorganisations, double spending, spending transactions from alternate branches and duplicate transactions.

$ cd libbitcoin/tests/fakechain/
$ ./run.sh
$ ../../examples/display-last

It works by mining the blocks from the genesis block. It submits a chain of 100 blocks then does the reorg test on top. This is because you cannot spend coinbase transactions that are less than COINBASE_MATURITY (100 blocks).

[phelix]

Trying to build on windows/MinGW for Zeroreserve (https://bitcointalk.org/index.php?topic=295930)

Installed MinGW with gcc 4.8.1

What I got so far:

Code:

mingw-get install autoconf
mingw-get install automake
mingw-get install libtool
in configure.ac: AM_PROG_AR  --> AC_CHECK_TOOL([AR], [ar], [false])     [no idea what I am doing here but seems to work]

---> "autoreconf -i" works ok

Now I am stuck with "./configure --enable-leveldb" --> "could not detect boost libraries"
I have boost installed and can compile Bitcoin but I can not convince libbitcoin configure...   any ideas?

Has anybody ever compiled libbitcoin on windows?

Tried $BOOST_ROOT, --with-boost, --with-bost-libdir and various combinations, e.g.:
./configure --enable-leveldb --with-boost=./libs/boost_1_54_0 --with-boost-libdir=./libs/boost_1_54_0/stage/lib
./configure --enable-leveldb --with-boost=./libs/boost_1_54_0 --with-boost-libdir=./libs/boost_1_54_0/stage
./configure --enable-leveldb --with-boost=./libs/boost_1_54_0 --with-boost-libdir=./libs/boost_1_54_0

What parts of boost should I compile? (chrono, filesystem, program-options, system, thread is what I got now; static linking)

Using boost 1.54.0

[genjix]

Trying to build on windows/MinGW for Zeroreserve (https://bitcointalk.org/index.php?topic=295930)

Installed MinGW with gcc 4.8.1

What I got so far:

Code:

mingw-get install autoconf
mingw-get install automake
mingw-get install libtool
in configure.ac: AM_PROG_AR  --> AC_CHECK_TOOL([AR], [ar], [false])     [no idea what I am doing here but seems to work]

---> "autoreconf -i" works ok

Now I am stuck with "./configure --enable-leveldb" --> "could not detect boost libraries"
I have boost installed and can compile Bitcoin but I can not convince libbitcoin configure...   any ideas?

Has anybody ever compiled libbitcoin on windows?

Tried $BOOST_ROOT, --with-boost, --with-bost-libdir and various combinations, e.g.:
./configure --enable-leveldb --with-boost=./libs/boost_1_54_0 --with-boost-libdir=./libs/boost_1_54_0/stage/lib
./configure --enable-leveldb --with-boost=./libs/boost_1_54_0 --with-boost-libdir=./libs/boost_1_54_0/stage
./configure --enable-leveldb --with-boost=./libs/boost_1_54_0 --with-boost-libdir=./libs/boost_1_54_0

What parts of boost should I compile? (chrono, filesystem, program-options, system, thread is what I got now; static linking)

Using boost 1.54.0

I'll message you via email and we'll sort this out.

[genjix]

Time for a new libbitcoin release courtesy of unSYSTEM!

Shout outs to everyone involved in no particular order: Tristan, Pablo
Castellano, caedes, Santi (d3), Vitalik, Robert, Jaromil, Grazcoin and
others.

Links:

  http://libbitcoin.dyne.org/download/libbitcoin-1.4.tar.bz2

  http://libbitcoin.dyne.org/download/obelisk-0.2.tar.bz2

  http://sx.dyne.org/download/sx-0.2.tar.bz2

This is the first public release of obelisk (scalable blockchain query
infrastructure) and sx (empower the Bitcoin sysadmin) to the world!

Safe journeys fellow travellers!

[virtualcodewarrior]

I just installed Ubuntu 13.10 and rebuild libbitcoin after getting the latest changes from github.
It didn't give any errors as far as I could see but when I tried to build the examples I got the following errors :

Code:

/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_free'
/usr/local/lib/libbitcoin.so: undefined reference to `curl_easy_perform'
/usr/local/lib/libbitcoin.so: undefined reference to `d2i_ECPrivateKey'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_add'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_get0_group'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_set_negative'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_POINT_free'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_sub'
/usr/local/lib/libbitcoin.so: undefined reference to `RIPEMD160_Final'
/usr/local/lib/libbitcoin.so: undefined reference to `curl_easy_init'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_CTX_free'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_set_word'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_CTX_new'
/usr/local/lib/libbitcoin.so: undefined reference to `RIPEMD160'
/usr/local/lib/libbitcoin.so: undefined reference to `SHA1'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_set_public_key'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_cmp'
/usr/local/lib/libbitcoin.so: undefined reference to `SHA256_Final'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_rand'
/usr/local/lib/libbitcoin.so: undefined reference to `SHA256_Update'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_new'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_clear_free'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_POINT_get_affine_coordinates_GFp'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_POINT_new'
/usr/local/lib/libbitcoin.so: undefined reference to `CRYPTO_free'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_mpi2bn'
/usr/local/lib/libbitcoin.so: undefined reference to `RIPEMD160_Init'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_get_word'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_bin2bn'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_GROUP_new_by_curve_name'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_POINT_mul'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_copy'
/usr/local/lib/libbitcoin.so: undefined reference to `SHA256_Init'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_copy'
/usr/local/lib/libbitcoin.so: undefined reference to `curl_easy_cleanup'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_div'
/usr/local/lib/libbitcoin.so: undefined reference to `i2o_ECPublicKey'
/usr/local/lib/libbitcoin.so: undefined reference to `ECDSA_size'
/usr/local/lib/libbitcoin.so: undefined reference to `ECDSA_sign'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_mul'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_num_bits'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_POINT_set_affine_coordinates_GFp'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_bn2hex'
/usr/local/lib/libbitcoin.so: undefined reference to `i2d_ECPrivateKey'
/usr/local/lib/libbitcoin.so: undefined reference to `curl_easy_setopt'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_bn2bin'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_bn2mpi'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_generate_key'
/usr/local/lib/libbitcoin.so: undefined reference to `RIPEMD160_Update'
/usr/local/lib/libbitcoin.so: undefined reference to `SHA256'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_get0_private_key'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_lshift'
/usr/local/lib/libbitcoin.so: undefined reference to `ECDSA_verify'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_dup'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_set_private_key'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_GROUP_get_order'
/usr/local/lib/libbitcoin.so: undefined reference to `o2i_ECPublicKey'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_GROUP_free'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_init'
/usr/local/lib/libbitcoin.so: undefined reference to `BN_free'
/usr/local/lib/libbitcoin.so: undefined reference to `EC_KEY_new_by_curve_name'

I hope someone can help me with this.
Did I forget some step or am I missing some library?

[phelix]

Any news on the windows build? 

[Mike Hearn]

virtualcodewarrior, looks like you're missing OpenSSL.

[virtualcodewarrior]

openssl is installed as far as I can tell (did a reinstall to make sure) and it seems that "curl_easy_perform" should be in libcurl which is also installed but that function is also listed as undefined .

ldconfig has the libraries listed and the "libssl.so.1.0.0" and "libcrypto.so.1.0.0" both exist in "/lib/x86_64-linux-gnu/" directory.

using nm to see the exports from libcrypto.so.1.0.0 does list EC_KEY_free as being exported from it

and libbitcoin.pc has Libs.private: -lcrypto
listed

I am new to using c++ on linux so I might have missed something obvious.

Update:
This problem is caused by changes made between Ubuntu 13.04 and 13.10.
I tried a clean install of both versions of Ubuntu in virtualbox and on 13.04 it will build fine
while on 13.10 it will fail with the undefined references.

I will try and see if I can spot any differences between the two versions.

[genjix]

Hmm strange, anyway I'm upgrading to latest Ubuntu 13.10. Will post a fix here afterwards.

Windows/Mac: in around 1 month (next release). I've got both platforms working before but unsupported.

[genjix]

Fixed.

https://github.com/spesmilo/libbitcoin/commit/e543599b3ac37e4f30370bbf7c1323de1c71fba2

I patched this on the website tarball too.

[virtualcodewarrior]

Thanks,

That indeed did the trick.
It seems I still have a lot to learn about development on linux 

[genjix]

Thanks,

That indeed did the trick.
It seems I still have a lot to learn about development on linux 

No worries, we're all at different stages of beginner.

[phelix]

Hmm strange, anyway I'm upgrading to latest Ubuntu 13.10. Will post a fix here afterwards.

Windows/Mac: in around 1 month (next release). I've got both platforms working before but unsupported.

cool

[jedunnigan]

genjix, I am curious what you think about this post by satoshi:

I don't believe a second, compatible implementation of Bitcoin will ever be a good idea.  So much of the design depends on all nodes getting exactly identical results in lockstep that a second implementation would be a menace to the network.  The MIT license is compatible with all other licenses and commercial uses, so there is no need to rewrite it from a licensing standpoint.

[genjix]

Shocked Satoshi would think that since he's often showed a lot of foresight.

And I was shut out of development and prevented from doing work in the early days. So I went my own way because I see Bitcoin development differently to the current Bitcoin establishment.



----------

Good news everyone! The Python front is progressing nicely!

http://libbitcoin.dyne.org/labs.html

People creating cool stuff. My goal now is to get these projects all merged together into one pure Python package for wallet developers.

[kjj]

Satoshi is exactly right about alternate implementations.  They are hard to do right, and can cause a hell of a mess when done wrong.

But, it comes down to which of the two bad options do we like least.  Virtually no one currently active on the project wants a monoculture.  We are willing to take on the challenges and risks of multiple implementations because we feel that the risks of having just a single client are even worse.

Perhaps he'll turn out to have been right about this too, in the long run.  We should be able to answer that question fairly well in another 10 or 20 years.

[marcus_of_augustus]

Shocked Satoshi would think that since he's often showed a lot of foresight.

And I was shut out of development and prevented from doing work in the early days. So I went my own way because I see Bitcoin development differently to the current Bitcoin establishment.



----------

Good news everyone! The Python front is progressing nicely!

http://libbitcoin.dyne.org/labs.html

People creating cool stuff. My goal now is to get these projects all merged together into one pure Python package for wallet developers.

The Dude Abides.

[btcven]

----------

Good news everyone! The Python front is progressing nicely!

http://libbitcoin.dyne.org/labs.html

People creating cool stuff. My goal now is to get these projects all merged together into one pure Python package for wallet developers.

Dead link, I wan't to check it out

[lclc]

Dead link, I wan't to check it out

It's just that the website seems to be down right now..

BTW: Great work, I think having more than one implementation of the protocol is very much needed.

[genjix]

back up again!

btw hope all friends here have seen our crowdfunding campaign by now:

http://www.indiegogo.com/projects/bitcoin-dark-wallet

if you can get the word out about libbitcoin and all that we're trying to achieve here, that's much appreciated. thanks!

[hivewallet]

Satoshi is exactly right about alternate implementations.  They are hard to do right, and can cause a hell of a mess when done wrong.

But, it comes down to which of the two bad options do we like least.  Virtually no one currently active on the project wants a monoculture.  We are willing to take on the challenges and risks of multiple implementations because we feel that the risks of having just a single client are even worse.

Perhaps he'll turn out to have been right about this too, in the long run.  We should be able to answer that question fairly well in another 10 or 20 years.

Talk to us about the kind of mess it could potentially cause. Satoshi's "menace to the network" statement was maybe a bit vague.

[jedunnigan]

Satoshi is exactly right about alternate implementations.  They are hard to do right, and can cause a hell of a mess when done wrong.

But, it comes down to which of the two bad options do we like least.  Virtually no one currently active on the project wants a monoculture.  We are willing to take on the challenges and risks of multiple implementations because we feel that the risks of having just a single client are even worse.

Perhaps he'll turn out to have been right about this too, in the long run.  We should be able to answer that question fairly well in another 10 or 20 years.

Talk to us about the kind of mess it could potentially cause. Satoshi's "menace to the network" statement was maybe a bit vague.

On the discussion of alternative implementations, I am torn; I think it's a double edged sword, largely because the potential upside is real. That being said, Satoshi has a point.

The hell of a mess he was referring to would be a fork, or some other disastrous scenario where an alt implementation is not kept up to date with changes to source or was poorly implemented in the first place. Or perhaps it's not prepared for some fringe race condition. Who knows... what might happen exactly depends on the context.

I don't take a side in this debate, because honestly it wouldn't be appropriate given my programmatic experience (or lackthereof). Regardless, these are real issues with real consequences.

[hivewallet]

On the discussion of alternative implementations, I am torn; I think it's a double edged sword, largely because the potential upside is real. That being said, Satoshi has a point.

The hell of a mess he was referring to would be a fork, or some other disastrous scenario where an alt implementation is not kept up to date with changes to source or was poorly implemented in the first place. Or perhaps it's not prepared for some fringe race condition. Who knows... what might happen exactly depends on the context.

I don't take a side in this debate, because honestly it wouldn't be appropriate given my programmatic experience (or lackthereof). Regardless, these are real issues with real consequences.

Right. Vague memory of a conversation between Peter Todd, Greg Maxwell, and Pieter Wuille in San Jose this year about that very subject...

[MoonShadow]

Satoshi had a point.  He hasn't been here for some time, and his comments were relevent to the state of Bitcoin at that time.  The main client was still, itself, very rough.  Very much still beta.  IT's also a monolithic implimentation.  Which is fine for early work.

We are beyond early work.  I have long complained that the main bitcoin client didn't have simple unix like tools.  Libbitcoin seems to intend to be that set of unix tools.  I want to be able to pipe a transaction to a file, move it with a usb sneakernet, and pipe it to the bitcoin network on another machine.  I want a client that can do this for me, automaticly.  I want a client that I can create the keypairs on one machine, and manage the funds on a headless server half way around the world, without either of those machines even knowing the IP address of the other.  I want an online wallet service client that can logic bomb my funds onto the bitcoin network if tripwire is activated.  If the tools exist to perform each of the simple operations that the bitcoin client performs; then clients that can do a great many things can be developed to suit small group needs.

[jedunnigan]

Satoshi had a point.  He hasn't been here for some time, and his comments were relevent to the state of Bitcoin at that time.  The main client was still, itself, very rough.  Very much still beta.  IT's also a monolithic implimentation.  Which is fine for early work.

We are beyond early work.  I have long complained that the main bitcoin client didn't have simple unix like tools.  Libbitcoin seems to intend to be that set of unix tools.  I want to be able to pipe a transaction to a file, move it with a usb sneakernet, and pipe it to the bitcoin network on another machine.  I want a client that can do this for me, automaticly.  I want a client that I can create the keypairs on one machine, and manage the funds on a headless server half way around the world, without either of those machines even knowing the IP address of the other.  I want an online wallet service client that can logic bomb my funds onto the bitcoin network if tripwire is activated.  If the tools exist to perform each of the simple operations that the bitcoin client performs; then clients that can do a great many things can be developed to suit small group needs.

Very good points. I am in agreement in many respects; I think modular nodes are vital (see: Bits of Proof). We can't wait for the core devs to do everything, that's not fair. We just need to make sure we are very careful and meticulous in our preparations...

[kjj]

Satoshi is exactly right about alternate implementations.  They are hard to do right, and can cause a hell of a mess when done wrong.

But, it comes down to which of the two bad options do we like least.  Virtually no one currently active on the project wants a monoculture.  We are willing to take on the challenges and risks of multiple implementations because we feel that the risks of having just a single client are even worse.

Perhaps he'll turn out to have been right about this too, in the long run.  We should be able to answer that question fairly well in another 10 or 20 years.

Talk to us about the kind of mess it could potentially cause. Satoshi's "menace to the network" statement was maybe a bit vague.

Each node validates network messages.  If some nodes think that a block or transaction is valid, while other nodes think it is invalid, you can have a fork.  This is what happened when the BDB bug hit a while back.

In the case of the BDB bug, the people running the network (aka, "us") decided that we'd invalidate one fork to re-merge the network.  This option is not likely to be possible or practical for all such forks in the future.  In particular, as more implementations start up, the fraction of the network hurt by any given fork will grow smaller and smaller and at some point the majority will refuse to insert bug compatibility.

In practice, this will cause accidental or intentional forks at small numbers of nodes.  As in, someone may be able to carefully craft a transaction or block that will knock one node out of sync, with the intention of doing mischief to that node.  Equally possible will be that such things will just happen by chance, and some times they will go unexploited, but other times people will lose money because of it.

The only real safety is to be running the exact same software as the majority, and in the exact same environment.  Note that the BDB thing didn't hit all nodes that it potentially could have hit, because it depended partially on the environment.*

The monoculture carries different risks, but it is largely resistant to these sorts of problems.

Are you familiar with the Brown M&Ms thing from rock history?  It was parodied in Wayne's World II.  But it was there for a reason.  Van Halen put on a huge show, and to do it safely, they had a list of detailed technical requirements they presented to the local promoter.  The brown M&Ms were a canary.  If the promoter forgot that step, it was assumed that the other things in the contract had also been neglected, so they had to check everything personally.  If the promoter presented the M&Ms correctly, it was likely that they were careful with everything else too.

In a similar way, the people that are most familiar with the software have a sort of list of likely forks points in the code, where if someone gets it wrong, their node will (or at least can) eventually fork from the rest of the network.  When someone posts a new implementation, they can check a few of those, and if they aren't done right, the devs are pretty safe saying "Your code has big obvious flaws.  It probably has other more subtle flaws too.  Please do not let anyone use this code until you can show that you really understand the reference software."

*  I run one ancient node for amusement.  By some quirk of fate, it was not hit by the BDB thing, even though it really should have.  I think that all of my nodes that are merely old did get hit by it.

[MoonShadow]

Satoshi is exactly right about alternate implementations.  They are hard to do right, and can cause a hell of a mess when done wrong.

But, it comes down to which of the two bad options do we like least.  Virtually no one currently active on the project wants a monoculture.  We are willing to take on the challenges and risks of multiple implementations because we feel that the risks of having just a single client are even worse.

Perhaps he'll turn out to have been right about this too, in the long run.  We should be able to answer that question fairly well in another 10 or 20 years.

Talk to us about the kind of mess it could potentially cause. Satoshi's "menace to the network" statement was maybe a bit vague.

Each node validates network messages.  If some nodes think that a block or transaction is valid, while other nodes think it is invalid, you can have a fork.  This is what happened when the BDB bug hit a while back.

In the case of the BDB bug, the people running the network (aka, "us") decided that we'd invalidate one fork to re-merge the network.  This option is not likely to be possible or practical for all such forks in the future.  In particular, as more implementations start up, the fraction of the network hurt by any given fork will grow smaller and smaller and at some point the majority will refuse to insert bug compatibility.

In practice, this will cause accidental or intentional forks at small numbers of nodes.  As in, someone may be able to carefully craft a transaction or block that will knock one node out of sync, with the intention of doing mischief to that node.  Equally possible will be that such things will just happen by chance, and some times they will go unexploited, but other times people will lose money because of it.

The only real safety is to be running the exact same software as the majority, and in the exact same environment.  Note that the BDB thing didn't hit all nodes that it potentially could have hit, because it depended partially on the environment.*

The monoculture carries different risks, but it is largely resistant to these sorts of problems.

Are you familiar with the Brown M&Ms thing from rock history?  It was parodied in Wayne's World II.  But it was there for a reason.  Van Halen put on a huge show, and to do it safely, they had a list of detailed technical requirements they presented to the local promoter.  The brown M&Ms were a canary.  If the promoter forgot that step, it was assumed that the other things in the contract had also been neglected, so they had to check everything personally.  If the promoter presented the M&Ms correctly, it was likely that they were careful with everything else too.

In a similar way, the people that are most familiar with the software have a sort of list of likely forks points in the code, where if someone gets it wrong, their node will (or at least can) eventually fork from the rest of the network.  When someone posts a new implementation, they can check a few of those, and if they aren't done right, the devs are pretty safe saying "Your code has big obvious flaws.  It probably has other more subtle flaws too.  Please do not let anyone use this code until you can show that you really understand the reference software."

*  I run one ancient node for amusement.  By some quirk of fate, it was not hit by the BDB thing, even though it really should have.  I think that all of my nodes that are merely old did get hit by it.

This is all fine opinion, but this is still all conjecture.  It's also one reason that we have the running testnet, so alternate clients can test their functions against a running network before going live on the main network.  Libbitcoin isn't the first alternate client either, as a couple of different simple clients have been written in Java and Python.  Notablely, those clients were developed after Satoshi disappeared, and they managed to not break anything.  The network is rather resilient, actually.  The bugs are only a problem when they are discovered in a client that has been widely adopted already.  The BDB bug was in the main reference client.  It's impossible to protect against all screwups.

[MykelSilver]

Feel very stupid to ask this question: Can the bitcoins of the original bitcoin-qt client and the libbitcoin client be transferred to each other? Or are the coins completely different?
I really a start for dummies about libbitcoin. Any other links than http://libbitcoin.dyne.org/ and http://bitcoinmagazine.com/6234/what-libbitcoin-and-sx-are-and-why-they-matter/ are welcome.  Reason I am interested in libbitcoin is that I am worried about my stack of bitcoins being destroyed by any possible government intervention in the future. Looking for a possible exit in case of an emergency. Thanks in advance.
EDIT: Just found http://www.indiegogo.com/projects/bitcoin-dark-wallet
Really interesting stuff.....

[K1773R]

Feel very stupid to ask this question: Can the bitcoins of the original bitcoin-qt client and the libbitcoin client be transferred to each other? Or are the coins completely different?
I really a start for dummies about libbitcoin. Any other links than http://libbitcoin.dyne.org/ and http://bitcoinmagazine.com/6234/what-libbitcoin-and-sx-are-and-why-they-matter/ are welcome.  Reason I am interested in libbitcoin is that I am worried about my stack of bitcoins being destroyed by any possible government intervention in the future. Looking for a possible exit in case of an emergency. Thanks in advance.
EDIT: Just found http://www.indiegogo.com/projects/bitcoin-dark-wallet
Really interesting stuff.....

you dont transfer coins around, you would transfer privkeys. as the protocol is the same, you just have to import the privkey and you (obviously) have access to all the coins

[MykelSilver]

you dont transfer coins around, you would transfer privkeys. as the protocol is the same, you just have to import the privkey and you (obviously) have access to all the coins

Very cool. Thx for sharing.

[Red Emerald]

I've managed to get libbitcoin built on osx!  I'm still having trouble with obelisk and sx, but libbitcoin and pybitcointools seem to be working!

https://github.com/WyseNynja/homebrew-bitcoin

[ngenko]

I find this initiative very interesting, especially with respect to what is happening at the foundation and the regulator going to put pressure on the bitcoin community.

I'm not a developer, so I cannot really help you guys in enhancing the implementation. However, I would be happy to use my BFL miner to mine libbitcoins.

As far as I understood, there is no mining pool and nothing plug-and-play to start mining libbitcoin.
Is there any plan to realase it?

[lclc]

However, I would be happy to use my BFL miner to mine libbitcoins.
As far as I understood, there is no mining pool and nothing plug-and-play to start mining libbitcoin.
Is there any plan to realase it?

I think you misunderstand libbitcoin. It's not a altcoin, it's a alternative implementation of Bitcoin but with the same blockchain, same protocol etc etc.
So it's the same Bitcoin, the same coins.
It's still a very important project (more important than yet another altcoin) as it is absolutely independent from the foundation.
You might just donate some of the mined Bitcoins you made to the DarkWallet fundraising, which is also from them (and using libbitcoin as a server-backend) to support it.

[ngenko]

I think you misunderstand libbitcoin. It's not a altcoin, it's a alternative implementation of Bitcoin but with the same blockchain, same protocol etc etc.
So it's the same Bitcoin, the same coins.
It's still a very important project (more important than yet another altcoin) as it is absolutely independent from the foundation.
You might just donate some of the mined Bitcoins you made to the DarkWallet fundraising, which is also from them (and using libbitcoin as a server-backend) to support it.

Right, there is something that I misunderstand then. If it's an alternative implementation but with the same blockchain, what does it mean?
using libbitcoin means using another wallet than the official one (using darkwallet instead of Multibit?)?

[MoonShadow]

I think you misunderstand libbitcoin. It's not a altcoin, it's a alternative implementation of Bitcoin but with the same blockchain, same protocol etc etc.
So it's the same Bitcoin, the same coins.
It's still a very important project (more important than yet another altcoin) as it is absolutely independent from the foundation.
You might just donate some of the mined Bitcoins you made to the DarkWallet fundraising, which is also from them (and using libbitcoin as a server-backend) to support it.

Right, there is something that I misunderstand then. If it's an alternative implementation but with the same blockchain, what does it mean?
using libbitcoin means using another wallet than the official one (using darkwallet instead of Multibit?)?

libbitcoin is a programming library for bitcoin client functions.  It's not a client unto itself, but makes the building of clients easier and more uniform.  Darkwallet is, indeed, based upon libbitcoin.

[RayG]

Has anyone ever managed to build libbitcoin library for Windows? I'm offering 0.5 BTC for working solution here https://bitcointalk.org/index.php?topic=358298.0.

[genjix]

New library for interfacing with Obelisk servers and doing Bitcoin functionality.

Pure Python so library is easily portable to platforms that can run a Python interpreter.

https://bitcointalk.org/index.php?topic=354607

https://github.com/darkwallet/python-obelisk/

See examples/ directory.

This is a collaboration between Pablo Martin's ZMQ/Obelisk implementation, Robert Williamson's blockalchemy library and code from ThomasV's Electrum.

Obelisk is a scalable blockchain query infrastructure. See the setup guide. There is a public server available at 37.139.11.99:9091 but I wouldn't use it for anything production (besides testing).

A pure Python library is great because it can be deployed easily without requiring dependencies. This allows code to be ported across many platforms such as the mobile phone with Kivy, desktops or any platform running Python.

Features:

* Blockchain queries and navigation through Obelisk servers.
* Construct & sign transactions.
* BIP32 hierarchical wallet.
* Traverse the blockchain using a syntax similar to Django ORM / SQL alchemy.

Use cases:

* Website payment backend (store and manage the keys yourself).
* Wallet software.

[genjix]

I've created a new library which will have non-critical code (like
deterministic wallets, key formats and selecting outputs) ported from
libbitcoin to libwallet.

This will keep libbitcoin focused on its core functionality. The client
side stuff is less important considering all the wallet development done
in Python, JavaScript .etc libbitcoin is aimed at server software.

https://github.com/spesmilo/libwallet

The install-sx.sh has been updated to reflect this. Users simply need to
re-run the script to update.

-------------

BTW we have a mailing list: https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/libbitcoin

[genjix]

Hi!

I've made significant performance increases for the blockchain by creating a custom database that outperforms the more conventional LevelDB. This is still a work in progress and won't be in libbitcoin at least for some time. You can read more about more research here:

https://wiki.unsystem.net/index.php/Libbitcoin/Blockchain

More optimisation is possible too.

Writes:



Reads:

[maaku]

genjix, your wiki is incorrect. LevelDB is a log-structured merge tree which is an append-only data structure periodically compacted into new files. I just took a cursory glance, but mmht looks like it would have similar performance characteristics. Have you benchmarked HyperLevelDB? I would expect that the difference in performance is probably mostly due to *LevelDB's compaction and durability features (which aren't something you can hand-wave away.. I don't want my server to suddenly go down for 24 hours because the db got corrupted and it has to resync the block chain).

[genjix]

blaa blaa blaa *I just took a cursory glance* blaa blaa blaa

http://symas.com/is-lmdb-a-leveldb-killer/

[lclc]

Looks good, I'll try it.

Do you know RocksDB?  Facebook tooked LevelDB and made some improvements that it runs faster on system with less memory. Maybe you can find some ideas there too: http://rocksdb.org/
I don't understands DB internals, just read it in the news.

[Mike Hearn]

maaku is correct that LevelDB does not use b-trees internally. Your wiki claims that it does.

[genjix]

https://wiki.unsystem.net/index.php/Libbitcoin/Open_tasks

feel free to send me an email <myusername>@riseup.net

[genjix]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi!

I've just released new versions of libbitcoin, obelisk, and sx.
This marks the first major release of Obelisk and SX. The version number
of libbitcoin has been bumped to 2.0, and I recommend everyone to upgrade.
This release has important bug fixes and code quality improvements.
I strongly recommend people to upgrade Obelisk if they were running the
unstable tarball.

These packages will be maintained for some time, with bug fixes backported.
I'll also help with upgrading code to newer versions as the API changes
(just send me a message).

sha256sum of tarballs:

3dfcf1cbe12c1d04443f1a467302e4a7ddd81a9e5afe16a3eae3bfdfd921e0be  libbitcoin-2.0.tar.bz2
940782033118e65a44a645291bfb5def96b7b423933e4de330c9d9852a4c166d  libwallet-0.4.tar.bz2
8a76e2075f6488bc8bc6487e09d9485986304c939945771102a92859d2bf0f42  obelisk-1.0.tar.bz2
3dc8688a511c9f5d698920ee2d47ae81c7dfd76f80307a1b97a04cbbebb07746  sx-1.0.tar.bz2

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBAgAGBQJSyoUmAAoJELA70IOMMKqa/uQQAKc4TXILScdBmzL2CQY9i4GG
KPwDxGPEnocPQgx+EOY6W5HSPiR6MFh4G4FGB2atPug4hpArCYUZAOSq5prjhpvu
Zag5+2xa/qaPUa4CZujIZ7+LBv51bM5Sgs1KjIg7AcTGI9vpjNYangG1kKNCzTKA
zjFf1tpoTzoFKVyYDUJaNN8fvFoGVQzkIJN4VC1udiG4x6gHEsVMDS4SmeEnAaEd
ec0VaU9HyZvErQ81aWqZe/IUlYXCQH0XQBJj1o74o/5zcqQzYunYv5/YMF0hhKmH
IyeCMEc3BhDaZJpzCrBTIaxwcVFVI1fsMB87dK/9tscQTCd+zBMtYnGNfNZfv8+3
E7aCbjpZI4H/sPtzMCY6VWBVRD62sUBo9X7rFX5k2ajY2ntyvaOUvvhAElFcyphd
PJp2YCkn1hid4QP0DvnWjVGHjmw86QojtOlqCTipFTZJTd3tY1fJRI4JVCD/95tX
cxSJLlhgcC33oi559dQKoqral6tLmqbxT0Zl1LMuZDfN5EngPlpQSylZkf8jY0zn
kwo1i73W5i6F94hz7d8s7V32WoG+N+bbMXNTPYwbIUwu4qeawi8sCyREHgvG9Irg
aIYEGdYr2by1fXKM/7sbnHI8x1ctP+zeSl3HR+KvpMc7Y1kcBxhGRrSiDIgYU4yd
hFMCGD21Q2tUTK95cD27
=UsJt
-----END PGP SIGNATURE-----

[genjix]

WARNING: I just upgraded libbitcoin to new style compressed public keys. This will produce incompatibility with generating addresses on old-style Electrum deterministic wallets. Generating private keys is still fine, but the addresses will look different. I suggest moving funds off your wallet before upgrading and then moving them back after.

Anyone who needs the old style uncompressed keys, can use the latest libbitcoin tarball release off the website rather than the Git repo.

[grux]

Can I apply github flavored markdown to make fancy it up, like on bitcoin's github repo?

[Red Emerald]

I'm trying to get this working on OS X again now that new versions are out.

https://github.com/WyseNynja/homebrew-bitcoin/issues/19
https://github.com/WyseNynja/homebrew-bitcoin/issues/20
https://github.com/WyseNynja/homebrew-bitcoin/issues/21

[Prayer]

I'm trying to build libbitcoin on FreeBSD 10.  I gave up on clang due to several issues in the source (missing braces and such) and installed gcc 4.9.

Things are looking a bit better, but I'm running into a different problem.  This isn't libbitcoin specific, but this is the source I'm trying to build and haven't had any luck with google.

In network.lo, I get an error that to_string is not a member of std.  I verified this to be a gcc problem by making a small test file and compiling it with various options.  None are working, but I can compile cleanly with clang.

Any pointers on how to build on FreeBSD (with clang or gcc)?

Thanks,

[genjix]

I'm trying to build libbitcoin on FreeBSD 10.  I gave up on clang due to several issues in the source (missing braces and such) and installed gcc 4.9.

Things are looking a bit better, but I'm running into a different problem.  This isn't libbitcoin specific, but this is the source I'm trying to build and haven't had any luck with google.

In network.lo, I get an error that to_string is not a member of std.  I verified this to be a gcc problem by making a small test file and compiling it with various options.  None are working, but I can compile cleanly with clang.

Any pointers on how to build on FreeBSD (with clang or gcc)?

Thanks,

Are you able to provide a virtual machine image for me to take a look? BTW I'm genjix on #darkwallet on Freenode IRC

[genjix]

Can I apply github flavored markdown to make fancy it up, like on bitcoin's github repo?

yes, do as you want

[genjix]

I'm trying to get this working on OS X again now that new versions are out.

https://github.com/WyseNynja/homebrew-bitcoin/issues/19
https://github.com/WyseNynja/homebrew-bitcoin/issues/20
https://github.com/WyseNynja/homebrew-bitcoin/issues/21

let me know what you need. I'm available on Freenode IRC #darkwallet

[lewisg]

I love the way they code libbitcoin.

[genjix]

thanks lewsig.

Stealth blockchain stuff is now part of libbitcoin, obelisk .etc

More info: https://wiki.unsystem.net/index.php/DarkWallet/Stealth#Toys

You can lookup the transactions in the blockchain, do the comparisons and find candidate results.

Documentation about everything is on the wiki for developers.

Code:

$ sx fetch-stealth
Usage: fetch-stealth NUMBER_BITS BITFIELD [FROM_HEIGHT]
$ sx fetch-stealth 2 deadbeef
DEBUG: ephemkey: 024398667c6a11652ae80fe6370e140cc67d4f82fb8310122cdaddae1524dad9e0 address: 1Nw1EKu8Y6mPGhMGyrKPS9TZWDyTPLvi8a tx_hash: 6a6246ccc7cb9427efee85dd3c7b80164f8a61213a7ce357b8cfd3816f59aab9
DEBUG: ephemkey: 02828c82feeae6ea5526e1b17adf3a7e9a1ee53680fc0ba651a3e7c1a99529ef24 address: 1QJSNUqt8HRygcNvbyYHQw5NizMCBkixsb tx_hash: 75885e50418cee0e527316df1f740ef6bc6b60a4b325c6d8d20e00a053cd27f9
DEBUG: ephemkey: 03a3e941099eda82cee7727e238e34283611e71debe063ed5029b51336060de080 address: 1KnByqTSjWbh5gb2K74Nkdf6oU5VMtfJqf tx_hash: 6deba5f43c85d881fa88fd40f53ab2c825682d4829db0154982ddf9f60a9a0c7
DEBUG: ephemkey: 024b4d9a7564644deb94743872e6823f87c1ce8cca7c2ba84cd926dea5498a0a53 address: 14DcERFvDPYTr1PHqphqvQuawvAoFTsKgH tx_hash: ba058ad8330e56b5cb18c08d98c465a91d7ce4d1bc5ac10f1384a0619a3559f4

[Prayer]

I'm trying to build libbitcoin on FreeBSD 10.  I gave up on clang due to several issues in the source (missing braces and such) and installed gcc 4.9.

Things are looking a bit better, but I'm running into a different problem.  This isn't libbitcoin specific, but this is the source I'm trying to build and haven't had any luck with google.

In network.lo, I get an error that to_string is not a member of std.  I verified this to be a gcc problem by making a small test file and compiling it with various options.  None are working, but I can compile cleanly with clang.

Any pointers on how to build on FreeBSD (with clang or gcc)?

Thanks,

Are you able to provide a virtual machine image for me to take a look? BTW I'm genjix on #darkwallet on Freenode IRC

If provided an environment, would you be willing to work the code to build cleanly on FreeBSD 10 using the native toolchain?

Taken further, if you had a permanent, up to date environment, would you be willing to create and maintain ports for libbitcion and your other tools?

I'm willing to provide initial funding and administration for a FreeBSD environment until such a time that it can be supported through donations.  The offer to provide administration is extended until such a time that it's no longer economical to do so.  Any donations above operating costs would be split of course, but we can talk about the exact split at a later time.

Any/all developers on the project will have full root access to the machine so they can perform full test installations for the libraries and tools being created?

Let me know!

-Prayer

PS: I may be willing to do the same for other bitcoin/altcoin projects.  Contact me via PM to discuss any details.
PPS: While additional work would still be needed for MacOS, porting to FreeBSD first should get you most of the way there.

[genjix]

yes, if I have an SSH and some help, I would make sure everything compiles as I develop the tools.

I have an admin who can provide the environment and support, but he will need assistance to setup all the dependencies and build system. Once everything is ready and I can try to compile libbitcoin (coordinating with you), then I can see what fixes need (if any) to be made to the code and maintain that by having my admin compile it every so often.

Red Emerald does mac ports here, and I'd need help for packaging. I'd just guarantee the code compiles + works.

I'm on Freenode IRC #darkwallet as genjix (admin is d3), or alternatively my email is genjix at riseup dot net (cc d3 at riseup dot net)

[benjyz]

docker.io is a linux container which might be worth looking into. it's moving to 1.0, and can be used for maintaining various tools. usually its used for complex webstacks (ebay and others use it). other tools in this area: packer.io, vagrantup.com, drone.io, saltstack (build on zmq). overall web/server integration and buildmanagement in bitcoind is not very good. with something like containers we get language/distro independent frameworks. you ship a whole thin OS with it, so you have really only one single dependency. on the downside its overhead to get into.

[dharma]

Hey!!

Yes, we can do this! Send us a email to coordenate the work. And we can use the wiki too.

My GPG key ID: 89AA3C9A

[sfireman]

Can I use libbitcoin to parse/query primecoin blockchain of live client?
i.e. client is running all the time and I want to read and query his blockchain.
Any advices?

[sfireman]

Is anyone alive here?!?!

[dharma]

Now we have VM's dedicated to libbitcoin/obelisk/sx development and test.

OpenBSD Debian Ubuntu

Who want to participate read the link above and send me a PM.

more info:

https://wiki.unsystem.net/index.php/Resources_Documentation

Cheers

[Timerondsc]

I cannot even look at this code due to the possibility of legal risk to my company!

[sshapiroNJ]

I cannot even look at this code due to the possibility of legal risk to my company!

Pardon me, didn't get you, what risk do you mean?

[PhantomPhreak]

Maybe someone can help me with this. I'm having a problem building libbitcoin-bindings on Arch Linux. I PMed genjix, but he hasn't gotten back to me yet. (I'm sure he's very busy.)

Here is the output I get when running make. I symlinked include/bitcoin/ from the libbitcoin source directory into the current directory. Also, I had to change the calls to pkg-config in the Makefile to use python3 explicitly.

Googling around led me to this closed GitHub issue for another project. Maybe it's related? I don't know how to duplicate the fix, though.

Thanks.

[genjix]

Maybe someone can help me with this. I'm having a problem building libbitcoin-bindings on Arch Linux. I PMed genjix, but he hasn't gotten back to me yet. (I'm sure he's very busy.)

Here is the output I get when running make. I symlinked include/bitcoin/ from the libbitcoin source directory into the current directory. Also, I had to change the calls to pkg-config in the Makefile to use python3 explicitly.

Googling around led me to this closed GitHub issue for another project. Maybe it's related? I don't know how to duplicate the fix, though.

Thanks.

The bindings are old and unmaintained. Sorry.

If you want to use Obelisk from Python, we have a pure Python library:

https://github.com/darkwallet/python-obelisk

[PhantomPhreak]

Maybe someone can help me with this. I'm having a problem building libbitcoin-bindings on Arch Linux. I PMed genjix, but he hasn't gotten back to me yet. (I'm sure he's very busy.)

Here is the output I get when running make. I symlinked include/bitcoin/ from the libbitcoin source directory into the current directory. Also, I had to change the calls to pkg-config in the Makefile to use python3 explicitly.

Googling around led me to this closed GitHub issue for another project. Maybe it's related? I don't know how to duplicate the fix, though.

Thanks.

The bindings are old and unmaintained. Sorry.

If you want to use Obelisk from Python, we have a pure Python library:

https://github.com/darkwallet/python-obelisk

It's no problem. Thanks for getting back to me!

I'll look at using Obelisk instead.

[jedunnigan]

Amir, just wanted to pop in and thank you for the hard work!

[Red Emerald]

I forgot to post an update here.  Thanks to wozz at Github, libbitcoin, libwallet, obelisk, sx, and obelisk now all build on OSX with https://github.com/WyseNynja/homebrew-bitcoin

[marcus_of_augustus]

Always good to see progress with important FOSS projects like these ...

[FasterStronger]

I am really keen with Libbitcoin and they've got easy to understand for new clients instruction, that's great   

[genjix]

import info for devs:
https://wiki.unsystem.net/index.php/Libbitcoin#split_project
will happen in around 2 weeks time.

[5flags]

Where are we on a windows builld?

[genjix]

Where are we on a windows builld?

eric voiskull (evoiskull) has successfully built it under MSVC. the blockchain uses mmap() which is only for posix systems so it's been commented for now.

[Xello]

I am really keen with Libbitcoin and they've got easy to understand for new clients instruction, that's great   

Totally agree with you and more over - they had pretty nice logo

[genjix]

Hi,

We will keep the master branch for periodic merges with some guarantee of stability for our users.

Development will happen on the develop branch.

This is a simplified version of the git-flow branching model:

   http://nvie.com/posts/a-successful-git-branching-model/

We can adopt the fuller scheme later if need be.

[genjix]

Per convos with devs in real life and over IRC, I've moved the repos now to:

https://github.com/libbitcoin/

Still waiting on SX because there's tons of open issues and lots more
people are using it. Hopefully we can slowly migrate sx over by
deprecating the old repo (just don't commit to it anymore).

[Gyrsur]

how is the development of "libbitcoin" coordinated with the development of "Bitcoin Core"? I mean in this way like "bitcoinj" is coordinated with "Bitcoin Core".

https://bitcointalk.org/index.php?topic=279249.msg6510490#msg6510490

[genjix]

how is the development of "libbitcoin" coordinated with the development of "Bitcoin Core"? I mean in this way like "bitcoinj" is coordinated with "Bitcoin Core".

https://bitcointalk.org/index.php?topic=279249.msg6510490#msg6510490

They try to fuck me multiple times and push me out of the process.

[Gyrsur]

how is the development of "libbitcoin" coordinated with the development of "Bitcoin Core"? I mean in this way like "bitcoinj" is coordinated with "Bitcoin Core".

https://bitcointalk.org/index.php?topic=279249.msg6510490#msg6510490

They try to fuck me multiple times and push me out of the process.

not nice!

[genjix]

ok i have a huge list of development fixes / tasks to do, but here is my primary list this month:

* resolve why testnet unconfirmed txs don't seem to work.
* sx commands for stealth. see https://wiki.unsystem.net/index.php/Sx/Stealth
* merge develop to master
* blockchain dev

[jedunnigan]

how is the development of "libbitcoin" coordinated with the development of "Bitcoin Core"? I mean in this way like "bitcoinj" is coordinated with "Bitcoin Core".

https://bitcointalk.org/index.php?topic=279249.msg6510490#msg6510490

They try to fuck me multiple times and push me out of the process.

Can you elaborate on that or point me in the direction of some links? That's very disappointing to hear, I know you've mentioned it before. Shameful, really.

[genjix]

example, saying/pretending one thing in public and doing another thing in private,

https://bitcointalk.org/index.php?topic=322328.msg3460051#msg3460051

[genjix]

hey all! we have a new forum for libbitcoin + related projects... check it out:

https://forum.unsystem.net

we also welcome other collaborators + projects to join and use the space.

[inBitweTrust]

https://www.youtube.com/watch?v=_mMZls_c7JM

SF Bitcoin Devs Seminar: A deep dive into Libbitcoin, BX, and Obelisk

[RamirezX]

Hi,

synchronization of blockchain is very slow above height 300+
Is there any direct way to speed up, with use fully synchronized blockchain of Bitcoin Core, running on the same subnet ?
Adding BC as peer doesn't help with speed ...
 
Exists any tool to convert db's ?

Thanks
R.

[h0g0f0g0]

Hi,

synchronization of blockchain is very slow above height 300+
Is there any direct way to speed up, with use fully synchronized blockchain of Bitcoin Core, running on the same subnet ?
Adding BC as peer doesn't help with speed ...
 
Exists any tool to convert db's ?

Thanks
R.

Did you find any solution? It's a real pain to sync over 300k and it looks like my computer does heavy reads (maybe needed for indexing or db cleanup).
Any way to make this faster?

[lclc]

Hi guys

We've implemented Litecoin in libbitcoin. Libbitcoin gives the Litecoin community:

- A high performance blockchain query server; libbitcoin-server

- A LTC command line tool

- LTC C++ libraries for developers to build apps

You can find more information here: https://medium.com/feathercoin/new-litecoin-full-node-as-a-gift-from-feathercoin-665e0cce52e4
(Also my tweet for this: https://twitter.com/lucas_lclc/status/1040562444404502528 )

This means you can use the same library - even the same binary - for Bitcoin and Litecoin apps built with libbitcoin.

Happy coding!

[Waldschrat2]

Eric gave some insights on the motivation and goal behind Libbitcoin in the most recent interview for Bitcoin Saigon ahead of the CryptoEcon2020 event in Hanoi at the end of this month:

The libraries matured greatly over this period, in terms of reduced dependencies, build automation, test coverage and reporting, code and interface rationalization and clean factoring. While Libbitcoin is not an application, it exposes an extensive production-quality command line tool (Bitcoin Explorer - BX) for working with various Bitcoin primitives. It also exposes a command line full node with a high performance query interface (Bitcoin Server - BS). The roadmap consists largely of evolving the node/server to support mining and wallets. It’s a very innovative implementation, never a fork of Satoshi’s code.

While others talk about factoring the Satoshi client into independent components, this is what we have done. It’s a very significant effort, comprising over a half million lines of code, supporting Linux, macOS and Windows. The code is designed to be readable and performant. We don’t see this as a tradeoff. Many of the performance optimizations in the Satoshi client are simply unnecessary, and would be deoptimizations in Libbitcoin given the architecture. The simplified, rationalized and well-factored code base also makes it an ideal tool for education and research.

We have built a lasting and committed community, essential for others to take the dependency. I’ve managed to secure funding and custom development contracts for full time, highly-skilled independent developers. Recently Tom Pacchia, Lucas Betschart and I established the Libbitcoin Institute, a US 501(c)3 non-profit to formalize and advance this effort.