I've had the feeling this was the direction we've been moving in for a while. Now the eager response of Hive is a welcome addition.
The Linux operating system came together when Linus' kernel was combined with the GNU system. If we wish to build a resilient and successful opensource ecosystem, then it will need the combination of projects responsible for a small piece of turf. It will force us to serve each other and create something very powerful.
When I got involved with Electrum, my goal was to replace the backend with libbitcoin. I saw Electrum as the future. Electrum's community and technology get me very excited. It's an open community, with lots of solidarity (despite the small size) and people helping out with infrastructure in a decentralised self-organised way.
As I started to develop on Electrum (which involved refactoring the server code to support pluggable backends), I became concerned with the UI. Over 2-3 months I did my own usability studies with friends and people, and started to focus on how Electrum should look like. Then there was more refactoring of the client code so we could integrate another GUI. During this time also saw Animazing becoming involved in development and he also had lots of ideas.
The original libbitcoin Electrum backend, was a fullnode integrated straight into Electrum server. However I didn't believe this was a scalable model. After the bitcoin2012 conference, I took a big break from Bitcoin for some months (the conference was too much). During this time, I started to formulate new ideas about a backend, slowly hacking prototypes at first, theorising, writing more code, rewriting, trying different approaches, ... The moving from Apache Thrift to ZeroMQ was a massive leap. I have a hero worship problem with the ZMQ creator Pieter Hintjens
I think he's a genius.
http://zguide.zeromq.org/page:all#toc7And obelisk was born.
The community around libbitcoin started to grow, and we slowly developed our software. We started to plan between ourselves the tools and code needed to build a functional wallet. I still wasn't sure whether we'd go with Electrum or make something new. I remember when trying to integrate the light GUI into Electrum, it wasn't so modular (tight coupling between GUI and wallet).
Then we had the Barcelona Electrum developer meeting, and Electrum's new release was impressive. Thomas had re-architected Electrum into a modular framework, and was rapidly integrating features like HD wallets. We also saw the Electrum Android preview from Andy which was awesome. Never seen such a great wallet on mobile phones - and it's based off Electrum and Python! Andy is working with Kivy developers to build the thing.
The new aim for Electrum is a modular framework for creating wallets. The framework provides the features and you just hook them into an interface. Thomas demoed a terminal Electrum app to illustrate this.
I spoke with Pablo because I wasn't sure how to support Electrum. Before my intention had been to make an adapter that provided the current Electrum API (so clients could remain compatible with minimal changes to the Electrum codebase), but this was sub-optimal and doesn't take advantage of the parallel asynchronous scalability features (which needs some knowledge from the API user to use effectively). Also with multiple workers you need to be able to handle failure cases to retry a different host or know how to properly query results.
Pablo suggested the way forwards in replacing the client side Electrum blockchain query code with obelisk API. This was a problem though since Electrum is a Python-only project and the current obelisk was a C++ library. Also even if we made a Python obelisk then it will still depend on ZMQ. Since there was the Electrum Android Kivy project (and we're interested in phones too), we tried building ZMQ on the mobile phone. It was a nightmare and we never got it working (I don't think it's ever been managed). Pablo did the unthinkable and rewrote ZMQ in pure Python! Pieter Hintjens (project maintainer for ZMQ) was excited and wants to make it an official part of ZMQ. So we have a pure Python obelisk library.
By moving some of the Electrum core utils (such as bitcoin.py) into his project, we have a full Python wallet software library. Also Vitalik made Pybitcointools for working with keys, and Robert (a libbitcoin contributor) created a good abstraction layer for the blockchain which makes it look like a Django database ORM (for the blockchain). So we can move this functionality into a library, and prefer friend's code (so they own that piece of the ecosystem).
The technology in Electrum is solid, but it needs a decent UI. My only concern is that if we have worker -> balancer -> desktop core -> interface, that's a lot of overhead for transferring wallet history but maybe it's not such a big deal in the end. Also I'm working on some important issues to do with the blockchain database. I've been developing a custom database.
This is database read and write speed for Bitcoin transactions vs LevelDB. Note these results are merely illustrative. There's a lot more complex things happening underneath that's difficult to explain (I will write a manual). But by exploiting the properties of the blockchain leads to some interesting results such as hot backups, lock-free reads and lack of a separate database cache (instead use madvise() to tell the kernel how to cache from disk). Anyway this is experimental stuff. Read speed is my priority.
Together we can create a big group of talented developers freeing up the others to focus on our particular development interests. I see the potential in Hive for security features and concepts Thomas is working on in Electrum. There is the old conflict between the top-down cathedral with the ability to mobilise huge resources through command hierarchies, vs the bottom-up bazaar that provides the creative and open environment for merit to rise to the top. Debian is the Linux kernel, the GNU system tools, the graphics layer, the Gnome desktop environment and the various applications. I am optimistic that we can create the same resiliency and strength for the future of money.
