Amiko development progress

[cjp]

(last status update: Dec 4th 2014)

I'd like to use this thread to keep you informed about development progress of Amiko. Whenever practical (e.g. software development) issues pop up, I'd also like to discuss them here. Concept choices are better discussed in the threads that describe these concepts.

You don't know Amiko? It's basically an implementation of the ideas in these threads:
https://bitcointalk.org/index.php?topic=94674.0
https://bitcointalk.org/index.php?topic=152334.0

You can find the latest source code on Github:
https://github.com/cornwarecjp/amiko-pay

Code:

Plan + completion status:
[90%] Concept design
[75%] Development of proof of concept prototype
[ 0%] Security hardening
[ 0%] Development of usability features
[10%] Documentation
[ 0%] Beta testing, processing user feedback
[ 0%] Deployment, marketing etc.

Code status (will improve in time):
* Usefulness: no known use cases
* Security level: has known security holes
* Documentation level: current feature set is mostly documented

For now I have some practical questions:

1) What is the best way to connect to the bitcoind RPC interface from C++? Is it a good idea to just copy some RPC code from the Bitcoin source code?

2) Since Amiko will depend on bitcoind for a lot of things already, I'm considering using Bitcoin as a private key storage for Amiko. That way, I don't have to re-invent the wheel for a secure (encrypted etc.) storage method. Is this a good idea? It would mean that the private keys have to be transferred through the (unencrypted) RPC connection from Bitcoin to Amiko (e.g. with the dumpprivkey command). It would also mean that Amiko has to have the passphrase for unlocking the Bitcoin wallet; how should the passphrase be stored? Maybe it shouldn't be stored at all, but provided by the user at start-up of Amiko?

3) Are there any resources on how to best use testnet? E.g. I've read there were some problems in the past with testnet mining; do I need to do my own mining? Is there a place to get some testnet coins for free?

[1714072039]

1714072039

[1714072039]

1714072039

[1714072039]

1714072039

[cjp]

I'm bumping this thread to let you know that, after 6 months of inactivity (due to personal reasons), I've restarted working on this project.

[genjix]

saluton amiko!

have you tried obelisk?

c++ lib: https://github.com/spesmilo/obelisk/blob/master/include/obelisk/client/blockchain.hpp
python lib: https://bitcointalk.org/index.php?topic=354607

based off libbitcoin: http://libbitcoin.dyne.org/

also http://sx.dyne.org/

[cjp]

saluton amiko!

have you tried obelisk?

c++ lib: https://github.com/spesmilo/obelisk/blob/master/include/obelisk/client/blockchain.hpp
python lib: https://bitcointalk.org/index.php?topic=354607

based off libbitcoin: http://libbitcoin.dyne.org/

also http://sx.dyne.org/

Thanks for the links.
I was about to ask you whether this is the same libbitcoin as the one developed by Amir Taaki, but then I realized who you are ... 

Your blockchain.hpp seems too limited for my needs, but if the underlying architecture is what I want, then I might just use it as a starting point, make my own extensions and contribute them back to you. After all, that's how open source is supposed to work, isn't it?

In terms of architecture, what I'm really looking for is this:

• Use standard bitcoind as back-end. In the short term, this is more likely to be trusted than alternative Bitcoin implementations; also, altcoins are likely to be based on bitcoind, so porting Amiko to altcoins will be simple. In the longer term, interfaces to alternative Bitcoin implementations are necessary, especially bitcoinj for Android implementations.
• Low-level layer: the RPC protocol used by bitcoind: connection open/close and serialization/deserialization of messages.
• High-level layer: the actual bitcoind API, exposed as C++ methods and data structures.
• Very high level layer: the set of functions actually required by Amiko. Note that this is probably the best layer to be re-implemented when porting from bitcoind to an alternative Bitcoin implementation with a different API. Essentially, Amiko needs to deal with microtransaction channels, so it needs to do thinks like multisignature transactions and nLockTime + non-final transactions.

[cjp]

c++ lib: https://github.com/spesmilo/obelisk/blob/master/include/obelisk/client/blockchain.hpp
python lib: https://bitcointalk.org/index.php?topic=354607

based off libbitcoin: http://libbitcoin.dyne.org/

also http://sx.dyne.org/

I've had a closer look at these things, and they all seem to be using libbitcoin. In itself that's not bad: at this moment, I consider libbitcoin to be architecturally and politically superior to the Satoshi client. However, my trust in software is based on what the code does and how many people have reviewed it, and not on who wrote it; I'd have to give it the same code review I once gave to the Satoshi client. Besides, what's even more important is what potential users of Amiko are trusting. In areas of uncertainty, people tend to "follow the crowd", so I think the most frequently used implementation has an advantage here (unfortunately).

In the long term I certainly consider it important to have support for libbitcoin, because of the importance of having multiple independently maintained Bitcoin implementations. However, I have to "choose my fight", and in the short term I consider it more important to get Amiko up and running with the Satoshi client.

[cjp]

I'd like to give an update about the progress I've made so far:

A discussion with some other people made me realize that the basic architecture of the C++ code was sub-optimal. Basically, every network protocol link is modeled as a dialogue with blocking "wait for message" actions. To keep the application responsive, it was designed in a massively multithreaded way, which created its own problems. I started to believe that an event-based architecture is more appropriate for this project. This can initially be made (mostly) single-threaded, but a future version might have multiple event-handling threads.

I also realized that, even on an architectural level, this code is much more experimental than I'd admitted so far. In my experience, Python is much more suitable for such experimental code than C++, so I basically re-wrote the entire application in Python, with a new event-based architecture.

So far, that seems to have been a very good decision. I managed to re-create most of the functionality of the C++ code, in only 10% of the size of the code, and with a much shorter development time. I'm thinking I might just stick with Python for a while, and just get rid of the C++ version as soon as I don't need it anymore.

One problem: there doesn't seem to be ECDSA support in Python, or in any common Python package. What would be the best way to do ECDSA in Python? I've found some ECC/ECDSA implementations in Python on this forum, but I could also try to call the openSSL implementation from Python (with ctypes?). The last one seems to be better, since it will be faster, and it is frequently used, well-known and well-supported open source code, which probably makes it more secure than other implementations.

The same question applies to getting a source of secure random data.

[cjp]

WOOHOOOOOOO!!!!!

I managed to submit the first Bitcoin transaction, created by the Amiko software:
https://blockchain.info/tx/8f04cac419d838551106eb496f9871f3fb6d37aff3e3b25d412ab1b066018564

This is still a relatively boring transaction, with standard input/output script, but you have to start somewhere. I consider it quite an achievement that I got the transaction signing right. Going from here to more complicated scripts should be a smaller step.

This transaction was created the following way:
Amiko asks Bitcoin for unspent outputs
Amiko selects some unspent outputs
Amiko asks Bitcoin for the corresponding private keys
Amiko constructs the transaction and signs is
I submitted the serialized transaction at blockchain.info. I could have submitted it through Bitcoin, but I didn't want to risk corrupting my local Bitcoin setup with a bogus transaction; now the risk was on blockchain.info 

I will soon extend this functionality to multi-signature transactions, and then integrate it into the already implemented Ripple-style payment system.

Some technical information:
The latest Amiko code is 100% Python. I connect to Bitcoin using Jeff Garzik's AuthServiceProxy module. I use OpenSSL through ctypes for hashing and ECDSA signatures.

[cjp]

I've implemented multi-signature transactions (the basic building block of a microtransaction channel:
https://blockchain.info/tx/c4767cc0ce7adc4630c424d1230fdddc7721a0c449a16a8b79579806ea06f69f
https://blockchain.info/tx/aebe8ad10790943dbf263183309c13f2943781b1b620576eee24133e8b6203d1

Note: while the "spending" transaction seems to be accepted by Bitcoin-Qt and by blockchain.info, it isn't included into a block yet. Blockchain.info mentions a warning that the transaction has a non-standard input. This sounds strange to me: "OP_0 sig1 sig2" seems to be the standard way to sign this type of transaction, and I've even seen something like that in the Bitcoin source code.

Edit: apparently, the transaction is now confirmed, apparently by "AntPool". I had to wait 45 minutes for that block!!! Now, the warning on blockchain.info is also gone.

[cjp]

If you wish to experiment with Amiko Pay: I've made that a bit easier by updating the manual:
https://github.com/cornwarecjp/amiko-pay/blob/master/doc/manual.md

Even without experimenting, the manual should give you an impression of the current set of features.

The previous version of the manual was written for the C++ version, which has not received an update for a long time. The Python prototype now has so much more functionality than the C++ version that I decided to remove the C++ version from the source tree. Of course, you can still access it by checking out old revisions with Git.

[cjp]

I made the first real Amiko transactions!

• I created a transaction link to myself (the two sides of the link are named "A" and "B")
• I deposited 1 mBTC on side A of the link. This is the deposit transaction.
• I transferred 0.2 mBTC from side A to side B with an Amiko transaction.
• I transferred 0.05 mBTC back from side B to side A.
• I withdrawed the bitcoins from the link. This is the withdrawal transaction. Notice that only the final balances of the link end up in the block chain.

Log files:
Console output
debug.log

Since it only involves a single link, this doesn't yet demonstrate routing capabilities. Some very primitive routing is implemented though, so if you wish, you can try to beat me in performing the very first routed Amiko transaction.

Many checks, handling of failure modes etc. aren't implemented yet, so you shouldn't consider it secure in its current state. Still, I think I should be working towards making a first release. Now that a very minimal set of features is present, I think I should focus on making the software secure, well-tested and well-documented. To speed up development process I should avoid making features that aren't needed; to discover what features are needed, I need to have a community of users. Hopefully, if this minimal set of features can be provided by secure software, I might already attract some early adopters...

[runeks]

Hi

I just want to let you know that I'm following the development with interest. What has happened since the last update?

One question: is it not the case that transactions aren't risk free yet, because of transaction malleability, or is this problem avoided somehow?

EDIT: I just found https://github.com/cornwarecjp/amiko-pay/blob/master/doc/vulnerabilities.md -- it answers all my security questions. Wonderful documentation .

[cjp]

What has happened since the last update?

The most significant thing that happened was that the Lightning Network concept was published, which is very similar to the original Amiko Pay concept. I examined the Lightning Network and came to the conclusion that it achieves the same things as Amiko Pay, while it requires much smaller changes to Bitcoin to become completely "trust-free". So, my plan is now to implement the Lightning Network concept in Amiko Pay.

Since the Lightning Network still requires some Bitcoin features that currently don't exist yet, I went looking for ways to deal with this absence. Read this thread for a description of the design I made for this.