cbitcoin - Bitcoin implementation in C. Currently in development.

[MatthewLM]

Hello.

I am currently making a bitcoin library named cbitcion. This is a very incomplete library in the early stages of development. The point of the library is to bring powerful bitcoin features directly to the C programming language, with a focus on documentation, portability and powerful features. Hopefully, considering this is C, it should have good performance as well. Perhaps the library could be used or modified for use in embedded device applications.

The library will be low level. It is not intended to be a client library for easy bitcoin functionality. Instead it should give programmers the tools to make bitcoin applications from basic abstraction on the bitcoin protocol. The design of cbitcoin should hopefully allow programmers to make a diverse range of bitcoin applications.

Feedback and contributions are welcome. This is the code: https://github.com/MatthewLM/cbitcoin/

I've given up developing iPhone apps to put a lot of my time towards this, so hopefully it will come along reasonably quickly.

There will be inevitable costs to this project. If anyone would like to help this project financially, donations are very welcome: 1D5A1q5d192j5gYuWiP3CSE5fcaaZxe6E9

Do not hesitate to contact me regarding this project via the email address cbitcoin@thelibertyportal.com

Thanks!

Matthew M

[hamdi]

great

[Pieter Wuille]

If you're going to rewrite C++ in C (by building virtual function tables yourself), why not code it in C++ in the first place? If compatibility is a problem, you can always expose a C interface to the code.

[MatthewLM]

The number one reason is that I've never liked C++. I get on with C much better.

C++ programmers can pretty much read and use C code with little problem. C++ code is not so easy for solely C programmers. Not only can the library be used by C programmers, it can be written by C programmers that don't get on with C++.

And implementing OOP features in C is not as hard as it sounds once you know how.

But yes, that's a common question.

[etotheipi]

I fully approve of this effort.  Many told me that it was a waste of time to reimplement Bitcoin, but there's no better way to learn everything than to do it yourself.  It's not always pleasant (especially that goddamned endianness!), but it is extraordinarily educational and rewarding by the time you do get it.  I feel like anyone who's going to participate in serious discussions about protocol, client design/features, or contribute to any Bitcoin project at all, should have this kind of experience.

I started about 10 months ago, and did a little bit of documentation as I went.  I expect you'll find this useful

https://bitcointalk.org/index.php?topic=29416.0

Feel free to PM me if you have any implementation questions.  I've been through just about all of it by now.  Half of it in C++, half in Python...
(though I never bothered with bignum/base58 in C++... I track everything by hash160 values and let Python deal with Base58 using its native bignum capability).

[finway]

I fully approve of this effort.  Many told me that it was a waste of time to reimplement Bitcoin, but there's no better way to learn everything than to do it yourself.  It's not always pleasant (especially that goddamned endianness!), but it is extraordinarily educational and rewarding by the time you do get it.  I feel like anyone who's going to participate in serious discussions about protocol, client design/features, or contribute to any Bitcoin project at all, should have this kind of experience.

I started about 10 months ago, and did a little bit of documentation as I went.  I expect you'll find this useful

https://bitcointalk.org/index.php?topic=29416.0

Feel free to PM me if you have any implementation questions.  I've been through just about all of it by now.  Half of it in C++, half in Python...
(though I never bothered with bignum/base58 in C++... I track everything by hash160 values and let Python deal with Base58 using its native bignum capability).

+1

[jgarzik]

I'm currently looking at the bignum division: http://stackoverflow.com/questions/10522379/bignum-division-with-an-unsigned-8-bit-integer-c The current algoritm was rather much a curiosity, only now have I looked for a better algorithm. This will go into the base 58 converter functions which is what I'll be working on (Maybe those can be done differently also?).

Why?  Just use OpenSSL for bignum and move on.  People have already written that part.

Bitcoin-specific data structures for storing blocks, transactions, etc. are the fundamental building blocks.

[casascius]

Why?  Just use OpenSSL for bignum and move on.  People have already written that part.

Bitcoin-specific data structures for storing blocks, transactions, etc. are the fundamental building blocks.

If he manages to pull this off without a dependency on OpenSSL, his code is going to be able to run on my credit card machines.

[jgarzik]

Why?  Just use OpenSSL for bignum and move on.  People have already written that part.

Bitcoin-specific data structures for storing blocks, transactions, etc. are the fundamental building blocks.

If he manages to pull this off without a dependency on OpenSSL, his code is going to be able to run on my credit card machines.

It is rarely a good idea to reinvent the wheel, when it comes to security-sensitive, time-tested, well-reviewed-by-experts cryptography code.

Once bignum has been reinvented, what about ECDSA and similar gadgetry?

[Pieter Wuille]

From the perspective of being educational, I have absolutely no problem with people reimplementing code. They may stumble upon bugs, and even a partial implementation can have niche purposes (for example only the networking code for a crawler, or fast blockchain manipulation code for rescanning addresses).

That said, trying to understand existing code also has great educational value; in particular when you're trying to improve things, because that confronts you with things you thought you understood but didn't. In my opinion, this is a better way for helping the community than by starting from scratch - that just takes longer before it is useful.

[DeathAndTaxes]

It is rarely a good idea to reinvent the wheel, when it comes to security-sensitive, time-tested, well-reviewed-by-experts cryptography code.

This.  Also why is avoiding dependency on OpenSSL important?  Something I am missing?

[mb300sd]

Looks like your writing this to work on 8-bit CPUs? This may be very useful if you can keep it C-only and 8-bit compatible... Single chip embedded bitcoin client anyone?

[casascius]

It is rarely a good idea to reinvent the wheel, when it comes to security-sensitive, time-tested, well-reviewed-by-experts cryptography code.

This.  Also why is avoiding dependency on OpenSSL important?  Something I am missing?

If you ripped the code from OpenSSL it would work fine.  The devices just don't have gobs of memory for libraries and they have their own compiler that has idiosyncrasies that make it not compile everything that can be compiled in Linux.  Cause it is not Linux.

This would be a 32 bit CPU, no need for 8 bits.

[MatthewLM]

This library aims to have zero dependencies except the standard C library making it small and easily portable. This doesn't mean I will reinvent the wheel in every single case. There are many open source implementations of the things my library will need. I can adapt these tried and tested algorithms to suit the specific needs of this library. In some cases it may be more or less copy and paste and in other cases it may require tweaks or substantial modifications.

At the moment I'm implementing specialist functions that include operations between a bignum and an 8 bit integer. This is for the base-58 encoder/decoder. If anyone knows a better solution to the base-58 stuff, please tell me or feel free to work on it yourself. When I'll be going operations between two bignums it should be simple to derive my code from bignum libraries and then I can derive the cryptography code from open-source libraries such as OpenSSL too.

If he manages to pull this off without a dependency on OpenSSL, his code is going to be able to run on my credit card machines.

Well wouldn't that be interesting. You would need an implementation of standard library features including dynamic memory allocation but I plan absolutely no additional dependencies other than the standard C library. Of-course, feel free to contribute anything, including ideas.

I fully approve of this effort.  Many told me that it was a waste of time to reimplement Bitcoin, but there's no better way to learn everything than to do it yourself.  It's not always pleasant (especially that goddamned endianness!), but it is extraordinarily educational and rewarding by the time you do get it.  I feel like anyone who's going to participate in serious discussions about protocol, client design/features, or contribute to any Bitcoin project at all, should have this kind of experience.

I started about 10 months ago, and did a little bit of documentation as I went.  I expect you'll find this useful

https://bitcointalk.org/index.php?topic=29416.0

Feel free to PM me if you have any implementation questions.  I've been through just about all of it by now.  Half of it in C++, half in Python...
(though I never bothered with bignum/base58 in C++... I track everything by hash160 values and let Python deal with Base58 using its native bignum capability).

Thanks. I am doing this largely to learn as well as to create something functional. The images you have made a pretty nice. Would I be able to use them in my documentation? I do hope to build a nice documentation for my library.

For the next few days I'll be busy with other things but I'll when I'm back to this library I hope to sort of the base-58 code and also modify the OOP so that the virtual function tables are initialised at compile time which makes more sense.

Looks like your writing this to work on 8-bit CPUs? This may be very useful if you can keep it C-only and 8-bit compatible... Single chip embedded bitcoin client anyone?

The library should be compilable with C compilers providing C99 with the standard library is supported (At least everything the library uses). Perhaps the library will need minor tweaks for compilers that aren't compatible with some features in C99 but that's not going to cause many headaches I sure. The biggest headache would be to implement the standard library features for microcontrollers with no out of the box standard C library support.

But are you saying some 8-bit architectures come with compilers that only allow 8 bit types? That would be pretty rubbish. The compilers should support up-to 64 bit integer types (int64_t). I believe long long types should be at least 64 bits so a compiler that keeps to the C specification properly should support it.

If anyone understands microcontrollers well and can advise on how the library can be designed with microcontroller portability in mind, please share your knowledge.

[etotheipi]

Thanks. I am doing this largely to learn as well as to create something functional. The images you have made a pretty nice. Would I be able to use them in my documentation? I do hope to build a nice documentation for my library.

Go ahead.  Attribution is preferable, but I'm really just happy if other people get use out of them.  Maybe I can save you the 2 days I spent battling endianness and the OP_CHECKSIG wiki page trying to figure out how the hell to implement it!

[MatthewLM]

Under the images I can put "Pictures created by Alan Reiner, developer of Bitcoin Armory (http://bitcoinarmory.com/)"?

I'm hoping following bitcoinj will also help (It's very easy to read), the issue is that bitcoinj doesn't implement everything so I'll have to look into other code that does implement things like the script properly. The C++ client has the code but it harder to read. Documentation no doubt helps me understand what I'm doing rather than do a piece by piece port, so I can make better code.

[jgarzik]

It is rarely a good idea to reinvent the wheel, when it comes to security-sensitive, time-tested, well-reviewed-by-experts cryptography code.

This.  Also why is avoiding dependency on OpenSSL important?  Something I am missing?

If you ripped the code from OpenSSL it would work fine.  The devices just don't have gobs of memory for libraries and they have their own compiler that has idiosyncrasies that make it not compile everything that can be compiled in Linux.  Cause it is not Linux.

This would be a 32 bit CPU, no need for 8 bits.

Ummm...

OpenSSL is heavily used in the resource-constrained, embedded space, and runs on Windows, Linux, uclinux (embedded linux), VxWorks (an embedded OS), OSX, Solaris, HPUX, OSF1/Tru64, IBM mainframes, ancient MS-DOS systems, netware, and others old and new.  It works very well on almost every known 32- and 64-bit  platform in existence.

It does not require "gobs of memory"; the state structures for AES or SHA algorithms are almost always smaller than 1,024 bytes.  Compiled code size is already small and compact.

Copying OpenSSL code into your own codebase will not save any memory, regardless of linking style.  Libraries are internally split into modules and demand-paged in.  At best it will save disk space for unused code, at a cost of always lagging behind whenever security problems in the code are found.

[doldgigger]

If you're going to rewrite C++ in C (by building virtual function tables yourself), why not code it in C++ in the first place? If compatibility is a problem, you can always expose a C interface to the code.

In order to have a super-lean runtime and dev environment, for example...

[casascius]

It is rarely a good idea to reinvent the wheel, when it comes to security-sensitive, time-tested, well-reviewed-by-experts cryptography code.

This.  Also why is avoiding dependency on OpenSSL important?  Something I am missing?

If you ripped the code from OpenSSL it would work fine.  The devices just don't have gobs of memory for libraries and they have their own compiler that has idiosyncrasies that make it not compile everything that can be compiled in Linux.  Cause it is not Linux.

This would be a 32 bit CPU, no need for 8 bits.

Ummm...

OpenSSL is heavily used in the resource-constrained, embedded space, and runs on Windows, Linux, uclinux (embedded linux), VxWorks (an embedded OS), OSX, Solaris, HPUX, OSF1/Tru64, IBM mainframes, ancient MS-DOS systems, netware, and others old and new.  It works very well on almost every known 32- and 64-bit  platform in existence.

It does not require "gobs of memory"; the state structures for AES or SHA algorithms are almost always smaller than 1,024 bytes.  Compiled code size is already small and compact.

Copying OpenSSL code into your own codebase will not save any memory, regardless of linking style.  Libraries are internally split into modules and demand-paged in.  At best it will save disk space for unused code, at a cost of always lagging behind whenever security problems in the code are found.

Oh, yes, I should clarify.  VeriFones use a persistent battery-backed RAM-based file system, so disk space essentially equals RAM.  I am referring to the size of the library, not how much memory it allocates.  And some of the other devices I've developed for have very limited flash storage where fitting the entire compiled openssl lib takes up significant space.  I suppose it's laughable to consider a couple megabytes to be "gobs" but in the context of these cheapo devices, it is.  Also for the VeriFones they don't use GCC so sometimes things take refactoring for their dinosaur compiler to be able to handle it (and makefiles in particular).  Of course I understand having outdated tools is sucky, but that's also why you can find these things on eBay for so cheap.  The Omni 3200 model has an even older DOS-based compiler, but someone who bites the bullet and makes a working app for it suddenly breathes new life into a $30-on-eBay all-in-one-computer with a printer to boot.

Even with security bugs (not saying security bugs are OK), but possession is nine tenths of the law with these old school devices.  Suppose somebody finds a way to be able to perform a side channel attack on it and steal keys with RF emissions or something, they also need to gain physical access to the device to do it, making a widespread attack not all that feasible.  Their network support is rudimentary (often the units with Ethernet ports are merely Ethernet-to-Serial converters on an internal tty that is completely silent unless you write an app to access it) so the odds of someone making a key-disclosing network attack against them is pretty much moot.

[MatthewLM]

Would people be more universally happy if my library required developers using the library to pass cryptographic functionality to the library using function pointers? That way people can use whatever cryptography library they like that matches their needs, and my library will focus on everything else unique to bitcoin.

Good idea?