Reverse-engineering and documenting Bitcoinica

[davout]

I've just started a technical document aimed at describing the way the bitcoinica codebase works, how the trading works and how one could potentially setup a clone. Feel free to help !

It's all here

[jim618]

After the amount of heartache and financial loss Bitcoinica has caused to its users and the wider Bitcoin community you would be better off throwing it away and starting again.

[notme]

After the amount of heartache and financial loss Bitcoinica has caused to its users and the wider Bitcoin community you would be better off throwing it away and starting again.

Not really.  The code needs a security audit, but the trading code is decent.

[davout]

Started thoroughly studying the trading part, here's my understanding of the trade matching code.

My comments start with "D :", there are like 2 comments made by ZT

http://pastie.org/4257541

[jimbobway]

I don't recommend Ruby on Rails, but perhaps the logic could be recreated in another more reliable language in addition to beefing up the security.

[Andrew Vorobyov]

Guys, I'm doing margin trading exchange at https://github.com/santacruz123/node-bitcoin-exchange

It's PostgreSQL\NodeJS based. Order matching done through SQL triggers...

I would accept some criticism on early stages

[notme]

I don't recommend Ruby on Rails, but perhaps the logic could be recreated in another more reliable language in addition to beefing up the security.

Cool story.  What makes Ruby unreliable/insecure?

[grue]

Started thoroughly studying the trading part, here's my understanding of the trade matching code.

My comments start with "D :", there are like 2 comments made by ZT

http://pastie.org/4257541

doesn't ruby have something like enums? why make slow string comparisons?

[davout]

Started thoroughly studying the trading part, here's my understanding of the trade matching code.

My comments start with "D :", there are like 2 comments made by ZT

http://pastie.org/4257541

doesn't ruby have something like enums? why make slow string comparisons?

I'm interested in the semantics, not the aesthetics.

[jimbobway]

I don't recommend Ruby on Rails, but perhaps the logic could be recreated in another more reliable language in addition to beefing up the security.

Cool story.  What makes Ruby unreliable/insecure?

Here are some reasons not to do it in Ruby on Rails:

1.) Ruby on Rails is a scripting language built on top of another language.  Any flaws or bugs in the foundation language can propagate to the scripting language.  It takes time to fix these changes until it is fixed and compiled in the Ruby language.  (PHP works the same way where functions in PHP are mostly wrappers to functions in other libraries.)

2.) Ruby on Rails hasn't been around as long as some other web languages.  It's less proven.

3.) There are less Ruby on Rails developers then other languages.  In the case with Bitcoinica, the code was passed to Intersango who had no experience with Ruby on Rails.

4.) Ruby on Rails attempts to write code automatically for you.  It's possible the automatically written code could be overlooked.

5.) There specific security issues with RoR.  (I guess you could Google it.)

[dogisland]

I don't recommend Ruby on Rails, but perhaps the logic could be recreated in another more reliable language in addition to beefing up the security.

Cool story.  What makes Ruby unreliable/insecure?

Here are some reasons not to do it in Ruby on Rails:

1.) Ruby on Rails is a scripting language built on top of another language.  Any flaws or bugs in the foundation language can propagate to the scripting language.  It takes time to fix these changes until it is fixed and compiled in the Ruby language.  (PHP works the same way where functions in PHP are mostly wrappers to functions in other libraries.)

2.) Ruby on Rails hasn't been around as long as some other web languages.  It's less proven.

3.) There are less Ruby on Rails developers then other languages.  In the case with Bitcoinica, the code was passed to Intersango who had no experience with Ruby on Rails.

4.) Ruby on Rails attempts to write code automatically for you.  It's possible the automatically written code could be overlooked.

5.) There specific security issues with RoR.  (I guess you could Google it.)

Let me clarify if I may as I think you've dismissed ruby and ruby on rails without acknowledging the benefits.

Ruby is a dynamic language  http://en.wikipedia.org/wiki/Dynamic_programming_language

Rails is an MVC based architecture for the rapid development of web applications. The Rails architect is built using the Ruby language and that's why it's called Ruby on Rails. (RoR).

Let me address some of the points you've raised.

1. The ruby interpreter is written in C I believe, it's been around since the mid 90's and is stable and mature.

2. The Rails architecture has been around since 2005 and is at version 3. It powers a large number of websites including twitter, github, scribd, shopify and many more.

3. There are plenty of ruby on rails developers, just that intersango didn't have that skill set.

4. Rails dynamically creates methods at run time so that you don't have to, it can be argued that this is a benefit as the developer writes less code and therefore can make less mistakes.

5. There have been specific security issues with Rails. They get addresses quickly in my experience.

I've been developing web apps for over 12 years and RoR is easily the quickest and most succinct way to build web applications in my experience.

[notme]

I don't recommend Ruby on Rails, but perhaps the logic could be recreated in another more reliable language in addition to beefing up the security.

Cool story.  What makes Ruby unreliable/insecure?

Here are some reasons not to do it in Ruby on Rails:

1.) Ruby on Rails is a scripting language built on top of another language.  Any flaws or bugs in the foundation language can propagate to the scripting language.  It takes time to fix these changes until it is fixed and compiled in the Ruby language.  (PHP works the same way where functions in PHP are mostly wrappers to functions in other libraries.)

2.) Ruby on Rails hasn't been around as long as some other web languages.  It's less proven.

3.) There are less Ruby on Rails developers then other languages.  In the case with Bitcoinica, the code was passed to Intersango who had no experience with Ruby on Rails.

4.) Ruby on Rails attempts to write code automatically for you.  It's possible the automatically written code could be overlooked.

5.) There specific security issues with RoR.  (I guess you could Google it.)

Let me clarify if I may as I think you've dismissed ruby and ruby on rails without acknowledging the benefits.

Ruby is a dynamic language  http://en.wikipedia.org/wiki/Dynamic_programming_language

Rails is an MVC based architecture for the rapid development of web applications. The Rails architect is built using the Ruby language and that's why it's called Ruby on Rails. (RoR).

Let me address some of the points you've raised.

1. The ruby interpreter is written in C I believe, it's been around since the mid 90's and is stable and mature.

2. The Rails architecture has been around since 2005 and is at version 3. It powers a large number of websites including twitter, github, scribd, shopify and many more.

3. There are plenty of ruby on rails developers, just that intersango didn't have that skill set.

4. Rails dynamically creates methods at run time so that you don't have to, it can be argued that this is a benefit as the developer writes less code and therefore can make less mistakes.

5. There have been specific security issues with Rails. They get addresses quickly in my experience.

I've been developing web apps for over 12 years and RoR is easily the quickest and most succinct way to build web applications in my experience.

Thanks for saving me the time.  Ruby kicks ass.  Rails takes names.

[Peter Todd]

Started thoroughly studying the trading part, here's my understanding of the trade matching code.

My comments start with "D :", there are like 2 comments made by ZT

http://pastie.org/4257541

doesn't ruby have something like enums? why make slow string comparisons?

I can't speak for Ruby or that exact case specifically, but often in modern languages string comparisons are a lot faster than they look. For instance, a typical enum replacement would look like this:

Code:

a = "foo"

if a == "foo":
    do stuff

The trick is that if strings are immutable, and short strings are guaranteed to have unique memory locations, the a == "foo" comparison can actually be implemented as a direct pointer comparison rather than a slow string comparison. This is basically just as fast as a traditional enum, and in practice takes up the same amount of space. (integers in structures are usually not packed) Of course, comparing three letters is pretty quick as well, especially in the context of an interpreted language. FWIW if I'm not mistaken Python strings work this way, and it's considered "Pythonic" to use strings to replace enums.

Premature optimization is the root of all evil.

[notme]

Started thoroughly studying the trading part, here's my understanding of the trade matching code.

My comments start with "D :", there are like 2 comments made by ZT

http://pastie.org/4257541

doesn't ruby have something like enums? why make slow string comparisons?

I can't speak for Ruby or that exact case specifically, but often in modern languages string comparisons are a lot faster than they look. For instance, a typical enum replacement would look like this:

Code:

a = "foo"

if a == "foo":
    do stuff

The trick is that if strings are immutable, and short strings are guaranteed to have unique memory locations, the a == "foo" comparison can actually be implemented as a direct pointer comparison rather than a slow string comparison. This is basically just as fast as a traditional enum, and in practice takes up the same amount of space. (integers in structures are usually not packed) Of course, comparing three letters is pretty quick as well, especially in the context of an interpreted language. FWIW if I'm not mistaken Python strings work this way, and it's considered "Pythonic" to use strings to replace enums.

Premature optimization is the root of all evil.

Ruby symbols would be preferable to string comparisons, but yeah, it's really an unnecessary optimization.

[grue]

The trick is that if strings are immutable, and short strings are guaranteed to have unique memory locations, the a == "foo" comparison can actually be implemented as a direct pointer comparison rather than a slow string comparison. This is basically just as fast as a traditional enum, and in practice takes up the same amount of space. (integers in structures are usually not packed) Of course, comparing three letters is pretty quick as well, especially in the context of an interpreted language. FWIW if I'm not mistaken Python strings work this way, and it's considered "Pythonic" to use strings to replace enums.

Premature optimization is the root of all evil.

Premature optimization may be the root of all evil, but only if it makes the code harder to read, aka replacing the strings with status numbers. Enums doesn't reduce code readability, as it is essentially the same as strings, (except without the quotes). Secondly, enums are superior to strings because they are strongly type checked to prevent any undefined behavior, which may result from typos. In addition, IDEs can recognize enums and provide additional features such as auto completion and error detection, which boosts productivity.

[BasementMiner!]

I don't recommend Ruby on Rails, but perhaps the logic could be recreated in another more reliable language in addition to beefing up the security.

Cool story.  What makes Ruby unreliable/insecure?

Here are some reasons not to do it in Ruby on Rails:

1.) Ruby on Rails is a scripting language built on top of another language.  Any flaws or bugs in the foundation language can propagate to the scripting language.  It takes time to fix these changes until it is fixed and compiled in the Ruby language.  (PHP works the same way where functions in PHP are mostly wrappers to functions in other libraries.)

2.) Ruby on Rails hasn't been around as long as some other web languages.  It's less proven.

3.) There are less Ruby on Rails developers then other languages.  In the case with Bitcoinica, the code was passed to Intersango who had no experience with Ruby on Rails.

4.) Ruby on Rails attempts to write code automatically for you.  It's possible the automatically written code could be overlooked.

5.) There specific security issues with RoR.  (I guess you could Google it.)

I'm not sure if you're a Rails troll and trying to spread some FUD but I'm going to clear some things up for you.

1.) Ruby on Rails is a scripting language built on top of another language.  Any flaws or bugs in the foundation language can propagate to the scripting language.  It takes time to fix these changes until it is fixed and compiled in the Ruby language.  (PHP works the same way where functions in PHP are mostly wrappers to functions in other libraries.)

Ruby on Rails is not a scripting language. It is a MVC framework built on top of Ruby. Just like other frameworks, like CodeIgniter, Kohana, Yii, ASP.NET MVC 3, it aims to provide agile development.

Many applications have been built with it. Sites such as GitHub (which hosts the core Bitcoin project), Yellow Pages, Groupon, and most of the latest hip start up companies use it.

2.) Ruby on Rails hasn't been around as long as some other web languages. It's less proven.

Have you been living under a rock? http://rubyonrails.org/applications

3.) There are less Ruby on Rails developers then other languages.  In the case with Bitcoinica, the code was passed to Intersango who had no experience with Ruby on Rails.

The community & ecosystem is huge. Large enough for many companies to use it, and for many many libraries to be written for it. https://github.com/rails/rails

4.) Ruby on Rails attempts to write code automatically for you.  It's possible the automatically written code could be overlooked.

It does not write code for you. It's a framework.

5.) There specific security issues with RoR.  (I guess you could Google it.)

Just like any other Framework or language, there will always be potential security vulnerabilities. All of these have been fixed quickly.

However for vulnerabilities such as XSS, SQL injection, CSRF attacks, it is up to the developer to fix these. Rails already takes advantage of the best practices so these are almost impossible to leak through for any app. You won't have to go boast to your visitors that you use 'prepared statements'.

[jimbobway]

Sure there are sites that use Rails such as Twitter.  The primary reason why I say it's less proven is because I don't see very many financial websites using Rails.

Etrade, Ameritrade, Chase, Bank of America, PayPal, etc. don't use Rails.

But, I am willing to give Rails the benefit of the doubt and maybe I am wrong this time.

[Peter Todd]

The trick is that if strings are immutable, and short strings are guaranteed to have unique memory locations, the a == "foo" comparison can actually be implemented as a direct pointer comparison rather than a slow string comparison. This is basically just as fast as a traditional enum, and in practice takes up the same amount of space. (integers in structures are usually not packed) Of course, comparing three letters is pretty quick as well, especially in the context of an interpreted language. FWIW if I'm not mistaken Python strings work this way, and it's considered "Pythonic" to use strings to replace enums.

Premature optimization is the root of all evil.

Premature optimization may be the root of all evil, but only if it makes the code harder to read, aka replacing the strings with status numbers. Enums doesn't reduce code readability, as it is essentially the same as strings, (except without the quotes). Secondly, enums are superior to strings because they are strongly type checked to prevent any undefined behavior, which may result from typos. In addition, IDEs can recognize enums and provide additional features such as auto completion and error detection, which boosts productivity.

See, in a strongly typed language, I'd agree with you. But Ruby and Python are dynamically typed, which greatly reduces the advantages of traditional enums because there is no mechanism to type-check them anyway.

Note though, the actual preferred Ruby enum approach is apparently symbols - as notme mentioned, see http://stackoverflow.com/questions/75759/enums-in-ruby - which are pretty much immutable strings with some syntactical sugar. (and potentially namespacing) Still using strings may make sense in some cases, like interfacing to external code. Python meanwhile doesn't even have the concept of a symbol.

Is dynamic typing system the right approach? That's a very complex question... For one thing, remember that typing systems can be a lot more complex than the simple C/C++ model of mostly incompatible types. Look as Haskell's inferred typing for instance.

[BasementMiner!]

Sure there are sites that use Rails such as Twitter.  The primary reason why I say it's less proven is because I don't see very many financial websites using Rails.

Etrade, Ameritrade, Chase, Bank of America, PayPal, etc. don't use Rails.

But, I am willing to give Rails the benefit of the doubt and maybe I am wrong this time.

This most likely because when the back-ends of those companies were built, Rails was not seen as a viable tool at the time. Rails was released in 2004, and most of those companies were founded before this time.

As a side note, I have not witnessed a single incident of Bitcoinica that was caused by a Rails security vulnerability or by the 17 year-old developer's code. The hacks were were related to mail servers, infrastructure break-ins (Linode) and general stupidity (source code containing api key).

A brief skim at the source code however makes me realize the code is using floats (.to_f) rather than Ruby's BigDecimal class. This worries me because floats are prone to rounding errors, BigDecimal is perfect for monetary transactions as it keeps precision.

[davout]

As a side note, I have not witnessed a single incident of Bitcoinica that was caused by a Rails security vulnerability or by the 17 year-old developer's code. The hacks were were related to mail servers, infrastructure break-ins (Linode) and general stupidity (source code containing api key).

This.

Also please let the trolls die. A part of me boils with nerd rage when I see some retarded things written about Ruby or Rails but I refrain from answering because it's a pure loss of time.

A brief skim at the source code however makes me realize the code is using floats (.to_f) rather than Ruby's BigDecimal class. This worries me because floats are prone to rounding errors, BigDecimal is perfect for monetary transactions as it keeps precision.

Yes, this is a concern for me. I don't care if floats are only used for display purposes (and not storage), but it starts bothering me when I see them everywhere as a sign of developer lazyness.