[Edit 131016 : A nice fix is currently being tested, so it seems like namecoin will be back in business. Cudos to the "snailbrain" and "phelix" for cooking it together and acting fast. So current status is - don't buy a domain from someone, and don't trust any important key-value pair in namecoin before the fix has been rolled out! - Will update once it is there, but could take days to deploy at miners.]

This is the postmortems and obituary over namecoin. In fact it never really existed, but by block: 139872 it became clear. However, if you haven' t noticed yet read on...

Edit 131016 : A nice fix is currently being tested, so it seems like namecoin will be back in business. Cudos to the "snailbrain" and "phelix" for cooking it together and acting fast. So current status is - don't buy a domain from someone, and don't trust any important key-value pair in namecoin  before the fix has been rolled out! - Will update once it is there, but could take days to deploy at miners.

Namecoin has always been my favorite alt-coin - it had a clear purpose, different from Bitcoin, offering a nice way to keep a de-central registry of key-value pairs. About a month ago I had a closer look at namecoin, to integrate it into libcoin on the Kraken exchange. Libcoin is a complete other story, it is a library supporting bitcoin as well as several of the alt coins, enabling easy construction of anything from light weight wallets to full server wallet solutions for exchanges and merchant sites. However, back to namecoin...

I have integrated several alt coins, and I know the machinery pretty well by now. The engine of any bitcoin based crypto currency is the ConnectBlock / ConnectInputs methods in main.cpp. They keep the rules of when to accept a block and when to reject a block, and it is there you make patches to enable anything from alternative hashing algorithms (litecoin) to merged mining (namecoin and others) as well as add new features and rules. Namecoin keep a reasonable separation through the definitions of hooks, implementing the actual rules in a separate file, namecoin.cpp.

So the real interesting stuff in namecoin is happening in namecoin.cpp in the ConnectInputs method. This one is called from ConnectInputs in main.cpp and hence have the ability to change and add rules.

All namecoin rules are kept hidden from the bitcoin script rule engine through OP_DROP opcodes, i.e. some special opcodes and data is entered, followed by a matching chain of OP_DROP commands, so the normal script rule engine will simply ignore anything namecoin'ish. The special op codes of namecoin are:

Code:

OP_NAME_NEW
OP_NAME_FIRSTUPDATE
OP_NAME_UPDATE

The reason for the

Code:

OP_NAME_NEW/OP_NAME_FIRSTUPDATE

setup is to avoid domain opportunists listening for new domain reservations and issuing competing reservations to later sell the domain back. So first you issue a:

Code:

OP_NAME_NEW << hash << OP_2DROP

Where the hash is composed of a random number and the domain, hashed. You are not allowed to issue a first-update, finally registering the domain, before after 12 blocks, ensuring no block reorganizations can enable a domain opportunists to steal your domain. In the name_new/name_firstupdate RPC calls this rule is nicely enforced, however, when you look in the ConnectInputs method you find rules enforcing a fee, rules enforcing the 12 blocks, but NO RULES ENFORCING THE HASH! [namecoin.cpp line 1874-1907] ]. So any name_new can be used as input for ANY name. This means that the domain reservation is not enforced at all leaving namecoin completely open for domain opportunists.

Clearly the patient is bleeding and in urgent need for help, but brace yourselves, this is not affecting already registered domains so it is fixable by a proper patch, and a recommendation to not reserve any new domains before the patch is in effect. Relieved that there was a cure I continued with the standard examination, to check if the rest was ok.

The key lines in namecoin.cpp are probably 1930 to 1949, this is the very core of namecoin. This is the enforcing of a name_update - a name update is the script:

Code:

OP_NAME_UPDATE << vchName << vchValue << OP_2DROP << OP_DROP

So, take an already registered name and update that with a new value. Now you would expect some code enforcing that only an input of that name can be update to another value - but NO! Again there is no enforcing of the core ruleset. So you can in fact update the value of any name in namecoin by any other input name. And after that you own it, or well, as much as you can actually own a name who anyone can update.

The final test was to try it out - (sorry) - I might had overlooked something, so, I changed the name_update algorithm to enable such takeovers, and did a:

Code:

./namecoind name_fakeupdate d/postmortem d/bitcoin "Namecoin died October the 15th 2013, coinslayer"

Try name_history on d/bitcoin and see for yourselves - there is no enforced integrity of the key value pairs in namecoin. So namecoin looses its entire purpose. The problem is that there is no fix to this - it is similar to being able to randomly take ownership of other peoples money, all the value is gone. I tried, initially, a silent fix contacting namecoin developers and key users more than a month ago, but I never got any answers back. Perhaps, the best future for namecoin now is a rebirth with a new genesis, or just a cancel of all the name reservations starting from some future block ?

I should also note that up until block 139872, no one have exploited the bugs. The libcoin code actually enforced the above rules, and I was able to download and verify the entire chain, now I have added a flag, ignore_rules, to get pass block 139872.

Coinslayer aka Michael, Chief Operation Officer, Payward Inc. kraken.com

The amount of "dust" in the block chain is getting large and it is growing all the time. Currently 11% of unspent tx outputs (UTXO) are of 1Satoshi (0.00000001BTC), 32% is less than 0.0001BTC and 60% is less than 0.001BTC. (Thanks to Jan for digging out these numbers!)

This means that a huge part of the block chain is used for essentially nothing - e.g. the sum of the 11% is worth roughly 2 US cents !

The main source for these 1 Satoshi payouts is Sahtoshi Dice. And nothing wrong with that, however, we should work on ensuring that too many too small payments will not kill the size of the blockchain in the end - further, they are essentially too small to be included in other transaction as the added fee will often make it more expensive to remove them. Hence, there is no incentive to get rid of them.

I have an idea for a possible mitigation of this problem - introduction of demurrage - not as in it normal meaning as a percentage over time (see: http://en.wikipedia.org/wiki/Demurrage_(currency) btw, this has also been tried in freicoin), but as a mean to recycle pennies over time. The proposal is simple - UTXOs age out if not re-transacted - the smaller the coin the faster the aging:
1-99 Satoshi: lives for 210 blocks
100-9999 Satoshi: lives for 2100 blocks
10000-999999 Satoshi: lives for 21000 blocks
1000000-99999999 Satoshi: lives for 210000 blocks

Only amounts above 1BTC lives forever - (or we could even impose aging on those too..)

The aged coins are simply included in the block mining reward, creating another incentive for miners. Further, if we include all coins in this recycle scheme coins will never be lost forever.

This scheme will impose some lifetimes also on e.g. colored coins (hence you need to use a certain amount to borrow space on the blockchain for the time needed, or simply transact them).

If you like this I would be happy to write it into a BIP.

Thoughts ?

I just stumbled over this article:

http://www.forbes.com/sites/janetnovack/2012/03/26/24-states-moving-towards-decision-on-taxing-groupon-livingsocial-deals/

The discussion is, as I see it, also relevant for for bitcoins. Digital currencies like bitcoin are, for a very near analogy to a gift card or a group on voucher. You buy your proof of value and then later you use it in a store that accepts it. The decision by New York and other states to charge tax on the value and not the gift card value is actually accepting a gift card as a for of currency or a proof of depth. I expect bitcoins to be treated similarly resulting in nor sales tax on bitcoins.

We have had a similar discussion in the Nordic/EU forum, but I think the article from forbes indicates that it is a relevant discussion for US too.

Cheers,

Michael

Debugging code is both tedious and contagious. You are held in suspense between your curiosity, stubbornness and immense boredom. It is one of these needle in a haystack tasks that can last for days and during the endeavor you feel awfully unproductive. However, there are often some lessons to be learnt, and sharing debugging stories is for code gurus like sharing fishing stories for fishermen.

A famous story is the one from Fortran that caused a satellite to crash (Mariner Venus) due to '-' for '~' error... This story is in C++ and about a missing '&', and like the satellite story it also begins with a crash.

I have had my customized bitcoin server (based on libcoin) running on different mac and windows desktop machines running for weeks with no issues, it survived reorganizations and the recent date based protocol changes (BIP0016/BIP0030) silently, and seemed pretty stable. I hence installed it in its production environment a Ubuntu machine at Linode. It had compiled and functioned several times before on linux, except that sometimes running on small virtualized machines I had seen crashes due to memory issues. Nothing to be really alarmed about, and the instance running on an amazon large instance had never crashed. I hence compiled to code and started it on Linode. Blockchain download... a reorganization and crash! Reason was bad_alloc, so most likely the memory issue, and afterall the Linode instance had only 1Gig of RAM. In my custom server I also keep some rather large data structures in memory, so I disabled these during chain download and tried again. Reorganize, and crash! Again due to bad_alloc.

Annoyingly, following the crashes the databases got corrupted as well. I still believed that the crashes were due to low memory, but I wanted to make sure in the future that I could at least recover gracefully from database corruptions. So I tried to figure out what state the database was left in.

In the reorganize method of the BlockChain class (basically similar to the reorganize method in bitcoin/bitcoin) there are a lot of nice tests to check if things fails - if blocks can't be read from disk or if disconnect fails. If this is the case the series of accumulated database transactions are aborted (TxnAbort) leaving the database in the same state as before, and ensuring that even thought the reorganization failed (and will fail again) the database is left in the same state and the same error will reappear on restart and can hence easily be debugged.
In my case it was a memory exception thrown in the loop where all the txes are pushed on a stack to be resurrected after the chain reorganization. The exception is caught, but it is not caught before in the message processing method (ProcessMessage in bitcoin, handleMessage in libcoin). Here it results in an error message, but the program continues and the database transaction is never aborted (TxnAbort). The failed reorganize will repeat it self and the database transaction log fills up till 500 uncommitted transactions and then the BDB makes the program die. However, at some point in between, at least in my code, the best height was sat in the database and that caused the corruption.

I fixed the issue by inserting a try catch around everything in reorganize up to the TxnAbort ensuring correct behavior even in case of a memory exception.

Instead of rebuilding the database (from scratch download) I tried to rebuild the database: Forcing a reorganize 10 steps back and cleaning the blockchainindex for all non main chain blocks. However, that immediately caused another reorganize crash due to a bad_alloc ! This time the reason was clearly not a filed up memory. I debugged the code and the same loop as before was the culprit:

Code:

    // Queue memory transactions to resurrect
    BOOST_FOREACH(const Transaction& tx, block.getTransactions())
        if (!tx.isCoinBase()) {
            vResurrect.push_back(tx);
        }

The Block::getTransactions() was defined like:

Code:

    const TransactionList getTransactions() const { return _transactions; }

libcoin encapsulates all classes, and hence looping over a (const version) of transactions in a block is done this way. I have other part of the code (in Block) where:

Code:

    BOOST_FOREACH(const Transaction& tx, _transactions)
        // do something with tx

is called multiple times with no problem, yet the code crashed there and it was tx that was corrupted! I changed the loop to a normal for loop using block.getNumTransactions() and block.getTransaction(int) - now the code ran perfectly up to block 171091 - which every bitcoinerd will recognize as the first block of March the 15th ! BIP0030... My BIP0030 code is:

Code:

        if (pindex->nTime > _chain.timeStamp(Chain::BIP0030))
            BOOST_FOREACH(const Transaction& tx, block.getTransactions()) {
                TxIndex txindexOld;
                if (ReadTxIndex(tx.getHash(), txindexOld))
                    BOOST_FOREACH(const DiskTxPos &pos, txindexOld.getSpents())
                    if (pos.isNull())
                        return false;
            }

Hey, again the BOOST_FOREACH loop! That construct was apparently somehow polluted - a google for 'BOOST_FOREACH const reference' gave this like:

http://www.richelbilderbeek.nl/CppBOOST_FOREACHExample1.htm

Note from Richel's page:

Code:

//BAD: Compiles and crashes program
  //std::clog << __func__ << '\n'; BOOST_FOREACH(const std::string& s, f()) { t+=s; }

So you cannot use a function returning a copy of a vector in BOOST_FOREACH, however a const reference to it works, but will spew a compiler warning. Changing Block::getTransactions() to :

Code:

    const TransactionList& getTransactions() const { return _transactions; }
//                       ^ note the single &

made everything work smoothly again. So the culprit was a missing '&' or more correctly that BOOST_FOREACH that didn't behave correctly. However, it does behave correctly when compiled on visual studio and Xcode. Hence I never saw that error before...

What is the lesson learnt then ? I think there are two:
* Use either exceptions OR false/true functions in a program not both! Somethimes it is better to crash than to keep trying.
* Use macros (BOOST_FOREACH) with great care, the behavior is often hard to predict and also hard to debug. I will over time change all BOOST_FOREACH in libcoin to normal for / while loops.

Thanks for reading this - I hope you found it useful!

Cheers,

Michael