Gocoin - totally different bitcoin client with deterministic cold wallet

[piotr_n]

Hi Piotr

Thank you for your work on this. I am trying it out, but getting an error on block 00000000000000560719da7088be587ea9c3018cfbbfae2dd2540682185c034b (#233046). It says:

.....
7474.8MB of data processed. We are at height 234765. Processing speed 10.507MB/sec, recent: 12853.3KB/s
7582.1MB of data processed. We are at height 235421. Processing speed 10.510MB/sec, recent: 10967.8KB/s
7710.2MB of data processed. We are at height 236155. Processing speed 10.541MB/sec, recent: 13109.1KB/s
7850.1MB of data processed. We are at height 236949. Processing speed 10.588MB/sec, recent: 14325.8KB/s
7958.2MB of data processed. We are at height 237625. Processing speed 10.586MB/sec, recent: 10700.6KB/s
8087.1MB of data processed. We are at height 238516. Processing speed 10.616MB/sec, recent: 13185.7KB/s
CheckBlock failed: CheckBlock: Block 00000000000000560719da7088be587ea9c3018cfbbfae2dd2540682185c034b hooks too deep into the chain: 233046/238760 000
00000000000e76cddac91bfdbf51a73f31ec740e07d651fba24f381289965

Do you maybe know what could be wrong. Im on windows 8, 64bit golang. I chose not to verify scripts.

Thanks ! : )

Thanks for reporting.

It looks like it aborted importing blocks from the satoshi's db, because there was some very orphaned block inside.
But if you re-launch the client after this - it should download all the further blocks from the internet, haven't it?

You can also take a new client/init.go and see how it goes then - in the updated version it prints the error but continues with converting the db.

[orz.dk]

Hi Piotr

Thanks for your help. If i restart it goes:

Using NewEC wrapper for EC_Verify
Gocoin client version 0.8.1
Opening blockchain... (Ctrl-C to interrupt)
Blockchain open in 35.300 seconds
open d:\data\gocoin\btcnet\wallet\DEFAULT: The system cannot find the file specified.
open d:\data\gocoin\btcnet\wallet.txt: The system cannot find the file specified.
Starting WebUI at 127.0.0.1:8833 ...
> ListenTCP listen tcp4 0.0.0.0:8333: Only one usage of each socket address (protocol/network address/port) is normally permitted.

When i go to the webui the last blockheight is 238773, so at first glance it does not seem like it's downloading anything beyond the orphaned block where it crashed.

Can you help me on the mechanisms behind one block being more orphaned than another? , just for my own understanding, thx. For example height 226794 also splits, but it is "less orphaned" than 233046. Is it because there is too many blocks between the output being created and then later referenced in a spend? If yes, why does it occur on a orphan, could it also have occured on a main chain block?

I will try commenting out the errors as in the new submit. Would everything still be in sync, theres some comment about unwind buffer not being able to handle...?

Thanks : )

 

[piotr_n]

open d:\data\gocoin\btcnet\wallet\DEFAULT: The system cannot find the file specified.
open d:\data\gocoin\btcnet\wallet.txt: The system cannot find the file specified.

This just tells you where to put a wallet - a text file with your public addresses (each address in new line).

Starting WebUI at 127.0.0.1:8833 ...
> ListenTCP listen tcp4 0.0.0.0:8333: Only one usage of each socket address (protocol/network address/port) is normally permitted.

This is unusual. Do you have any other app bound to TCP port 8833?
Aren't you by a chance running two instances of the client, at the same time?

When i go to the webui the last blockheight is 238773, so at first glance it does not seem like it's downloading anything beyond the orphaned block where it crashed.

When you go to the Network tab - are there any net connections established?

Can you help me on the mechanisms behind one block being more orphaned than another? , just for my own understanding, thx. For example height 226794 also splits, but it is "less orphaned" than 233046. Is it because there is too many blocks between the output being created and then later referenced in a spend? If yes, why does it occur on a orphan, could it also have occured on a main chain block?

I will try commenting out the errors as in the new submit. Would everything still be in sync, theres some comment about unwind buffer not being able to handle...?

Yes - it keeps a history of which unspent outputs were added/removed with each block, in case if there was a fork and it had to unwind the changes while switching to a different  branch.
But it is now set to 5000 blocks - I'm not aware of such a deep forks ever happening in the past.

Anyway, if you want to restart importing the DB, fetch the new init.cpp (this one does not abort on error) and delete the entire folder d:\data\gocoin\btcnet
See how it goes then.
And make sure that you are not running two instances of the client.

[orz.dk]

Ok, now it finished.

- ok with wallet error, not using wallet yet
- network socket error message is gone now, check
- network tab shows many connections, check

And it has all the blocks now. Great !  It also has 6GB of my RAM and 30% CPU, so i might have to move it to a server : )

I plan to maybe use it for my address monitor (my hobby version of triplespeeder's bicoinmonitor.net) - he is using a bitcoind fork that hits a url on new transactions, can your tool do something similar? Also how do you recommend querying gocoins version of the utx base for address balances? (the tools/fetchbalance.go uses blockchain.info api, would like to get it all locally)

Thanks again !

[piotr_n]

And it has all the blocks now. Great !  It also has 6GB of my RAM and 30% CPU, so i might have to move it to a server : )

Try to exit it and restart. The memory consumption should go lower. Normally it does not go above 3GB at my PC.
As for the 30% CPU - it should not happen in between new blocks. Unless you are receiving lots of txs and the node is busy verifying them.
You can disable relying transactions by editing the config file - then the CPU woudl only be busy while verifying new blocks (every 10 minutes on average).

I plan to maybe use it for my address monitor (my hobby version of triplespeeder's bicoinmonitor.net) - he is using a bitcoind fork that hits a url on new transactions, can your tool do something similar? Also how do you recommend querying gocoins version of the utx base for address balances? (the tools/fetchbalance.go uses blockchain.info api, would like to get it all locally)

If you play with the source code, that should be easy to add.
Just put the addresses that you want to monitor inside the wallet and look into client/main.go, where it does:

Code:

	if wallet.BalanceChanged {
		wallet.BalanceChanged = false
		fmt.Println("Your balance has just changed")
		fmt.Print(wallet.DumpBalance(nil, false))
		textui.ShowPrompt()
	}

This is printing some message each time a balance on any of the wallet's addresses changes.
Add there whatever you need  e.g. a call to http.Get(...) and you should have the functionality you seek.

Also remember that the wallet.BalanceChanged is set only when a new relevant txs has been mined into a block (so not for yet-unconfirmed transactions).

[orz.dk]

Super, it's down to ~2.2GB now and ~1% CPU after restart. I will try to play around with the wallet and/or code.

[piotr_n]

have fun

[piotr_n]

Update on the recent development.

The last one (from October) was on version 0.8.0 - now we are on 0.8.5.

Wallet
* Can now decode and sign files containing raw transaction data (either hex-dump or a raw binary format).
* Is able to sign a transaction partially - only signing the inputs that it "owns" (knows a private key for).
* Checks for GOCOIN_WALLET_CONFIG environment variable (useful for a global config file).

Client
* You can include a content of another (watch) wallet inside a wallet file (use "@filename"), so you can have like a sub-wallets and a master wallet that includes them.
* Starting from 0.8.5 switching between wallets should be much quicker, since there is a monitoring (cache) of all the addresses that are not in the currently selected wallet but have been checked for a balance ever since the node was started.
* Inside the payment.zip (the one you download from MakeTx tab) there is additional file tx2sign.txt, which caries the unsigned transaction data - so you can e.g. sign it using a different wallet or give it to someone else for his part of a so-called CoinJoin signing.
* P2SH addresses should now be properly decoded (but that only matters when you view a transaction details).
* Some changes in WebUI's look & feel.

Privacy
* Now you can tell WebUI to broadcast own (locally loaded) tx only to a single random peer.
* Never broadcast own tx to peers that have not sent an inv to us.
* Disconnect peers that are not sending any invs (they are useless for us and might just be there to spy).
* The entire 129.132.230.0/24 network is banned, since I don't know what these guys are up to (sorry).
* fetchbalance (renamed to fetchbal) now supports getting the blanace via tor's socks5 proxy
* A user can setup his node to report any User-Agent he wants (the string that goes out in the "version" message).

[adamas]

I'm looking for a safe way to create offline LTC transactions. Could your client be ported to Litecoin to serve this purpose?
If not, do you know a way how to manage this?

[piotr_n]

Hi. I'm not a programmer, but I'm looking for a safe way to create offline LTC transactions, and broadcast them. Could your client be ported to Litecoin to serve this purpose?
If not, do you know a way how to manage this?

I'm not sure about the specific differences between bitcoin and litecoin protocol, but I don't see why it could not be ported.

I believe you can use the default litecoin-qt client and its raw transaction API to sing transactions offline (by another litecoin-qt).
https://en.bitcoin.it/wiki/Raw_Transactions

Unfortunately this probably still requires moving the entire blockchain from the online to the offline PC, since I don't believe "signrawtransaction" will work otherwise.
So (after moving the blockachain), you might just as well make the offline payment using the GIU and then "getrawtransaction" to get the hexdump followed by "sendrawtransaction" in the online to broadcast it.

Remember that your actual wallet is on the offline PC - whatever wallet you have in your online PC should not matter.
Just make sure to not overwrite the wallet.dat when moving the blockchain data.

[adamas]

Thanks for this. 

[gorgorom]

Hey guys, that's just not fair that today many bitcoin-focused media are screaming about a "bitcoin alternative made in Go", that is planed to be released "soon", by some Conformal (whoever they are), while my yesterday's announcement has been completely ignored.

What is wrong with you? 
You don't believe me, that my software actually works? It really does - feel free to see for yourself.

Of course my implementation is not a "bitcoind alternative", but only because I am not a kind of person that would be spending his time trying to re-invent a wheel.
I mean: why would I need a bitcoind alternative, if I already have bitcoind?

So my implementation has not much to do with bitcoind, but instead it is focused on fulfilling my own personal needs, which might (by a chance) be similar with your needs - think about it.

And that is, for instance, why my node does not relay transactions.
I know that it is bad for the network, but I am very sorry for having only 1000kbps of upstream at my DSL, and so I am not going to waste 50% of it, plus lots of my CPU, just to relay your satoshidice bets. So I don't - and what can you do about it?

Another feature that I needed, and implemented it into Gocoin, was a cold deterministic wallet, that is based on a password-seed and thus does not require any backups.
My implementation is designed for such a wallet(s), which is of course an opposite of how it is made in bitcoind.

Moreover, Gocoin has an option to limit the bandwidth usage, which was also very important for me.
I do not know if you were checking the actual bandwidth that your bitcoin client takes, but all I can tell you is that my browser stops loading pages, when I have it running and you guys have just mined a new block. So for me a bandwidth limit is not a minor feature.

To wrap it up. I really don't understand why a news about an implementation of bitcoind in Go, that is supposed to be released in some undefined feature, happens to be so much more of a headline than a news about a Go implementation of a bitcoin node with some unique features, that can already be tried. I guess it's because I just don't know the right people in the theater...

In my humble opinion, you're a genius.

[piotr_n]

nee. sathoshi was a genius - I'm just writing code.
though, I see where you're coming from; the reference industrial benchmark for a development of a bitcoin software is so weak these days that my hobby looks like a master performance next to it
but the truth is that there are thousands of kids out there who can make code even quicker - most of them are like half my age and I would not even dare to compete with them, unless on 'who has a bigger mouth'..

[KriszDev]

Can i use this for any altcoin? If yes what i need to modificate?

[piotr_n]

Can i use this for any altcoin? If yes what i need to modificate?

The parts that need to be different for the altcoin of your choice.

If you want a specific answer I'm gonna need a specific question.

[KriszDev]

Can you make it to a multiwallet (you can store different coins not just bitcoin)?

[piotr_n]

Can you make it to a multiwallet (you can store different coins not just bitcoin)?

I don't make things that I don't use, so I will make it if I only need it one day.

[piotr_n]

0.9.0

The version 0.9.0 (after 0.8.5) brings performance improvements to the block chain parsing.
If you do "client -r", forcing it to re-index UTXO database, it should go much faster now (like below one hour to re-build the entire UTXO DB)


Block-chain downloader

Additionally I created a new tool called downloader.
It speeds up the initial block-chain download, even to one hour or less, depending on your internet speed and performance of your PC.
The main trick is to skip verification of scripts, up to the "trusted" block. Normally the trusted block gets discovered automatically, but to improve security you can use a command line switch, like e.g.:

Code:

./downloader -t 000000000000000144785afb658fbe3ad2ce9ec2c5827886934c9a678d6942fd

This would assume all the block up to #282757 as trusted and only all the scripts from further blocks will be verified.

You can also download the blocks without even applying their content to the UTXO database. In such case just use "-b" switch. Then you are basically limited only by the speed of your internet connection. If you do it this way, the UTXO database will be automatically rebuilt after you start the "client" with the new content of the blocks database. As I had said, the full re-indexing usually takes below one hour now, though it does need some system memory, so I recommend at least 8GB of RAM these days.


Other changes

As always, you can see more detailed list of changes/fixes in the README.md


Open issues

Nobody reports any issues, so either nobody even uses this s/w (weird since the github repo already has 68 stars) or you find no issues.
And if you find no issues than you either suck in testing, or this is a really good piece of software
But I also find no issues (except of the ones I fix on daily basis), so who am I to blame you anyway..


Road-map

As for the future of this project.
I have been thinking hard about the part that would need an improvement the most and IMO that would be the UTXO database engine.
The one that I use now, I made myself, but it takes a hell lot of memory and the occupation of the disk is also far from being optimal.
But currently I do not really have any breakthrough idea in how to move it on, maybe except re-writing it from Go to C, which should at least let me to save system memory, though without improving the disk usage.

So if anyone had an idea on improving the UTXO DB engine, please share it - I will surely take it seriously and you can keep the credit for inventing it.
LevelDB is out of the question - I need to be able to browse through all the UTXO records as quickly as possible, so the entire DB index needs to always be in the memory.
Otherwise (if I loose performance just to save memory), it would not have been an improvement to me.

[piotr_n]

Getting advantage of the recent malled transactions hysteria (not that I'm behind it), I'd like to use the opportunity to advertise my Gocoin client as a wallet that is 100% resistant for this kind of "attacks". Gocoin really doesn't give a shit of whether anyone wants to alter its transaction before it gets confirmed or not - feel free to alter them all you want!

So, since apparently there are some ongoing disturbances in the network which affect the official wallet, if you don't want to be exposed to them, just know that there are alternative wallets, for which thins kind of problems don't even exist.

To be honest, myself I don't even see it as a problem - and MtGox should be really ashamed that they caused so much panic by literally claiming that there is a problem with a bitcoin protocol. The only problem there is, is inside a lame implementation of the MtGox software. Shame on them, BTW.

[Tovadnok]

Hi, when I run

> ./wallet -l -v

and enter 1234 as the passphrase, I get the following addresses:

mtMzKRALsjU3QUW9f7mrNdFkKuR32rRVJk TypC 1
ms8dDiiQh92J13Sb5FGxTcLbxfeyKTXAkT TypC 2
mjUi7buZNg93whuD1qeohVm7TKeBgs1G8E TypC 3
mhYcRqxfUJkEMCNQXPZueCJpnWxuGg3m7k TypC 4
mgrkKPPFUppG9WNySxiYtPA77yPVQk4x9j TypC 5
muTGvkfyRDQ1n5otCqukPK5RX1Spnyf5qG TypC 6
n3AgSXhgVt4BuunbzLhjc2bKbdxHmUkGET TypC 7
mzvpZH8jhgYs5JGHSF7HZBnvYudP8xQwWa TypC 8
mofUnUzEJDFNHfhbY51sqRqzKARWbwuEhr TypC 9
moJ4HS4NYtDRd9KqeG282qferp4kXHbbDH TypC 10
mkTDZxJTM6oR88zTtV1gWE6qNnEQvxT5wD TypC 11
mzKNtwNaeScnY71zFqghA69iTTgZX5gh6b TypC 12
mw1JJpDtTLRdkC7akZoYt15wsSifGP89vf TypC 13
moUuYTsFA69LAuwyVSNKJiHrTkGp5XPmPm TypC 14
mpA8fNq4Qt6gd3sPNvoFne85wz7KHHsUb7 TypC 15
mxF4F2V1ySQMrR1tdsA4u5fQdmDdaVgiW5 TypC 16
mgstWQteoRTQnkAMXU2viRmHu1AYXHbBx8 TypC 17
mzkhGFopVmfeLAw84YcVx3uZgfbo3XYmGR TypC 18
mjivnvukgajoHCmJ55dXh1QyN84WhVEVD1 TypC 19
muGkLzrQ4CrBo91ecMdkcjbET4LmB6aYPw TypC 20
mnjxHuncYvNNzr71T1gTKqUE17jHMjUHqz TypC 21
n48TXP4PiRy9wDcZJFKFNNYnPgkss2WAb1 TypC 22
mjX8qm14PQntBqwa2AazzA58hqccpzmHit TypC 23
mgJJQ5QPWYqB1p1KauNtEZUawqBL7JURU5 TypC 24
n26Ncr3fKHNTsANMi1cdDfB4uNyxvXKs7A TypC 25

These are not valid bitcoin addresses. What am I doing wrong?