A full shell script implementation of bitcoin ?

[grondilu]

The thread
http://bitcointalk.org/index.php?topic=2459.0

make me feel like suggesting a full shell implementation of bitcoin.  I've actually been thinking about this for some time but I'd like to see if other people would be interested in such a crazy project.

Here is the kind of architecture I'd like to see :

- nodes communicate transactions via IRC, XMPP or similar (a shell script communicating with such a protocol should be easy to implement) ;
- each node publishes its blocks via its own http server (possibly via a TOR hidden service to avoid NAT traversal problems) ;
- each node also publishes the list of nodes it is currently connected to ;
- nodes store their blocks using GnuNet or maybe a NoSQL database such as MongoDB  (easily scripted) ;
- cryptographic functions are performed via the openssl command line ;
- blocks and wallets are stored in an human readable ASCII format ;

[1680221770]

1680221770

[1680221770]

1680221770

[1680221770]

1680221770

[BioMike]

That would be one big dependency hell...

[MoonShadow]

At least it would be trivial to have small CL tools to perform bitcoin operations in this system, such as tools to import and export transactions, blocks and keypairs to/from normal files.  *nix CL shells, such as Bash, are centered around the manipulation of data streams as files, and it's an incredibly powerful model.

[grondilu]

At least it would be trivial to have small CL tools to perform bitcoin operations in this system, such as tools to import and export transactions, blocks and keypairs to/from normal files.  *nix CL shells, such as Bash, are centered around the manipulation of data streams as files, and it's an incredibly powerful model.

Exactly.

I also wonder if all communications could not be done using HTTP.  Blocks would be published via http GET method (giving the hash of the preceding block), and transactions could be sent via POST method.

One advantage of using  http would be that it would be very hard for governments to forbid.

[jgarzik]

Shell script would be fine for a connect-work-disconnect method of working.

But it is grossly ineffective for maintaining a long-running P2P network node, where long-lasting TCP connections are preferred.

[bitcoinex]

also, Kademlia network works fine without kickstarters and other things like this

[jgarzik]

also, Kademlia network works fine without kickstarters and other things like this

Do you mean bootstrapping?  Kademlia network needs bootstrapping, just like other P2P networks.

[MoonShadow]

Shell script would be fine for a connect-work-disconnect method of working.

There is a growing demand for a client that can function disconnected from the Internet.  This might be a step in this direction.

[pj]

Shell script would be fine for a connect-work-disconnect method of working.

But it is grossly ineffective for maintaining a long-running P2P network node, where long-lasting TCP connections are preferred.

One could work around that inefficiency in at least a couple of ways:
 1) Background a task that has a file descriptor open back to the active shell, that can manipulate and communicate with sockets, or
 2) Use a shell that provides this builtin, such as the net/socket module in zsh as described at  http://www.cims.nyu.edu/cgi-systems/info2html?%28zsh%29The%2520zsh%2Fnet%2Fsocket%2520Module

[davout]

Could anyone educate me about the interest of coding a bitcoin client in shell script ? (protip : "just for fun" is a valid answer:) )

[pj]

Could anyone educate me about the interest of coding a bitcoin client in shell script ? (protip : "just for fun" is a valid answer:) )

Someone else asked the question, so I can just guess from what they said.

My guess is that the interest is not so much in coding the guts of the client in shell script (not exactly a high performance engine for computationally intensive tasks), but rather in exposing the data flows and stores to shell script manipulation.

In other words, as I have often done myself in other tasks, write several tools that each have specific capabilities, and which have interfaces that work well when combined in shell scripts with each other and other *nix utilities.  One shot commands are handled in that environment by forking/execing a command that is passed command line arguments, environment variables and files or file descriptors to read or write.  Higher performance for repetitive tasks is obtained by using commands in a series of filters, operating on a stream of line oriented text.  The new, individual tools would each be written in C or Python or some other such language that provided adequate performance and library support for their particular purpose.

[grondilu]

I'm trying to implement the base58 functions.  I'm almost there but I have difficulties with the checksum.

Code:

#!/bin/bash

base58=({1..9} {A..H} {J..N} {P..Z} {a..k} {m..z})

EncodeBase58() {
    # 58 =0x3A
    bc <<<"ibase=16; n=${1^^}; while(n>0) { n%3A ; n/=3A }" |
    tac |
    while read n
    do echo -n ${base58[n]}
    done
}

Hash160ToAddress() {
    ADDRESSVERSION=00
    EncodeBase58 "${ADDRESSVERSION}$1$(Checksum "$1")"
}

Checksum() {
    xxd -p -r <<<"$1" |
    openssl dgst -sha256 -binary |
    openssl dgst -sha256 -hex |
    cut -d\  -f2 |
    sed -r "s/^.*((..){4})/\1/"
}

Hash160() {
    echo -n "$1" |
    openssl dgst -sha256 -binary |
    openssl dgst -rmd160 -hex |
    cut -d\  -f2
}

H=0057b0dc5aac7c215a9a458d6c3c85cd21089af8
Hash160ToAddress $H
# I should get     112p3sLidyEptFEfx3C2RCvFoRPK89HyBT
# I actually get     2p3sLidyEptFEfx3C2RCvFoRPK7JQWdq

[grondilu]

Yes !  It works 

Code:

#!/bin/bash
#
# Requires bc, openssl, xxd
#

base58=({1..9} {A..H} {J..N} {P..Z} {a..k} {m..z})

EncodeBase58() {
    # 58 =0x3A
    bc <<<"ibase=16; n=${1^^}; while(n>0) { n%3A ; n/=3A }" |
    tac |
    while read n
    do echo -n ${base58[n]}
    done
}

Hash160ToAddress() {
    printf %34s $(EncodeBase58 "00$1$(checksum "00$1")")|
    sed "s/ /1/g"
}

checksum() {
    xxd -p -r <<<"$1" |
    openssl dgst -sha256 -binary |
    openssl dgst -sha256 -hex |
    cut -d\  -f2 |
    sed -r "s/^((..){4}).*/\1/"
}

Hash160() {
    openssl dgst -sha256 -binary |
    openssl dgst -rmd160 -hex |
    cut -d\  -f2
}

PubKeyToAdress() { Hash160ToAddress $(tail -c 65 |Hash160) ; }

[davux]

I like the idea: having a series of small tools that do one task, do it well, and can work together through stdin/stdout, the filesystem, sockets, etc.

I'd start off with strict sh compatibility, though, not bash, unless there's really a big advantage of requiring bash over lighter posix-compliant sh implementations (e.g. dash).

[grondilu]

I'd start off with strict sh compatibility, though, not bash, unless there's really a big advantage of requiring bash over lighter posix-compliant sh implementations (e.g. dash).

Writing in POSIX is much less fun.  To me GNU is a better standard de facto.

Also, what's the point of those excellent GNU tools if we can never use them ?

[mikegogulski]

I'd start off with strict sh compatibility, though, not bash, unless there's really a big advantage of requiring bash over lighter posix-compliant sh implementations (e.g. dash).

Not that I've got any skin in this game, but it would be worth considering targeting the more limited toolset offered by Busybox:

http://busybox.net/

The profusion of embedded linux devices these days means Busybox is everywhere.

[grondilu]

I've added two functions : decodeBase68 and checkBitcoinAddress.

I think it might be usefull to check the validity of a bitcoin address in pure bash.

Requires "dc", the unix desk calculator.  Few people use this reverse polish notation calcultor but it's much easier to handle in scripts.

Code:

#!/bin/bash
#
# Requires bc, dc, openssl, xxd
#

base58=({1..9} {A..H} {J..N} {P..Z} {a..k} {m..z})
bitcoinregex="^[$(printf "%s" "${base58[@]}")]{34}$"

decodeBase58() {
    s=$1
    for i in {0..57}
    do s="${s//${base58[i]}/ $i}"
    done
    dc <<< "16o0d${s// /+58*}+f"
}

encodeBase58() {
    # 58 =0x3A
    bc <<<"ibase=16; n=${1^^}; while(n>0) { n%3A ; n/=3A }" |
    tac |
    while read n
    do echo -n ${base58[n]}
    done
}

checksum() {
    xxd -p -r <<<"$1" |
    openssl dgst -sha256 -binary |
    openssl dgst -sha256 -hex |
    sed 's/^.* //' |
    head -c 8
}

checkBitcoinAddress() {
    if [[ "$1" =~ $bitcoinregex ]]
    then
        h=$(decodeBase58 "$1")
        checksum "00${h::${#h}-8}" |
        grep -qi "^${h: -8}$"
    else return 2
    fi
}

hash160() {
    openssl dgst -sha256 -binary |
    openssl dgst -rmd160 -hex |
    sed 's/^.* //'
}

hash160ToAddress() {
    printf %34s "$(encodeBase58 "00$1$(checksum "00$1")")" |
    sed "y/ /1/"
}

publicKeyToAddress() {
    hash160ToAddress $(
    openssl ec -pubin -pubout -outform DER 2>/dev/null |
    tail -c 65 |
    hash160
    )
}

[grondilu]

I have modified dirtyfilthy's program so that it can export the whole wallet in an ascii format.

http://github.com/grondilu/bc_key


Just use "ALL" instead of a particular bitcoin address.

Be careful with this command, for it shows all your private keys in clear !  In particular, you should never redirect the output of this program to a file.  Unless you use an encrypted file system.

Anyway, I'll add a few bash functions that can be used to create a new address, sign a transaction, and so on...

Here is a function to create a new address already :

Code:

#!/bin/bash

. base58.sh

wallet="$HOME/.bitcoin-bash/wallet.dat"

generateNewAddress() {

    privkey="$(openssl ecparam -name secp256k1 -genkey)"

    openssl ec -pubout <<<"$privkey" 2>&- |
    publicKeyToAddress 2>&- |
    if [[ -f "$wallet" ]]
    then
        tee -a "$wallet"  # storing and showing the address
        printf "%s" "$privkey" >> "$wallet"  # storing the private key
    else
        # showing address and private key
        cat
        printf "%s" "$privkey"
    fi

}

[grondilu]

I know a bit about this software and I think it's pretty cool.  I really would like to see an implementation of bitcoin using this as a database handler.


One of the reason why I think it would fit bitcoin nicely is the fact that mongodb uses JSON directly to represent data.   An other reason is that it is very easy to connect to an other mongodb on the net.  Thus, the network handling would be easy.

Also, it's very easy to handle from a bash script.


www.mongodb.org

[Nefario]

What is it with you and bash scripts? 
Can you do nothing else?