CIYAM - Project Plan Outline and Progress Updates

[CIYAM]

Some exciting news is that I have successfully tested an ACCT CLTV redeem using Bitcoin (regest)!!!

The script to do this has been added to the CIYAM project so provided that you install Bitcoin you will be able to run this using the ciyam_client application.

Usage is as follows (make sure you have bitcoind running before doing this):

Code:

./ciyam_client -echo -quiet -no_prompt < test_acct_cltv.cin

The script contents is as follows:

Code:

; Copyright (c) 2016 CIYAM Developers
;
; Distributed under the MIT/X11 software license, please refer to the file license.txt
; in the root project directory or http://www.opensource.org/licenses/mit-license.php.
;
;CIYAM Bitcoin regression test script for ACCT CLTV redemption
;-------------------------------------------------------------
;
;NOTE: In order to utilise the following script you will first
;need to install Bitcoin either into the directory where CIYAM
;has been compiled or if not then set the TBITCOIN environment
;variable accordingly (check the 'tbitcoin' script to work out
;how to do this and don't forget the '-regtest' option). First
;make sure that any existing 'regtest' data has been destroyed
;then start 'bitcoind'.
;
~tbitcoin generate 101
;
; Define an address and matching WIF private key for testing
;
ADDR=mp7zjTrEszZezRBM1cUmTvaK2zf3MuDAHt
WKEY=cPR4DC5NXVkR37wEbCuUVuvuFxSozsMr8SHYteB2iqG8yV3uHdFR
;
; To simplify testing just uses the same address for redemption
;
crypto_addr_hash $ADDR
PKH1=$OUTPUT
PKH2=$OUTPUT
;
; Setup an ACCT P2SH script and then get its address and script hash
;
SCTH=c775e7b757ede630cd0aa1113bd102661ab38829ca52a6422ab782862f268646
~tbitcoin importprivkey $WKEY
TXID=@~tbitcoin sendtoaddress $ADDR 50
BLKN=7f
SCRIPT=76a820%SCTH%87637576a914%PKH1%88ac6701%BLKN%b17576a914%PKH2%88ac68
crypto_p2sh_addr TBTC $SCRIPT
P2SH_ADDR=$OUTPUT
crypto_addr_hash $P2SH_ADDR
P2SH_HASH=$OUTPUT
;
; Send funds to the P2SH address
;
~tbitcoin generate 1
TXID=@~tbitcoin sendtoaddress $P2SH_ADDR 50
OLD_TX=@~tbitcoin getrawtransaction $TXID
~tbitcoin decoderawtransaction $OLD_TX
;
; (check that the VOUT matches or change the next line)
; (also verify the P2SH scriptPubKey's script and hash)
;
VOUT=0
#$P2SH_HASH
~tbitcoin getblockcount
LOCK=127
crypto_p2sh_redeem $TXID $VOUT $SCRIPT $ADDR 5000000000 $WKEY $LOCK
RAW_TX=$OUTPUT
~tbitcoin decoderawtransaction $RAW_TX
~tbitcoin signrawtransaction $RAW_TX
;
; Note this attempt will fail due to being non-final (i.e. locktime)
;
~tbitcoin sendrawtransaction $RAW_TX
;
; Increase the block height and then redeem funds from the P2SH UTXO
;
~tbitcoin generate 25
~tbitcoin getblockcount
TXID=@~tbitcoin sendrawtransaction $RAW_TX
NEW_TX=@~tbitcoin getrawtransaction $TXID
~tbitcoin decoderawtransaction $NEW_TX
~tbitcoin generate 1

In order to use it you will need to delete any existing 'regtest' stuff first.

This is only the CLTV redeem (or "refund redeem") - so the next step will be the "reveal secret redeem".

[vbcs]

Great job Ian! Congrats! I will try to write a howto tutorial about this in the following days. Many folks here are not aware how to run CIYAM.

[CIYAM]

I extended the P2SH redeem command to allow for "extra data" to be pushed onto the scriptSig stack which means the "secret reveal" method can also now be used - modified script is as follows:

Code:

; Copyright (c) 2016 CIYAM Developers
;
; Distributed under the MIT/X11 software license, please refer to the file license.txt
; in the root project directory or http://www.opensource.org/licenses/mit-license.php.
;
; CIYAM Bitcoin test script for ACCT using either a CLTV or secret reveal redemption tx
; -------------------------------------------------------------------------------------
;
; NOTE: In order to utilise the following script you will first need to install Bitcoin
; into the directory where CIYAM has been compiled or else set the TBITCOIN environment
; variable accordingly (check the 'tbitcoin' script to work out how to do this and when
; doing this don't forget the '-regtest' option). Check that there is no 'regtest' data
; before starting 'bitcoind' and then running this script.
;
;SECRET_REDEEM=1
;
@ifndef $SECRET_REDEEM
#Redeeming using CLTV refund...
@else
#Redeeming using secret reveal...
@endif
~tbitcoin generate 101
;
; Define an address and matching WIF private key for testing
;
ADDR=mp7zjTrEszZezRBM1cUmTvaK2zf3MuDAHt
WKEY=cPR4DC5NXVkR37wEbCuUVuvuFxSozsMr8SHYteB2iqG8yV3uHdFR
;
; To simplify testing just uses the same address for redemption
;
crypto_addr_hash $ADDR
PKH1=$OUTPUT
PKH2=$OUTPUT
;
; Setup an ACCT P2SH script and then get its address and script hash
;
SCTH=c775e7b757ede630cd0aa1113bd102661ab38829ca52a6422ab782862f268646
~tbitcoin importprivkey $WKEY
TXID=@~tbitcoin sendtoaddress $ADDR 50
BLKN=7f
SCRIPT=76a820%SCTH%87637576a914%PKH1%88ac6701%BLKN%b17576a914%PKH2%88ac68
crypto_p2sh_addr TBTC $SCRIPT
P2SH_ADDR=$OUTPUT
crypto_addr_hash $P2SH_ADDR
P2SH_HASH=$OUTPUT
;
; Send funds to the P2SH address
;
~tbitcoin generate 3
TXID=@~tbitcoin sendtoaddress $P2SH_ADDR 50
OLD_TX=@~tbitcoin getrawtransaction $TXID
~tbitcoin decoderawtransaction $OLD_TX
;
; (check that the VOUT matches or change the next line)
; (also verify the P2SH scriptPubKey's script and hash)
;
VOUT=0
#$P2SH_HASH
~tbitcoin generate 3
~tbitcoin getblockcount
LOCK=127
@ifndef $SECRET_REDEEM
crypto_p2sh_redeem $TXID $VOUT $SCRIPT $ADDR 5000000000 $WKEY $LOCK
@else
crypto_p2sh_redeem $TXID $VOUT $SCRIPT $ADDR 5000000000 $WKEY -e=31323334353637383930
@endif
RAW_TX=$OUTPUT
~tbitcoin decoderawtransaction $RAW_TX
~tbitcoin signrawtransaction $RAW_TX
@ifndef $SECRET_REDEEM
;
; Note this attempt will fail due to being non-final (i.e. locktime)
;
~tbitcoin sendrawtransaction $RAW_TX
;
; Increase the block height and then redeem funds from the P2SH UTXO
;
~tbitcoin generate 20
@endif
~tbitcoin getblockcount
TXID=@~tbitcoin sendrawtransaction $RAW_TX
NEW_TX=@~tbitcoin getrawtransaction $TXID
~tbitcoin decoderawtransaction $NEW_TX
~tbitcoin generate 1

So the same script can redeem either way (set the SECRET_REDEEM environment variable for the "secret reveal" method).

[CIYAM]

Added support for "unix timestamp" CLTV redemption and other minor changes were made to the script (https://github.com/ciyam/ciyam/blob/master/src/test_acct_cltv.cin).

The following shows the three different ways that you can redeem the CLTV P2SH UTXO:

NOTE: Before running each and any of the following first ensure that bitcoind is not running then delete the the regtest folder (if it exists) and then start bitcoind.

1. The "secret reveal" method (which will redeem immediately).

Code:

USE_SECRET_REVEAL=1
USE_UNIX_TIMESTAMP=
./ciyam_client -echo -quiet -no_prompt < test_acct_cltv.cin

2. The CLTV method using a block number (which will redeem after X blocks).

Code:

USE_SECRET_REVEAL=
USE_UNIX_TIMESTAMP=
./ciyam_client -echo -quiet -no_prompt < test_acct_cltv.cin

3. The CLTV method using a unix timestamp (which will redeem after X seconds).

Code:

USE_SECRET_REVEAL=
USE_UNIX_TIMESTAMP=1
./ciyam_client -echo -quiet -no_prompt < test_acct_cltv.cin

[CIYAM]

Whilst I've been taking a bit of a break from the core parts of the development of the CIYAM project I created a new tiny computer language that I have named FISSILE (https://github.com/ciyam/ciyam/blob/master/src/console_commands.cpp#L188).

Basically it's a Turing complete extension to the CIYAM console handler (which is part of any CIYAM application that has a console) and fits in with the environment variable handling that was already present in the console handler.

For those that have an interest in "pure mathematics" the way that this language understand maths is purely symbolically (i.e. it doesn't do any normal math ops in the code).

Here as a sample of how this works:

Code:

> *#=10

> *#
10

> *#2
1010

> *#8
12

>*#16
a

> *$
..........

and note the following:

Code:

> **

# (raw string) 10#
* (raw string) 10000
10#0 (raw string)
10#1 (raw string) .
10#2 (raw string) ..
10#3 (raw string) ...
10#4 (raw string) ....
10#5 (raw string) .....
10#6 (raw string) ......
10#7 (raw string) .......
10#8 (raw string) ........
10#9 (raw string) .........
16#0 (raw string)
16#1 (raw string) .
16#2 (raw string) ..
16#3 (raw string) ...
16#4 (raw string) ....
16#5 (raw string) .....
16#6 (raw string) ......
16#7 (raw string) .......
16#8 (raw string) ........
16#9 (raw string) .........
16#a (raw string) ..........
16#b (raw string) ...........
16#c (raw string) ............
16#d (raw string) .............
16#e (raw string) ..............
16#f (raw string) ...............
2#0 (raw string)
2#1 (raw string) .
8#0 (raw string)
8#1 (raw string) .
8#2 (raw string) ..
8#3 (raw string) ...
8#4 (raw string) ....
8#5 (raw string) .....
8#6 (raw string) ......
8#7 (raw string) .......

(the latter information is how the former numerical outputs were performed - there is no code that is directly doing any hex, octal or binary conversions with numbers but instead just code that is doing symbolic operations upon strings which works extremely fast)

[CIYAM]

A few improvements were made to the ACCT demo script (https://github.com/ciyam/ciyam/blob/master/src/test_acct_cltv.cin).

Previously the actual "secret in hex bytes" (and its SHA256 hash) were just hard-coded but now after adding some new environment variable transformation functions the secret can be provided like this:

Code:

SECRET=abcdef
USE_SECRET_REVEAL=1
./ciyam_client -echo -quiet -no_prompt < test_acct_cltv.cin

(so it will now automatically work out the hex bytes given an ASCII secret as well as its SHA256 hash)

[CIYAM]

A prototype Trade package (for being able to list, locate and lock into ACCT trades) has been developed.

Some screen snapshots can be found here: https://bitcointalk.org/index.php?topic=1316199.msg13571110#msg13571110

[CIYAM]

Further progress has been made with the new Trade package (it's nearing functional completion) and with the help of four other batch/script files to use Bitcoin with the "-regtest" (with one of four different wallets) the following script https://github.com/ciyam/ciyam/blob/master/src/bct_acct_test.cin simulates an ACCT between two users.

In order to turn this into a usable decentralised exchange platform the Wallet package will need to be enhanced to perform the functionality that is being done manually through the CIYAM script.

[CIYAM]

One of the good things about having a project as large as this one is that when you get bored working on one thing there are many other things you can work on to keep your motivation going.

Today's bit of fun for me was to extend the console user input behaviour to support a single key "choice" from a set of options.

A console user input line begins with an ampersand character (&) with what follows typically being in the following form:

Code:

&VAR=

When executed by a CIYAM console the above would prompt the user with:

Code:

VAR=

and the user would type in the value and press enter (even if standard input was redirected from a file this would be direct console input).

But let's say we wanted the user to decide whether or not to continue something. It would be nice to just be able to type a simple 'y' or 'n' character (with no need to hit an Enter key) and this is what the "choice" addition I made today allows.

Here is an example of what you would put into the console script:

Code:

&Continue? [yes=1!Yes|no=!No] (choose one)

When the console command handler reads this you'll see the following output to the console:

Code:

Continue? [y]es, [n]o (choose one)

Assuming you hit 'y' then the above line would instantly change to this:

Code:

Continue? Yes

And the actual "command" issued would be this:

Code:

Continue=1

which then could be used with say an @ifdef conditional such as the following:

Code:

@ifdef $Continue
#we are continuing...
@else
#we are not continuing...
@endif

Obviously extending the console conditional handling to check values (rather than just whether empty or not) is something that will need to be added in order to make this new feature more usable but that will have to wait for another day when I'm bored.

[CIYAM]

The Trade package is now at the point of final testing and so should be ready any day now.

In the meantime I discovered this: https://github.com/tromp/cuckoo

There is a whitepaper for this algorithm as well and it so far looks very promising to me - it could be the final piece in the CIYAM blockchain jigsaw that I have been looking for.

[CIYAM]

Although Cuckoo is still a likely candidate for now a simpler algorithm has been implemented (the key loop is here: https://github.com/ciyam/ciyam/blob/master/src/crypt_stream.cpp#L440).

Using this algorithm CIYAM blocks now require a "nonce" that is created by trying different values until one that produces an SHA256 hash with a leading zero hex digit is found. As per any other "proof of work" implementation verifying the nonce is much simpler than finding it.

As the main point of the CIYAM blockchain is not "proof of work" this is simply being added for NAS prevention (and the "difficulty" is kept very low).

The same POW will later be used for account creation and verification (a key part of how a CIYAM blockchain is secured) and for that purpose the difficulty will be greatly increased (current testing indicates that two leading hex digits should suffice).

[CIYAM]

Further work on the Wallet package was completed today - now it handles the "spendable" attribute that was added to the "listunspent" output for Bitcoin (for "watch only" addresses).

Currently the Wallet package works with four different kinds of wallets being:

Private (where the keys belong to the application and are generated randomly)
External (where the keys belong to an external application such as bitcoind and are generated by it)
Watch Only (where keys are considered as belonging to "cold storage" and are therefore are never generated)
Deterministic (where keys are generated from a "seed" which is all that is required to back up and later restore such a wallet)

[CIYAM]

A key file that is only generated the first time that the CIYAM application server is run is called ciyam_server.sid and this file is unique to every CIYAM system and is used as a source of entropy for encryption (technically it is what is known a UUID).

Backing up this file is essential if you are going to be able to restore a CIYAM system in case of it being damaged and it is also essential if you were wanting to migrate to new hardware so a commit was pushed today that will display a "fingerprint" of this UUID the first time that you run Meta (which you should record as well as more importantly making sure that you have backed up the ciyam_server.sid file).

The next important focus is going to be on extending the Wallet package to be able to work with Atomic Cross-Chain Transfer transactions (that use CLTV).

[CIYAM]

Initial work on the Wallet and Transaction packages to support ACCT (using CLTV) has been completed.

Currently it requires a fair bit of manual copy and pastes which should be able to be removed by optionally tying the Trade package to the Wallet and Transaction packages (which I think is going to be essential for this to be generally usable).

I expect this will take at least a couple of weeks after which work on being able to turn this into a combined local and blockchain application will commence.

[CIYAM]

Working is still continuing although it was paused for a while due to shifting home and taking a holiday.

Some preparatory commits for tying the Trade package to the Wallet and Transaction packages have already been pushed and work on the Trade package extensions has now begun.

[CIYAM]

Holidays were extended as I had pretty much lost motivation on the project ("burn out").

Recently I have been doing some low-level DB stuff as a tangent but hopefully some further work on the Trade and Wallet packages will be coming soon.

[CIYAM]

The Trade package has been extensively improved to work along with the Wallet and Transaction packages so that nearly all of the ACCT operations (such as creating the required wallets and receive or refund transactions) are now automated.

A demo of a working ACCT dialog will be made available soon (although at this stage only to those involved with the CIYAM project).

Others wanting to get involved with the project further should send me an email address in order to get a Slack invitation.