How to do document timestamping with the block chain?

[cjp]

I want to create something like the "Bitcoin message service", but I think I need non-standard transactions for that. I know some of you would disapprove, but what would be the easiest way to do this?

And do I need to forward these transactions to certain specific pool operators, who are known to accept non-standard transactions? Who would that be, and how would I forward my transactions to them?

My own version of a "messaging service" would put a very low load on Bitcoin: maybe only a single, small transaction once every 24 hours. To prove that load won't be too high, I am willing to use unusually high transaction fees. Think of something like a 0.1 .. 1 BTC transaction fee.

(Edit: changed subject)

[cjp]

Sorry for replying to myself, but I have some more info:

I've found the protocol that is used by that "Bitcoin message service":
http://btcmsg.staticloud.com/btcmsg_protocol_v1.txt

BTCmsg Protocol v1 (2011-09-18)
===============================

Each message is represented by multiple payment which is calculated by
the following algorithm:
1. Two first chars for message type ('01' for md5, '02' for ascii).
2. Then the message in hex (python binascii.hexlify).
3. Split the long string to groups of 4 hex digits.
4. Each group of 4 hex (e.g. 2 ascii letters from the message) is
   represented by a payment in satoshi (maximum 0xffff=65535).
5. The service fee (composed of a fixed part and a variable part times
   the amount of small payments required, minus the transaction fee),
   can be seen within the first transaction.

Enjoy!

Actually that's pretty smart, and I feel a bit stupid for not having thought of that!

I can do the same of course, and nobody will be able to stop me from doing it (except by convincing me it's a bad idea of course), since this scheme only uses "standard transactions". This does indeed look like the easiest way to achieve my goal, although I need to put a sha256sum into the transaction, so I'll use a new message type ID.

As I said before, I'll use a very low number of transactions (max once every 12 hours in the current design), so my load on the block chain shouldn't be too big. Still, the scheme as used by the "Bitcoin message service" is far from optimal. I think the transaction containing the message could be much smaller if a special transaction type was used, that just pushes some unused data on the stack.

The policy of only accepting "standard transaction" and not "messages" really works counterproductive here. Since nobody can stop people from using the block chain for messaging anyway, wouldn't it be better to have a standard transaction that allows for efficient insertion of small messages? A sha256sum would be the minimum that should be supported IMO. That would allow people like me to insert secure hashes of documents of arbitrary size.

BTW, this reminds me of some people I used to know who played a game of chess by making small bank transfers to each other and putting the chess moves in the transaction comments  .

[Sukrim]

Well, noone can stop you from doing this, however in the end, all you're doing is burning bitcoins (and giving a few to miners) for the "benefit" of having something (most likely) unprunable embedded on every machine that will process bitcoins in the future.

Maybe you can think of something else how to reach your goal(s) - whatever they are - in a less obtrusive way? Something like sending specific amounts of coins to a special address + giving proof that oyu own this address by signing a message with it's key?

[grue]

or use OP_DROP

[cjp]

Well, noone can stop you from doing this, however in the end, all you're doing is burning bitcoins (and giving a few to miners) for the "benefit" of having something (most likely) unprunable embedded on every machine that will process bitcoins in the future.

I'm not burning bitcoins: I'm just sending them to myself, in very specific amounts. Yes, I will be giving a few to miners, but I think it will be worth it.

Maybe you can think of something else how to reach your goal(s) - whatever they are - in a less obtrusive way? Something like sending specific amounts of coins to a special address + giving proof that oyu own this address by signing a message with it's key?

What I want to do is a little bit different from what the "Bitcoin message service" does. My goal is to make is a trust-free timestamp service. I want to create evidence that a certain document (or a document with attached name, signature etc.) existed at a certain time. Just "signing with my key" is not sufficient, since it can still be done at a later moment.

The block chain acts as a "timestamp server", and I want to use it as such. I don't believe in creating alternative block chains for alternative purposes, because I believe a trust-free, Bitcoin-style timestamp server needs the monetary incentive to function. What I want to do instead, to reduce load on the block chain to an absolute minimum, is to create my own Merkle trees, which have their root in a Bitcoin transaction.

or use OP_DROP

Please tell me more about this. I already know a couple of things about Bitcoin scripting, but is there a standard transaction type for this? Otherwise, it brings me back to my original question.

[CIYAM]

Another possible approach that had been mentioned before is to make a "fake" bitcoin address that is appears to be valid but instead of being a real address it is actually a user provided hash (after the 1) and then a suffix (to ensure it is parsed as being valid).

You then burn a small amount of BTC by sending to this address (this amount will never be able to be claimed and so cannot be pruned) and then you would have for example the hash of a document permanently recorded in the blockchain.

I guess perhaps even better would be to include a URL in the tx (as a message) so that the tx indicates the original document location (or perhaps just its name or something else).

[Sukrim]

Maybe use the hash of the document/message/whatever as private key and make a transaction to and from the resulting address? This would even have the benefit that the hash doesn't get public but it's trivial to check if the resulting address was used at a specific time. Also this might be pruneable (as you leave an empty address) and you can do as many transactions as you like as soon as you want. As long as you pay transfer fees I guess that could be considered part of normal bitcoin operations.

Should someone want to know if the document with the hash "abc123" existed prior to today, he would simply use "abc123" as private key and check if the corresponding address received any money already (maybe even send the filesize in Satoshis to be extra sure?).

[bitlotto]

Code:

#!/bin/bash
# 
# Timestamping program
# Returns a valid bitcoin address made from stdin
# To timestamp, just send 0.01 BTC to this address
#
# Requires bc, openssl, xxd
#

base58=({1..9} {A..H} {J..N} {P..Z} {a..k} {m..z})

EncodeBase58() {
    # 58 =0x3A
    bc <<<"ibase=16; n=${1^^}; while(n>0) { n%3A ; n/=3A }" |
    tac |
    while read n
    do echo -n ${base58[n]}
    done
}

checksum() {
    xxd -p -r <<<"$1" |
    openssl dgst -sha256 -binary |
    openssl dgst -sha256 -hex |
    cut -d\  -f2 |
    sed -r "s/^((..){4}).*/\1/"
}

Hash160() {
    openssl dgst -sha256 -binary |
    openssl dgst -rmd160 -hex |
    cut -d\  -f2
}

Hash160ToAddress() {
    printf %34s "$(EncodeBase58 "00$1$(checksum "00$1")")" |
    sed "s/ /1/g"
}

Hash160ToAddress $(Hash160)

From https://bitcointalk.org/index.php?topic=2358.0 made by grondilu

Just creates a hash in Bitcoin address format. Sent a small amount to this address. The likelihood of an address existing that matches the hash of your file are pretty darn small!!! Great to prove that something has existed since at least the time of the tx.

[cjp]

Thanks. That script looks promising.

Is there a way to reclaim the spent bitcoins? If I understand it correctly, the hash is actually used as private key, so it shouldn't be too difficult to format the hash in such a way that I can import it into a Bitcoin wallet, right? I already have pyWallet, and I believe it allows me to import private keys...

[bitlotto]

Thanks. That script looks promising.

Is there a way to reclaim the spent bitcoins? If I understand it correctly, the hash is actually used as private key, so it shouldn't be too difficult to format the hash in such a way that I can import it into a Bitcoin wallet, right? I already have pyWallet, and I believe it allows me to import private keys...

The script creates a Bitcoin address only and that address is the hash of the file. There is no private key go along with it. So any BTC you send to that address is gone. It would be pretty much impossible to create a private key to go along with it. BUT, it does create a visible hash that's permanently set in the block chain.

[CIYAM]

Is there a way to reclaim the spent bitcoins? If I understand it correctly, the hash is actually used as private key, so it shouldn't be too difficult to format the hash in such a way that I can import it into a Bitcoin wallet, right? I already have pyWallet, and I believe it allows me to import private keys...

I think if you want to be sure that the tx is permanently stored then the funds cannot be reclaimed (otherwise it could be pruned in the future), however, another approach you could perhaps consider is the following:

1) Use the file hash as a private key then generate a pubic key and send BTC to an empty wallet (that has had this address imported into it).

2) After getting the BTC immediately forward it to another address in the same wallet (if a tx fee is required you will have to lose this amount).

3) Publish the private key of the original tx (after enough confirmations have passed with the last step).

Assuming you don't spend the money you forwarded to yourself then I think that would keep a permanent record with the money eventually being able to be reclaimed (perhaps after a time when you no longer care about the proof).

[cjp]

I think if you want to be sure that the tx is permanently stored then the funds cannot be reclaimed (otherwise it could be pruned in the future), however, another approach you could perhaps consider is the following:

1) Use the file hash as a private key then generate a pubic key and send BTC to an empty wallet (that has had this address imported into it).

2) After getting the BTC immediately forward it to another address in the same wallet (if a tx fee is required you will have to lose this amount).

3) Publish the private key of the original tx (after enough confirmations have passed with the last step).

Assuming you don't spend the money you forwarded to yourself then I think that would keep a permanent record with the money eventually being able to be reclaimed (perhaps after a time when you no longer care about the proof).

My intended way of dealing with pruning of spent transactions is simple: keep a local copy of the relevant part of the Merkle tree of the block. If I am correct, the root of the Merkle tree is part of the block header, which will be kept forever. And actually spending the coins is not just good for me: it also allows for the pruning to take place, which will help save disk space in the future. I already feel a bit like a socially responsible blockchain-environmentalist  .

If it is possible to use a SHA256 hash as private key and generate a public key and Bitcoin address from that, it seems like an efficient method (less block chain pollution  than the "Bitcoin message service" approach). Of course I'll need to transfer the coins to another address with a second transaction, before publishing the SHA256 hash of the document, or otherwise the coins are free to be grabbed by anybody who knows my method and the published hash.

I think I can make an implementation of this, but I'd appreciate some help on the following:

• How can I generate a private and public key from a SHA256 hash, e.g. using bash and openssl?
• Is it possible to import such a key pair into Bitcoin while bitcoind is running? I have no experience with the JSON API, but from what I read, it doesn't seem to support this.

[Pieter Wuille]

• Is it possible to import such a key pair into Bitcoin while bitcoind is running? I have no experience with the JSON API, but from what I read, it doesn't seem to support this.

In 0.6.0 this is possible via the "importprivkey" RPC call.

[FreeMoney]

Someone correct me if I'm wrong, but pruning is just the act of a miner forgetting something they don't need to know anymore. The tx will still have happened and still be in the chain and any service doing lookups of hashes of documents for the purpose of time stamping will be smart enough to not prune. Or maybe you can mark it in some way that lets timestamp checking services forget everything else and remember only the non-financial tx.

If people are worried about the size of the chain, maybe a way to do it is for a company to accumulate documents to be stamped, hash them all together and just include 1 hash per block/hour/day.

[cjp]

Someone correct me if I'm wrong, but pruning is just the act of a miner forgetting something they don't need to know anymore. The tx will still have happened and still be in the chain and any service doing lookups of hashes of documents for the purpose of time stamping will be smart enough to not prune. Or maybe you can mark it in some way that lets timestamp checking services forget everything else and remember only the non-financial tx.

If people are worried about the size of the chain, maybe a way to do it is for a company to accumulate documents to be stamped, hash them all together and just include 1 hash per block/hour/day.

I think you are correct, and that is exactly what I am planning to do:
I will create "timestamp certificates" that contain:

• Some explanation of the methods used (only part that is not machine-readable)
• Version number of the timestamping method
• The Bitcoin block index
• Maybe the entire block header, if it's not too large
• The block timestamp
• The block Merkle tree root
• The part of the block Merkle tree that is needed for the timestamp transaction
• The timestamp transaction
• The timestamp Merkle tree root, as extracted from the transaction
• The part of the timestamp Merkle tree that is needed for this particular certificate
• A short message, together with the secure hash of a document

I am planning to create max. 2 timestamp transactions per 24 hour, so that every document will be timestamped within 24 hours. And pruning is not a problem, since the certificate contains a local copy of the relevant parts of the block.

On the methods of encoding a hash value into a transaction: for simplicity, I'll first use the "Bitcoin messaging service" method, and later, as soon as I switch to Bitcoin 0.6, I'll use the more efficient method of using the hash as a private key.

[Red Emerald]

In order for this to work, the timestamp has to be easy to verify by a third party.  Keep that in mind when designing your system.  It can't be 100% trusted if it relies on any more than the document and the blockchain IMO.  If you hash a whole bunch of documents together, than all of those documents will need to be available to the third party.  This seems like it would be a problem.

[FreeMoney]

In order for this to work, the timestamp has to be easy to verify by a third party.  Keep that in mind when designing your system.  It can't be 100% trusted if it relies on any more than the document and the blockchain IMO.  If you hash a whole bunch of documents together, than all of those documents will need to be available to the third party.  This seems like it would be a problem.

Right, I was thinking it would be fine because he'd have a 'master document' with the days stamped docs, but people won't want to pass around the master so they'll go to his site I suppose, but that sucks because now you need him. You don't have to trust him, but you need him, or at least that doc. Still it could be useful, you could push torrents up every day with the day's stamped docs. Doesn't seem like it would scale well at all though.

[fergalish]

In order for this to work, ... If you hash a whole bunch of documents together, than all of those documents will need to be available to the third party.  This seems like it would be a problem.

The "master document" need not actually be all the day's collected documents - just a list of their hashes.  i.e. on day X at time Y in block B, a transaction occurred to a private key equal to the hash of master document Z, which therefore existed at that time.  In document Z, presented here Your Honours, we can see that there is the hash of "bob's_contract_of_employment.doc" exactly as shown here.

[Red Emerald]

In order for this to work, ... If you hash a whole bunch of documents together, than all of those documents will need to be available to the third party.  This seems like it would be a problem.

The "master document" need not actually be all the day's collected documents - just a list of their hashes.  i.e. on day X at time Y in block B, a transaction occurred to a private key equal to the hash of master document Z, which therefore existed at that time.  In document Z, presented here Your Honours, we can see that there is the hash of "bob's_contract_of_employment.doc" exactly as shown here.

But how do you have a master document without having to trust anyone? What happens if the person with the master document disappears? IMO, one great part of bitcoin is the complete lack of need to trust a centralized party, but with a system with a "master document" you have created a centralized party.

[mila]

In order for this to work, ... If you hash a whole bunch of documents together, than all of those documents will need to be available to the third party.  This seems like it would be a problem.

The "master document" need not actually be all the day's collected documents - just a list of their hashes.  i.e. on day X at time Y in block B, a transaction occurred to a private key equal to the hash of master document Z, which therefore existed at that time.  In document Z, presented here Your Honours, we can see that there is the hash of "bob's_contract_of_employment.doc" exactly as shown here.

But how do you have a master document without having to trust anyone? What happens if the person with the master document disappears? IMO, one great part of bitcoin is the complete lack of need to trust a centralized party, but with a system with a "master document" you have created a centralized party.

the master document will have to be published. period

everybody will be able to see the content of it (should be anyway only the list of document IDs and their hashes) and the hash of the document itself would be in the blockchain.
this way anyone could verify if the document of his interest is in any of the master documents, if yes, which one and then test/compare the hashes and verify in blockchain.

edit: this approach will be used for official eu publications. they will sign 1 document containing list of hashes of all files published that day and anybody can verify that a pdf comes from the publication office by checking the hash of pdf, checking if the hash matches the info in the list which was signed and then verify signature (certificate used, if it was not revoked, etc).
the bitcoin/blockchain timestamping is kind of different, has no certicate issuer nor not known yet how the master document (daily list) would be published. but it should be public and tamper proof