tlsnotary - cryptographic proof of fiat transfer for p2p exchanges

[waxwing]

Hello I am keen to do some testing if you like.  I have downloaded and run the python script on my linux box, but I think I need a key from you?

PM or BM dansmith for the key. Thanks for giving it a try.

[greBit]

Just a note for linux testers.  The alphatest.txt file should be created with the most minimal permissions, if the permissions are too 'open' SSH will fail

[waxwing]

Just a note for linux testers.  The alphatest.txt file should be created with the most minimal permissions, if the permissions are too 'open' SSH will fail

Good call. Yes openssh insists on, if I recall correctly, 600 permissions for private key files (or at least 0,0 for group,world).
We're going to try to move all alphatesting technical discussion to this thread.
I will copy your comment there grebit.

We'll keep this thread reserved for discussions, questions and ideas on the software/architecture.

[dansmith]

Moved from a different thread. This thread is more appropriate for discussion of the architecture.

Hi waxwing and dansmith,

first, thank you for this alpha test. I am currently trying to understand the oracle code and trying to set it up on my own AWS accound for initial testing.
I am not very familiar with Amazon EC2 but I was able to setup the oracle as you described in the INSTALL-readme.
Now I am trying to understand the points in the readme how someone can make sure that the oracle was not maliciously modified.
First, I have maybe a stupid question regarding the public key in the file https://github.com/themighty1/ssllog/blob/alphatest/oracle/authorized_keys. What is the corresponding private key and why do we need this here? Shouldn't we try to block all ssh access to the oracle?

I think it will take some more days until I really understand the other checks what you describe in the INSTALL file...

@nomailing, thank you for trying it out.
The authorized key you're pointing out is the default key which allows the escrow to logon to the oracle for the very first time. Then the escrow is expected to change that key using oracle's register_escrow command.
The corresponding private key is called escrow_key.

Shouldn't we try to block all ssh access to the oracle?

Yes, we should and we are. We explicitely tell sshd not to spawn a new shell and to run our custom command with the line "no-pty,no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding,command=/usr/bin/python /home/ubuntu/stub.py..." in authorized_keys

There are a number of reason why we use sshd instead of our own code:
1. sshd code is definitely more trusted than any other code we would write
2. sshd provides a simple authentication mechanism to enable only those with correct keys to use the oracle
3. If it was not for sshd, the user would have to connect directly to stcppipe's outward facing port (we need stcppipe to log ssl session on oracle, remember?). So if it wasn't for sshd, we would have to juggle with iptables rules in order to protect stcppipe's outward facing port from DOS.

[nomailing]

Moved from a different thread. This thread is more appropriate for discussion of the architecture.

Hi waxwing and dansmith,

first, thank you for this alpha test. I am currently trying to understand the oracle code and trying to set it up on my own AWS accound for initial testing.
I am not very familiar with Amazon EC2 but I was able to setup the oracle as you described in the INSTALL-readme.
Now I am trying to understand the points in the readme how someone can make sure that the oracle was not maliciously modified.
First, I have maybe a stupid question regarding the public key in the file https://github.com/themighty1/ssllog/blob/alphatest/oracle/authorized_keys. What is the corresponding private key and why do we need this here? Shouldn't we try to block all ssh access to the oracle?

I think it will take some more days until I really understand the other checks what you describe in the INSTALL file...

@nomailing, thank you for trying it out.
The authorized key you're pointing out is the default key which allows the escrow to logon to the oracle for the very first time. Then the escrow is expected to change that key using oracle's register_escrow command.
The corresponding private key is called escrow_key.

Shouldn't we try to block all ssh access to the oracle?

Yes, we should and we are. We explicitely tell sshd not to spawn a new shell and to run our custom command with the line "no-pty,no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding,command=/usr/bin/python /home/ubuntu/stub.py..." in authorized_keys

There are a number of reason why we use sshd instead of our own code:
1. sshd code is definitely more trusted than any other code we would write
2. sshd provides a simple authentication mechanism to enable only those with correct keys to use the oracle
3. If it was not for sshd, the user would have to connect directly to stcppipe's outward facing port (we need stcppipe to log ssl session on oracle, remember?). So if it wasn't for sshd, we would have to juggle with iptables rules in order to protect stcppipe's outward facing port from DOS.

Ahh ok, that makes sense. Now I get the overall architecture a bit more. Thank you for the explanation.

[nomailing]

I am now trying to figure out how exactly I have to setup the escrow after I first started the snapshot using the ec2-api-tools and running the command "ec2-run-instances ami-a73264ce ...".

I tried to get some rough overview about the code: It seems I have to first register my new escrow_key with the register_escrow command and then I have to add users using the add_pubkey command. These commands will be sent via ssh to stub.py on the ec2-instance. From there the commands transmitted via ssh are forwarded via a socket to oracle.py. In oracle.py the command is handled by the escrow_thread or 

I am not quite sure if I am supposed to use test_oracle.py to do this initialization of the oracle. It seems that test_oracle will initialize the ssh session to localhost and not to the EC2-instance? It would be very nice, if you could explain the steps I have to take to initialize the oracle after the first start. Or did I overlook some file where it is explained maybe?
Thank's a lot.

[waxwing]

Hi nomailing.

If you've already done register_escrow and add_pubkey for a test user, you should be able to start paysty up directly, as long as you've put that test user's pubkey in alphatest.txt.
Let us know if it doesn't work and we can figure it out together.

Oh and edit: have you constructed the query strings yet with aws_query.py?

[nomailing]

Hi nomailing.

If you've already done register_escrow and add_pubkey for a test user, you should be able to start paysty up directly, as long as you've put that test user's pubkey in alphatest.txt.
Let us know if it doesn't work and we can figure it out together.

Oh and edit: have you constructed the query strings yet with aws_query.py?

I couldn't do register_escrow so far, because somehow the ssh login to the oracle is not working.

Are you sure that the public key in
https://github.com/themighty1/ssllog/blob/alphatest/oracle/authorized_keys
matches the private key in
https://github.com/themighty1/ssllog/blob/alphatest/oracle/escrow_key?

[waxwing]

Yes, it's the same key. You can verify by doing a ssh-keygen -y on the escrow_key.

If you want to communicate more real-time to solve the issue, PM me and we can set up some kind of channel.

There are lots of fiddly details in getting the oracle to work. It took me quite a while the first time too!

[nomailing]

My tshark version in ubuntu does not have the parameter "-Y".
It is tshark version 1.6.7.

I tried replacing -Y by -R, but now it complains about wrong time format:

Code:

tshark: "2013-11-22 01:35:45" is not a valid absolute time. Example: "Nov 12, 1999 08:55:44.123"

[waxwing]

My tshark version in ubuntu does not have the parameter "-Y".
It is tshark version 1.6.7.

I tried replacing -Y by -R, but now it complains about wrong time format:

Code:

tshark: "2013-11-22 01:35:45" is not a valid absolute time. Example: "Nov 12, 1999 08:55:44.123"

Yes, the latest Wireshark version is 1.10.x, -R was deprecated a few versions back. As for the date, probably the same issue, never seen that.

Is this just for various kinds of manual testing? tshark executable is shipped with Paysty so shouldn't be any issue there, unless I missed something.

[nomailing]

My tshark version in ubuntu does not have the parameter "-Y".
It is tshark version 1.6.7.

I tried replacing -Y by -R, but now it complains about wrong time format:

Code:

tshark: "2013-11-22 01:35:45" is not a valid absolute time. Example: "Nov 12, 1999 08:55:44.123"

Yes, the latest Wireshark version is 1.10.x, -R was deprecated a few versions back. As for the date, probably the same issue, never seen that.

Is this just for various kinds of manual testing? tshark executable is shipped with Paysty so shouldn't be any issue there, unless I missed something.

I am using ubuntu, so the executable is not really an option.
Anyway, I will try to update to the latest version.

[dansmith]

Thank you for your patience nomailing, github updated to support tshark v1.6. Download from the usual location:

https://github.com/themighty1/ssllog/tree/alphatest

I also changed the oracle machine location from US to South America, because I learned the hard way that accessing your non-US financial institution from a US IP address is reason enough to get your account suspended.

[Mike Hearn]

Probably the oracle should proxy traffic back through the client machine rather than connect to the bank directly. That way the bank can't see anything different.

[waxwing]

Probably the oracle should proxy traffic back through the client machine rather than connect to the bank directly. That way the bank can't see anything different.

We looked at a number of architectures, but we rejected this one because we didn't see it being possible to trust the buyer's network traffic. In this scenario the buyer would have access to the master secret that's been negotiated by the handshake, which means they could fake traffic post-handshake.

On the other hand, dansmith has been looking at the possibility of splitting the master secret in such a way that this kind of arch (buyer-oracle-buyer-bank) might actually work, with the oracle holding the mac key (but not the encryption key). Watch this space for more, I guess.

Edit: I guess I should mention that we also spent a long time looking at (buyer-oracle/escrow-seller-bank) architectures where the idea was that using the seller as a proxy provided an extra safety for the seller (they keep an encrypted record of the traffic) and helped with the peer-peer element since the IP addresses from which traffic originates are distributed (and kept local if buyers and sellers are in same locality). This approach doesn't seem to be the preferred one by most people in the discussion though, since it involves a "synchronization" of the seller being online in some sense, plus many people say as buyers they don't want their traffic going through unknown nodes (to me that concern makes no sense, but OK).

Btw we never went into buyer-seller-bank architectures because of NAT traversal issues. Of course a full P2P network can overcome that too.

[Mike Hearn]

Ah, I see.

Then yes, relays in various cities might make sense. I would suggest Tor nodes or other proxies, but those would give banks even more of an allergic reaction.

This seems like a fairly major problem. The master secret is normally an AES or RC4 key, right? I don't see how it's possible to split that. And Amazon definitely won't have datacenters in most geographies. Even if they did, I know from experience that all kinds of dodgy things can come out of AWS so some banks would likely treat that as inherently suspicious.

Hmmmm.

[waxwing]

So a couple of things: first, the TLS RFC section 6.3 states that the master secret is partitioned into 4 16 byte blocks: client/server encryption key, client/server mac key.

This is the basis of the idea of "splitting"; if someone other than the buyer holds the server mac key, then the authentication can be done by another party.
About AWS oracles: yes, there isn't enough geographic distribution, and in any case it isn't a good idea to have a lot of (or maybe any) bank traffic flowing from there. But I think we all agree it can flow *through* there.

We are looking at this "splitting" idea right now. It's possible we can do this without the oracle, or with it.

Edit: as of right now I see some issues with this splitting idea. Probably best leave it alone until we have something more definite, or we decide it can't be done.

[Mike Hearn]

What if it was relayed via the seller instead? In many cases, buyer and seller will be in the same country.

[waxwing]

What if it was relayed via the seller instead? In many cases, buyer and seller will be in the same country.

Yes, see the third paragraph of post 175 just above, this was the first model we looked at seriously.

[Mike Hearn]

Oops, sorry, you're right.