mgw

[jl777]

oldnbold's summary: https://bitcointalk.org/index.php?topic=345619.msg5526665#msg5526665

James

[jl777]

https://bitcointalk.org/index.php?topic=345619.msg5500225#msg5500225

http://arstechnica.com/security/2014/03/critical-crypto-bug-leaves-linux-hundreds-of-apps-open-to-eavesdropping/

The above describe real world security fails. Critical that we make sure no such thing ever happens.

the problem is that protocols of gen1 coins are pretty complicated, leaving lots of room for errors:

https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_Calls_list
https://en.bitcoin.it/wiki/Raw_Transactions

I am designing things to minimize changes needed to add support for another BTC (or even LTC) fork. However, since I am utilizing some advanced features, it is not clear which coins actually support what is needed. One major task that really is the thing that creates the most value is to simply install the daemon for as many coins as possible (in order of importance) and make sure it supports the required low level functions. A desktop with a few terabytes and good bandwidth is all that is needed.

Once we have someone that has dozens of daemons to test things with, we will need to make some small tools for him to be able to easily test if the required functions are there. It took me a while to figure out how to properly create 2 of 3 multisig accts, not to mention just making a rawtransaction. Whoever signs up for being the tool maker, I will PM the reference src needed to get started.

Ultimately, we will have the following topology:


We are currently working on the code for the three gateway servers. All relevant transactions will leave a record in blockchain using AM. So, we need somebody to monitor all the relevant AM's and create a "block explorer" equivalent. This will allow not only us to debug things, but it will create the transparency needed for a trustless system. Whoever signs up to do the website for this, I will PM the internal AM codes. For now, I will assume that Evil Bob wont be able to directly communicate with the gateway servers as they will only accept connections from the 100 guardians.

This means all inputs to the gateways are via AM. At first, I had a lot of client visible calls, but in the interest of simplicity, I reduced to down to just 2 for sending data to gateway and 1 for receiving data back:

#define GET_COINDEPOSIT_ADDRESS 'g'
#define BIND_DEPOSIT_ADDRESS 'b'
#define SET_COINWITHDRAW_ADDRESS 'w'

I expanded GET_COINDEPOSIT_ADDRESS to include the ability to set the withdraw address, so SET_COINWITHDRAW_ADDRESS is only needed to change the withdraw address, but for most people this will be rare. The most common use case is that the client software sends a GET_COINDEPOSIT_ADDRESS with their acct # and withdraw address. They then poll all the AM's for a BIND_DEPOSIT_ADDRESS AM to find out what their deposit address is.

After the wallet addresses are linked to account, everything is designed to be transparent. just send coin to the deposit address and it appears in the AE. just transfer from AE to original issuer (or gateway ID) and it appears in the coin wallet. I tried to achieve Apple level usability so as long as the GUI makes the address binding process as simple as the balance/finance pages on the centralized exchanges, nobody should get confused.

I had to put in a workaround due to the fact that assets do not have fractions at this point. Not sure if this will ever change, so any deposit that isnt a round number will get issued milliALT (or microBTC) for the fraction. I am hoping that assets will support fractions, so this is just a temporary workaround. When we can denominate AE bids in terms of other assets, we will be able to put in a bid of .001 ALT for 1 milliALT and this would allow people to easily convert their millis (and micros) to the full unit asset.

Now the ability to set the withdraw address is definitely an attack vector. Imagine if someone is able to change the wallet address where your withdrawals go to. We need to make sure we identify all attack vectors and really pound on the system to make sure we are not vulnerable. I think looking at the prior security incidents we can create a list of potential attack vectors and a test plan based on them. I need somebody to take charge of this as I will have blind spots on attack vectors on my design. I imagine we will need to use test programs that create AM's en masse to make sure Evil Bob wont succeed.

Finally, if anybody can figure out how to add an optional "comment" field to the transferAsset API http://wiki.nxtcrypto.org/wiki/Nxt_API#Transfer_asset I will be able to implement a AE level atomic exchange between any two assets. It will require a private prearranged trade, but for large direct exchanges I dont think this is a barrier. As soon as the gateway is secure, assets are basically 1:1 for the real thing.

I am not familiar with everyone's skill set, so I am hoping that you will all self select tasks from the list and we can coordinate and clarify things on this thread.

a) massive installs and tests of bitcoin forks
b) tool maker
c) transaction explorer
d) attack vectors and Evil Bob attack programs
e) adding "comment" support to core

James

P.S. doing a dogecoind -reindex took a long time, but the unconfirming tx is gone and all three are in sync, so we now have a functional trio ready for testing. I havent verified https://github.com/jl777/multigateway/blob/master/get_dogeaddr.c is up to date, so maybe someone can do that?

[l8orre]

heya! ironing out last f***img wrinkles from AE client, but I'm alert!

 

[mcjavar]

I am technically not fit to help here, but I will take over the organizational stuff. Just let me know what is needed so you can focus on coding.

Could the others please also drop a line about your understandings of what James wrote in his post above?

[knightcoin]

I am technically not fit to help here, but I will take over the organizational stuff. Just let me know what is needed so you can focus on coding.

Could the others please also drop a line about your understandings of what James wrote in his post above?

It's not clear for me exactly what is all about, my area is networking, so hopefully I can help on that field.. Well you can fairly simulate a large network at very low cost. To Networking prototyping, it is desired to have Flexibility, Scalability, Low-Cost and Realism then basically you can have a networks simulators  like ndnSIM, OMNeT++ , opnet but their are mathematical models. You can also have testbeds (expensive, they are experimental infrastructures that provide real resources for test and pilot scenarios eg.: Geni, Planetlab) and finally emulators just like testbeds,  they are realistic because they run real code (applications,  OS  kernel,  etc).   

To emulate that network above I would suggest then ..

The Common Open Research Emulator (CORE) it has a nice gui, network tools, etc and have a documentation
http://www.nrl.navy.mil/itd/ncs/products/core   

[mcjavar]

I am technically not fit to help here, but I will take over the organizational stuff. Just let me know what is needed so you can focus on coding.

Could the others please also drop a line about your understandings of what James wrote in his post above?

It's not clear for me exactly what is all about, my area is networking, so hopefully I can help on that field.. Well you can fairly simulate a large network at very low cost. To Networking prototyping, it is desired to have Flexibility, Scalability, Low-Cost and Realism then basically you can have a networks simulators  like ndnSIM, OMNeT++ , opnet but their are mathematical models. You can also have testbeds (expensive, they are experimental infrastructures that provide real resources for test and pilot scenarios eg.: Geni, Planetlab) and finally emulators just like testbeds,  they are realistic because they run real code (applications,  OS  kernel,  etc).   

To emulate that network above I would suggest then ..

The Common Open Research Emulator (CORE) it has a nice gui, network tools, etc and have a documentation
http://www.nrl.navy.mil/itd/ncs/products/core   

Thank you for your input!

@jl777 can we use such a tool?

[jl777]

I am technically not fit to help here, but I will take over the organizational stuff. Just let me know what is needed so you can focus on coding.

Could the others please also drop a line about your understandings of what James wrote in his post above?

It's not clear for me exactly what is all about, my area is networking, so hopefully I can help on that field.. Well you can fairly simulate a large network at very low cost. To Networking prototyping, it is desired to have Flexibility, Scalability, Low-Cost and Realism then basically you can have a networks simulators  like ndnSIM, OMNeT++ , opnet but their are mathematical models. You can also have testbeds (expensive, they are experimental infrastructures that provide real resources for test and pilot scenarios eg.: Geni, Planetlab) and finally emulators just like testbeds,  they are realistic because they run real code (applications,  OS  kernel,  etc).   

To emulate that network above I would suggest then ..

The Common Open Research Emulator (CORE) it has a nice gui, network tools, etc and have a documentation
http://www.nrl.navy.mil/itd/ncs/products/core   

Thank you for your input!

@jl777 can we use such a tool?

Maybe, but I already have access to ~180 servers, at least for now, to setup a real environment. Not sure if we need anything more elaborate, especially with our limited personnel.

[btc2nxt]

i have read all James'  posts, many are full of invention.
6 years ago ,i coded a small program with object-c in linux, now use delphi in windows. In China you know  a few people use linux/unix.
Because i am not familiar with PHP in web design, i think  i can help comment on core and make some test tool.
btw, my english is very fool.

[jl777]

i have read all James'  posts, many are full of invention.
6 years ago ,i coded a small program with object-c in linux, now use delphi in windows. In China you know  a few people use linux/unix.
Because i am not familiar with PHP in web design, i think  i can help comment on core and make some test tool.
btw, my english is very fool.

As long as you can understand that is the most important.
The client software I wrote should be simple enough to port to whatever other language. I think the first step is for someone who is not me to be able to write programs that create gateway AM

James

[mcjavar]

i have read all James'  posts, many are full of invention.
6 years ago ,i coded a small program with object-c in linux, now use delphi in windows. In China you know  a few people use linux/unix.
Because i am not familiar with PHP in web design, i think  i can help comment on core and make some test tool.
btw, my english is very fool.

As long as you can understand that is the most important.
The client software I wrote should be simple enough to port to whatever other language. I think the first step is for someone who is not me to be able to write programs that create gateway AM

James

Could you please specify that?

[jl777]

i have read all James'  posts, many are full of invention.
6 years ago ,i coded a small program with object-c in linux, now use delphi in windows. In China you know  a few people use linux/unix.
Because i am not familiar with PHP in web design, i think  i can help comment on core and make some test tool.
btw, my english is very fool.

As long as you can understand that is the most important.
The client software I wrote should be simple enough to port to whatever other language. I think the first step is for someone who is not me to be able to write programs that create gateway AM

James

Could you please specify that?

Somebody on the team needs to recreate the doge client demo using just the AM data structure. Most of the code is just a hardcoded parser for API JSON return strings. I bet someone can make an equivalent program in much fewer lines of python or any other language using hansson JSON lib.

[mcjavar]

i have read all James'  posts, many are full of invention.
6 years ago ,i coded a small program with object-c in linux, now use delphi in windows. In China you know  a few people use linux/unix.
Because i am not familiar with PHP in web design, i think  i can help comment on core and make some test tool.
btw, my english is very fool.

As long as you can understand that is the most important.
The client software I wrote should be simple enough to port to whatever other language. I think the first step is for someone who is not me to be able to write programs that create gateway AM

James

Could you please specify that?

Somebody on the team needs to recreate the doge client demo using just the AM data structure. Most of the code is just a hardcoded parser for API JSON return strings. I bet someone can make an equivalent program in much fewer lines of python or any other language using hansson JSON lib.

Ok, anyone capable of this? Guys?

[antanst]

Ok. I'm going to make a whole lot of questions, and some of them might seem strange, but I skipped many posts of the huge NXT thread...so I appreciate your patience :-)

1. What does "AM" stand for? Asset Management?

2. Why 2-3 multisig transactions? Is there support for M-N multisig transactions, where 2>M>N, at least in the RPC API of dogecoind or in {Bit,Lite}coind?

Can you explain the general workflow, and the input/outputs of each computer in the workflow?

After the wallet addresses are linked to account, everything is designed to be transparent. just send coin to the deposit address and it appears in the AE. just transfer from AE to original issuer (or gateway ID) and it appears in the coin wallet.

Let's say I have some Doge's. I talk to the input server. I assume that the "Input Server" will be the one that hosts a web site or an API server, right? So I'm given a DOGE M-N multisig address, in which I should deposit the coins. This address has been generated with all the N private keys coming from the N Trusted Gateways. I also give my NXT account, in which I want my assets transferred to.

I deposit the doge's in the address. The trusted gateways coordinate together, M of them sign the transaction and withdraw the DOGE's into a private address. Then some (how many?) "DOGENXT" assets are created, and given back to the user to his NXT account.

Are the above right so far?

[jl777]

Ok. I'm going to make a whole lot of questions, and some of them might seem strange, but I skipped many posts of the huge NXT thread...so I appreciate your patience :-)

1. What does "AM" stand for? Asset Management?

2. Why 2-3 multisig transactions? Is there support for M-N multisig transactions, where 2>M>N, at least in the RPC API of dogecoind or in {Bit,Lite}coind?

Can you explain the general workflow, and the input/outputs of each computer in the workflow?

After the wallet addresses are linked to account, everything is designed to be transparent. just send coin to the deposit address and it appears in the AE. just transfer from AE to original issuer (or gateway ID) and it appears in the coin wallet.

Let's say I have some Doge's. I talk to the input server. I assume that the "Input Server" will be the one that hosts a web site or an API server, right? So I'm given a DOGE M-N multisig address, in which I should deposit the coins. This address has been generated with all the N private keys coming from the N Trusted Gateways. I also give my NXT account, in which I want my assets transferred to.

I deposit the doge's in the address. The trusted gateways coordinate together, M of them sign the transaction and withdraw the DOGE's into a private address. Then some (how many?) "DOGENXT" assets are created, and given back to the user to his NXT account.

Are the above right so far?

1. AM means Arbitrary Message, it is really a way to publish data on the blockchain. I check the first four bytes for the GATEWAY_SIG code and if it matches, assume it is a data packet for the gateway. All communications to the gateway are via the blockchain.

2. I wanted to do 4 of 5, but the current bitcoin network treats anything over 3 signers as a non-standard transaction and few of the miners will propagate it. For dogecoind, it simply gags on anything over 3.

The only way you can talk to the gateway is by using a sendMessage NXT API
http://wiki.nxtcrypto.org/wiki/Nxt_API#Arbitrary_Message_System_Operations

The data needs to be a gateway_AM data structure. The C code has char *AM_get_coindeposit_address(int timestamp,int gatewayid,char *nxtaddr,char *withdrawaddr,char *userpubkey)

This shows exactly how to create and send a request to bind NXTaddr with DOGE withdrawal addr. It will then publish a corresponding AM with the deposit address. The C code loops scanning all AM's for the gateway signature and when it finds the one with the deposit address, it terminates.

Now there are two completely separate processes, deposit and withdraw.

Deposit
1. user sends DOGE to the address returned by gateway in the AM data
2. the selected gateway (each gateway has different deposit address) detects the deposit and transfers the amount to the shared multisig acct and at the same time issues the corresponding amount of DOGE and milliDOGE assets. Amount is rounded to nearest .001 and TXFEE is subtracted

Withdraw
1. user transfers DOGE or milliDOGE asset to the gateway (or DOGE asset issuer)
2. the selected gateway notices the withdraw request and posts an AM with the details
3. all three gateways process the withdraw request and add it to an internal queue
4. All withdraws are put into a single global queue for all withdraws to avoid any parallel changes to balances
5. the three servers agree on which withdraw request to process and generate a rawtransaction to satisfy it
6. all three servers compare their rawtransaction with the other servers and if it is IDENTICAL to the byte, the selected server signs a rawtransaction that was already signed by one of the other gateways and broadcasts it to DOGE network and also sends an AM to the NXT blockchain that the withdraw was completed
7. all three servers mark the withdraw request as completed and go on to the next on in the queue

So, quite a lot of stuff happening behind the scenes, but as far as the user is concerned, the DOGE deposit address is directly linked to the NXT AE and the NXT AE "transfer" link is directly connected to their DOGE wallet registered as the withdraw address.

I designed this to be the easiest to use gateway out of all the gateways that are out there, but one that is more secure.

James

P.S. If you point out that the above does not match the picture, you got me. I just thought the pic was cool and it is something that might or might not get done in a parallel project

[l8orre]

i have read all James'  posts, many are full of invention.
6 years ago ,i coded a small program with object-c in linux, now use delphi in windows. In China you know  a few people use linux/unix.
Because i am not familiar with PHP in web design, i think  i can help comment on core and make some test tool.
btw, my english is very fool.

As long as you can understand that is the most important.
The client software I wrote should be simple enough to port to whatever other language. I think the first step is for someone who is not me to be able to write programs that create gateway AM

James

Could you please specify that?

Somebody on the team needs to recreate the doge client demo using just the AM data structure. Most of the code is just a hardcoded parser for API JSON return strings. I bet someone can make an equivalent program in much fewer lines of python or any other language using hansson JSON lib.

yup. I'll be putting FreeRider up on github tomorrow. I really want to do s.t. else for a break. That sounds good!



 

[jl777]

I cleaned up the code a bit and now the get_dogeaddr.c is around 100 lines of C

git clone https://github.com/jl777/multigateway
cd multigateway
chmod +x run.sh
./run.sh
./get_dogeaddr <NXT addr> <NXT acct passkey> <coin withdraw addr> [gatewayid]

James

[eB101]

I am technically not fit to help here, but I will take over the organizational stuff. Just let me know what is needed so you can focus on coding.

Could the others please also drop a line about your understandings of what James wrote in his post above?

I will dig into this for some time, and let you know how I can contribute afterwards.
This thread is great to get started, thank you James and mcjavar.

[btc2nxt]

it is time to recall c,linux and java.

[antanst]

Deposit
1. user sends DOGE to the address returned by gateway in the AM data
2. the selected gateway (each gateway has different deposit address) detects the deposit and transfers the amount to the shared multisig acct and at the same time issues the corresponding amount of DOGE and milliDOGE assets. Amount is rounded to nearest .001 and TXFEE is subtracted

So the user deposits DOGE to an address that is solely handled by one gateway, and afterwards the amount is transferred to a multisig account? What if this gateway gets compromised? Why not give the user directly a multisig acount to deposit in the first place?

I designed this to be the easiest to use gateway out of all the gateways that are out there, but one that is more secure.

What gateways are you referring to? Ripple's?

[jl777]

So the user deposits DOGE to an address that is solely handled by one gateway, and afterwards the amount is transferred to a multisig account? What if this gateway gets compromised? Why not give the user directly a multisig acount to deposit in the first place?

Each specific deposit is handled by one gateway and then swept into multisig, but any of the gateways can be selected for handling the deposit.

For deposits, my design is as vulnerable as a centralized exchange, but this is only for the brief time that a deposit is in transit. People can mitigate against this by splitting up their deposit into three and sending increments to all three.

I thought about letting the users directly send to the multisig, but then we would need to limit each user to using a single source address for making deposits as there would be no other way to correlate deposits with customers. It also exposes the multisig account to uncontrolled unspent txouts, which impacts the method of generating withdrawals. There are some scenarios where deposits would have to be delayed for a bit to clear out withdrawals and if we didn't control the deposits going in, we would not be able to manage this. What happens after the server gets restarted? Every single direct deposit to the mutltisig would have to be correlated with NXT acct, I think this would require some sort of database, which brings up a whole host of issues that I avoid with my simpler design.

I could have created a different multisig for each person, but that opens up an entirely new level of complexity and would introduce a much higher chance of bugs, which is really indistinguishable from a security risk. The simpler the design, the fewer chances for bugs. I wanted the gateway to be as stateless as possible.

It is a tradeoff, but since the user has a way of mitigating risk, all of the other factors won out for this method. For the truly paranoid, we can have a client automatically breakup the deposit into three sets and then incrementally send fragments of that. At any given time, the risk is limited to the time the deposit is in transit only. It is not like the funds will be sitting there for any length of time.

What gateways are you referring to? Ripple's?

Certainly much easier than any of ripples half dozen+ gateways, but I am also viewing all of the various exchanges balances/finances page as gateways. I think relying on NXT account # in AM is more secure than email confirmations.
I guess my thinking is that it is much easier to use than any of the ripple gateways and just as easy to use as centralized exchange gateways, but more secure than they are

James