barterDEX testing found connectivity issues between nodes as the main problem. This required totally revamping the low level networking and actually totally changing how the pubkey based communications are done.
I decided to go to a broadcast method, which is theoretically more bandwidth intensive, but due to the redundancy needed to ensure the time critical packets got to the destination, it was using the bandwidth anyway. And if the design is a broadcast method, it means that we can add point to point encryption for all the pubkey based comms.
So I did.
I also streamlined the broadcast mechanism and added a broadcasting service that the LP nodes do for client nodes. This allows any client to request a broadcast of an encrypted packet to the network. Now I am tracking the number of redundant packets sent to make sure we aren't flooding the network.
PULL dup.1 (30 / 139) 21.6% encrypted.0 recv.4015931715 [7b 22] vs 43 41 U.102
About 20% redundant packets, which is not bad at all. Since the atomic swap is using pubkey based messaging, this means that the atomic swaps are done in the dark. Of course there are still indirect means that can be used to determine who did a trade at a certain time, just by looking at the blockchains, so it isnt anything like zeroknowledge level privacy, but having all the trade details under encryption certainly makes it harder for others to know what is going on.
This will of course create a bit of a problem when trying to gather stats, but I have an idea on how to do that without infringing on the privacy.
As I have discussed before nodes that cant bind ports that other nodes can connect to (virtually all home isps!) is the hardest case. Of course we want to support this case as the alternative is that anybody that wants to post an order long term would need to run a vps. The new broadcast mode is designed to work much more reliably with such nodes. And in fact was able to list some JUMBLR for sale and then fill it from another node!
This means that even now, anybody can run a barterDEX client, post orders and have it filled, without requiring the LP nodes to be involved in anything but the routing of packets (which do not have any ip address information of the client). Now if a client uses canbind:<nonzero> then it will need to give its ip address to the nodes it trades with, but with the trade details encrypted it is only to the counterparties to a trade. Still if you do a lot of trades maybe this is not so good, and canbind:0 is the way to go, as the ip address you give to the counterparties is the ipaddress of the LPnode that created a psock for you.
A psock is a paired socket that the LPnodes binds on two ports. These ports are then published and exactly one node can connect to each one. This also provides a mutual exclusion functionality and with the connection address under encryption it makes any wannabe interlopers job much more difficult, even if they manage to become a man in the middle. The LP node does not even get the ip address in the request or even the pubkey of who is requesting it, it just allocates a psock and then whenever there is data available on either port just sends it from one port to the other. Of course, given wireshark level data the LP node that makes the psock for you will be able to determine your IP address, but with proper precautions you can make it so that it wont matter, ie use a vps node as your client.
Anyway, I have been busy getting barterDEX production ready and it is almost passing all the required musthave test points. We still need to do the stress testing and illegal input type of tests along with disconnection during atomic swaps, so not done yet, but the finish line is now appearing visible.
I tried doing a JUMBLR/BTC atomic swap and it actually almost worked, it seems the autofee calculation is a bit wonky, but considering this was the first BTC swap I even tried, it is not a bad result
dexfee:
https://blockchain.info/tx/f827cde2f79fa67118529567f023ecaf1907cd3324dedc3f45eb01ea9d275d04 0.0001 BTC
bobdeposit: 6cefc5325c6d606362d91187b9066aca8c28f15b71daab8a8131040d25faf3c4 1.12498750 JUMBLR
alicepayment:
https://blockchain.info/tx/957e3bdb555e79206c91ea7d0110826867e3780866ef5795136def0443202397 0.0044996 BTC
bobpayment: e573f7aa397d4d38580c4886f3008214fb80a4616971215fd2b2a73dd0592c30 1.00000000 JUMBLR
alicespend: 9254b848a7e0644c553605394c910d90b39405a03fca8ffa68ceec61790ee4c3 1.00000000 JUMBLR (this is a bug where the txfee was not deducted)
bobspend: 01000000019723204304ef6d139557ef660878e367688210017dea916c20795e55db3b7e9500000
000db0048304502210099723f1d7912b8ce431aa28e3a6b809aa591c5afeb789e0529fc90866b0b
960802204cf5d62f140808491ceeef21d8ccddfa6f7d1ee638a1f0b6d8b5d89c9fc645850148304
502210090696d279ac68121e9b72b35b2064947a998f0ab3b24e79e27a85cc7f05f4bba02200583
d64b4a1d6563ee65f54702ef371b121fdbbf539168b3b13ca88942f40ceb0147522102725643f29
e4139071c632c6cf0ff84108070074eec46ba11376052492e13406f210383d62d25e9ef6087dc92
a4d16c40eb21272f3407feb65f772ecd59539dc329da52aeffffffff01a8dd0600000000001976a
914efdfb7a012b73ad7a2aecfa013d231dd9b6736c088ac00000000
error due to insufficient priority trying to spend 449960 sats when input is the same
bobrefund: 01cb13f6fa9e0c2f4ddcfc0ff9642c49f13e7f2b632dfc47cbcdfc1a2b8a4611 1.12488750 JUMBLR
We seem to be down to some small issues with the error recovery code. I am using the same code for error recovery as for the mainstream swap cases to minimize the amount of code needed and maximize the code path coverage.
fixed the txfee issue for BTC and with the new version, it automatically was able to recover the pending alicepayment:
So I think BTC is now properly supported by barterDEX, though we do have to test it more
