PROFITS - ATPx [Automated Trading Platform eXtended]

[Nova!]

Ok so the time has come to discuss the peer to peer networking aspects of this application.

I was originally going to use a system similar to bitmessage where an anonymous encrypted message is passed around from machine to machine, each machine making an attempt to decrypt the message.  Obviously if you can decrypt the message then it must be meant for you.

While that's a nice idea, the system load at scale is horrific.  Bitmessage drains my laptop from a fullcharge to 0 in less than 30 mins.  I normally get 4 hours of regular use on a charge.  Also I simulated it and realized that all it would take is someone flooding the system with junk messages and the entire network could be denied.  Thus routing in this method is not optimal for our particular situation.  I still love bitmessage, it's just not the right tool for what we're accomplishing here.

So here is the solution I've worked up and what will be in the v1 when it's released.

The basic trading principle is one of aggregation and diffusion or put another way, scatter & gather.
There is a common channel where broadcast messages (trades) are sent to all highly connected peers by other highly connected peers.
A trade request enters the broadcast channel and is diffused into a series of smaller trade requests through a collection of closed loop tightly connected peers (called a peernet).
The peers in a peernet quickly work amongst themselves to attempt to facilitate as much of the trade as possible by matching their own internal resources against the trade. If the trade matches some threshold that is set on a per peer basis (by it's owner) then the peer produces a partial match.
These smaller partial trades are aggregated by the primary node and presented to the channel as a match.
The two primary match nodes then communicate directly to find a settlement option that works for both of them and then they perform settlement according to the agreed upon rules.

Here is an example.
I am connected to MtGox & BTC-e, my bot notices a $10 descrepancy and wants to arbitrage that for as long as the window remains open.
I have 100BTC and $500USD.  My bot directly arbitrages the trade within it's own darknet but sees that the window is still open.
My trading window has netted me 150BTC and I now have $0 USD to purchase more.

My bot would make an announcement (to it's peer controller, more on that later) of 150BTC@USD:BO:SEPA|OKPAY|PAYPAL|DWOLLA|ACH (BO is best offer, the others are proposed settlement methods in order of preference).
This is carried onto the primary channel where it is picked up by 4 primary nodes and diffused/scattered to 40 secondary nodes.

The first node is connected to SEPA, the second through OKPAY etc.(They could be multiconnected, but for now let's assume a single node/settlement method).
Rather than any single node taking the full risk of the transaction, each secondary node comes back to it's primary node with a partial match according to it's own internal rules.

Node1, subnode 1 matches 1BTC@110USD:SEPA, subnode 2 matches 1BTC@109.95USD:SEPA etc.  For a total of 40BTC@105USD:SEPA
Node2, subnode 1 matches 1BTC@117USD:OKPAY, subnode 2 matches 1BTC@116.90USD:OKPAY etc for a total of 90BTC@110USD:OKPAY
...

This goes on until 150BTC have been matched.
My node then presents settlement instructions to the primary nodes directly.  My node then sends the offered BTC to each primary node.  The primary nodes perform the fulfillment and distribute the BTC to their secondary nodes and settle according to their own negotiated rules.

Obviously there is a bit of trust involved here.  However there are rules in place to prevent a node from becoming a primary node if it has trouble with fulfillment, furthermore each primary node is in charge of rating it's secondary nodes to the network.  I call this trustnet and it's something I'll get into in another post, but I sort of touched upon it before.

Now as to the network itself...
When the application is launched for the very first time it will generate a UUID and an encryption certificate.

It will join 4 or 5 IRC servers on a specific channel, let's say #expressnet
Unless you have selected the darknet option and put in your friends UUIDs, once joined, it will poll a list of all nodes (clients) sitting on that channel and perform a DCC offer to each client currently sitting there. 
For our purposes a peernet and a darknet are essentially the same thing. 
The difference is that a darknet is a "pre-authorized" peernet, for instance you and a group of close friends would constitute a darknet.
It's a self agreed network operating to the exclusion of others, it is "dark" to the outside world.
In the trustnet setting, a darknet node is automatically given a trustscore that you assign.  If you fully control it you may as well assign it a 10.

A good example for using a darknet might be someone with nodes located close to each major exchange who just wants to create a network directly between those nodes for the purposes of direct arbitrage.  Personally I would see this option being used as a way of becoming a "major player" on the expressnet, due to the highspeed interconnects and direct connection to sources of liquidity, but I digress.

If you've selected the darknet option, it will only poll for pre-authorized UUIDs and only communicate with those pre-authed IDs.
(There is a mixed dark/open option too, but it's waiting on the trustnet implementation which will be a few more days before I can talk about in depth).

Back to the implementation details...

A DCC roundtrip time will be calculated between every node.
A new client will then sort the nodes into an array ordered by roundtrip time.
The default nodelist size is currently 12, but that number is configurable. 
On a quad core that's probably the max you want, with about +2 nodes per additional core if your machine is going to be fully dedicated to this task. 
If this machine won't be fully dedicated to trading, I recommend cutting that size in half.  Eventually I'll put in a self optimizing algorithm, but for now this is what we've got.

DCC is a direct connection and it is setup between two peers, bypassing the IRC server once the connection has been established.
It is beneficial to use this method rather than rolling our own because most firewalls and routers understand it and will co-operate, there is therefore no need to try and explain ports and configuration which frankly is more of a headache than it's worth.
 
However it's unencrypted and none of the libraries I could find knew how to use  transport layer encryption without puking in the event of a self signed cert.
Therefore we will need to implement a TLS style, Diffie Helman key-exchange and secure the message body itself while in transit.

This will produce a series of loosely connected closed loop peernets (or darknets) that are great for high speed clearing between themselves, but will suck for long range message traversal.  Obviously messages under this scheme will come in, but they won't come out.

For most purposes this is what we want, we want to trade frequently with people in close physical proximity to ourselves.
However there are two classes of message, standard and broadcast.
Standard messages are internal control and prep signaling, essentially the system is going to try and aggregate multiple smaller trades in the closed loop peernets.
It will also work in the reverse, that is to say if a big offer is detected, it's going to try and fulfill it by diffusing it through the local darknet and presenting it as a single completed trade to the network.  (darknet & peernet are interchangeable here).

Broadcast messages are reserved for the big trades.

To correct for this, one peernet node will remain in channel and when it receives a broadcast message it hasn't seen before and cannot be completed internally, it will relay the unfulfilled portion back on to IRC, exactly once (ideally aggregated with other internal offers that cannot be fulfilled internally).

Only one node per peernet will be given voice or authority to speak on behalf of the peernet.  This is to limit inchannel traffic to only large trades that cannot be diffused or aggregated through a local peernet.
 
It will be enforced at least in the beginning by a bot operator that will set the voice attribute only on peers that have been particularly long lived on the channel.
Note that everything I've described above is extremely similar to how bitcoin does peer discovery, except that it doesn't do the voice exit to IRC option.

I hope this is exciting and I'm posting it here to invite feedback.  It is mostly code complete, however if anyone sees a significant problem with it, I would be glad to take it into consideration, I'm open to suggestions here.

Enjoy!

[joesmoe2012]

Still standing by to help test!

[Ianwhitbread]

Yes. I would like to read this post a few times more to get my head around it

But excited

[Ianwhitbread]

The darknet seems a good idea

But my question is

If you have no friends, or you simply dont trust anyone  then can you run multi Instances of this software to get the same goal

Or just configure multiple exchanges yourself?

[Nova!]

I'll be setting up the system with pure crypto to crypto exchanging in the next few days.
The darknet option will be temporarily disabled, trustscore will default to 1 (0 is untrusted, 10 is fully trusted).
The system is setup to directly feed the ticker data from BTC-e into the primary channel and treat it as actual offers.
It will attempt to order match and clear transactions immediately, then it will attempt to refund each transaction within an hour during the testing period.
Trustscore gains (and losses) will carry over from testing to live.

Out the gate I'll be supporting BTC/LTC as the first pair on this and will introduce other currencies as the modules for them become ready.
Since I'm using the RPC API for BTC & LTC and I suspect other currencies use the same/similar API, are there any requests for other wallets?

Ok it's probably time to explain trustscore.

Trustscore is our way of tracking credibility.
Every time a transaction is agreed to by two nodes, a hash of the transaction is generated and signed by both nodes.
This information contains the publickeys of both sides and the currencies to each side but not the amounts.
This is signed and broadcast to the network and kept by all nodes.

It looks something like

Code:

UTC Timestamp
BTC|sender|recipient
LTC|sender|recipient
TXHASH
SIGHASH1 (timestamp+line1+line2)
SIGHASH2 (timestamp+line1+line2)

Once broadcast, each party to the transaction gets 1 credit.

Credits are valid for 1 year then they expire out, that way no one needs to have gigs of trustscore blocks on hand.
Trustscore goes up logarithmically, from 0 to 1 takes 10 transactions, from 1 to 2 takes 100 etc.
Trustscore gains are limited to no more than 1 point gained rolling 7 day period regardless of the number of transactions performed.

Also the time the last transaction was performed factors into it.  
If you go 1 week without any transactions your trustscore drops an entire point for each week of inactivity, but this can be gained back after 7 days of daily activity.

If either node has a problem with the transaction (for instance once side fails to live up to their end) a "dispute" is notated by referencing the previous hash and announcing it as a dispute to the network, where it is added to the rest of the trustchain.

Code:

timestamp
TXHASH
recipientID
SIGHASH

Disputes stay on the record for 1 year and they drag you down to the next lowest trust point.  
Thus if you were a 3 and you get a dispute you are now a 2.
In this instance 900 credits would be essentially wiped out for 1 dispute.
To negate the risk of bad actors, only 1 dispute can be lodged per trading pair.  
It is expected that if you have a dispute that you stop trading with that node, the client implements this behaviour by default but it can be overridden.

A dispute affects both equally in that it drags them both down by a whole point, this ensures that only disputes which are "worthwhile" will be reported.
There is no dispute resolution method, once filed it's there for a year.  
The default settings prevent the client from trading with a node that has a dispute open in the last 30 day rolling period, however existing relationships are not directly effected, just the risk model.

Now here is how the client implements risk modelling.
The trustscore translates directly into a percentage of assets at risk.  0 is completely untrusted, 1 is 1%, 2 is 2% etc.
It will cap at 10%, thus at no time is a trade to go through for more than 10% of liquid assets.
This can be overridden in the configuration and settings panel.  In fact for someone to get to 10 they would have to have 10,000,000,000 successful transactions with no disputes, so it can be considered safe.

In the next version trustscore will be broken out by currency pair and settlement method which will be much more informative, but right now we have a single combined score.

That's it for now folks.
PM me if you want access during the testing period.

[Nova!]

The darknet seems a good idea

But my question is

If you have no friends, or you simply dont trust anyone  then can you run multi Instances of this software to get the same goal

Or just configure multiple exchanges yourself?

Yes you can still run solo on a single exchange and you can also run your own darknet and arbitrage multiple exchanges.
You can do it as multiple instances, or a single instance, it's your decision.
It's not the preferred method, because we would like you to participate in the ExpressNET OTC board, but it is designed to still do the same things the original ATP was.

[Askit2]

Nova! what would be better from a personal setup type system. I am thinking having one bot per exchange would make each bot more responsive to that exchange. I would assume that the benefit to one bot per currency pair would be limited. I also assume one bot per user limits its speed. I am not sure that I have it right but I think I do.

I am excited about this release. It seems to have a lot of potential

[Nova!]

Nova! what would be better from a personal setup type system. I am thinking having one bot per exchange would make each bot more responsive to that exchange. I would assume that the benefit to one bot per currency pair would be limited. I also assume one bot per user limits its speed. I am not sure that I have it right but I think I do.

I am excited about this release. It seems to have a lot of potential

Yeah it's one bot per exchange by default.  However there is nothing preventing more than 1 per exchange, nor multiple exchanges per bot.  It's whatever you have resources for.
Not sure where you got the idea about a single bot per currency pair.  That was never my intent and I hope I didn't say anything like that.  The closest I can think of is the older ATP design where it was one pair per thread.  But that had serious limitations.

We're using an observer pattern with publish & subscribe as well as DSP.

[Nova!]

FYI folks, I'll be changing nicks again soon.
I get too many inquiries about NovaCoin which has nothing to do with me.  However I am working on a new coin too (not coming very soon so don't hold breath) and the email traffic makes it hard to sort who's asking about what.

I've always liked A Midsummer Night's Dream, so I'll be likely be taking up the name Auberon here shortly (unless of course it's taken already too).  If they don't let me change nicks here, I may end up noobifying myself again like I did after the Isis incident.  Sure wish I could hold onto a Nick for awhile.

On another note.
Come the 15th of July, I'll be opening boards for ATPx on the expressvest.com site.
If you click before then, you'll notice the distinct absence of anything resembling a site.

I just re-secured the rights to the expressvest name.  It was originally for a fully licensed banking/trading/investment site and part of the OpenPay & OpenBank framework, but the investors fled the minute MtGox got their US assets seized.  I just managed to finally get the keys to the domain name handed back to me from the dissolution of the partnership.

We'll be adopting the ExpressVest name for the new business that will be sponsoring this project.  The software will be re-branded expressvest and whispernet will be probably be re-branded to something like ExpressNet.  Here's hoping Citizens Bank doesn't send me a C&D for trademark infringement.  I don't see them as closely related, but it wouldn't be the first time I've had to change direction due to a legal filing.

I'm open to other names for ExpressNet, if anyone feels like racking their brain on it for awhile.

[Askit2]

Nova! what would be better from a personal setup type system. I am thinking having one bot per exchange would make each bot more responsive to that exchange. I would assume that the benefit to one bot per currency pair would be limited. I also assume one bot per user limits its speed. I am not sure that I have it right but I think I do.

I am excited about this release. It seems to have a lot of potential

Yeah it's one bot per exchange by default.  However there is nothing preventing more than 1 per exchange, nor multiple exchanges per bot.  It's whatever you have resources for.
Not sure where you got the idea about a single bot per currency pair.  That was never my intent and I hope I didn't say anything like that.  The closest I can think of is the older ATP design where it was one pair per thread.  But that had serious limitations.

We're using an observer pattern with publish & subscribe as well as DSP.

Sorry no you didn't say anything like that. I was just thinking out loud on the one per pair. I was thinking of having one specific host per bot per exchange. Then I thought maybe there would be even less latency if it was one bot per pair. Sadly at some point the connection would saturate leaving the same latency issue just farther along. That and having a number of bots all on one API and keypair likely wouldn't help either setting up multiples would work but even then its more load on the server for maybe a small boost in trading speed.

Anyways on ATPx once more exchanges are added it would be fairly simple to add another bot to handle say MTGoX then have them able to arbitrage the difference.

If I understood you correctly you are going to have a streaming ticker on ATPx as well.

[Nova!]

Nova! what would be better from a personal setup type system. I am thinking having one bot per exchange would make each bot more responsive to that exchange. I would assume that the benefit to one bot per currency pair would be limited. I also assume one bot per user limits its speed. I am not sure that I have it right but I think I do.

I am excited about this release. It seems to have a lot of potential

Yeah it's one bot per exchange by default.  However there is nothing preventing more than 1 per exchange, nor multiple exchanges per bot.  It's whatever you have resources for.
Not sure where you got the idea about a single bot per currency pair.  That was never my intent and I hope I didn't say anything like that.  The closest I can think of is the older ATP design where it was one pair per thread.  But that had serious limitations.

We're using an observer pattern with publish & subscribe as well as DSP.

Sorry no you didn't say anything like that. I was just thinking out loud on the one per pair. I was thinking of having one specific host per bot per exchange. Then I thought maybe there would be even less latency if it was one bot per pair. Sadly at some point the connection would saturate leaving the same latency issue just farther along. That and having a number of bots all on one API and keypair likely wouldn't help either setting up multiples would work but even then its more load on the server for maybe a small boost in trading speed.

Anyways on ATPx once more exchanges are added it would be fairly simple to add another bot to handle say MTGoX then have them able to arbitrage the difference.

If I understood you correctly you are going to have a streaming ticker on ATPx as well.

I promise to get a proper architecture document up when I release this thing to testing.  I'll probably make a wiki and link it to the javadoc or something. 
Arbitrage is just 2 (or more) bots talking to eachother, which is why we have the expressnet p2p network.  If you're going to arbitrage 2 or more exchanges you just setup 2 or more bots and have them connect to eachother (you can darknet this by preloading their keys which will mean they don't talk to other bots).

[daybyter]

How do these bots communicate the total funds to know if arb is feasable?

[Askit2]

Judging by the description I would say they don't. One bot puts up an ask another if there is enough money bids for part of the transaction. No bot needs to know about an opportunity directly. If there is an opportunity the ask will be filled.

[daybyter]

Uhm...sorry, I don't get it...

Let's say, I want to do arb from btc-e to gox. I buy btc at btc-e and sell at gox. What matters is the difference of the price at the 2 exchanges, so no bot can place an order without knowing about the other bot?

Or did I misunderstand the whole concept?

Pardon my ignorance...

[Askit2]

Your BTC-e Bot can place an order to the whole network. An order asking to buy some amount of BTC for some other currency. Another bot, yours or someone else's, can take part of your order and complete the trade. You wouldn't need a Gox account to Arbitrage MTGoX. Your MTGox bot could help with or complete the arbitrage for you depending on size. It sees an order for 100USD for some 1 BTC say. The gox bot sends some amount of that order. Lets say the price on Gox is 107, the bot jumps on the order noting a 7% profit. So on Gox some slightly under 1 BTC is sold and 100$ is sent to your BTC-e account. If there isn't enough interest in the order or profit the Gox Bot would take a pass and not trade. if there isn't enough funds then the Gox bot would trade part of the order.

If you are using the darknet option and depending on how the upper controls work it is possible that a total price and account values would be available at your controller. It sends some signal to your bots and they handle it. Initially there isn't a darknet option working so I don't know for sure. I would know more when it gets released.

[daybyter]

But the btc-e bot must stop the order, if there's no profit! Not just buy the btc at any price! It has to know the data from the gox bot?

[Askit2]

Not as much as know what it is willing to pay to keep a trade going. Yes likely it would to some extent either learn or at the very least catch up to where no one would sell at a price the bot would pay.
As long as BTC-e bot says buying 100$ at 100$ per BTC and the MTGox bot knew GoX price was 107 the trade would likely continue until the bots had a very similar price.
I do believe there is an overarching control system that likely would know what tickers values are at the exchanges you have bots at. But it wouldn't require a bot to make money from it as long as your bot is buying when its profitable for your bot.

EDIT:
Likely scenario would be reversed. Gox bot being higher price would post an ask for USD at say 105 with the current price at 107. BTCe bot would jump on it noting that it would make a 5% profit.....

[Nova!]

How do these bots communicate the total funds to know if arb is feasable?

The answer is, it depends on your configuration.

There is only a single bot per exchange.  This is to reduce system load.
However there is a highspeed, hightrust option that you can configure.  
This sets up up a direct connection between your bots and causes the bots to be intimately aware of each other's internal state.
Currency balances are considered internal state.
In this setup the bot's represent a currency as source.symbol
e.g. MTGOX.BTC or BTCE.BTC
It also causes them to preference the desires of the other bot.

This configuration is intended specifically for exchange to exchange arbitrage on a single pair.  BTCE.USD/MTGOX.BTC
However I haven't finished the closed loop solver on it so it can only arb as long as there are funds available at the source.
When the closed loop solver is complete it will seek to match the exit method of one exchange to a supported entry method on another exchange.
Example (note, that : means transfer and | means trade in this example)

Code:

OKPAY.USD:BTCE.USD|BTCE.BTC:MTGOX.BTC|MTGOX.USD:OKPAY.USD

The closed loop solver will ONLY be enabled in for owned bot arbi mode.
It's not helping any that OKPAY doesn't want to do business with exchanges anymore.  I expect the exchange modules are going to be the biggest upkeep responsibility here.

[Nova!]

Not as much as know what it is willing to pay to keep a trade going. Yes likely it would to some extent either learn or at the very least catch up to where no one would sell at a price the bot would pay.
As long as BTC-e bot says buying 100$ at 100$ per BTC and the MTGox bot knew GoX price was 107 the trade would likely continue until the bots had a very similar price.
I do believe there is an overarching control system that likely would know what tickers values are at the exchanges you have bots at. But it wouldn't require a bot to make money from it as long as your bot is buying when its profitable for your bot.

EDIT:
Likely scenario would be reversed. Gox bot being higher price would post an ask for USD at say 105 with the current price at 107. BTCe bot would jump on it noting that it would make a 5% profit.....

Your description appears to be correct for open net trades.

Your bot would just participate as a regular trader on the p2p network and function as a filter against the exchange's pricing.
It would read in the current pricing, add in the profit that it wants to see, and handle the negotiations on both sides.

Effectively shouting in a room either
"Hey guys, I got BTC, fresh hot BTC only $105 USD, I take paypal, okpay, or sepa, come an get 'em!"
or
"Hey guys, I got USD, warm, smooth and easy on the digestion, I'll give ya 105 of 'em for just 1 BTC, that's right folks I said 1 BTC for 105 of these tasty USD's, come over and see me!"

[Nova!]

Good news everyone!

As you may or may not be aware, I really do suck badly at art.
I wanted this product to have a fun, fresh, clean & professional feeling to it.  I didn't want it to be loaded down with unprofessional programmer art, so I set out to find an artistic director.  I needed someone with a strong portfolio of artwork that would lend itself to exactly the look and feel I wanted to give the product.
Someone who can be responsible for the artistic aspects of the site, the documentation and the product.

I found one and boy what a catch! 
Everyone please welcome aboard Alex Wendzel

Alex is probably best known for his webcomic Out At Home, www.out-at-home.com
A fantastic little comic about a retired baseball allstar, his family & friends and their adventures.

I was inspired to ask him to join us after seeing this comic http://www.out-at-home.com/archives/2940 which pretty much sums up my own fears about how complex this product is starting to get. 
So everyone please welcome Alex aboard and check out his site, it's a lot of fun and he does a great job.