Do you use a bot for trading?

[wilson76]

hm.... might mixing things up of the transport layer, the protocol layer and how an exchange API is implemented.
For sure there is a connection established when I want to invoke the API. But the http requests and responses do not have
to be synchronous.

There are tons of simple examples of working with REST Api's in a asynchronous way, e.g.:

Future<Response> response1 = client.target("https://bittrex.com/api/v1.1/market/cancel")
                               .request()
                               .async().get(Response.class);

Future<Response> response2 = client.target("https://bter.com/api/1/private/cancelorder")
                               .request()
                               .async().get(Response.class);

// now you can do other stuff or sit and wait till you have both responses.

[deisik]

For example, you can send a new order request and at the same time, without waiting for a reply to this request

Yeah this is multithreading

The question is not about multithreading

The question is about exchange api's and how they work in practice, surely not about how exactly your code can send requests. These gory details are ultimately irrelevant for most practical reasons since this is not where problems typically arise. With a synchronous api, you can't possibly do that (remember I've been there), and it doesn't matter if you are using a true multithreading or just software interrupts (or whatever) since you will most likely just receive an error from the exchange if you try that (this refers only to authenticated requests). But you seem not to understand that

[Sampey]

There are tons of simple examples of working with REST Api's in a asynchronous way, e.g.:

Future<Response> response1 = client.target("https://bittrex.com/api/v1.1/market/cancel")
                               .request()
                               .async().get(Response.class);

Future<Response> response2 = client.target("https://bter.com/api/1/private/cancelorder")
                               .request()
                               .async().get(Response.class);

Of course, this is async at APPLICATION LEVEL.
This is an high level library but i can also write that function by mystelf

- Prepare Call
--------- NEw Thread(PerformCall -> Give the Reponse to Method X)
- Continue with main flow

I use java futures for dedicated process like monitoring or checking running algorithms.

But if we look into NEw Thread(PerformCall -> Give the Reponse to Method X) we will discover that a Sync call is happening :-) It's invisible to you, but technically is a Sync call.

There's a lot of mysunderstanding about the concept of Sync/Async because many time it depends by the level of the view.
The code you post is surely async but the real http call will be performed in sync way.

A REAL async http Call is this :



As you can see 1-2 is the SEND Routine (that can be completed with a 200 http OK Code)
Then 3-4 is the real async response, where you set a service for the response.

[wilson76]

To come back to the topic  

Yes, I tried several bots in the past and this made me the decisson the implement my own trading platform. I'm not talking
of a "bot" anymore.

Each bot I tried, had its issues and parts I wanted to improve: different strategies, faster execution, diving into new technologies etc.
Here's a list of bots I tried:
- C.A.T.  
- SpreadBot (a German project)
- leoArdo
- poloniexlendingbot
- currently having a close look how gunbot is performing
- and some github open source projects I forgot


 

[Sampey]

I think i have the DEFINITIVE Reponse.
I remember about http return codes.

When CALL Is Synchronous you get HTTP 200
When CALL Is Asynchronous you get HTTP 202 (because it only means accept)

https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

As i remember i never seen something different than HTTP 200 after the REST Connection 

[wilson76]

I've never have seen a 202 response code either.

And maybe I pushed the discussion to the wrong side: the client-side.
For me the more interesting part is the BTer server-side
and the behaviour @deisik described. Still thinking about what he said

PS: Despite all a nice discussion so far

[Sampey]

And maybe I pushed the discussion to the wrong side: the client-side.

But my discussion was on Client HTTP <-> API layer.
So we lost time all of us. Good 

[wilson76]

And maybe I pushed the discussion to the wrong side: the client-side.

But my discussion was on Client HTTP <-> API layer.
So we lost time all of us. Good  

Yay okay. As you said, there are different ways of implenting stuff on the client side. Synchronous calls, sending a request and waiting for
the response. and so on. Sending the request and asynchronously waiting for the response. So far we do agree Just different styles
of implementation. Indeed all the exchange API's I know so far are implementing simple request/response pattern. But thats a different topic
and not the point where the discussion started.

@deisik brought up this BTer async thing And I do not plan to implement BTer so I don't care about the behaviour that much. have
some thoughts on my own, but that is just speculating how they implemented their API.

PS: lost 3hours I could spend in coding 

[deisik]

And maybe I pushed the discussion to the wrong side: the client-side.
For me the more interesting part is the BTer server-side
and the behaviour @deisik described. Still thinking about what he said

In my opinion, asynchronous api is much better overall

Though it may be harder to code since you have to keep track of the requests that you made internally. As I said, I worked with proprietary api's, and they specifically distinguish between asynchronous and synchronous modes of operation in the sense I mean it (that's basically how such modes are defined). Below is the relevant part which I copied from the developer's guide of one them (namely, quik trading api):

Cyщecтвyeт двa cпocoбa пepeдaчи тpaнзaкций – cинxpoнный и acинxpoнный, кoтopыe peaлизyютcя oтдeльными фyнкциями:
1. Пpи cинxpoннoй пepeдaчe тpaнзaкций выxoд из фyнкции ocyщecтвляeтcя тoлькo пocлe пoлyчeния oтвeтa oт cepвepa QUIK. Cлeдoвaтeльнo, oтпpaвлять cинxpoнныe тpaнзaкции мoжнo тoлькo пocлeдoвaтeльнo, дoжидaяcь oтвeтa o кaждoй oтпpaвлeннoй тpaнзaкции – этoт cпocoб пpoщe и бoлee пoдxoдит для пpoгpaммиcтoв c нeбoльшим oпытoм paзpaбoтки пpoгpaмм.
2. Пpи acинxpoннoй пepeдaчe тpaнзaкций выxoд из фyнкции ocyщecтвляeтcя cpaзy жe. Для пoлyчeния oтвeтa oб oтпpaвлeнныx acинxpoнныx тpaнзaкцияx иcпoльзyeтcя фyнкция oбpaтнoгo вызoвa (callback). Фyнкция вызывaeтcя пpи кaждoм пoлyчeнии oтвeтa oб иcпoлнeннoй или oтвepгнyтoй тpaнзaкции. Знaчeния cтaтycoв иcпoлнeния тpaнзaкции, кoтopыe вoзвpaщaютcя в фyнкции oбpaтнoгo вызoвa, пpивeдeны в oпиcaнии к тeм жe cтaтycaм, кoтopыe иcпoльзyютcя пpи oтпpaвкe тpaнзaкций чepeз фaйл

You can use Google Translate, but this piece essentially says that there are two modes of sending transactions, i.e. synchronous and asynchronous. In the former case (synchronous mode), the calling function waits for the reply from the server, while in the latter case (asynchronous mode), the function exits immediately and the reply is received through a callback function. The emphasized part repeats what I said before, i.e. synchronous transactions can be sent only sequentially after receiving a reply from the previous request. As I remember, the guide specifically mentions that if you try to send another synchronous transaction (say, from another thread) while still waiting for a reply, the behavior is undefined. I tried that, and it just crashed the dll library which was used for transacting (trans2quik.dll if you are curious)

[carap]

Wow, what a discussion

Our bots are generally made of two parts:
1. The trading part - the one that is responsible for placing orders and check their statuses is purely synchronous. We are sending request and waiting for the response, we analyze this response to make sure there is no error and everything run smoothly. Yes this approach is slower, but anyway our bots are not the high frequency ones. At the same time we are not generating a load of requests and we never exceed any limits, even when we trade multiple pairs.

2. The data-gathering part  - this one collects data to be used for analysis, it runs on the public API, so we are free to do what we like, multiple simultaneous threads collecting info about exchange rates of different pairs, collect data about order book, etc.

Regarding the 1st part - I honestly wouldn't feel safe by making it in asynchronous way. You can call me (and my team members) the lame programmers for that, we are not going to change our approach. It could be all about our skills, we are sure we can make it safe in totally synced manner, analyzing responses seems crucially important to us. On top of that, this seems to be working on all markets we working with and we see no practical reasons for making it differently.

Regarding the 2nd part - Oh yes, here we can be really nasty Paralleling requests, using multiple servers (and VPNs) to overcome imposed limits. Anyway we normally have 1-2 instances of the data-gathering for each market, while sometimes we have more than 200 separate instances of the trading part (different strategies / parameters / trading pairs / accounts).

My respect to Sampey, he managed to cover 18 (20 soon) different markets, we are not that advanced, we currently have only 5 and various little differences between them is already a significant headache.

[emezh10]

I thought it would be interesting to know if you guys use a bot to trade bitcoin or altcoins..

Whether or not you do, if you were to make one, what rules would it based on?

I don't use one myself.. but am looking to get one built, and would welcome any suggestions on features and rules that you think would work..

Thanks!

It more interesting if you don't use bots , In trading you will one who see the trading and it more thrill when you're the one who gonna buy and sell the coins as of now i do small trades for altcoins and ico's but if im giving a chance to create bot i probably build one your just making a big mistake making human tobecome more lazier

I don't consider to use bot in trading. Personally i don't use bot i manage my tradings all by my self in my own time . I handle it and manage it will all my best. And in fact many site don't allow us to use bot cause it's like cheating. Aside for that it's much better to work to earn . And if you have goals it much better if you are the who will pursue not just the bot you will use. Don't be lazy ! Work hard to earn

[Sampey]

My respect to Sampey, he managed to cover 18 (20 soon) different markets, we are not that advanced, we currently have only 5 and various little differences between them is already a significant headache.

Thank you so much man 

Real Api Number is 24 

18 In current Version
2 (Bitex/Liqui) in the coming version
3 Dismissed due to Exchange Close (Cryptsy/Cryptomic/RawX)
1 Never Go Live due to Exchange Scam (Mintpal)

 

Don't forget that the trick to interface so many exchanges is that you must create a layer

MAIN PROGRAM <-> API LAYER -> External Api Sistem

When

MAIN PROGRAM doesn't know anything about the API LAYER. Nothing in specific. Only the Interface the API Layer must respect.
Then you will be able to write the API LAYER For any exchange without many difficulties.

[deisik]

Wow, what a discussion

Our bots are generally made of two parts:
1. The trading part - the one that is responsible for placing orders and check their statuses is purely synchronous. We are sending request and waiting for the response, we analyze this response to make sure there is no error and everything run smoothly. Yes this approach is slower, but anyway our bots are not the high frequency ones. At the same time we are not generating a load of requests and we never exceed any limits, even when we trade multiple pairs

I could easily mention a few cases when this approach can lead to losses

Or less profits, for that matter. Imagine a situation where your bot is placing a new order, and at the same time it analyzes in parallel data streams (which you yourself mention). Then it sees that the market quickly changes, but it has to wait until this order gets placed (or does something else which requires waiting), and thus not being able to do anything with other orders (at the same exchange) may cost dear. For example, it sees that it is profitable to buy or sell some other coin at the exchange due to an arbitrage opportunity springing up that lasts only a few moments (just enough to place another order). These orders may not even be related to each other. If the bot were running asynchronously (and the exchange accepted asynchronous requests), it could send a new order immediately and grab the opportunity. You see, 1 second may be an eternity within which fortunes can be made as well as lost

[Sampey]

I could easily mention a few cases where this approach can lead to losses

Or less profits, for that matter. Imagine a situation when your bot runs into a situation where it is placing a new order, and at the same time analyzes in parallel data streams (which you yourself mention). Then it sees that the market quickly changes, but it has to wait until this order gets placed, and thus not being able to do anything with other orders (at the same exchange) may cost dear. For example, it sees that it is profitable to buy or sell some other coin at the exchange due to an arbitrage window that lasts only a few moments (just enough to place another order). If it were running asynchronously (and the exchange accepted asynchronous requests), and it could send a new order immediately. You see, 1 second may be an eternity within which fortunes can be earned as well as lost

The sync api call will be performed in the parallel data stream.
Sync doesnt means that your sistem must wait another API Call to be completed. Sync means that when the http protocol is invoked your program must wait for the response.

I have customers who heavily trade at the same time on over 100 markets.
Don't you really think that only 1 call is performed by my bot using a FIFO queue.

Solutions is simple : 1 Open Market = 1 GUI + 1 (or more) algorithm and 1 Algorithm = 1 or more dedicated thread. All call are sync at http level.

You example is true, but has nothing to do with sync/async call 

[deisik]

@deisik brought up this BTer async thing And I do not plan to implement BTer so I don't care about the behaviour that much. have
some thoughts on my own, but that is just speculating how they implemented their API.

PS: lost 3hours I could spend in coding 

Just think that it may not have been worth doing at all

And this is real life. I spent literally months on doing some very intricate stuff at the max of my mental capacities (and sometimes beyond that) which I have never used afterwards. I would certainly change these months of non-stop coding and thinking for just one day of reading sense which would have prevented me from basically wasting my time



"Give me six hours to chop down a tree and I will spend the first four sharpening the axe"

[carap]

Don't forget that the trick to interface so many exchanges is that you must create a layer

MAIN PROGRAM <-> API LAYER -> External Api Sistem

When

MAIN PROGRAM doesn't know anything about the API LAYER. Nothing in specific. Only the Interface the API Layer must respect.

Sure, we already have it this way But it takes time to come up with universal approach to cover all the little differences. For example, one exchange returns the list of executed orders from any desired day, another one returns only one month-back and for the lengthy strategies you have to sync your history. Little differences with various limits: minimum order sizes, minimum-maximum prices like on btc-e, number of digits after the comma, commission (absolute value versus percent), handling of partially executed orders... Anyway, we have resolved most of these issues and we made something really stable and universal.

But 24APIs - you are a Maaaaan! Honestly, I praise your patience!
Have you ever looked outside the crypto-trading? Like commodities, options?

I could easily mention a few cases when this approach can lead to losses

Or less profits, for that matter. Imagine a situation where your bot is placing a new order, and at the same time it analyzes in parallel data streams (which you yourself mention). Then it sees that the market quickly changes, but it has to wait until this order gets placed (or does something else which requires waiting), and thus not being able to do anything with other orders (at the same exchange) may cost dear. For example, it sees that it is profitable to buy or sell some other coin at the exchange due to an arbitrage opportunity springing up that lasts only a few moments (just enough to place another order). These orders may not even be related to each other. If the bot were running asynchronously (and the exchange accepted asynchronous requests), it could send a new order immediately and grab the opportunity. You see, 1 second may be an eternity within which fortunes can be made as well as lost

It's all about implemented trading strategy! In our case speed is not of the great importance.
Depending on settings (mainly risk and greed), trading cycles on some pairs may take minutes, while others may last for weeks.
We have a gentleman-bot who walks less because he plans better You know, its like comparing a machine gun against a sniper rifle.

[Sampey]

But 24APIs - you are a Maaaaan! Honestly, I praise your patience!
Have you ever looked outside the crypto-trading? Like commodities, options?

No only cryptoworld 
I haven't much time to look into other things, real life is full 

[veleten]

ha this thread happened to be very informative (unexpectedly)
thanks go to both deisik and campey and of course others who contributed to the discussion
a question to sampey,is it possible for a person without any experience in trading to earn anything with your bot?

[deisik]

It's all about implemented trading strategy! In our case speed is not of the great importance.
Depending on settings (mainly risk and greed), trading cycles on some pairs may take minutes, while others may last for weeks.
We have a gentleman-bot who walks less because he plans better You know, its like comparing a machine gun against a sniper rifle

Excuse me but that doesn't make much sense

At first you say that you go all out to get the smallest chunks of data possible querying an exchange a few (hundred) times a second using whatever tools are available (e.g. multiple servers and VPNs to overcome imposed limits), and then you continue to say that trading cycles take minutes to weeks and your bots are not the high frequency ones anyway



It is like saying that you opt for a sniper rifle while actually taking a machine gun belt as ammo

[carap]

At first you say that you go all out to get the smallest chunks of data possible querying an exchange a few (hundred) times a second using whatever tools are available (e.g. multiple servers and VPNs to overcome imposed limits), and then you continue to say that trading cycles take minutes to weeks and your bots are not the high frequency ones anyway

Have you noticed that I mentioned our bot is made of two major parts?

Our strategy is not the high frequency one. Trading is very careful and targeted. Part 1 of the bot. We may check market like once every minute or even less often (depending on parameters).

Data-gathering is a 2nd part, it doesn't use private API and that one may work as a machine gun making lots of requests. However, that behavior is not always necessary (depends on a number of pairs, on market itself, on additional strategy adjustments). At the present moment we are working well within allowed limits.