[ANN] MarginBot - A Bitfinex Margin Lending Management Bot

[ASutas]

Guys, can we keep this thread focused on MarginBot please?  I don't want people getting confused about what script your discussing.  If you want to discuss or need support on the other script, please start a thread for it.

Totally agree! Sorry for hijacking

[1714071426]

1714071426

[1714071426]

1714071426

[1714071426]

1714071426

[1714071426]

1714071426

[1714071426]

1714071426

[Amph]

cool , do you have enough for another job based on php? i need a programmer

[HowardF]

cool , do you have enough for another job based on php? i need a programmer

Absolutely.  I'm a freelance programmer.  Contact me at howard@fuckedgox.com and I'll be happy to discuss any projects you're working on.

[PRab]

Can I have a little more help setting this up? I have ubuntu server installed with the LAMP stack. I extracted MarginBot to /var/www/html, and went to the webpage. Everything is looking good and I tried to run through the setup procedure. I got a little hung up because it appears that you have to actually create the DB in mysql before the setup will build the tables and such. Everything is still looking good, so I log in to MarginBot using the username/password I setup.

It is now complaining that the the cron scripts haven't run. I waited at least 10 minutes, but it was still showing the same error. Is there anything special I have to do to allow MarginBot "Access to add a cronjob"?

[HowardF]

It is now complaining that the the cron scripts haven't run. I waited at least 10 minutes, but it was still showing the same error. Is there anything special I have to do to allow MarginBot "Access to add a cronjob"?

You'll need to set up the crons yourself, unfortunately MarginBot can't do that for you.  If you don't know how to set up a cronjob,  i'd take a look at the Ubuntu docs for your specific distribution version.

This also gives a good intro to cron:

https://help.ubuntu.com/community/CronHowto

For MarginBot you'll want to add 2 lines, one to run the 10 minute cron, and one to run the hourly cron.  It should give you the details on the exact lines to add in the error when you log in to MarginBot.   Something like:

5 * * * * wget -qO- http://yoururl.com/HourlyCron.php >/dev/null 2>&1

and

01,11,21,31,41,51 * * * * wget -qO- http://yoururl.com/TenMinuteCron.php >/dev/null 2>&1

[PRab]

I added those lines (from index.php) to my crontab, waited 10 minutes, but still nothing. Does that cron job have to run for a specific user? Also, if I just wget that url, would that be the equivalent of the cron job running? I tried it, but wget just returned an empty file. (Sorry if these questions are stupid, I'm a programmer, but I don't use Linux often and have never used php)

Another observation, after the 2nd page of setup, it just left me at a blank screen. Is that correct?

[HowardF]

I added those lines (from index.php) to my crontab, waited 10 minutes, but still nothing. Does that cron job have to run for a specific user? Also, if I just wget that url, would that be the equivalent of the cron job running? I tried it, but wget just returned an empty file. (Sorry if these questions are stupid, I'm a programmer, but I don't use Linux often and have never used php)

If you run wget manually it should get rid of the warning for a little while, but the point of the cron is to run it every 10 minutes, and I'm assuming you wouldn't want to have to run it manually every 10 minutes...

Another observation, after the 2nd page of setup, it just left me at a blank screen. Is that correct?

No, this is definitely not normal.  It sounds like something didn't work right.  When you load the index or MarginBot, does it show your balance correctly?  Do you see any data in the database tables it created?  Try opening the cron urls in a web browser as well, if something isn't working there it should give you an error message of some kind.

[PRab]

I see 4 tables:
BFXLendBot_CronRuns
BFXLendBot_Tracking
BFXLendBot_Users
BFXLendBot_Vars

All of the tables appear to be empty.

When I open the cron urls in a web browser, it just displays an empty page.

Edit: Missed answering one of your questions. I do not see my balance anywhere. First it shows me a login page. When I login, it gives an error "Your Username or Email was not found! Please Try Again." and shows what looks like the headers to a table.

[HowardF]

I see 4 tables:
BFXLendBot_CronRuns
BFXLendBot_Tracking
BFXLendBot_Users
BFXLendBot_Vars

All of the tables appear to be empty.

When I open the cron urls in a web browser, it just displays an empty page.

Edit: Missed answering one of your questions. I do not see my balance anywhere. First it shows me a login page. When I login, it gives an error "Your Username or Email was not found! Please Try Again." and shows what looks like the headers to a table.

something didnt work during install.  I'm guessing maybe your config file wasnt writeable, or else you entered something about your database incorrectly...  Try going through the install.php process again and see what happens

[jabetizo]

Another observation, after the 2nd page of setup, it just left me at a blank screen. Is that correct?

Does your PHP installation have curl enabled?

I'm having trouble installing... Step 1 and 2 go through smoothly but after clicking "Complete Install", I just get a blank page.

You don't seem to have curl support enabled on your PHP install, which is typically standard.  CURL is required to use the bitfinex API.

See http://php.net/manual/en/curl.installation.php for help compiling with CURL

[PRab]

Looks like Ubuntu doesn't install php5-curl by default. Running

Code:

sudo apt-get install php5-curl

and restarting the webserver fixed it. Thanks for all the help!

Edit: After installing php5-curl, I had to drop all the tables and rerun the install script.

[PRab]

Is there a way to make MarginBot also lend out my BTC balance? All I see are options for USD. Right not I have my money stored in Bitfinex split between both USD and BTC because I'm not sure which direction the market is going to go.

[HowardF]

Is there a way to make MarginBot also lend out my BTC balance? All I see are options for USD. Right not I have my money stored in Bitfinex split between both USD and BTC because I'm not sure which direction the market is going to go.

Currently it only supports USD.  In a future update I may add BTC and LTC back in, but when the bot was written BTC/LTC lending rates were so low it wasn't worth lending them (and the rates are quickly slipping back into the not worth lending area again.....).

[nrd525]

Looks like a great idea and a good implementation. I'm a php/mysql developer, though not a security expert - so a lot of my security concerns may be totally wrong and are partially for my own education.  Some comments:

BTC lending is becoming a bigger deal.  Who knows if it will last.

Hourlycron.php and TenMinuteCron.php - are mostly identical and could be refactored.

What's the risk of storing a plain text bitfinex api key in a mysql database?

Is the fact that PasswordHash.php function: crypt_private() uses md(5) a security issue?

Do you have any plans to support leap years? Just kidding =)

It's really a feature that BFX should offer - but it might be nice to have multiple auto-lend rates.  So you could set aside $1000 at 0.03%, $1000 at 0.04%, and so on.  I'm guessing this bot could be customized to do this.  It's similar to your high hold concept, but a bit more elaborate.

Is any kind of collision attack possible using the install function?  As it will tell if you a BFX key is already used.

Are there any places you might want to use strip_tags() to exclude javascript (or other) attacks?

[HowardF]

Looks like a great idea and a good implementation. I'm a php/mysql developer, though not a security expert - so a lot of my security concerns may be totally wrong and are partially for my own education.  Some comments:

BTC lending is becoming a bigger deal.  Who knows if it will last.

Hourlycron.php and TenMinuteCron.php - are mostly identical and could be refactored.

probably could, but they work fine as is...  when they were originally coded, for my own internal use, there were reasons they were seperated out.  most of those reasons didn't make it to the open source code.

What's the risk of storing a plain text bitfinex api key in a mysql database?

Low, but not non-existant.  api keys don't allow removal or any other direct access to currency on bitfinex, only orders and balance checks basically.  So someone could potentially set a low order or something along those lines with your key, but they couldn't withdraw any currency for example.  One way encryption isn't feasible since I need to pass the key, and I'm generally of the opinion that simple encrpytion isn't especially useful if someone is able to get access to your DB, since they probably have access to the code that contains the encryption salt at that point as well.

Is the fact that PasswordHash.php function: crypt_private() uses md(5) a security issue?

It doesn't.  thats just a fallback that shouldn't come into play.  Read more about the framework here:
http://www.openwall.com/phpass/

Do you have any plans to support leap years? Just kidding =)
It's really a feature that BFX should offer - but it might be nice to have multiple auto-lend rates.  So you could set aside $1000 at 0.03%, $1000 at 0.04%, and so on.  I'm guessing this bot could be customized to do this.  It's similar to your high hold concept, but a bit more elaborate.

I'm available for hire  ;-P

Is any kind of collision attack possible using the install function?  As it will tell if you a BFX key is already used.

Thus the "Delete install.php" nag when its installed.


As I state in the install direction, people should really consider security as they see fit when installing this app. I consider it pretty low risk, since the API is pretty limited at bitfinex, but on my own server, I have several extra layers of protection, as I would any financial app.

[nrd525]

Thanks for all your responses!  Also, please reply to my PM which found something that I didn't want to share publicly in case it was a real bug.

How is the bot performing in the recent upsurge in rates? I've been feeling that the USD rates would increase for a week or two - and just now they're hitting 0.07%/day.   Unfortunately I gave up waiting and have most of my money lent out at 0.0288%/day for the next 5 days.

We're probably in another spike and then slow decline phase.

I've been lending out USD swap on BFX since Jan or Feb 2013.  Got around 100% in 2013 which was pretty crazy, and 40% in 2014.  This year, I'm guessing that I might make as little as 10%

[HowardF]

Thanks for all your responses!  Also, please reply to my PM which found something that I didn't want to share publicly in case it was a real bug.

I'll reply tomorrow when I've had a few less beers, since i just came back from a Mardi Gras party, but suffice it to say, I don't particularly consider what you found a risk.  The strings are always escaped before actual use in a database.  

EDIT:  I feel like I should elaborate here for anyone now concerned about this.  The userid string is set to a session variable here:
accounts.php line 46:   $this->userid = $_SESSION['userid'];

He was concerned that I pass this string to the bitfinex object without escaping it later on, which is true, but every time I use the session object, or the passed setter variable in the bitfinex class, I ALWAYS escape it before passing it to the database...  so, there is no real security concern here.  This is verifiable throughout the bitfinex class, on lines:
132, 150, and 363  (the only places the userid variable is ever passed to a database in the bitfinex class).  could it have been escaped earlier?  yes.  but i prefer to always escape when doing a sql query, not before.

How is the bot performing in the recent upsurge in rates? I've been feeling that the USD rates would increase for a week or two - and just now they're hitting 0.07%/day.   Unfortunately I gave up waiting and have most of my money lent out at 0.0288%/day for the next 5 days.

We're probably in another spike and then slow decline phase.

the latest results on a very basic setup are always available here, as mentioned earlier in this thread:
http://fuckedgox.com/MBot/

Yesterdays return was %0.040010, but i imagine todays will be better (current estimate is 0.05790%).  Automation bots like margin bot are very good at splitting your total investment across many small loans, so you always have at least some money ready to catch waves like we've had the last few days.  On my larger account the last few days return is even higher.

I've been lending out USD swap on BFX since Jan or Feb 2013.  Got around 100% in 2013 which was pretty crazy, and 40% in 2014.  This year, I'm guessing that I might make as little as 10%

I started in mid 2013, but I've been running the bot (in a very early form) since Jan 14.  in 2014 I made .1277% / day averaged over the year (About 50% for the year).  earlier in this thread you can see most of my returns posted in csv format, though i was doing other things with the account during part of that time, so its not 100% accurate.

[Sukrim]

I recently updated my spreadsheet (since now the FRR wall has been broken a few times) to compare your bot returns with 30 day auto-FRR:

The statement on your website ("* FRR Return Difference Assumes a 70% Efficiency for time money spends lent out. This is actually MUCH higher than most users get using the FRR, but seems like a reasonable comparison") is completely wrong, here are actual differences between your retuns and mine in percent ( -(100%-my_returns/your_returns) ) since you report them on your page:

Code:

01.12.2014	-7,443460%
02.12.2014	-10,054463%
03.12.2014	-15,504663%
04.12.2014	-16,924298%
05.12.2014	-10,553673%
06.12.2014	-27,546879%
07.12.2014	-29,276524%
08.12.2014	-33,408200%
09.12.2014	-33,379851%
10.12.2014	-25,599959%
11.12.2014	-11,808594%
12.12.2014	-11,556969%
13.12.2014	-11,719647%
14.12.2014	-11,798841%
15.12.2014	-9,980899%
16.12.2014	2,099330%
17.12.2014	1,963954%
18.12.2014	1,618318%
19.12.2014	9,932038%
20.12.2014	10,659478%
21.12.2014	17,783933%
22.12.2014	19,632251%
23.12.2014	-0,274929%
24.12.2014	36,185919%
25.12.2014	47,920128%
26.12.2014	21,771841%
27.12.2014	33,091841%
28.12.2014	24,557408%
29.12.2014	42,748676%
30.12.2014	22,251068%
31.12.2014	22,583313%
01.01.2015	19,038134%
02.01.2015	16,775189%
03.01.2015	18,753871%
04.01.2015	20,981465%
05.01.2015	10,475503%
06.01.2015	-4,716642%
07.01.2015	-9,402876%
08.01.2015	2,937789%
09.01.2015	1,264892%
10.01.2015	-7,299879%
11.01.2015	4,262874%
12.01.2015	6,635660%
13.01.2015	7,094768%
14.01.2015	3,554590%
15.01.2015	12,009196%
16.01.2015	8,244013%
17.01.2015	0,757309%
18.01.2015	1,362987%
19.01.2015	0,743891%
20.01.2015	-3,301981%
21.01.2015	3,499526%
22.01.2015	-3,648495%
23.01.2015	-17,951652%
24.01.2015	-3,792983%
25.01.2015	1,869511%
26.01.2015	4,668778%
27.01.2015	1,901741%
28.01.2015	3,410024%
29.01.2015	5,961179%
30.01.2015	-0,522030%
31.01.2015	-6,395268%
01.02.2015	-4,071584%
02.02.2015	-2,634039%
03.02.2015	0,237169%
04.02.2015	-3,479075%
05.02.2015	-7,313430%
06.02.2015	-11,208395%
07.02.2015	-5,065939%
08.02.2015	3,776792%
09.02.2015	1,576461%
10.02.2015	-5,731081%
11.02.2015	-9,831616%
12.02.2015	-7,923571%
13.02.2015	4,206058%
14.02.2015	1,688207%
15.02.2015	-5,998144%
16.02.2015	-25,304583%
17.02.2015	-18,174385%
18.02.2015	-9,773438%
19.02.2015	-3,527300%
20.02.2015	-4,175197%
21.02.2015	-5,676997%
22.02.2015	-4,951486%
23.02.2015	-10,979872%
24.02.2015	-16,248698%
25.02.2015	-25,884357%
26.02.2015	-19,517822%
27.02.2015	-14,240242%
28.02.2015	-31,930358%
01.03.2015	-36,312944%
02.03.2015	-30,285468%
03.03.2015	-17,201920%

-30% for example means that I earned 30% less that day in swaps than your bot did, +30% means I earned more than your bot. While FRR seems to definitely underperform if the FRR wall is broken (like recently), in a downtrend the FRR seems to be actually the better choice (last week of December) or at least not much different from your bot (January until mid-February).

I'd recommend a less aggressive strategy when the rates are trending down (FRR falls slower than a bot undercutting it) but it seems to be great in situations like right now where FRR might be a loss. To determine the strategy, you might want to sample and store the currently lowest rate on the books as well as the current FRR rate every 10 minutes, then calculate the return of the last 24 hours with each strategy, and either place FRR bids (FRR wall existing, FRR returns of the last 24h were above lowest rate returns) or undercut the current best offer (no FRR wall, lowest rate return of the last 24h was above FRR return). Maybe with an optional setting to adjust the time window (24 hours, 72 hours, 1 hour...)

[najzenmajsen]

cool , do you have enough for another job based on php? i need a programmer

Absolutely.  I'm a freelance programmer.  Contact me at howard@fuckedgox.com and I'll be happy to discuss any projects you're working on.

That was one hell of a proffesional email adress you have there seriously tho , i think i might be contacting you into near future if that's a real email i need a php aswell : >

[HowardF]

I recently updated my spreadsheet (since now the FRR wall has been broken a few times) to compare your bot returns with 30 day auto-FRR:

The statement on your website ("* FRR Return Difference Assumes a 70% Efficiency for time money spends lent out. This is actually MUCH higher than most users get using the FRR, but seems like a reasonable comparison") is completely wrong, here are actual differences between your retuns and mine in percent ( -(100%-my_returns/your_returns) ) since you report them on your page:

-30% for example means that I earned 30% less that day in swaps than your bot did, +30% means I earned more than your bot. While FRR seems to definitely underperform if the FRR wall is broken (like recently), in a downtrend the FRR seems to be actually the better choice (last week of December) or at least not much different from your bot (January until mid-February).

This is the information I got from a VERY reliable source.     If you're doing better great, but I trust the information to be accurate, at least as an average, which is my primary concern, since most people using the bot will be concerned about beating the average.  ( I think the main difference is you do 30 Day FRR Lends, which very few other people do.  Most people don't seem to want their money tied up for a full 30 days, unless they are getting exceptional rates. )

I'd recommend a less aggressive strategy when the rates are trending down (FRR falls slower than a bot undercutting it) but it seems to be great in situations like right now where FRR might be a loss. To determine the strategy, you might want to sample and store the currently lowest rate on the books as well as the current FRR rate every 10 minutes, then calculate the return of the last 24 hours with each strategy, and either place FRR bids (FRR wall existing, FRR returns of the last 24h were above lowest rate returns) or undercut the current best offer (no FRR wall, lowest rate return of the last 24h was above FRR return). Maybe with an optional setting to adjust the time window (24 hours, 72 hours, 1 hour...)

Theres always room for improvement in my overall strategy, but as a basic out of the box bot, I'm fairly sure MarginBot is currently unbeaten in returns (including annualized against even the best FRR users).  There is a new version which will be released soon which will make a few tweaks to the core engine.  In my internal testing, its performing about 3-5% better than the current version of MarginBot.  Not a huge difference, but certainly worthwhile.

One thing I will say, my bots will never support the FRR.  It would be completely counter intuitive to the entire point of this bot, which is to lend money quickly and catch uptrends.  The only way to get a reasonable return in FRR is to keep the money lent out, and the only way to do that is with 30 day loans.  2 day FRR loans sit for a very long time, especially in down markets, making no returns at all.  And 30 day loans would miss any rate rises, and significantly reduce annual returns.  As it stands, those FRR wall breaks and rapid rate rises more than make up for any slightly lower days MarginBot has.  It only takes a few days where rates jump to .45% to beat months of FRR being .005% higher than MarginBot rates.

Besides, as a general rule, I have serious issues with the manipulative concept of an "Averaged" open market.