[ANN] MarginBot - A Bitfinex Margin Lending Management Bot

[BeastOfBodmin]

Ok, its ready!  MarginBot v0.1.07 is now live on github!  Full Crypto Margin lending!  Currently it supports all currencies that Bitfinex supports for margin.

For a few weeks now, my marginbot setup has not been picking up account history, nor unused funding, but the cron job runs were being added to the BFXLendBot_CronRuns table.

I spent some time on it yesterday, and didn't get any ideas as to what might be wrong. Not least because my php binary will not execute a .php file.

But the problem shows up as a lack of history on http://.../marginbot/index.php

and a lack of entries in the table BFXLendBot_Tracking. The last entry was 2017-08-19.

So I noticed this morning 1.07b was available on the master branch.

I followed the update instructions, and the following occurred:

A new table BFXLendBot_CurPairs is created, and the 4 existing tables are backed up to copies with the suffix _BACKUP_v17

But I get this error on the screen http://.../marginbot/update.php?doUpdate=2

THERE HAS BEEN A DATABASE ERROR
Please Contact Support If this error persists.
1264 - Out of range value for column 'dep_balance' at row 7

I am using

5.7.16
MySQL Community Server (GPL)

Poking around the MySQL tables, it seems the table definitions are still the same: decimal(12,2). But I can see some ALTER TABLE statements in update.php, that switch to decimal(12,8).
When I paste the ALTER TABLE for the Tracking table into MySQLWorkbench it thinks there is a syntax error in both MODIFY COLUMN parts of the SQL statement.

Code:

ALTER TABLE `BFXLendBot_Tracking`
ADD COLUMN `trans_cur`  varchar(10) NULL AFTER `user_id`,
MODIFY COLUMN `dep_balance`  decimal(12,8) NULL DEFAULT NULL AFTER `date`,
MODIFY COLUMN `swap_payment`  decimal(12,8) NULL DEFAULT NULL AFTER `dep_balance`,
DROP INDEX `uniquieKeys` ,
ADD UNIQUE INDEX `uniquieKeys` (`user_id`, `trans_id`, `trans_cur`) USING BTREE;

Code:

CREATE TABLE `BFXLendBot_Tracking` (
  `id` int(12) NOT NULL AUTO_INCREMENT,
  `user_id` smallint(4) DEFAULT NULL,
  `trans_id` int(12) DEFAULT NULL,
  `date` date DEFAULT NULL,
  `dep_balance` decimal(12,2) DEFAULT NULL,
  `swap_payment` decimal(12,2) DEFAULT NULL,
  `average_return` decimal(8,6) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `uniquieKeys` (`user_id`,`trans_id`)
) ENGINE=MyISAM AUTO_INCREMENT=171 DEFAULT CHARSET=latin1;

CREATE TABLE `BFXLendBot_Tracking_BACKUP_v17` (
  `id` int(12) NOT NULL AUTO_INCREMENT,
  `user_id` smallint(4) DEFAULT NULL,
  `trans_id` int(12) DEFAULT NULL,
  `date` date DEFAULT NULL,
  `dep_balance` decimal(12,2) DEFAULT NULL,
  `swap_payment` decimal(12,2) DEFAULT NULL,
  `average_return` decimal(8,6) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `uniquieKeys` (`user_id`,`trans_id`)
) ENGINE=MyISAM AUTO_INCREMENT=171 DEFAULT CHARSET=latin1;

I am also wondering it it makes more sense to use decimal(16,8), as that way you can still support numbers up to 999,999,999. decimal(12,8) only goes up to 9,999.

[HowardF3]

Ok, its ready!  MarginBot v0.1.07 is now live on github!  Full Crypto Margin lending!  Currently it supports all currencies that Bitfinex supports for margin.

For a few weeks now, my marginbot setup has not been picking up account history, nor unused funding, but the cron job runs were being added to the BFXLendBot_CronRuns table.

I spent some time on it yesterday, and didn't get any ideas as to what might be wrong. Not least because my php binary will not execute a .php file.

But the problem shows up as a lack of history on http://.../marginbot/index.php

and a lack of entries in the table BFXLendBot_Tracking. The last entry was 2017-08-19.

So I noticed this morning 1.07b was available on the master branch.

I followed the update instructions, and the following occurred:

A new table BFXLendBot_CurPairs is created, and the 4 existing tables are backed up to copies with the suffix _BACKUP_v17

But I get this error on the screen http://.../marginbot/update.php?doUpdate=2

THERE HAS BEEN A DATABASE ERROR
Please Contact Support If this error persists.
1264 - Out of range value for column 'dep_balance' at row 7

I am using

5.7.16
MySQL Community Server (GPL)

Poking around the MySQL tables, it seems the table definitions are still the same: decimal(12,2). But I can see some ALTER TABLE statements in update.php, that switch to decimal(12,8).
When I paste the ALTER TABLE for the Tracking table into MySQLWorkbench it thinks there is a syntax error in both MODIFY COLUMN parts of the SQL statement.


I am also wondering it it makes more sense to use decimal(16,8), as that way you can still support numbers up to 999,999,999. decimal(12,8) only goes up to 9,999.

Aye, good catch, that should definitely be at least decimal(16,8).   I was building the updater on a relatively new install that didn't have much data in it, and 12,8 worked for me (only a few thousand $ in the test account), but it will fail for anyone with > $9,999 in their account.  I will update this in github.

[HowardF3]

Just updated github to 1.07c.  Fixed the above update.php issue (i used decimal(18,8), which should cover even the largest of investors), and made a quick fix to balances updating after loans are canceled ( Thanks jbhuang ! ).

The only 3 files that changed from v1.07b are:
/update.php
/inc/version_info.php
/inc/ExchangeAPIs/bitfinex.php

if updating from that version, you should only need to copy over those three files.  If updating from an older version, or doing a new install, follow the standard procedures.

[BeastOfBodmin]

Any chance of adding Litecoin (LTC)?

It's fantastic that you've generalised to more currencies.

I was seriously thinking about hacking your bot myself to get multi-currency support in.

[prototype_nsx]

Right, when I try to run from the browser the page appears messy like the previous screenshot, i am not sure if you can see that?

http://imgur.com/a/umbny

Hmm.  something is seriously not right with your server setup.  From that screenshot it looks like its dumping raw code rather than executing it, which shouldn't even be possible, unless its not actually being parsed by PHP.  have you been able to correctly run any other PHP code on the server?

I build a new VM and the issue went away so it's all good thanks to you

Can you or anyone recommend settings for 1k usd lending?

Many thanks

[BeastOfBodmin]

Ok, its ready!  MarginBot v0.1.07 is now live on github!  Full Crypto Margin lending!  Currently it supports all currencies that Bitfinex supports for margin.

For a few weeks now, my marginbot setup has not been picking up account history, nor unused funding, but the cron job runs were being added to the BFXLendBot_CronRuns table.

I spent some time on it yesterday, and didn't get any ideas as to what might be wrong. Not least because my php binary will not execute a .php file.

But the problem shows up as a lack of history on http://.../marginbot/index.php

and a lack of entries in the table BFXLendBot_Tracking. The last entry was 2017-08-19.

I solved the lack of updates problem.

I put in a print_r statement that told me my nonce was too small.

That made me think maybe I was reusing some keys (1 iMac, 2 android devices, two BFX accounts).

I was right!

[HowardF3]

I solved the lack of updates problem.

I put in a print_r statement that told me my nonce was too small.

That made me think maybe I was reusing some keys (1 iMac, 2 android devices, two BFX accounts).

I was right!

Yeah, the way the bitfinex API enforces nonce is strange.  I usually just use microtime on just about every other API in the world and it works fine, but on really busy BFX keys, there seems to always be lots of conflicts.  Easiest way around it is to generate a new API key for every device / bot / app.

[HowardF3]

Any chance of adding Litecoin (LTC)?

It's fantastic that you've generalised to more currencies.

I was seriously thinking about hacking your bot myself to get multi-currency support in.

Well that's awkward... Not sure how I managed to miss that one...  I added it to to the newest build, just uploaded to github.  However, for anyone who already has v1.07b or higher, you'll need to manually run a sql query to add it to your database:

Code:

INSERT INTO `[YOURDBPREFIX]CurPairs` VALUES ('13', 'LTC', 'Litecoin', '1');

make sure to change [YOURDBPREFIX] to the correct prefix ( BFXLendBot_ if you used the default setting ).

I also made a few other updates to the newest version just uploaded (1.07e).  Mainly it allows pausing lending for a single coin or currency, rather than pausing the entire account....

[HowardF3]

Also, if Bitfinex starts supporting margin for a new currency in the future, and you don't want to wait for me to add support to it, all you'd need to do is add it to the CurPairs table.  Just need to add the currency code to curSym, the name of the coin to the curName row, and set status to 1 (leave the id blank and it will auto increment).  After that is should be an option in the dropdown menu.  Its also possible to hide currency that you don't care about, just set the status to 0 on any you want to hide.

[prototype_nsx]

Update on my setup I have started with small amounts and my earnings below so far

USD

http://imgur.com/a/l6G47

http://imgur.com/a/6DNSe

BTC

http://imgur.com/a/4e9xY

I have a question, how do I find View Overall Returns for other currencies besides USD?

Thanks again HowardF3

[HowardF3]

Update on my setup I have started with small amounts and my earnings below so far

The earnings are slow, but they add up over time.  Better than just about any other investment you'll find... 

I have a question, how do I find View Overall Returns for other currencies besides USD?

I haven't put it in yet.  Making those charts is a bit of a pain, but I'm working to get it into the next build, and clean up the charts overall...

[prototype_nsx]

Agreed, I find it less stressful than day trading.

No worries, will wait for the next release

Cheers

[BeastOfBodmin]

Update on my setup I have started with small amounts and my earnings below so far

The earnings are slow, but they add up over time.  Better than just about any other investment you'll find... 

I think of it as being akin to selling picks and shovels during a gold rush. Apart from the currency risk (if USD is not your home currency), the only volatility is in the daily interest rate.

You don't lose money, you just make it at different rates.

[HowardF3]

I think of it as being akin to selling picks and shovels during a gold rush. Apart from the currency risk (if USD is not your home currency), the only volatility is in the daily interest rate.

You don't lose money, you just make it at different rates.

Exactly.  I've always said don't be the gold miner, be the store that supplies the gold miner.  Let others take the big risks.  Some will make a fortune, lots more will lose their asses.  But either way, the suppliers (or in this case margin providers) make money.

[HowardF3]

I have a question, how do I find View Overall Returns for other currencies besides USD?

Just posted an update to github, v1.08.  This is a small update that primarily addresses the "View Overall Returns" page.  It is now tabbed to show all available Crypto returns.  I also added a function (at the bottom of this dropdown menu) to import older return History, in case you want to import your older return history from Bitfinex for tracking / stats reasons.

http://therovegroup.com/MBot/MarginBotUpdate1.08.PNG


For the new Import function, just go to View Overall Returns -> Grab Return History From Bitfinex.  Then select the currency you want to import and select the date you want to use as the start date for your historical data.  Make sure to enter the start date in the format of YYYY-MM-DD EXACTLY (ex: 2017-09-19 for the date I'm posting this).  If you enter a long period of time, be prepared for this to take several minutes to run.  While its running, it will probably look like its stuck, but its not.  I ran mine from 2014-01-01 to stress test it and it took about 2 minutes to run, but this could vary a lot base on your server, how busy Bitfinex is, etc.  I'd suggest not going back too much earlier than that, as it may time out the script, but honestly Bitfinex isn't too much older than that, so it shouldn't be an issue...

http://therovegroup.com/MBot/MarginBotUpdate108-2.jpg

To install this update, just back up your /inc/config.php, download the latest version from github, unzip and overwrite the existing files.  Then copy back your /inc/config.php and everything should work.  If you're updating from a version older than 1.07, you should be directed to an update process when you access the site.  If you were already on 1.07d+ you shouldn't need to go through the update process.

[HowardF]

1. First, let me get this out of the way.  I am HowardF, the original creator of MarginBot.  While I was away from this forum, it apparently got hacked, and they locked down all the accounts and sent out a change password thing.  Well, that change password thing isn't working, I can't request a new password from my old account, and I can't figure out any way around it.  So, I created this new account to use from now on.  If there's any concern whether its really me or not, I'm happy to prove it by changing either the old website at fuckedgox.com or I can make a little change in the Github repo or whatever (just tell me what you guys want, and I'll do it).  Also, if anyone has a better contact for an admin here that could reset the HowardF account for me so I can get back in, that would be awesome as well.  I'd love to get access back to that account, would save some trouble.

So, I finally got in touch with an admin here and was able to get the original account back!   Looks like I'll be posting from this account again, and this will allow me to update the original post when things get updated around here.  I'm going to post all the latest updates to that original message now.

[HowardF]

It seems there are too many people interested in giving borrow their bitcoin and not enough people interested in doing business with it. okey

Bitcoin margin lending rates have always been MUCH lower than USD.  All crypto for that matter has much lower rates. I was in a fairly long discussion about crypto vs USD lending several years ago, in which I advocated always converting crypto to USD to lend ( https://bitcointalk.org/index.php?topic=229438.msg9739115#msg9739115 ), but with prices where they are right now, turns out depending on when you got your BTC, I may have been wrong in my analysis ( honestly didn't see $4k bitcoin coming this quickly.... ).  That said, I still believe it is much safer and a far more reliable investment to lend USD.  Lending USD gives MUCH higher returns than crypto, and its fairly reliable, daily income.

But, I also do have some coin that I hold, just in case.  And I know a lot of other people do as well.  And having that coin sitting there doing nothing doesn't really make sense to me, so I added back in crypto lending.  Getting 0.01% daily returns on those coins is better than nothing, and the bot can still focus on my USD returns, where my real income is.

[prototype_nsx]

I have a question, how do I find View Overall Returns for other currencies besides USD?

Just posted an update to github, v1.08.  This is a small update that primarily addresses the "View Overall Returns" page.  It is now tabbed to show all available Crypto returns.  I also added a function (at the bottom of this dropdown menu) to import older return History, in case you want to import your older return history from Bitfinex for tracking / stats reasons.

http://therovegroup.com/MBot/MarginBotUpdate1.08.PNG


For the new Import function, just go to View Overall Returns -> Grab Return History From Bitfinex.  Then select the currency you want to import and select the date you want to use as the start date for your historical data.  Make sure to enter the start date in the format of YYYY-MM-DD EXACTLY (ex: 2017-09-19 for the date I'm posting this).  If you enter a long period of time, be prepared for this to take several minutes to run.  While its running, it will probably look like its stuck, but its not.  I ran mine from 2014-01-01 to stress test it and it took about 2 minutes to run, but this could vary a lot base on your server, how busy Bitfinex is, etc.  I'd suggest not going back too much earlier than that, as it may time out the script, but honestly Bitfinex isn't too much older than that, so it shouldn't be an issue...

http://therovegroup.com/MBot/MarginBotUpdate108-2.jpg

To install this update, just back up your /inc/config.php, download the latest version from github, unzip and overwrite the existing files.  Then copy back your /inc/config.php and everything should work.  If you're updating from a version older than 1.07, you should be directed to an update process when you access the site.  If you were already on 1.07d+ you shouldn't need to go through the update process.

EDIT

Just checked again and the charts are loading fine

BTC
https://imgur.com/a/qPQZh

USD

https://imgur.com/a/SII6w

[prototype_nsx]

It seems there are too many people interested in giving borrow their bitcoin and not enough people interested in doing business with it. okey

Bitcoin margin lending rates have always been MUCH lower than USD.  All crypto for that matter has much lower rates. I was in a fairly long discussion about crypto vs USD lending several years ago, in which I advocated always converting crypto to USD to lend ( https://bitcointalk.org/index.php?topic=229438.msg9739115#msg9739115 ), but with prices where they are right now, turns out depending on when you got your BTC, I may have been wrong in my analysis ( honestly didn't see $4k bitcoin coming this quickly.... ).  That said, I still believe it is much safer and a far more reliable investment to lend USD.  Lending USD gives MUCH higher returns than crypto, and its fairly reliable, daily income.

But, I also do have some coin that I hold, just in case.  And I know a lot of other people do as well.  And having that coin sitting there doing nothing doesn't really make sense to me, so I added back in crypto lending.  Getting 0.01% daily returns on those coins is better than nothing, and the bot can still focus on my USD returns, where my real income is.

Update on my earnings

Started with 1.67 BTC

BTC   0.00064869   9/21/2017 1:34
BTC   0.00084992   9/20/2017 1:34
BTC   0.00065268   9/19/2017 1:34
BTC   0.00028963   9/18/2017 1:33
BTC   0.00009418   9/17/2017 1:34
BTC   0.00038094   9/16/2017 1:33
BTC   0.00001963   9/15/2017 1:35
BTC   0.00020636   9/14/2017 1:33
BTC   0.00030252   9/13/2017 1:34
BTC   0.00022353   9/12/2017 1:33
BTC   0.00029864   9/11/2017 1:33
BTC   0.00027561   9/10/2017 1:33

Total   0.00424233   
---------------------------------------------------
Started with 52.48 USD

USD   0.01138437   9/19/2017 1:34
USD   0.02645962   9/18/2017 1:33
USD   0.02068842   9/17/2017 1:34
USD   0.03058716   9/16/2017 1:34
USD   0.03076744   9/15/2017 1:35
USD   0.03093932   9/14/2017 1:33
USD   0.03085982   9/13/2017 1:34
USD   0.0299652   9/12/2017 1:33
USD   0.03008847   9/11/2017 1:33
USD   0.0235216   9/10/2017 1:33
USD   0.03226006   9/9/2017 1:34
USD   0.03385569   9/8/2017 1:33
USD   0.03531509   9/7/2017 1:34
USD   0.01338319   9/6/2017 1:37

Total   0.38007545   

[BeastOfBodmin]

It seems there are too many people interested in giving borrow their bitcoin and not enough people interested in doing business with it. okey

Bitcoin margin lending rates have always been MUCH lower than USD.  All crypto for that matter has much lower rates. I was in a fairly long discussion about crypto vs USD lending several years ago, in which I advocated always converting crypto to USD to lend ( https://bitcointalk.org/index.php?topic=229438.msg9739115#msg9739115 ), but with prices where they are right now, turns out depending on when you got your BTC, I may have been wrong in my analysis ( honestly didn't see $4k bitcoin coming this quickly.... ).  That said, I still believe it is much safer and a far more reliable investment to lend USD.  Lending USD gives MUCH higher returns than crypto, and its fairly reliable, daily income.

But, I also do have some coin that I hold, just in case.  And I know a lot of other people do as well.  And having that coin sitting there doing nothing doesn't really make sense to me, so I added back in crypto lending.  Getting 0.01% daily returns on those coins is better than nothing, and the bot can still focus on my USD returns, where my real income is.

I hold some crypto, as well as gold. I buy into an asset as the price falls using pre-allocated fiat at various price points. As the price rises, I sell out at predetermined price points, but for a new buy, I always retain 33% of the asset as a core position which I will "never sell". My target for BTC is USD 10000, at which point I might sell my core position (accumulated when the price was BTC 1 was less than USD 1000).

I therefore have no problem lending out my core crypto positions, as well as the USD that are allocated for rebuys when BTC, etc. crash much, much lower.