Bitcoin Forum

Bitcoin => Bitcoin Discussion => Topic started by: BTC Turkiye on March 01, 2018, 01:48:34 AM



Title: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 01, 2018, 01:48:34 AM
I have a bot which gives an error sometimes and cancels the process then I`ll have to the run command again to re-start the bot.

Is there anyway to automatically re-run the same previous command if the process is cancelled and it goes back to the main terminal command line again?

I use Debian

Thank you in advance for any help


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 01, 2018, 09:59:43 AM
No one can help me on this?


Title: Re: How to repeat the same command automatically in Linux?
Post by: HeRetiK on March 01, 2018, 10:05:41 AM
Usually you can use the arrow keys up / down to navigate through your command history, if that's what you're looking for. This is true for both Linux and Windows machines. Or are you talking about writing a script that repeats the most recent command in case of a failure?


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 01, 2018, 10:07:36 AM
Are you talking about writing a script that repeats the most recent command in case of a failure?

That`s exactly what I`m looking for. Anytime it comes back to the command line, it should repeat the same command that previously just ran


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 01, 2018, 07:10:27 PM
Usually you can use the arrow keys up / down to navigate through your command history, if that's what you're looking for. This is true for both Linux and Windows machines. Or are you talking about writing a script that repeats the most recent command in case of a failure?

Love your Motto BTW :D
Awesome!


Title: Re: How to repeat the same command automatically in Linux?
Post by: HeRetiK on March 01, 2018, 10:52:46 PM
Are you talking about writing a script that repeats the most recent command in case of a failure?

That`s exactly what I`m looking for. Anytime it comes back to the command line, it should repeat the same command that previously just ran

Depending on your setup you either want to have a utility like daemontools watch the process and restart it on a crash:

https://unix.stackexchange.com/questions/336501/how-to-restart-a-process-automatically-when-it-killed-in-linux-centos


Or add a loop to your script that retries the same command until it succeeds:

https://serverfault.com/questions/80862/bash-script-repeat-command-if-it-returns-an-error
https://stackoverflow.com/questions/12321469/retry-a-bash-command-with-timeout

In case you're new to shell scripting: When an application (or command) runs successfully it returns a status code = 0; if it fails it returns a nonzero error code. These loops retry the same command with a slight delay inbetween (sleep) until it returns the status code = 0, ie. succeeds.


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 04, 2018, 06:38:19 AM
Are you talking about writing a script that repeats the most recent command in case of a failure?

That`s exactly what I`m looking for. Anytime it comes back to the command line, it should repeat the same command that previously just ran

Depending on your setup you either want to have a utility like daemontools watch the process and restart it on a crash:

https://unix.stackexchange.com/questions/336501/how-to-restart-a-process-automatically-when-it-killed-in-linux-centos


Or add a loop to your script that retries the same command until it succeeds:

https://serverfault.com/questions/80862/bash-script-repeat-command-if-it-returns-an-error
https://stackoverflow.com/questions/12321469/retry-a-bash-command-with-timeout

In case you're new to shell scripting: When an application (or command) runs successfully it returns a status code = 0; if it fails it returns a nonzero error code. These loops retry the same command with a slight delay inbetween (sleep) until it returns the status code = 0, ie. succeeds.


Ok. I`m a little confused now. I`m still very much of a n00b in Linux

I`ve been trying this daemontools thing but I couldn`t figure out how to install it on Debian or if it comes in default already.

So I moved to the other links you sent but they`re all in different codes and I`m not sure what to do with them. My script is in Python and it keeps running until sometimes it gives and error and close itself. I just want it to start again with the same command when it closes. So should I add those codes in the python files? if so which file?
or should I create a new file and add it to the scripts folder or something?

I`d really appreciate it if you can explain step by step what to do. I desperately need this

Thank you


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 04, 2018, 09:29:58 AM
Does your script create a pidfile in eg /var/run? If so, you can use a cronjob to see if the pid still exist ("kill -0 `cat /var/run/myprocess.pid`").
If there is no pidfile, you can use top to see if the process is still running ("pidof myprocess" or "ps -x | grep myprocess | grep -v grep | awk {'print $1'}")


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 04, 2018, 09:31:02 AM
I tried the PID method but PID changes everytime it runs


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 04, 2018, 12:38:33 PM
Well sure it does. You get a new pid assigned every time.
When you check it and see that it's not running, then your watchdog script just needs to run the command which starts your bot again after it crashed.

If your bot is running as a service, you might want to look into systemd; afaik you can set in a unit file that it automatically restarts a service.


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 04, 2018, 12:44:13 PM
Well sure it does. You get a new pid assigned every time.
When you check it and see that it's not running, then your watchdog script just needs to run the command which starts your bot again after it crashed.

If your bot is running as a service, you might want to look into systemd; afaik you can set in a unit file that it automatically restarts a service.


I don`t think I explained it right.
When I run the script now the PID is 23423 and the script I will run on cron is going to be like waht you wrote above. But when I`m not around and the script dies. Your code will look for that PID but that PID will disappear and wont exist anymore.

So how can your code run a script automatically when the PID changes every time it runs?

What am I going to write at this part of your code exactly?
Code:
"pidof myprocess" 


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 04, 2018, 01:04:09 PM
Your code will look for that PID but that PID will disappear and wont exist anymore.
No, the code checks either if a pid exists (given via eg a pidfile) or if a given process exists by name (as in, has a pid).
You don't know the pid before you start a process.

From the top of my head, you could try something like that as a cronjob:
Code:
#!/usr/bin/bash
if [[ ! $(pidof yourbot) ]]; then
echo "Bot down"
#command to restart your bot
fi

Look into the processlist to see under what name your bot is running (with "ps ax" for example).
Let's say it's "yourbot": then "pidof yourbot" will always return a pid, unless your bot has crashed and its process is gone.
As a side note: I assumed your bot is running only once; otherwise the approach needs to be changed a bit.


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 04, 2018, 01:15:42 PM
The confusing thing is

1- Should I create a file and name it test.sh and then put this in cron job? Will that just work like that?
2- Process name shows as just python. But what I write in terminal is like
Code:
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 04, 2018, 01:53:12 PM
If the command pgrep exists, try this:
Code:
#!/usr/bin/bash
if [[ ! $(pgrep -f "rtgferfd.py") ]]; then
echo "Bot down"
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34 &
fi

If not, this:
Code:
#!/usr/bin/bash
if [ "$(ps ax | grep "rtgferfd.py" | grep -vc grep)" -eq "0" ]; then
echo "Bot down"
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34 &
fi

Make sure you use full paths though, or you might run into troubles with restarting.
This script should run fine via cron. Put it into a file and tell cron to run it eg every minute.
Try it manually first: name it test.sh or whatever you want to call it and execute it.
If your bot is running, you should see not output. If it is not, your bot should appear in the processlist afterwards.


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 04, 2018, 01:56:27 PM
If the command pgrep exists, try this:
Code:
#!/usr/bin/bash
if [[ ! $(pgrep -f "rtgferfd.py") ]]; then
echo "Bot down"
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34 &
fi

If not, this:
Code:
#!/usr/bin/bash
if [ "$(ps ax | grep "rtgferfd.py" | grep -vc grep)" -eq "0" ]; then
echo "Bot down"
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34 &
fi

Make sure you use full paths though, or you might run into troubles with restarting.
This script should run fine via cron. Put it into a file and tell cron to run it eg every minute.
Try it manually first: name it test.sh or whatever you want to call it and execute it.
If your bot is running, you should see not output. If it is not, your bot should appear in the processlist afterwards.


You`re awesome!
I`ll try to see if it`ll work like this!
Thank you very very much!


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 05, 2018, 04:17:12 PM
It worked perfectly like below. But I have couple questions


1- Does this script checks the code every some time? because I see my script is running but then once every minute or so I see one more of the same python process goes on the list for couple seconds and then disappear.

2- I didn`t need to add it to cron job because when I run your code, it checks the script and can`t find it so It starts it and when I cancel it, it restarts it again. So as long as your code runs, the script keeps running no matter what. Is that normal and should I still add the .sh file to the cron job?

Code:
#!/usr/bin/bash
if [[ ! $(pgrep -f "rtgferfd.py") ]]; then
echo "Bot down"
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34 &
fi


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 05, 2018, 07:15:59 PM
1: No, it only checks when you execute it. It's a one-time check. If you see more processes, maybe your bot uses forks/threads.

2: Adding it to cron means the script will be run in the intervals you define, 24x7. So if you want your bot to be checked even when you're asleep, add it. But you will have to use full paths or it will fail via cron. As long as the bot runs you won't see anything, but when it gets restarted, cron should send off an email to you (if it's configured that way) and restart it.

Or, you could find out what causes the bot to crash and fix that.


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 08, 2018, 10:03:22 PM
Alright. After testing it for few days. Here`s the result.

You`re right. I have to add it to the cron job because bot just stops sometimes. But I realized that sometimes the bot just stops however it doesn`t go back to command line. It just freezes. So can you add a line in the code where it will restart the script if it just stops or if it just sees a specific line in on the terminal while running?
Like
If "LINE LINE LINE" appears on the terminal page while running
Stop the script, go back to the command line and restart the script again with the same command it did before


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 09, 2018, 02:18:44 PM
Instead of keeping a terminal open, send it to background and redirect the output into a logfile.

I would try something like this:
Code:
#!/usr/bin/bash

if [[ ! $(pgrep -f "rtgferfd.py") ]]; then
echo "Bot down"
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34 > /var/log/rtgferfd.log 2>&1 &
fi

if [ "$(grep -c "LINE LINE LINE" /var/log/rtgferfd.log)" -gt "0" ]; then
echo "Bot died"
pkill -f "rtgferfd.py"
python rtgferfd.py --solid --r --fr --quantity 35 --wait_time 34 > /var/log/rtgferfd.log 2>&1 &
fi

But really, fix the bugs in the bot instead of trying to bandaid them with workarounds.


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 10, 2018, 07:49:59 AM
Ok I`m having a problem now. Below script I got it saved in /home/kullanici/restart_script.sh

Code:
#!/usr/bin/bash
if [[ ! $(pgrep -f "python /home/kullanici/bot.py --wait_time 0.2 --stop_loss 0") ]]; then
    echo "Bot down"
    python /home/kullanici/bot.py --wait_time 0.2 --stop_loss 0 &
fi

So when I go to terminal and write bash /home/kullanici/restart_script.sh it runs perfectly. But when I put that file in /etc/cron.hourly/ it doesn`t run it. I also went to terminal and wrote bash /etc/cron.hourly/restart_script.sh and it doesn`t run like that either. It says

Code:
root@debian:/etc/cron.hourly# Traceback (most recent call last):
  File "/home/kullanici/bot.py", line 9, in <module>
    from Trading import Trading
ImportError: No module named Trading


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 10, 2018, 07:50:43 AM
I`m so close to succeed in this solution right now, I don`t want to start another way to do this now to be honest.


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 10, 2018, 08:09:09 AM
Why don't you try to put it into crontab with /home/kullanici/restart_script.sh?

I thought you had it working with cron already?


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 10, 2018, 08:12:00 AM
No. I was running it from it`s destination. I have never put anything in cron before and I checked it out but couldn`t get it to work when I write
crontab * * * * * /home/kullanici/restart_script.sh

It says app: Not a regular file


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 10, 2018, 09:24:20 AM
How about doing a little research first? There are tons of tutorials on how to add a crontab entry.
You need to learn how to handle the basics of your Linux install if you want to rely on it.


Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 10, 2018, 11:07:48 PM
I did. That`s why I tried that crontab command first. I did a lot of research and tried multiple commands but none of them worked for me. I know I`m a n00b and I shouldn`t ask for the basics but believe me I really research these things before I come here and ask for it.

That`s why first I tried to put the file in the cron.hourly folder and it didn`t work. Then I tried to run it from there and it didn`t work and then put the crontab command and it still didn`t work. Please don`t think that I`m looking for the basic and easy to find info here.


Title: Re: How to repeat the same command automatically in Linux?
Post by: Bitsky on March 11, 2018, 08:57:10 AM
A few hints/ideas:
- cron runs as its own user with its own ENV, so $PATH etc differs from other users
- try running it in cron as your own user (kullanici)
- cd into your homedir before trying to start the bot since it seems unable to find modules ("No module named Trading")
- set the directory for the modules (PYTHONPATH or sys.path.append)

If you know someone who is good with Linux, let him have a look directly at your system. It's hard to debug something if you cannot do anything yourself.
Only let that do someone you really trust and look over his shoulder to see what he is doing at any time.



Title: Re: How to repeat the same command automatically in Linux?
Post by: BTC Turkiye on March 11, 2018, 09:04:34 AM
A few hints/ideas:
- cron runs as its own user with its own ENV, so $PATH etc differs from other users
- try running it in cron as your own user (kullanici)
- cd into your homedir before trying to start the bot since it seems unable to find modules ("No module named Trading")
- set the directory for the modules (PYTHONPATH or sys.path.append)

Thank you. Will try...
Quote
If you know someone who is good with Linux, let him have a look directly at your system. It's hard to debug something if you cannot do anything yourself.
Only let that do someone you really trust and look over his shoulder to see what he is doing at any time.

That`s the reason I`m writing here instead of getting someone to look at my screen. I kinda have to do this all by myself but I keep getting smashed on my face, get up and try again