And in terms of accessing via SSH:
Install 'screen' - not strictly necessary, but useful in keeping a program running while you logout from shell
run 'screen'
run 'startx' - this startx an X Server on the machine
in a separate screen window ('Ctrl-a c') run:
export DISPLAY=:0
'your-miner-command-line'
Don't you need to run xhost + from an xterm before you can just connect to the X11 session?