Move ubuntu mining setup to a USB thumb drive

[Fletch]

0.5 BTC reward for anyone who can post exact instructions on how to do this (that actually work). I've tried myself, but I can't get it to work.

I have a fresh Ubuntu 11.04 install on a harddrive with everything setup for headless mining (used this guide). When I installed Ubuntu, I made sure to put it all on one single partition that is less than 8GB (the size of the USB thumb drive). I have no swap and I have also enabled tmpfs (for /tmp) and added noatime to fstab to reduce writes to the USB drive (once its moved over).

Now I would like to clone this partition over to the USB thumb drive, remove the harddrive and just have the rig boot off the thumb drive.

If I understand things correctly, I need to:

1. Boot from a Live CD (or a second USB thumb drive in my case).
2. Clone the partition from the harddrive to the target thumb drive (using dd).
3. Somehow make the computer boot of the thumb drive.

I think I've managed step 1 and 2, but I have no idea how to tackle step 3. I'm guessing I need to install a boot loader on the thumb drive, but this is where my Linux skills fall flat.

Anyone?

[drawoc]

Typically, you create a live CD/bootable usb drive with a program like Debian live (Which also works for Ubuntu).

Those live CDs contain a normal file system, which then contains a compressed filesystem (squashfs) for the contents of the RAM drive, as well as the linux kernel itself, an initrd, and a bootloader like isolinux.

I've never heard of it being done the way you're doing it. I'll look up on it more if you're intent on doing it that way.

Edit: there's also Remastersys, which aims to make a complete copy of your current system as a live cd, but it doesn't keep the ATI proprietary drivers working.

Edit 2: It looks like you can just use remastersys to create a bootable drive with the ATI drivers installed, and then when you boot up, re-run aticonfig and restart X. If you like, I can post a list of easy instructions.

[drawoc]

Never mind remastersys, I think I've found an easier way. (I haven't tried them because I don't have an 11.04 live CD available. I'm downloading one now.)

1. Follow the instructions Here to install Ubuntu to your USB drive (Use method one). Make the partition the same size as the partition on your hard drive. You might want to try booting the thumb drive, just to make sure it works.

2. Boot the live CD again. Plug in your USB drive. In a terminal, run:

Code:

ls /dev/disk/by-id

3. That will list a number of disks. Find the one that corresponds to your flash drive's main partition. (SHOULD END IN -part1). For example:
usb-SanDisk_U3_Cruzer_Micro_000016A078718359-0:0-part1

4. Run dd like this:

Code:

dd if=hdpartition of=usbpartition

Replacing hdpartition with the partition on your hard drive, and usb partition with the partition on your usb drive. For example:

Code:

dd if=/dev/sda2 of=/dev/disk/by-id/usb-SanDisk_U3_Cruzer_Micro_000016A078718359-0:0-part1

[malavita]

0.5 BTC reward for anyone who can post exact instructions on how to do this (that actually work). I've tried myself, but I can't get it to work.

I have a fresh Ubuntu 11.04 install on a harddrive with everything setup for headless mining (used this guide). When I installed Ubuntu, I made sure to put it all on one single partition that is less than 8GB (the size of the USB thumb drive). I have no swap and I have also enabled tmpfs (for /tmp) and added noatime to fstab to reduce writes to the USB drive (once its moved over).

Now I would like to clone this partition over to the USB thumb drive, remove the harddrive and just have the rig boot off the thumb drive.

If I understand things correctly, I need to:

1. Boot from a Live CD (or a second USB thumb drive in my case).
2. Clone the partition from the harddrive to the target thumb drive (using dd).
3. Somehow make the computer boot of the thumb drive.

I think I've managed step 1 and 2, but I have no idea how to tackle step 3. I'm guessing I need to install a boot loader on the thumb drive, but this is where my Linux skills fall flat.

Anyone?

This is how I clone USB sticks, should work the same and, depending on the PC and USB stick, it should take about 10 mins for 8GB:

1) Put your empty USB stick in a usb port, then issue a

Code:
dmesg

you should see something like:

Code:
[38443.728018] usb 3-4: new high speed USB device number 3 using ehci_hcd
[38443.863492] usb 3-4: New USB device found, idVendor=0951, idProduct=1624
[38443.863496] usb 3-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[38443.863498] usb 3-4: Product: DataTraveler G2
[38443.863500] usb 3-4: Manufacturer: Kingston
[38443.863501] usb 3-4: SerialNumber: 0013729B6EB8E9616000002A
[38443.863978] scsi15 : usb-storage 3-4:1.0
[38444.893201] scsi 15:0:0:0: Direct-Access     Kingston DataTraveler G2  PMAP PQ: 0 ANSI: 0 CCS
[38444.894329] sd 15:0:0:0: Attached scsi generic sg2 type 0
[38445.626356] sd 15:0:0:0: [sdb] 15679488 512-byte logical blocks: (8.02 GB/7.47 GiB)
[38445.626847] sd 15:0:0:0: [sdb] Write Protect is off
[38445.626850] sd 15:0:0:0: [sdb] Mode Sense: 23 00 00 00
[38445.626852] sd 15:0:0:0: [sdb] Assuming drive cache: write through
[38445.630964] sd 15:0:0:0: [sdb] Assuming drive cache: write through
[38445.654546]  sdb: unknown partition table
[38445.657168] sd 15:0:0:0: [sdb] Assuming drive cache: write through
[38445.657173] sd 15:0:0:0: [sdb] Attached SCSI removable disk

Here I've just inserted a Kingston 8Gb USB stick, note the name it gets in the line where it tells you the size

Code:
[38445.626356] sd 15:0:0:0: [sdb] 15679488 512-byte logical blocks: (8.02 GB/7.47 GiB)

So its name is sdb.

With a mount command you can verify that your corrent USB stick (the one you booted from) is not sdb,

Code:
mount
 

Code:
/dev/sda1 on /live/image type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp437,iocharset=utf8,shortname=mixed,errors=remount-ro)
/dev/sda2 on /live/cow type ext4 (rw,noatime,user_xattr,acl,barrier=1,data=ordered)

look for the lines that contain /live/image and /live/cow.

then, to do the copy

Code:
dd if=/dev/sda of=/dev/sdb bs=64K

which calls dd and gives it the input file (on linux everything is a file, even a disk) the if=, the output file of= and a block size bs= to make it go faster otherwise it does a sector by sector copy (512 bytes) and so it can take a fairly longer time.

PAY ATTENTION
If you make a mistake and put as output file your current USB stick (the one you booted from) you will erase it!! Be careful.


If it works for you, please send donation here:  197kuoutCZN4zFfou7Jrm8eRJ6En9nbQrB

[Fletch]

drawoc: I was kind of hoping that I wouldn't have to go through the install once more, but I will give it go this weekend.

malavita: As far as I can see, you posted instructions for cloning a USB stick on to another USB stick. That's not what I'm after. I have installed Ubuntu on a harddrive and need to migrate it on to a USB stick. I can't use e.g. "dd if=/dev/sda of=/dev/sdb" because the target disk is smaller than the source. The partition I'm interested in is the same size, but not the entire disk.

[RaTTuS]

you may be able to just dd the partition however, personally
I'd just install it again direct to the usb - it's all good practice and you'll be able to build a new build with ease after a couple of times ;-p

[drawoc]

Uck. I just realized that since grub.cfg is on the same partition as everything else, my guide above will fail (when you dd the hd's partition over, it will contain the config file from the hard drive, and confuse GRUB).

It's probably easiest to just install Ubuntu on the flash drive like in step 1 above, and reinstall everything from there. I have a feeling you'd spend more time figuring out how to get dd-ing your partition over to work than to just reinstall everything on the flash drive. Good luck.

[Fletch]

Hmm, ok. It's just that installing Ubuntu on to a USB drive takes forever. Let alone installing the AMD proprietary drivers and the stream SDK. Oh well, I guess I'll have to bite the bullet. Or buy a quicker USB thumb drive

It is somewhat surprising to me that this is so complicated/difficult.

[organofcorti]

Is there any reason not to use LinuxCoin (deb based) which is actually designed to be run from a thumb drive? I got LinuxCoin installed (with change persistence) in 10 minutes, running stably for the last week or so.


I know this isn't what you asked and there may be reasons to stay on Ubuntu. If so at least check out the instructions for installing LinuxCoin to a  thumbdrive in the LC thread - there might be some hints there.

[nomnomnom]

Hi there,

I never cloned a install like that, but recently installed a Slackware
straight to usbdrive, which took some afterwork to play nice.

Basicly, if you have say one big ext4 partition on the usb drive
and all your files are in there with the right permissions, you
"only" need to make it bootable imo

I would try something like this, give the partition on the usbstick
a LABEL (sometimes devices can be sda or sdb on usbboot, so
its good to use a LABEL or UUID so everthing runs smooth)

tune2fs -L USBROOT /dev/sdb1 (asuming sdb1 is the partition on your usbstick)
#verify your label with blkid

mkdir /mnt/usbroot
mount /dev/sdb1 /mnt/usbroot
mount -o bind /dev /mnt/usbroot/dev
mount -o bind /proc /mnt/usbroot/proc
mount -o bind /sys /mnt/usbroot/sys

#chroot into the usbstick
chroot /mnt/usbroot

modify /etc/fstab so that the rootpartition points to your LABEL, looks like that for me:
LABEL=USBROOT   /                ext4        defaults,noatime         1   1

One important thing, for usbbooting you need to set a delay, so the kernel has some
time to find the usb devices while booting,

add rootdelay=7 to the default boot options in /etc/default/grub
(sometimes you need longer delay, try 10 if 7 is not working)

# install grub
grub-install /dev/sdb (make sure this is your usbstick)
update-grub

cross fingers and try to boot

No guarantee, just wrote this down from my head, but maybe it works  .

[Fletch]

nomnomnom: I will give it a go.

Is there any reason not to use LinuxCoin (deb based) which is actually designed to be run from a thumb drive? I got LinuxCoin installed (with change persistence) in 10 minutes, running stably for the last week or so.

Not really. It's just that I have my Ubuntu install setup just the way I want it and I fear that with Linuxcoin I will lose a lot of control (lots of stuff going on "behind my back").