EXT4 file system for usb node.

[Jet Cash]

I'm slowly putting together a system to run a Bitcoin node on an old HP Netbook. The machine spec is

Intel Celeron N3060 @ 1.60 GHz with 2Gb of Ram - the ram is not expandable, and the chip can't be changed.
The OS is the latest openSUSE Leap with Gnome. WiFi and Bluetooth are available, but no Ethernet
USB 2 and 3 and SD cards are available for extra storage. The internal drive only has about 28Gb available after software installation.
Proposed external drive - 2Tb Samsung EVO 860 in a Sabrient case to allow connection via USB 3

I propose to repartition the external drive and change the file system. At the moment I am considering a single EXT4 partition. The current OS uses BTRFS as its default, but I suspect that the older EXT4 system will place fewer demands on the netbook, and hopefully I can use it for surfing whilst sync'ing the blockchain. Net connection will be via public WiFi, and the system will often have to rely on the Netbook battery for power.

Whilst many may not consider this as an optimum setup, it is useful as part of my experiments as a digital nomad.
My question is - Do you think that a single EXT4 partition is the best choice under the circumstances?

[1715419493]

1715419493

[1715419493]

1715419493

[1715419493]

1715419493

[1715419493]

1715419493

[1715419493]

1715419493

[1715419493]

1715419493

[DaveF]

There is really no "best" for this. It's going to be a "less bad" option. EXT4 will work fine.

With that being said:
There will probably be a lot of people who will disagree with me on this but the netbooks lack of RAM is going to be a big big issue.
Every time you are offline and it has to catch up that and with the initial sync 2GB RAM and the low power of the CPU Will really kill you.

Are you using it because of weight and you want to carry less or some other reason?
You might have mentioned it in some other post.

-Dave

[Jet Cash]

I  bought it as a cheap backup about 3 years ago. It was new, but in a clearance because it is in a light blue case, the alternative was pink.
It lasts for about 9 hours on the battery, and came with Windows 10, and i promptly overwrote that with Ubuntu. The latest Ubuntu doesn't recognise the keyboard, hence the move to openSUSE. I'm reviewing all of my computing kit at the moment, and I decided to keep this as a backup - backup means an alternative after running out of battery on my main notebook. Also, I thought the netbook would be handy if I wanted to check a Bitcoin payment via a face to face meeting, and for smaller transactions into a current wallet. I will move to hardware wallets for long term storage, and I need to make a decision on my main ( more powerful ) notebook. I will be installing a comms centre in the van, and I will base this on one of the new single board computers with 12 volt or USB power..

The blockchain and wallet I want to use is over 2 years out of date, and I hoped to copy it onto the SSD and sync it as a background job whilst I am in a cafe. I'm really taken with this digital nomad life, and if I could power a pc from my wood burner, then I would love that.

[Jet Cash]

I
If you're concerned about resource usage, IMO you shouldn't use GNOME

I tried the KDE/plasma option in openSUSE, but I had issues with that, and gave up trying to correct it. Gnome worked straight out of the box, and as this is only a backup experiment, I thought I would run with that. I'll probably continue to do website building and image processing on the Notebook. If I continue to feel comfortable with openSUSE, then I'll double the RAM in the notebook, and replace the HDD with a SSD. The HDD on the notebook has a bad spot, and there may be a problem with the graphics controller. I'll do some testing, and at worst, I'll just buy another notebook if I have to.

[Jet Cash]

I'm in a bit of a mess at the moment, and I've got a lot to sort out. The Kung 'flu isn't helping either - hence the move further towards off grid and battery power. I did tell it not to bother with Libre office, but it decided to install it anyway. Also, I use Thunderbird, but it installed Evolution as the default. I'll probably have a look at that for some of my most used addresses. I've got a load of hard drives, and I've put some of those into cases, so I can move a lot of my data between machines.

I wanted to put a bit of thought into the Blockchain storage, both for speed and longevity/reliability of the drive. I'm not bothered about Windows compatibility. Sometimes I run the netbook tethered to my mobile using a USB connection, but I might experiment with a blutooth connection to access the data between machines - but not this month.

[NotATether]

You'll definitely need to upgrade the RAM on that netbook if you are going to use GNOME. On my system it regularly leaks an extra gigabyte or so if you leave it running for an extended period of time.

GNOME will run, but just barely, with 2GB. You need to set aside memory for bitcoin core. I would've suggested you add a second DDR2/3 2GB memory stick in it, but you mentioned the memory is non-expendable.

IMO, the RAM is a more important issue than the file system. 2GB will struggle running GNOME and bitcoin core at the same time (and Linux swap space is notoriously slow, at least from my experience).

If you don't mind editing config files to tweak the desktop, icewm is lighter than XFCE and LXDE and has a taskbar and menu at the bottom. Only ~256MB RAM usage. And the kernel uses a similar amount which puts the total usage at ~512MB.

[Jet Cash]

I can put 8Gb in the notebook, but I really wanted to see if I could use ther netbook as a background machine. I can leave it running for 3 or 4 hours with just core active, and I was hooping I could tweak it to handle that.

The alternative would be a Raspberry Pi powered by a solar battery pack, but I was thinking of trying that as phase 4 or 5 rather than an immediate project.

[LoyceV]

My question is - Do you think that a single EXT4 partition is the best choice under the circumstances?

I'm not sure how EXT4 would handle an accidental disconnect when you unplug the USB cable during writes. Other than that, it's a solid file system.

Every time you are offline and it has to catch up that and with the initial sync 2GB RAM and the low power of the CPU Will really kill you.

I've used a netbook (with slightly slower Atom CPU) with a backup blockchain in the past, but it took far too long to catch up when I turned it on, so I gave up.

[NotATether]

I'm not sure how EXT4 would handle an accidental disconnect when you unplug the USB cable during writes. Other than that, it's a solid file system.

If the device is removed or the computer crashes without unmounting an EXT4/3/2 filesystem first, it gets corrupted and you have to run fsck on it to repair the partition.

NTFS on Linux using FUSE kernel driver also gets bad, unremovable, inodes (file handles) if you're copying files and you unplug it in the middle of the transfer, but it doesn't destroy the drive.

XFS is much more robust and doesn't get corrupted when things like that happen. I've been through, what, dozens of power outages and I never had a single error on my Red Hat's XFS drive.

[Jet Cash]

I think I ignored the problems of premature disconnection, but that is probably unwise. If I am in a cafe, then there is a;ways the possibility that someone will close the netbook. Hopefully it will go through a closedown procedure, but there is probably no guarantee. of this. What could be more likely is that I am on battery, and thus am liable to run out of power, that gives an instant shutdown. I had a nightmare half day trying to fix the notebook when Windows started a mandatory update when I was running out of power. Not only did I have to waste time trying ( unsuccessfully ) to negate the effects of the so-called update, but I had to sort out the disk corruption as well.

Thasnks for reminding me of this risk. It isn't so serious with Bitcoin Core, as it seems to be fairly well written, and of course there are umpteen backups of the blockchain around the net.  It's still worth taking measures to avoid having to download the whole blockchain again though.

[Carlton Banks]

NTFS on Linux

yes, NTFS is not good (nor is it a "new technology" file system any longer)

XFS is much more robust and doesn't get corrupted when things like that happen. I've been through, what, dozens of power outages and I never had a single error on my Red Hat's XFS drive.

XFS and btrfs are what you want if you're interested in handling data corruption. But I don't think the laptop suggested in the OP would handle it ideally, as they both use merkle-tree based checksumming to correct errors on disk (ext4 can detect, but not correct, errors on disk). These old budget CPUs probably don't have an in-built native processing unit corresponding to the hashing algorithm that XFS or btrfs use for the error detection/correction. Possibly the standard integer processing APUs will handle it ok, but you will be putting additional load on the CPU no matter what.

The linux kernel in your linux distribution (Suse/Ubuntu) might have XFS and/or btrfs enabled by default, and that would certainly be the most efficient implementation (you would have to check what config options the kernel uses). If not, you can enable them yourself by copying the standard config for your version of linux, changing the option for XFS/btrfs to 'yes', then compiling it (don't forget to install the userspace modules and regenerate the bootloader). You probably could've solved the issues with the keyboard and/or ethernet the same way (i.e. flipping some option in the standard kernel options, then re-compiling it)

[Jet Cash]

>..<

Thanks for that helpful and informative post.

The keyboard issue seems to be pretty standard in the latest version of Ubuntu,and there are various suggestions as to how to fix it. I started to try them, and found that it was still an intermittent problem. I took the view that if Ubuntu released a version that was claimed to be stable, but contained such errors, then it was time to look for an alternative distro. So far, I haven't found any problems with the Gnome openSUSE installation, but I accept that there may be an overhead with Gnome. I can accept that on a backup machine, as long as it continues to work without my having to update the system. I may install oprnSUSE on the Netbook ifI decide not to replace it,and that has 8Gb of RAM.

I believe that the default file system is btrfs, and I am still considering that as an alternative to EXT4. There doesn't deem to be much in it, but I got the impression that EXT4 is slightly more stable, and may be less demanding on the cpu. This was why I considered it. However, using the default system might be a better option in the long run. I'm assuming that I won't have any problems copying files onto another hard disk if I decide to use an alternative file system on the solar powered system in the van - if I ever get round to building it.

[Carlton Banks]

I believe that the default file system is btrfs, and I am still considering that as an alternative to EXT4. There doesn't deem to be much in it, but I got the impression that EXT4 is slightly more stable, and may be less demanding on the cpu.

both XFS and btrfs are newer than ext4, and sometimes still considered beta software. As is bitcoin FWIW, I use btrfs and XFS (but not for the same length of time), and didn't encounter any issues other than the processing issues I described (the performance penalty of using either btrfs/XFS it quite noticeable if you copy some huge file, or the swap file is using btrfs/XFS and in heavy use). In general, I'm using btrfs/XFS for data disks, not for system disks (because of the performance penalty). Perhaps a good compromise would be to use btrfs/XFS for the root partition, then set up a separate partition for swap that uses ext3 or ext4, as the error correction is less essential for volatile data (and performance penalty would be reduced), but I have thus far not tried such a setup

This was why I considered it. However, using the default system might be a better option in the long run. I'm assuming that I won't have any problems copying files onto another hard disk if I decide to use an alternative file system

it's possible that btrfs is either badly supported or not supported at all on alternative operating systems, I would check that if you intend to use anything else (obviously, most other people are using Microsoft or macOS). I'm pretty sure (not 100% though) that ext2-4 are well supported elsewhere.

[Jet Cash]

Well that is interesting.

A few months ago I bought another 2Tb EVO drive, and I replaced the internal drive on my Notebook with it. I installed Mint for evaluation and experimentation. I went back to the old Windows 10 HDD because I needed to use some of the software on it. I put the SSD into a slightly different Sabient case. This one has an aluminium case, and a protective sleeve, so it may have been a bit more expensive. I've just looked at that setup using files on the Netbook, and it reports it as EXT3/4, but it tells me that some files are unreadable. This is the first time that I have looked at it via USB. I don't think I am going to go back to Mint, as there were some problems with the WiFi in cafes.

It seems to be coming down to my acceptance of Carlton's advice to use btrfs. I'm going to repartition the Mint drive, and use that for my Bitcoin wallet. Is it best to set up a single partition, or should I split out the wallet info, and put it in a second partition. This could make things slightly easier if I am using multiple nodes on more than one machine. However, it may be better to have a single online node, and use hardware wallets for long term storage.

I'm a bit cautious about installing any more software on the Netbook, because of the limited internal disk space, but it would be handy to make Brackets available. I use this to maintain my websites.I'm not short of drives, so I could put a 512Gb drive into a case, and use that to store the web site pages and backups. This would make it easier to move machines as well as saving space on the system drive.

[Carlton Banks]

I'm going to repartition the Mint drive, and use that for my Bitcoin wallet. Is it best to set up a single partition, or should I split out the wallet info, and put it in a second partition. This could make things slightly easier if I am using multiple nodes on more than one machine. However, it may be better to have a single online node, and use hardware wallets for long term storage.

My suggestion would be to use Logical Volume Manager (LVM):

• setup LVM: one PV and one VG per physical disk
• setup several LV's inside your VG
• write your partitions to the LV's
• setup the file system you want for each LV partition

this method is very flexible, you can change the size of the partitions (impossible with the typical way of creating partitions) and add new partitions very easily. This will make it easy to experiment, to see what the best setup is for your hardware and how you want to use it, before you commit to the size of the partitions.

I'm not sure if the typical "Disks" gui application lets you use LVM, but I would steer you away from using that anyway. The CLI for lvm is pretty straightforward, and there's a good chance your base system is already using LVM by default

[Jet Cash]

OK - things are becoming a bit clearer. I took the base cover off the netbook, and cleaned out the dust. I think the screren problem may have been caused by overheating. It has 8Gb of RAM, and a spare slot. An extra stick is about £35 if I buy a Crucial or Samsung one, so that is an eassy upgrade, and could well make quite a difference. The hard disk spot is persistent, but I will keep that disk as a historic backup, and hopefully the new SSD won't have any problems. I've more or less decided to go with openSUSE, as I don't want to spend too much time on experimenting any more. I've got some property and other issues that need to be resolved, and I want to move into the van full time. That is until the world economies stabilise, and I can see where I want to be in the structure. Actually, I probably want to be outside it, as that is where all the healthy fun will be in my opinion.

So back to the OP. The new SSD is just for Bitcoin, so I think I will set it up as 1 partition, and hope that this will extend its potential life. I take the point about the logical volumes, and that could be a nice idea for the web site disk, but that will have to wait for a month of so.

Many thanks for all the help and info, and I'll let you guys know if there are any problems that crop up in the future. I'm going to have a look at WiFi boosters to see if that will improve the network speed, as that seems to be another issue.

[Jet Cash]

My suggestion would be to use Logical Volume Manager (LVM):

I'm really grateful to you for suggesting this, and I have been reading this article.
https://opensource.com/business/16/9/linux-users-guide-lvm

I'm not sure that it is suitable for my node, but I think it will be a great help with my domains, web sites and images. I've got maybe a dozen or so external drives that I have salvaged, and the latest of these is this Hitachi one -
https://www.expertreviews.co.uk/storage/internal-hard-drives/50686/hitachi-z7k320-review
It came out of a scrap Xbox, and I'm waiting for the delivery of a USB case so that I can test it.

It really is time I got some order into my site structures, and maybe this will give me a chance to do this.

[Carlton Banks]

happy to help LVM is very useful, and surprisingly old tech for Linux (early versions are from the early 2000's era)

[Jet Cash]

As there didn't seem to be much interest in my pruned node project, I decided to change it a bit for my own use. I've doubled the memory in the Notebook to 16Gb, and replaced the HDD with a 2Tb SSD, and I've installed open SUSE on that. In fact I'm posting this on that updated machine, as there don't seem to be any problems here either. I haven't tested everything though. I've got a wallet with about 0.4 Bitcoin in it, and a blockchain that is about 3 years out of date. That is stored on an external HDD, and I'll update that with this machine. Once it is sync'd I'll copy it onto yet another drive, and I'll prune it. Once that is stable, then I'll see if I can copy the blockchain onto the 320Gb drive, and I'll associate it with an empty wallet.

I don't think that one should store more than half a Bitcoin online, any extra coins should be kept in more secure locations such as a hardware wallet.

[NotATether]

Once that is stable, then I'll see if I can copy the blockchain onto the 320Gb drive, and I'll associate it with an empty wallet.

When you reach that stage, could you also make a test and copy it to an external SSD, while keeping the network speed the same as when you synced it on HDD? I'm interested in seeing the performance differences between using an external HDD and SSD, in terms of how many hours it takes to sync the blockchain.

You can get a 480GB SSD for around $55 to $85 on Amazon. This one's on the cheaper side: Pioneer 3D NAND External SSD (480 GB)-Portable Solid State Drive USB 3.1 Gen 1 (APS-XS03-480), though your laptop port will probably be limited to USB 3 or even 2 connectivity speeds, but that'll give you an opportunity to see how the USB port influences the speed.