|
Title: Raspberry Pi - corrupted blocks Post by: Rath_ on January 20, 2021, 04:59:05 PM Hey,
does anyone know if there are any Raspibolt (https://github.com/Stadicus/RaspiBolt)/Raspiblitz (https://github.com/rootzoll/raspiblitz) alternatives with c-lightning support instead of LND? I had no luck so far in finding any. I will end up installing everything manually if I can't find any alternatives by tomorrow. Title: Re: c-lightning on Raspberry Pi Post by: DaveF on January 20, 2021, 06:01:01 PM I have looked too but not found any. But, it was a while ago that I looked.
mynode is LND too Any reason why you want c-lightning in a device instead of LND? I wanted to learn c-lightning a bit more and figured a pre-done environment would be better to start to play with. Wound up just doing it myself anyway to learn. -Dave Title: Re: c-lightning on Raspberry Pi Post by: BitMaxz on January 20, 2021, 06:14:30 PM Take a look at this one alternative just found when searching on Google.
- https://github.com/mynodebtc/mynode Or you might end up with manual installation below. - https://github.com/bitembassy/home-node I don't know this one will work in raspberry but since it's related to ubuntu it might be work with a normal OS(Ubuntu.) Title: Re: c-lightning on Raspberry Pi Post by: Rath_ on January 20, 2021, 10:24:32 PM Any reason why you want c-lightning in a device instead of LND? I have already played around with LND when I was getting started with the Lightning Network, so I want to try something new. Also, I heard a lot of people complaining about LND being a mess now. The only thing I don't like about c-lightning is that it lacks decent clients. The only one I can recall is Spark (https://github.com/shesek/spark-wallet) while LND can be connected to Zap (https://apps.apple.com/pl/app/zap-bitcoin-lightning-wallet/id1406311960?l=pl) or even BlueWallet (if configured properly), and some other clients. -snip Thanks for your input. The first link is not what I am looking for - that project supports only LND. As for the other link, it might actually come in useful. Title: Re: c-lightning on Raspberry Pi Post by: Carlton Banks on January 21, 2021, 10:10:39 AM The only thing I don't like about c-lightning is that it lacks decent clients. The only one I can recall is Spark (https://github.com/shesek/spark-wallet) while LND can be connected to Zap (https://apps.apple.com/pl/app/zap-bitcoin-lightning-wallet/id1406311960?l=pl) or even BlueWallet (if configured properly), and some other clients. there's also Lightning-qt (https://github.com/darosior/pylightning-qt) and I think there was a Lightning client on Android called "Zeus" that could operate the c-lightning API as a client, but I didn't try it to see how well it worked. Title: Re: c-lightning on Raspberry Pi Post by: DaveF on January 22, 2021, 07:12:04 PM The only thing I don't like about c-lightning is that it lacks decent clients. The only one I can recall is Spark (https://github.com/shesek/spark-wallet) while LND can be connected to Zap (https://apps.apple.com/pl/app/zap-bitcoin-lightning-wallet/id1406311960?l=pl) or even BlueWallet (if configured properly), and some other clients. there's also Lightning-qt (https://github.com/darosior/pylightning-qt) and I think there was a Lightning client on Android called "Zeus" that could operate the c-lightning API as a client, but I didn't try it to see how well it worked. Watch your quoting I did not say that :-) Just did a c-lightning install, on a machine that had LND running. Yeah, don't do that. No clue what I was thinking. But obviously didn't work till I changed the port. Shutdown LND to play with c-lightning Once you have a wallet synced, it's all kind of the same. But LND does seem to have more support. -Dave Title: Re: c-lightning on Raspberry Pi Post by: Carlton Banks on January 22, 2021, 07:42:51 PM Watch your quoting I did not say that :-) oops, sorry Dave. edited Once you have a wallet synced, it's all kind of the same. But LND does seem to have more support. LND uses the same/similar resources? I can see that being the case, golang is essentially just c underneath, so it should perform pretty well if you know how to write it well (I imagine it's more forgiving than c) Title: Re: c-lightning on Raspberry Pi Post by: Rath_ on January 22, 2021, 08:08:18 PM I might end up going for RaspiBolt since I am having some trouble with setting up everything correctly. My chainstate folder got corrupted twice for no reason when launching bitcoind. I am not sure if it's because my external SSD was formatted to exFat or because I am also running a samba server pointing to that drive. I am running Ubuntu Server 20.04.1 and I followed this guide (https://medium.com/@meeDamian/bitcoin-full-node-on-rbp3-revised-88bb7c8ef1d1).
Is there any particular reason why you guys chose c-lightning over LND or vice-versa? Title: Re: c-lightning on Raspberry Pi Post by: DaveF on January 22, 2021, 10:52:31 PM LND uses the same/similar resources? I can see that being the case, golang is essentially just c underneath, so it should perform pretty well if you know how to write it well (I imagine it's more forgiving than c) Yes, both seem to use the same RAM / CPU. It's constrained since I was testing on a RPi but neither one seems to have more bloat. Is there any particular reason why you guys chose c-lightning over LND or vice-versa? I originally started to play with lightning with one of the node in a box setups. It used LND and I just ran with it. I keep wanting to spend more time with c-lightning and now that I have it running again in my office I probably will, but the main reason I have LND everyplace is that it was what I learned 1st... -Dave Title: Re: c-lightning on Raspberry Pi Post by: Carlton Banks on January 22, 2021, 11:28:40 PM My chainstate folder got corrupted twice for no reason when launching bitcoind. I am not sure if it's because my external SSD was formatted to exFat or because I am also running a samba server pointing to that drive.[/url]. I wouldn't choose exFat, but not sure if it's actually susceptible to data corruption. It's just that you can likely get something more reliable (ZFS or btrfs), or simply with more features/security (ext4) Is there any particular reason why you guys chose c-lightning over LND or vice-versa? I think people initially chose LND because it was pre-compiled, and it had more features. The features gap has largely closed, and c-lightning is now available pre-compiled too I chose c-lightning because they went for a lightweight modular design (and because they had a veteran Linux dev running the project). The LND team seemed to work very fast (easier when writing software with golang, or so I hear), but make alot of mistakes. Don't know to what extent they're still like that, not been keeping up with it Title: Re: c-lightning on Raspberry Pi Post by: Rath_ on January 26, 2021, 11:26:04 AM I wouldn't choose exFat, but not sure if it's actually susceptible to data corruption. It's just that you can likely get something more reliable (ZFS or btrfs), or simply with more features/security (ext4) I ended up splitting my SSD into two separate partitions (exFat and ext4). I need to keep the exFat one because I want to be able to occasionally use that SSD with my Windows laptop on the go. Now, I am trying to figure out how to force RaspiBlitz to mount and use only the ext4 partition. The automatic installer insists on making one large partition, so I can't proceed with the installation. Title: Re: c-lightning on Raspberry Pi Post by: DaveF on January 26, 2021, 12:29:38 PM I wouldn't choose exFat, but not sure if it's actually susceptible to data corruption. It's just that you can likely get something more reliable (ZFS or btrfs), or simply with more features/security (ext4) I ended up splitting my SSD into two separate partitions (exFat and ext4). I need to keep the exFat one because I want to be able to occasionally use that SSD with my Windows laptop on the go. Now, I am trying to figure out how to force RaspiBlitz to mount and use only the ext4 partition. The automatic installer insists on making one large partition, so I can't proceed with the installation. How large is your SSD? How much space do you need for the exFat? Do you need anything that is on it now? If it's a 1 TB and you don't need anything that's on it just let installer do its thing and then do a shutdown and re-size / shrink the partition and create a ex-Fat one. Keep in mind between the blockchain and the lightning DB and everything else I would no go below 650 or so GB otherwise you are going to be doing this again soon. Yes you can do it in 500GB, I am doing that now, but why create more work for yourself later? -Dave Title: Re: c-lightning on Raspberry Pi Post by: Rath_ on January 26, 2021, 12:45:36 PM How large is your SSD? How much space do you need for the exFat? Do you need anything that is on it now? 1 TB (931.50 GB available). I would like to keep at least 400 GB for the exFat partition. I don't have any personal files on that drive, so I can safely keep playing with it. I am currently uploading the blockchain data through SCP to the ext4 partition, though. If it's a 1 TB and you don't need anything that's on it just let installer do its thing and then do a shutdown and re-size. Sounds good, but I might need to modify some auto-mount setting later. Keep in mind between the blockchain and the lightning DB and everything else I would no go below 650 or so GB otherwise you are going to be doing this again soon. My blockchain data takes about 350 GB right now. I assume that it will grow at a rate of about 50 GB per year. How much does the lightning database weight? 450-500 GB for the ext4 partition sounds fine to me at the moment. Title: Re: c-lightning on Raspberry Pi Post by: NotATether on January 26, 2021, 02:34:12 PM My chainstate folder got corrupted twice for no reason when launching bitcoind. I am not sure if it's because my external SSD was formatted to exFat or because I am also running a samba server pointing to that drive. I am running Ubuntu Server 20.04.1 and I followed this guide (https://medium.com/@meeDamian/bitcoin-full-node-on-rbp3-revised-88bb7c8ef1d1). bitcoind should not care which filesystem is underlying the folder. If you pointed bitcoin's folder to some other partition, then there is either a problem with the way the filesystem was made, or, like this guide is doing, you're symlinking the folder from filesystem to another and it's giving the read/write system calls hiccups. If it's the latter, and it probably is if other folders on that filesystem are getting corrupted too, then the ext4 filesystem was made on the wrong device. You should be making it on one like /dev/sda1 or sda2, not /dev/sda (which is reserved for the partition table on-disk). For reference, this is the correct way to format a disk and create an ext4 filesystem on the first partition of a disk. Code: $ sudo fdisk /dev/sda You’ll be prompted to enter the partition number. Hit “Enter” to use the default value (1): Code: Partition number (1-128, default 1): Code: First sector (2048-500118158, default 2048): Enter +100G to set the partition size to 100 GiB: Code: Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-500118158, default 500118158): +100G... Save the changes by running the w command: Code: Command (m for help): w Code: The partition table has been altered. ... Code: $ sudo mkfs.ext4 -F /dev/sda1 Alternatively, you may use some other partitioning utility besides fdisk such as gparted. Adjust the instructions accordingly. Your samba share is for all purposes unknown to bitcoind and so should not be related to this problem. I ended up splitting my SSD into two separate partitions (exFat and ext4). I need to keep the exFat one because I want to be able to occasionally use that SSD with my Windows laptop on the go. Now, I am trying to figure out how to force RaspiBlitz to mount and use only the ext4 partition. The automatic installer insists on making one large partition, so I can't proceed with the installation. Perhaps you can create a docker container and pass the mount point of your ext4 partition as a volume and install raspiblitz inside that? Then it will only be able to see the ext4 partition. Title: Re: c-lightning on Raspberry Pi Post by: DaveF on January 26, 2021, 03:31:57 PM For an synced install with nothing else installed / configured:
Code: admin@raspberrypi:~ $ df So yes, 500GB is more then enough. If you go through and install all the optional items like ElectRS / BTCPay / mempool.space explorer / etc you can chew though some more storage. When I installed all of it, I was well over 400GB and the RPi was about to melt it was trying to so much. -Dave Title: Re: c-lightning on Raspberry Pi Post by: darkv0rt3x on January 26, 2021, 07:29:16 PM I'm using symlinks just to point ~/.bitcoin folder to my actual data folder and despite the fact that I also have corrupted data once or twice, I'm not sure I can say it's due to symlinks. But that's a good point.
I am using a 1 TB drive in ext4 and I think better than this, only btrfs or ZFS. But I'm not running my nodes in a Raspbi as the OP. Anyways, I only wanted to point out that I'm using symlinks too with not big issues. Title: Re: c-lightning on Raspberry Pi Post by: Rath_ on January 26, 2021, 11:35:31 PM If it's the latter, and it probably is if other folders on that filesystem are getting corrupted too, then the ext4 filesystem was made on the wrong device. You should be making it on one like /dev/sda1 or sda2, not /dev/sda (which is reserved for the partition table on-disk). The corruption occurred when my whole drive was formatted to exFat. Thanks for your instructions, though. Perhaps you can create a docker container and pass the mount point of your ext4 partition as a volume and install raspiblitz inside that? Then it will only be able to see the ext4 partition. That's a good idea. However, RaspiBlitz is only available as an OS image. I have decided to make a clean setup and install everything manually again. Title: Re: Raspberry Pi - corrupted blocks Post by: Rath_ on January 27, 2021, 10:06:53 AM Both Bitcoin Core compilation and SCP upload finished over night, so I decided to give it another try. All the blockchain data is now stored on the ext4 partition. Unfortunately, something is still wrong. Block #667874 was the latest one at that moment.
Code: 2021-01-27T09:54:38Z init message: Verifying blocks... Here (https://pastebin.com/raw/81MBNKXx)'s my complete debug.log. Title: Re: Raspberry Pi - corrupted blocks Post by: NotATether on January 27, 2021, 01:06:06 PM Both Bitcoin Core compilation and SCP upload finished over night, so I decided to give it another try. All the blockchain data is now stored on the ext4 partition. Unfortunately, something is still wrong. Block #667874 was the latest one at that moment. Do you think that the SCP upload got disrupted mid-copy and abruptly left some blocks partially-copied? I don't think SCP attempts to finish copying interrupted transfers. It's ssh so even if you have a fast network, it's still encrypting all the files it sends and maybe the speed was so slow that it stopped transferring the files. Given that the scp transfer took a whole night the speed probably wasn't that great. Possibly several dozen Kbytes/s? Check your block files with md5sum and make sure they're all the same as the ones you have locally. Try using rsync or some other copy tool that doesn't make swathes of encryption. Title: Re: c-lightning on Raspberry Pi Post by: DaveF on January 27, 2021, 01:24:30 PM Weird, since 667874 was just mined and causing the issue I wonder if something else was "touching" that information / block when you were moving the data.
Other than that the only thing I can think of is there is some hardware issue that is only showing itself when the system is under load, like when it's verifying the blockchain or a block. But, that is really a stretch. Wish I could help more, but I got nothing....:( -Dave Title: Re: Raspberry Pi - corrupted blocks Post by: Rath_ on January 27, 2021, 01:55:31 PM Do you think that the SCP upload got disrupted mid-copy and abruptly left some blocks partially-copied? I don't think SCP attempts to finish copying interrupted transfers. I don't think that SCP is the culprit. I have already tried copying all the files using Windows Explorer when the whole drive was formatted to exFat. I am also using WinSCP which I believe resumes interrupted transfers. Given that the scp transfer took a whole night the speed probably wasn't that great. Possibly several dozen Kbytes/s? It was about 18-20 MB/s, so the transfer took about 5-6 hours. With a samba server, I could get those speeds up to 100 MB/s. Check your block files with md5sum and make sure they're all the same as the ones you have locally. I could do that, but it is very likely to be a waste of time. The error indicates that there is something wrong with the latest block which I believe was not downloaded on my PC. Other than that the only thing I can think of is there is some hardware issue that is only showing itself when the system is under load, like when it's verifying the blockchain or a block. But, that is really a stretch. I could try running either -reindex or -reindex-chainstate as the debug.log suggests, but I guess that would take a few days. When the drive was formatted to exFat, after the corruption had occurred on the Pi, I tried connecting it to my PC and replacing the "chainstate" folder with a backup, but I couldn't modify that folder unless I ran a Windows built-in repair tool. I have configured my samba server with the exFat partition and everything seems to be working fine. I have transferred a few large files back-and-forth and there are no corruptions so far. Title: Re: Raspberry Pi - corrupted blocks Post by: xyzzy099 on January 27, 2021, 02:22:14 PM Probably a stupid question, but you're not copying the blockchain while the client is running? If so, the database will be changing as it's copied, and you will always get corruption.
Title: Re: Raspberry Pi - corrupted blocks Post by: Rath_ on January 27, 2021, 02:23:48 PM Probably a stupid question, but you're not copying the blockchain while the client is running? If so, the database will be changing as it's copied, and you will always get corruption. Of course, not. I have turned off Bitcoin Core for every single transfer. It was also not running on the Raspberry Pi. Title: Re: Raspberry Pi - corrupted blocks Post by: Rath_ on January 28, 2021, 11:53:12 AM Why don't you just copy the files directly with the external USB drive from another device? it's faster and you don't need to worry about unstable/slow connection. Windows does not support ext4 natively, so I can't do it again. I have already done that when the whole drive was formatted to exFat. Bitcoin Core loaded everything properly from that drive on my main PC, but it failed on the Pi. I have run -reindex-chainstate on the Pi and I am currently waiting for it to finish. I think that there might be a problem with my power supply, so I will try replacing it once the reindex has finished. Edit: My Pi's red LED has started blinking which means that the voltage is too low. I will try replacing both the power brick and the cable. Title: Re: Raspberry Pi - corrupted blocks Post by: DaveF on January 28, 2021, 12:22:56 PM Edit: My Pi's red LED has started blinking which means that the voltage is too low. I will try replacing both the power brick and the cable. Are you using a RPi3 or a 4, I'm on a mobile at the moment and I didn't see if you said which. I have had a 3 get stupid and trigger low voltage warnings when it was not really low. Was another issue, but I wasted a lot of time tracking it down. Found a few people on the RPi discussions that had the same problem. -Dave Title: Re: Raspberry Pi - corrupted blocks Post by: Rath_ on January 28, 2021, 12:29:18 PM Not sure if it's related, but what is version of linux kernel your Pi? I remember there's new/updated exFAT on version 5.7. I don't think it's related since my blockchain data is currently stored on the ext4 partition. My kernel version is 5.4. Are you using a RPi3 or a 4, I'm on a mobile at the moment and I didn't see if you said which. I have had a 3 get stupid and trigger low voltage warnings when it was not really low. I am using a RPi 4 with 4 GB of RAM. Title: Re: Raspberry Pi - corrupted blocks Post by: Rath_ on January 28, 2021, 09:15:34 PM Edit: My Pi's red LED has started blinking which means that the voltage is too low. I will try replacing both the power brick and the cable. I decided not to wait for the reindex to finish as it would take a few more days. I have replaced the power brick, formatted the SSD to exFat and copied all folders from my data directory again. It looks like the power supply was the culprit. My node works perfectly fine now. I am surprised that I was able to compile Bitcoin Core without any problems. The Pi was under load for about 3.5 hours and nothing was corrupted in the process. Thanks for your help everyone. |