The Lightning Network FAQ

[HeRetiK]

Does anyone have a c-lightning node being watched by an Eye of Satoshi? To my understanding those are the only watchtowers that are currently supported by c-lightning's watchtower plugin and all the public / private altruistic watchtowers I found seem to be LND based.

[darkv0rt3x]

So the other day one of my nodes had an issue and I shut it down:
https://bitcointalk.org/index.php?topic=5369431.msg58486063#msg58486063

Then a couple of posts later @Rath_ did point out that you could tweak the config files of both LND and c-lightning to change the lock times that you create and accept. I knew that but never really thought about it. We should probably come up with a list of "things you might not want to leave at default" that can be discussed.  Timelock / fees / min and max channel size /

-Dave

I have some of those setup. Time Lock an min channel size, iirc.
These are some of my settings:

Code:

"experimental-dual-fund": true,
"experimental-onion-messages": true,
"experimental-offers": true,
"watchtime-blocks": 720,
"max-locktime-blocks": 2016,
"funding-confirms": 3,
"fee-base": 1000,
"fee-per-satoshi": 10,
"max-concurrent-htlcs": 30,
"min-capacity-sat": 400000,

Most of these are default and are settings in c-lightningv0.10.2.

[Rath_]

Does anyone have a c-lightning node being watched by an Eye of Satoshi? To my understanding those are the only watchtowers that are currently supported by c-lightning's watchtower plugin and all the public / private altruistic watchtowers I found seem to be LND based.

I couldn't find any other watchtower software for c-lightning as well. I still haven't set it up, but I can give it a try if you have any particular questions about it.

[HeRetiK]

Does anyone have a c-lightning node being watched by an Eye of Satoshi? To my understanding those are the only watchtowers that are currently supported by c-lightning's watchtower plugin and all the public / private altruistic watchtowers I found seem to be LND based.

I couldn't find any other watchtower software for c-lightning as well. I still haven't set it up, but I can give it a try if you have any particular questions about it.

Ah it's fine, I was just wondering if maybe I missed something. The knowledge out there regarding c-lightning seems rather limited compared to LND. I currently almost regret getting started with c-lightning over LND since the latter seems to have a more mature ecosystem but it might just be a grass-is-greener-on-the-other-side type of situation.

[Rath_]

I currently almost regret getting started with c-lightning over LND since the latter seems to have a more mature ecosystem but it might just be a grass-is-greener-on-the-other-side type of situation.

I also have mixed feelings. While I love c-lightning's dual-funding and multifund, I also feel like I am missing out on something. On the other hand, I have seen complaints on Reddit that LND can't keep up anymore if you are running a medium sized node on a low-power device like Raspberry Pi, which many people do because of various out-of-the-box LND based nodes. Anyway, I believe that it's important to support implementation diversification.

Would you mind sharing your node's id?

[DaveF]

I currently almost regret getting started with c-lightning over LND since the latter seems to have a more mature ecosystem but it might just be a grass-is-greener-on-the-other-side type of situation.

I also have mixed feelings. While I love c-lightning's dual-funding and multifund, I also feel like I am missing out on something. On the other hand, I have seen complaints on Reddit that LND can't keep up anymore if you are running a medium sized node on a low-power device like Raspberry Pi, which many people do because of various out-of-the-box LND based nodes. Anyway, I believe that it's important to support implementation diversification.

Would you mind sharing your node's id?

All the 'nodes in a box' that I run are LND.
I have had over a dozen active channels on them with no issues.
These are all RPi4 devices with SSD. Even the one with the now dead fan is still up with no heat throttling.

So for personal stuff with less then .01BTC spread out *I* have found no performance issues.

IMO, beyond that you are risking a lot of time and effort to recover a bunch of funds if something goes wrong. Remember, no RAID for the storage, no redundant PS, and the OS sitting on an SD card. So once again IMO if you have more funds / channels then that you REALLY should be on "real" hardware.

But for home use they are fine.

Side note, because of the fan failure I am shutting them all down and moving them to a new location. Just if there is an issue having then at home, like the one with the bad fan, I will notice a lot sooner. Had it happened to the ones in the office now 5 days later I still would not have known and since the fan is dead I might not have till the next time of intense CPU activity....

-Dave

[HeRetiK]

I also have mixed feelings. While I love c-lightning's dual-funding and multifund, I also feel like I am missing out on something. On the other hand, I have seen complaints on Reddit that LND can't keep up anymore if you are running a medium sized node on a low-power device like Raspberry Pi, which many people do because of various out-of-the-box LND based nodes. Anyway, I believe that it's important to support implementation diversification.

Would you mind sharing your node's id?

I'd rather not post my node id publicly from my Bitcointalk account for now as I'm not yet entirely sure what this would mean for my privacy.

It's a small c-lightning node with channels in the single digits running on a basic RaspiBlitz setup. Currently I'm a bit torn between wanting to increase my node's liquidity and the fact that it's running on a Raspi 4 with "no RAID for the storage, no redundant PS, and the OS sitting on an SD card", to quote DaveF. Hence why I'm mainly looking into backups and failsafes for now.

What puts my mind somewhat at ease is that on RaspiBlitz c-lightning comes with the backup plugin already set up and running, mirroring the channel states both on the SSD and the SD card. Interestingly enough this isn't mentioned anywhere, I stumbled upon this by accident when trying to set up the plugin and finding it already preconfigured. What I also learned that day though was that even something as simple as connecting an USB drive can apparently mess with the Raspi's voltage enough to impair its stability, so I don't want to think about what could happen to system integrity in case of a power outage. To be fair it was some random old USB drive I found in one of my spare part boxes so who knows what its power drain was.

So in the end it's probably more the reliability of Raspis that is worrying me, rather than c-lightning. Generally speaking I'm happy with c-lightning's features, especially its ability of backing up the full channel state (which ironically now is my main source of paranoia as well), but the LND ecosystem seems to offer many useful tools that c-lightning seems to be lacking (e.g. easier ways of rebalancing, available Watchtowers). On the other hand I'm not sure how reliable those tools are in practice and whether I just need to dive deeper into the workings of c-lightning. Nonetheless I am considering setting up LND on my desktop PC, just for testing.

[DaveF]

...What puts my mind somewhat at ease is that on RaspiBlitz c-lightning comes with the backup plugin already set up and running, mirroring the channel states both on the SSD and the SD card....

That's great if you can get to them, and spin up a new node and sync it before the other people force close all the channels since you are offline.
I'm a tech geek, and a BTC geek and I admit it. Had something catastrophic happened to my node with the bad fan I could have been back up and running with spare hardware in under a day.
If you don't have the spare HW to do this, it IS something you should be thinking about. And definitely something we should be discussing more here.

-Dave

[Rath_]

I'd rather not post my node id publicly from my Bitcointalk account for now as I'm not yet entirely sure what this would mean for my privacy.

If you are running behind Tor then you should be concerned only about the origin of coins which you used to open your channels. Currently, all funding transactions are public. If you decide to stick with c-lightning and grow your node, we can open a dual-funded channel.

Note that some Lightning explorers log almost everything including IP address and alias changes. If you were running your node on clearnet in the past, you should close all of your channels, mix your coins and set up another node with a new public key.

[HeRetiK]

...What puts my mind somewhat at ease is that on RaspiBlitz c-lightning comes with the backup plugin already set up and running, mirroring the channel states both on the SSD and the SD card....

That's great if you can get to them, and spin up a new node and sync it before the other people force close all the channels since you are offline.
I'm a tech geek, and a BTC geek and I admit it. Had something catastrophic happened to my node with the bad fan I could have been back up and running with spare hardware in under a day.
If you don't have the spare HW to do this, it IS something you should be thinking about. And definitely something we should be discussing more here.

-Dave

That's partly why I'm setting up a lightning node on my desktop PC as well -- not just for testing, but so that I have a fully synced blockchain ready in case I'm in a pinch. My main worry is the channel state getting corrupted on both the SSD and the SD card at the same time. Which may seem a bit paranoid but I'm the kind of person that keeps at least 2 current backups at all times + periodical backups that are kept at physically separate locations. Accordingly the combination of SD card + Raspi makes me a bit uneasy.

To reduce the risk of outage-caused data loss I've actually been looking into UPS systems. However the consumer grade UPS systems seem rather unreliable (eg. occasional serious overheating problems; cases where the power on the UPS caused a power outage rather than preventing one) and the enterprise level UPS systems are too expensive for my purposes (might as well just build a proper server at that point). So for now I'm looking at what other contingencies I could put in place.

Along the way I found this guide on "How to recover funds from a unilateral channel closure" in case everything goes wrong, hopefully none of us will need to refer to this in the future
https://github.com/mandelbit/bitcoin-tutorials/blob/master/CLightningRecoverFunds.md

This recovery method hinges on the channel using option_static_remotekey though and while the c-lightning docs mention this option being the default since 0.7.3 and LND refers to it in its source code I have no idea how wide spread it is in practice -- maybe someone else can shed some light on this, I'm still very early in my journey.

I'd rather not post my node id publicly from my Bitcointalk account for now as I'm not yet entirely sure what this would mean for my privacy.

If you are running behind Tor then you should be concerned only about the origin of coins which you used to open your channels. Currently, all funding transactions are public. If you decide to stick with c-lightning and grow your node, we can open a dual-funded channel.

Note that some Lightning explorers log almost everything including IP address and alias changes. If you were running your node on clearnet in the past, you should close all of your channels, mix your coins and set up another node with a new public key.

I'm aware of these points, but they are definitely worth repeating.

Thing is, while I don't feel the need for strict opsec, I also don't want to make linking my lightning node to my Bitcointalk account as easy as a quick google search I might change my stance in the future or send you a PM once I got a firmer grip on running a node, but for now I don't want to post something that I can't un-post.

On a sidenote, thanks for starting this thread! It's been a treasure trove of practical knowledge that really helped me getting started.

[PrimeNumber7]

Ultimately, you need to backup your LN node immediately after your node receives a partially signed closing tx from your channel-peer for every transaction (including when you facilitate a transfer). If you don't want there is the potential you will be able to close any of your channels, and that if you try to close a channel, your entire balance will be lost when you try to close it when an old channel-state.

If someone has your partially signed closing transactions received from your channel peers, they are not going to be able to steal your coin, so keeping these backups online should be safe.

You should be able to backup your node for about $0.05 for every 10k backups, plus <$0.01 per month if using a storage bucket and creating a new object for each new backup. You can set the permissions for the bucket you are using for backups such that no one has read access, and the IP of your node has write-only access to your bucket. If your node crashes, you can edit the permissions, and manually find the appropriate backups.

[LoyceV]

To reduce the risk of outage-caused data loss I've actually been looking into UPS systems. However the consumer grade UPS systems seem rather unreliable (eg. occasional serious overheating problems; cases where the power on the UPS caused a power outage rather than preventing one) and the enterprise level UPS systems are too expensive for my purposes (might as well just build a proper server at that point). So for now I'm looking at what other contingencies I could put in place.

Have you considered a laptop? It's cheap, and comes with build-in UPS.

[HeRetiK]

Ultimately, you need to backup your LN node immediately after your node receives a partially signed closing tx from your channel-peer for every transaction (including when you facilitate a transfer). If you don't want there is the potential you will be able to close any of your channels, and that if you try to close a channel, your entire balance will be lost when you try to close it when an old channel-state.

If someone has your partially signed closing transactions received from your channel peers, they are not going to be able to steal your coin, so keeping these backups online should be safe.

You should be able to backup your node for about $0.05 for every 10k backups, plus <$0.01 per month if using a storage bucket and creating a new object for each new backup. You can set the permissions for the bucket you are using for backups such that no one has read access, and the IP of your node has write-only access to your bucket. If your node crashes, you can edit the permissions, and manually find the appropriate backups.

In c-lightning's docs it's explicitely pointed out that the SQLITE3 database containing the channel states is unencrypted and thus should not be stored in cloud storage. I assume the same is true for LND's channel database.

I'm not quite sure what an adversary could actually do with that file though? From my limited understanding they should only be able to read your channel balance and see what payments got routed through your node -- the latter which would probably allow for a reconstruction of network transaction flow if enough node databases are spied upon -- but I don't think it should enable any acutely damaging behaviour like triggering a penalty transaction by sending an old state, right?

To reduce the risk of outage-caused data loss I've actually been looking into UPS systems. However the consumer grade UPS systems seem rather unreliable (eg. occasional serious overheating problems; cases where the power on the UPS caused a power outage rather than preventing one) and the enterprise level UPS systems are too expensive for my purposes (might as well just build a proper server at that point). So for now I'm looking at what other contingencies I could put in place.

Have you considered a laptop? It's cheap, and comes with build-in UPS.

It actually did cross my mind! But only after spending countless hours researching possible UPS solutions for my current setup In hindsight it's a super obvious alternative so that's probably the route I'll take further down the road.

[Rath_]

This recovery method hinges on the channel using option_static_remotekey though and while the c-lightning docs mention this option being the default since 0.7.3 and LND refers to it in its source code I have no idea how wide spread it is in practice

All of my existing channels support it. You are very unlike to open a channel to someone with outdated or modified software.

Ultimately, you need to backup your LN node immediately after your node receives a partially signed closing tx from your channel-peer for every transaction (including when you facilitate a transfer).

Commitment transactions are updated more often than that. Even if there is no activity in the channel, new transactions are signed periodically with different fees based on the current mempool state.

[HeRetiK]

Ultimately, you need to backup your LN node immediately after your node receives a partially signed closing tx from your channel-peer for every transaction (including when you facilitate a transfer).

Commitment transactions are updated more often than that. Even if there is no activity in the channel, new transactions are signed periodically with different fees based on the current mempool state.

I was already wondering what else changes a channel state, other than routing transactions. Do changes in fee policy trigger a commitment transaction as well?

[PrimeNumber7]

Ultimately, you need to backup your LN node immediately after your node receives a partially signed closing tx from your channel-peer for every transaction (including when you facilitate a transfer). If you don't want there is the potential you will be able to close any of your channels, and that if you try to close a channel, your entire balance will be lost when you try to close it when an old channel-state.

If someone has your partially signed closing transactions received from your channel peers, they are not going to be able to steal your coin, so keeping these backups online should be safe.

You should be able to backup your node for about $0.05 for every 10k backups, plus <$0.01 per month if using a storage bucket and creating a new object for each new backup. You can set the permissions for the bucket you are using for backups such that no one has read access, and the IP of your node has write-only access to your bucket. If your node crashes, you can edit the permissions, and manually find the appropriate backups.

In c-lightning's docs it's explicitely pointed out that the SQLITE3 database containing the channel states is unencrypted and thus should not be stored in cloud storage. I assume the same is true for LND's channel database.

I'm not quite sure what an adversary could actually do with that file though? From my limited understanding they should only be able to read your channel balance and see what payments got routed through your node -- the latter which would probably allow for a reconstruction of network transaction flow if enough node databases are spied upon -- but I don't think it should enable any acutely damaging behaviour like triggering a penalty transaction by sending an old state, right?

The only thing an adversary could do with an old channel state would be to reconstruct your old transaction history, the majority of which wouldn't even be your transactions. So you would potentially lose some privacy, however, this information is already being stored online (on the server hosting your LN node).

I understand that the transactions are not valid as stored, as they are missing the signature from your LN node (you can trivially produce the signatures via the use of your private keys, which should not be stored in cloud storage). So if someone were to get the closing transactions, they could not do any financial damage.

[Rath_]

I was already wondering what else changes a channel state, other than routing transactions. Do changes in fee policy trigger a commitment transaction as well?

No, the fee policy affects only off-chain payments. I don't think that there is anything beside on-chain fees and off-chain payments that can trigger an update. Any other channel related parameters are negotiated before the funding transaction is broadcast and they cannot be changed.

[Wind_FURY]

Did anyone of you who runs a Lightning routing node experience someone connecting privately to you, route coins through you, then close the channel? I believe Lightning deniers foresee something that they don’t want every Bitcoin user to discover.

[Rath_]

Did anyone of you who runs a Lightning routing node experience someone connecting privately to you, route coins through you, then close the channel?

It didn't happen to me so I guess that only extremely large/well-connected nodes experience it.

I believe Lightning deniers foresee something that they don’t want every Bitcoin user to discover.

I am not sure why you mentioned Lightning deniers here. The above sounds to me like a use of someone's Lightning node to mix coins.

[Wind_FURY]

Did anyone of you who runs a Lightning routing node experience someone connecting privately to you, route coins through you, then close the channel?

It didn't happen to me so I guess that only extremely large/well-connected nodes experience it.

I believe Lightning deniers foresee something that they don’t want every Bitcoin user to discover.

I am not sure why you mentioned Lightning deniers here. The above sounds to me like a use of someone's Lightning node to mix coins.

No, not as mere mixers, but a censorship-resistant method to increase Bitcoin’s anon-set, and also the ability to reset the anon-set and increase it back again if needed.

Lightning routing node operators can charge higher fees for that, and incetivize others to run their own nodes.