What can the recent core lightning memory exhaustion vulnerability teach us about handling untrusted gossip?
So I recently came across two memory exhaustion vulnerabilities disclosure in core lighting which I found interesting. Basically an attacker could send information to a lightning node which will cause its internal state to continuously grow until it potentially runs out RAM and crash.
Malicious peer
↓
Network messages
↓
Core Lightning node
↓
Internal state grows
↓
Memory exhaustion
↓
Node becomes unavailable
The first vulnerability involved the message queue between
connectd and
gossipdNodes temporarily keeps channel_update messages it receives for channels it doesn’t know about and there is actually a valid reason for that, where the problem came about was when an attacker deliberately send enough messages to make the queue grow indefinitely. A fix was made for this which was the introduction of a cutoff of 500,000 messages which allows additional messages to be dropped instead of letting memory usage continue growing.
The second vulnerability was actually discovered while testing the fix for the first vulnerability, when a node receives an update for an unknown SCID it may assume it has missed the corresponding channel announcement and keep track of it so the missing information can be retrieved. An attacker could exploit this and send many different fake SCIDs and cause the internal map to keep growing.
Vulnerability 1
Attacker → many channel_updates → growing queue → RAM exhaustion
Vulnerability 2
Attacker → many fake SCIDs → growing SCID map → RAM exhaustion
So even after one path to memory exhaustion was addressed, further testing revealed another path to essentially the same problem.
A fix was made for this too but my main thought process was that nodes have a legitimate reason for accepting and retaining some information from untrusted peers, but at the same time any attacker could exploit that so should bounded resource consumption be treated as a general security requirement? I mean, instead of fixing individual memory exhaustion bugs when they are discovered, shouldn’t the design ensure no untrusted network input can ever cause such problems in the first place?
As the network keeps improving, maybe more efficient gossip or set reconciliation mechanisms could help reduce bandwidth and processing requirements and also the amount of state nodes need to retain.
