Title: [solved] LNBits won't start due to python error Post by: darkv0rt3x on October 28, 2021, 10:26:45 PM Hello
I'm trying to setup my LNBits manually without using the easy way which is installing Caddy. I want to set it up by hand and have full control of what is going on. My setup is the following: Device: RockPro64 Router: NetGear R7800 running OpenWrt OS: Debian BullsEye - Linux rockpro64 5.10.0-8-arm64 #1 SMP Debian 5.10.46-4 (2021-08-03) aarch64 GNU/Linux LN node: C-Lightning v0.10.1-38-g8f782b0 LNBits: repository cloned from github, so I think it's master branch version Web Server: nginx/1.18.0 I follwoned these steps to install LNBits, from their github: Code: git clone https://github.com/lnbits/lnbits.git I also watched this video, until Caddy part: https://www.youtube.com/watch?v=WJRxJtYZAn4 My LNBits .env file is like this (the relevant parts): Code: $ cat .env My nginx config file, which is in /etc/nginx/conf.d/my.awesome.lnbits.site.com is like this: Code: server { My /etc/nginx/sites-available/default files is the default one. Didn't touch it! My router firewall rule to forward the traffic to my rp64 is the following: Code: config redirect So, after all this, when I try to run the command: Code: ./venv/bin/hypercorn -k trio --bind 0.0.0.0:5000 'lnbits.app:create_app()' I get this error: Code: ✔️ CLightningWallet seems to be connected and with a balance of [i]not enough[/i] msat. If I use http://my.awesome.lnbits.site.ddns.net:5000, the site works, no idea how if LNBits is not running! Note: I changed some folder names so that I can keep some of my privacy. I also changed the site name for the same reasons. So, this location "/home/rp64/Downloads/ doesn't really exist, but you all get the point. Same for the site name my.awesome.lnbits.site.ddns.net and ssl certificates. Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on October 28, 2021, 10:37:31 PM Well, I need to reformulate this thread. It happens that I just noticed that I had one instance of LNBits running within screen.
So, the problem is now different. I can't make it work on https! I'll rework the original thread to reformulate the problem! Edited; Well, maybe I don't need to reformulate the original thread and instead just post here the new problem! So, the only different thing is that LNBits is running but only on http. I have the ssl certificates set up, I have them in nginx config file, but if I try to use https I get this error on my laptop web browser: Code: Secure Connection Failed Title: Re: LNBits won't start due to python error Post by: DaveF on October 29, 2021, 01:29:16 AM Can you post the config of the nginx file?
That error usually occurs when the browser thinks its connecting with ssl / https but the server is just putting out non https on that port. Going offline till the morning, but you can probably google it. -Dave Title: Re: LNBits won't start due to python error Post by: NotATether on October 29, 2021, 04:59:55 AM OSError: [Errno 98] Address already in use - This means you ran another service that is listening to the same port.
Code: Secure Connection Failed This means you are trying to connect to an HTTPS site using HTTP protocol (add "https://" at the beginning of the IP address). Also you should try to get LNBits to listen for HTTPS traffic - you can't just add a certificate to a site and it magically understands HTTPS traffic. Title: Re: LNBits won't start due to python error Post by: vv181 on October 29, 2021, 05:04:14 AM Code: proxy_pass https://localhost:5000/; # ----> directive that sets up the reverse proxy in nginx Code: ./venv/bin/hypercorn -k trio --bind 0.0.0.0:5000 'lnbits.app:create_app()' I think your nginx.conf should be: Code: proxy_pass http://localhost:5000/; The Hypercorn didn't manage the SSL certs. So your Nginx passing a HTTPS request onto the Hypercorn. Try to change your Nginx conf as I suggested above, lets see if that works. EDIT: In addition, my suggestion above would probably work if you access it from my.awesome.lnbits.site.ddns.net. Since you set up the Nginx configuration only as a reverse proxy for that domain. In another hand, the reason why it works on HTTP but not on HTTPS when you access it from the local network[192.168.1.153:5000] is because the connection didn't managed by Nginx, it comes from Hypercorn. The Nginx has no configuration for a local connection, thus the error you get is because you are accessing an HTTP site(LNBits) using an HTTPS protocol, and the Hypercorn got no idea about any SSL certs, etc. Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on October 29, 2021, 08:11:22 PM Can you post the config of the nginx file? That error usually occurs when the browser thinks its connecting with ssl / https but the server is just putting out non https on that port. Going offline till the morning, but you can probably google it. -Dave It is in my first post, but here it is: Code: server { OSError: [Errno 98] Address already in use - This means you ran another service that is listening to the same port. Code: Secure Connection Failed This means you are trying to connect to an HTTPS site using HTTP protocol (add "https://" at the beginning of the IP address). Also you should try to get LNBits to listen for HTTPS traffic - you can't just add a certificate to a site and it magically understands HTTPS traffic. Of course I used https:// before the IP address. How would I make LNBits to listen for HTTPS traffic? Change listen 5000 ssl http2 default_server; to listen 443 ssl http2 default_server; ?? Code: proxy_pass https://localhost:5000/; # ----> directive that sets up the reverse proxy in nginx Code: ./venv/bin/hypercorn -k trio --bind 0.0.0.0:5000 'lnbits.app:create_app()' I think your nginx.conf should be: Code: proxy_pass http://localhost:5000/; The Hypercorn didn't manage the SSL certs. So your Nginx passing a HTTPS request onto the Hypercorn. Try to change your Nginx conf as I suggested above, lets see if that works. EDIT: In addition, my suggestion above would probably work if you access it from my.awesome.lnbits.site.ddns.net. Since you set up the Nginx configuration only as a reverse proxy for that domain. In another hand, the reason why it works on HTTP but not on HTTPS when you access it from the local network[192.168.1.153:5000] is because the connection didn't managed by Nginx, it comes from Hypercorn. The Nginx has no configuration for a local connection, thus the error you get is because you are accessing an HTTP site(LNBits) using an HTTPS protocol, and the Hypercorn got no idea about any SSL certs, etc. This is how I have my /etc/nginx/conf.d/my.awesome.lnbits.site.ddns.net.conf Code: ssl_session_cache shared:SSL:10m; However, I get this error when I try to access to https://my.awesome.lnbits.site.ddns.net:5000 Code: An error occurred during a connection to 192.168.1.153:5000. SSL received a record that exceeded the maximum permissible length. Title: Re: LNBits won't start due to python error Post by: DaveF on October 29, 2021, 09:05:40 PM Comments below in red you have 2 web servers on the same port. That can't work. You can't have 2 services listen like that.
Change the listen port on nginx to 5001 or something else and restart it and try again. Quote server { listen 5000 ssl http2 default_server; <--- This is the port that nginx is trying to listen on listen [::]:5000 ssl http2 default_server; server_name my.awesome.lnbits.site.ddns.net; # ---------> this site name is setup in in noip.com to avoid IP changes, if they happen ssl_certificate /etc/nginx/ssl/my.awesome.lnbits.site.crt; ssl_certificate_key /etc/nginx/ssl/my.awesome.lnbits.site.key; ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; location / { proxy_pass http://localhost:5000/; # ----> directive that sets up the reverse proxy in nginx <---But it can't since this is already on port 5000 } } You should also have some logging setup on nginx to see what else is going on: Code: access_log /var/log/nginx/reverse-access.log; -Dave Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on October 29, 2021, 10:18:01 PM Comments below in red you have 2 web servers on the same port. That can't work. You can't have 2 services listen like that. Change the listen port on nginx to 5001 or something else and restart it and try again. Quote server { listen 5000 ssl http2 default_server; <--- This is the port that nginx is trying to listen on listen [::]:5000 ssl http2 default_server; server_name my.awesome.lnbits.site.ddns.net; # ---------> this site name is setup in in noip.com to avoid IP changes, if they happen ssl_certificate /etc/nginx/ssl/my.awesome.lnbits.site.crt; ssl_certificate_key /etc/nginx/ssl/my.awesome.lnbits.site.key; ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; location / { proxy_pass http://localhost:5000/; # ----> directive that sets up the reverse proxy in nginx <---But it can't since this is already on port 5000 } } You should also have some logging setup on nginx to see what else is going on: Code: access_log /var/log/nginx/reverse-access.log; -Dave log files have nothing meaningful. Just the same errors I see with sudo journalctl -eu nginx. And I tried my config file like this: Code: ssl_session_cache shared:SSL:10m; But I still can't connect to https://my.awesome.lnbits.site.ddns.net:5000. I get the same error: Code: Secure Connection Failed Title: Re: LNBits won't start due to python error Post by: HCP on October 29, 2021, 11:02:54 PM Code: ssl_certificate /etc/nginx/ssl/my.awesome.lnbits.site.crt; Code: ssl_certificate /home/rp64/certificates/server-cert-signed.pem; Have you been experimenting with the way you were creating the ssl certs? ??? Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on October 29, 2021, 11:10:19 PM Code: ssl_certificate /etc/nginx/ssl/my.awesome.lnbits.site.crt; Code: ssl_certificate /home/rp64/certificates/server-cert-signed.pem; Have you been experimenting with the way you were creating the ssl certs? ??? Openssl doesn't care about file names. It only cares about its contents, so the names are irrelevant, I guess. But the reason I changed the names, is because I've been using dummy names to keep some privacy such has server names, domains, folder structures, etc, but as I said, openssl doesn't care about file names, so the extension being .pem or .key or .crt is irrelevant! Edited; I just ran this check: Code: $ openssl s_client -connect my.awesome.lnbits.site.ddns.net:5000 I guess this is not good either! Title: Re: LNBits won't start due to python error Post by: DaveF on October 30, 2021, 12:46:02 AM But I still can't connect to https://my.awesome.lnbits.site.ddns.net:5000. I get the same error: Code: Secure Connection Failed Because nginx is now listening on 5001 not 5000 Quote Code: Secure Connection Failed Try going to https://192.168.1.153:5001 (assuming that the IP of that machine has not changed) you should get an SSL error about the name not matching but you should be able to connect. -Dave Title: Re: LNBits won't start due to python error Post by: vv181 on October 30, 2021, 06:32:55 AM Because nginx is now listening on 5001 not 5000 and I think @OP must change the destination port on the router config as below, right?Code: option dest_port '5001' I just ran this check: I believe it's because the site is still returning from the Hypercorn HTTP on port 5000, not from Nginx(5001).~ I guess this is not good either! Try to change the router config destination to 5001, then visit https://my.awesome.lnbits.site.ddns.net:5000 Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on October 30, 2021, 01:08:26 PM But I still can't connect to https://my.awesome.lnbits.site.ddns.net:5000. I get the same error: Code: Secure Connection Failed Because nginx is now listening on 5001 not 5000 Quote Code: Secure Connection Failed Try going to https://192.168.1.153:5001 (assuming that the IP of that machine has not changed) you should get an SSL error about the name not matching but you should be able to connect. -Dave Because nginx is now listening on 5001 not 5000 and I think @OP must change the destination port on the router config as below, right?Code: option dest_port '5001' I just ran this check: I believe it's because the site is still returning from the Hypercorn HTTP on port 5000, not from Nginx(5001).~ I guess this is not good either! Try to change the router config destination to 5001, then visit https://my.awesome.lnbits.site.ddns.net:5000 Ohhhh it is finally working, I guess! Let me tell the setup I did... Jeezzz, when you don't know enough about networking, you get pretty messed up head aches! So, nginx config file in /etc/nginx/conf.d/my.awesome.lnbits.site.ddns.net.conf is the following: Code: ssl_session_cache shared:SSL:10m; My LNBits .env files relevant line is: Code: PORT=5001 My router firewall config rule for LNBits is: Code: config redirect I think this is all... But now, I would love to understand what is behind the scenes and why I had this struggle with hypercorn (I have no idea what this software does)... So, if anyone can tell me how the traffic goes around these settings and also taking into account this hypercorn thing, I would love to hear about it! Because I can't use port 5000 and I'm not sure I understand why. I mean, I think I know why, but I don't understand it. I think I can't use port 5000 because I have it in nginx config file for the reverse proxy directive proxy_pass https://localhost:5000. Title: Re: LNBits won't start due to python error Post by: vv181 on October 30, 2021, 03:33:00 PM But now, I would love to understand what is behind the scenes and why I had this struggle with hypercorn (I have no idea what this software does)... In simple terms the traffic *should* goes around like this:So, if anyone can tell me how the traffic goes around these settings and also taking into account this hypercorn thing, I would love to hear about it! Because I can't use port 5000 and I'm not sure I understand why. I mean, I think I know why, but I don't understand it. I think I can't use port 5000 because I have it in nginx config file for the reverse proxy directive proxy_pass https://localhost:5000. Internet <> Router <> Nginx <> Hypercorn <> LNBits The reason why it got problems was that: 1. You are using port 5000 as a virtual server(Listening port) while port 5000 are already being used by Hypercorn(LNBits). Just like what @DaveF mention above As you've guessed. The reverse proxy directive is 5000 which it's the port brought up by Hypercorn(LNBits). So Nginx cant use port 5000, again. 2. Your initial router firewall configuration is bypassing Nginx, so, it directly connects into Hypercorn(LNBits) The dest_port should be pointed into the Nginx listen port, so it will be able to acknowledge the SSL setting. Since initially, you are using the 5000 port, either you are accessing from a local network or from the domain, it would always communicate to the HTTP site of LNBits. Using the above scheme, the traffic flow is like: Internet <> Router <> Hypercorn <> LNBits. To sum up, you are accessing LNBits, either from the domain or local[Internet], then your router firewall rule is port forwarding <IP>:5001 into 192.168.151:5001(src port, dest_ip:dest_port)[Router]. After that, the Nginx acknowledged what to do since the referred port are 5001, the one you set up, listen 5001 ssl...[Nginx]. And then it passed to Hypercorn and lastly LNBits. Title: Re: LNBits won't start due to python error Post by: DaveF on October 30, 2021, 03:49:37 PM Mildly OT, but basic networking simplified:
Every device on your network has an IP address. Lets use 192.168.1.100 On every address there 65,536 ports. A program can use many ports i.e. a webserver by default listens on 80 (http) and 443 (https) BUT only 1 program can use a port at a time. So once your webserver is using 80 nothing else can. Otherwise when another computer tries to talk to that computer on that port there would be no way of knowing which program it would be talking to. So on 192.168.1.100 you could have a webserver listening on 80 & 443 a mail server on 25 & 587 & 110 & 143 and a FTP server on 20 & 21 As for hypercorn it's a web server. So it is giving on web pages on whatever port you tell it to. In this case 5000. nginx is also a webserver. With the configuration you have it is listening on 5001 which you have encrypted and then passing everything blindly back to hypercorn on port 5000 which is unencrypted. Hypercorn then gives the info to nginx which then passes it back out to whoever connected to it on port 5001. This keeps hypercorn isolated from the rest of the internet. There are many how to guides on how to secure / harden nginx to only allow it to pass the traffic you want passed back to hypercorn (or wherever) A bit more detail. You can stop reading if you don't care. Tweaking nginx config can allow you to run several different webservers for different things and it will pass it pack as you tell it to in the config. So if you can have: my.website.com = A public IP address my.otherwebsite.com = the same public IP address and this.otherwebsite.com = the same ip address again You then in your firewall pass that public IP address back to the nginx server 192.168.1.100 80 And nginx will respond to them all on 192.168.1.100 port 80 but pass: my.website.com to 192.168.1.100 port 5000 my.otherwebsite.com 192.168.1.100 port 5000 this.otherwebsite.com another site someplace else on the internet Probably more in depth then you needed but now you know. As for why nginx as the public side and not hypercorn. It's just because it's a much more developed product that does a lot more. -Dave Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on October 31, 2021, 09:00:02 AM ... 2. Your initial router firewall configuration is bypassing Nginx, so, it directly connects into Hypercorn(LNBits) ... After reading @DaveF post, I'm not sure I understand this. @DaveF says that internet is passing traffic to Nginx (encrypted) and then Nginx passes it to Hypercorn (unencrypted) and then, Hypercorn passes it back to Nginx again and finally Nginx passes it back to whatever application is listening on that port. You say my router firewall is bypassing Nginx. I'm not sure I understand. Sorry. @DaveF From your post. Yes, the first part, I am aware. I mean, in a home network, all devices have their (what I call) internal IPs in some range like 192.168.1.1 up to 192.168.1.something. And each application in each device can listen in a port from (usually) >1024 up to 65534, since below 1024 there are default listening ports for pre-defined services such has ssh (21), ftp (22), http (80), https (443), etc, etc... But regarding the traffic flow in this case, I was not expecting that Hypercorn passes back to Nginx unencrypted data. I thought it would be Internet -> Nginx -> Hypercorn -> LNBits instead of Internet -> Nginx -> Hypercorn -> Nginx -> LNBits. Why there is this additional data forwarding from Hypercorn to Nginx. Title: Re: LNBits won't start due to python error Post by: DaveF on October 31, 2021, 11:07:41 AM But regarding the traffic flow in this case, I was not expecting that Hypercorn passes back to Nginx unencrypted data. I thought it would be Internet -> Nginx -> Hypercorn -> LNBits instead of Internet -> Nginx -> Hypercorn -> Nginx -> LNBits. Why there is this additional data forwarding from Hypercorn to Nginx. In theory there should never be any direct communication between nginx & LNBits. Nginx should only talk to Hypercorn and LNBits should only talk to Hypercorn Internet <--> nginx <--> Hypercorn <--> LNBits 5001 <--> 5001<-->5000 <--> 5000 <--> whatever / however hypercorn talks to LNBits However, if you have your firewall set to pass traffic from the internet to Hypercorn (port 5000) this entire process falls apart. Since now I can see hypercorn in the world. -Dave Title: Re: LNBits won't start due to python error Post by: HCP on October 31, 2021, 10:36:14 PM But regarding the traffic flow in this case, I was not expecting that Hypercorn passes back to Nginx unencrypted data. I thought it would be Internet -> Nginx -> Hypercorn -> LNBits instead of Internet -> Nginx -> Hypercorn -> Nginx -> LNBits. There isn't... it just perhaps wasn't explained very well.... and possibly the <---> isn't the best way to show things either.Why there is this additional data forwarding from Hypercorn to Nginx. Essentially you have "inbound" traffic flow that goes: Internet ---> nginx (port 5001) ---> Hypercorn (Port 5000) ---> LNBits Then the outbound flow that goes: LNBits ---> Hypercorn ---> nginx ---> Internet nginx is essentially receiving the inbound packets passing them to Hypercorn to do whatever it needs to do with them (pass to LNBits, dump, return webpage etc)... Hypercorn then passes it's response back to nginx, which then passes them back to wherever they need to be going on "The internet"™. Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on October 31, 2021, 10:46:41 PM But regarding the traffic flow in this case, I was not expecting that Hypercorn passes back to Nginx unencrypted data. I thought it would be Internet -> Nginx -> Hypercorn -> LNBits instead of Internet -> Nginx -> Hypercorn -> Nginx -> LNBits. There isn't... it just perhaps wasn't explained very well.... and possibly the <---> isn't the best way to show things either.Why there is this additional data forwarding from Hypercorn to Nginx. Essentially you have "inbound" traffic flow that goes: Internet ---> nginx (port 5001) ---> Hypercorn (Port 5000) ---> LNBits Then the outbound flow that goes: LNBits ---> Hypercorn ---> nginx ---> Internet nginx is essentially receiving the inbound packets passing them to Hypercorn to do whatever it needs to do with them (pass to LNBits, dump, return webpage etc)... Hypercorn then passes it's response back to nginx, which then passes them back to wherever they need to be going on "The internet"™. Yes, I understood it from @DaveF's post. It makes sense now when someone tells it like in plain text like @DaveF did. Technically, I still struggled to undderstand things the way @vv181 explained. But the 2 replies together, made it more clear, I think. The thing is that there is little explanation on how to set things up if you don't want to use 3rd party applications like Caddy, because apparently, Caddy does part of this job for you automatically! Title: Re: LNBits won't start due to python error Post by: vv181 on October 31, 2021, 11:56:34 PM 2. Your initial router firewall configuration is bypassing Nginx, so, it directly connects into Hypercorn(LNBits) You say my router firewall is bypassing Nginx. I'm not sure I understand. Sorry.Quote However, if you have your firewall set to pass traffic from the internet to Hypercorn (port 5000) this entire process falls apart. Since now I can see hypercorn in the world. Technically; What he means and also I said the word "initial", it pointed to: Code: (router firewall rule) config redirect But since you are now using the correct config: Code: (router firewall rule) option dest_port '5001' After reading @DaveF post, I'm not sure I understand this. @DaveF says that internet is passing traffic to Nginx (encrypted) and then Nginx passes it to Hypercorn (unencrypted) and then, Hypercorn passes it back to Nginx again and finally Nginx passes it back to whatever application is listening on that port. Regarding that, What *I think* he explains is he taking the context of the correct firewall rule config. and it is furtherly explained well by HCP.Anyway, sometimes I have a hard time to wording and explaining a technical term, so I hope you bear with me :P Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on November 01, 2021, 09:24:11 AM 2. Your initial router firewall configuration is bypassing Nginx, so, it directly connects into Hypercorn(LNBits) You say my router firewall is bypassing Nginx. I'm not sure I understand. Sorry.Quote However, if you have your firewall set to pass traffic from the internet to Hypercorn (port 5000) this entire process falls apart. Since now I can see hypercorn in the world. Technically; What he means and also I said the word "initial", it pointed to: Code: (router firewall rule) config redirect But since you are now using the correct config: Code: (router firewall rule) option dest_port '5001' After reading @DaveF post, I'm not sure I understand this. @DaveF says that internet is passing traffic to Nginx (encrypted) and then Nginx passes it to Hypercorn (unencrypted) and then, Hypercorn passes it back to Nginx again and finally Nginx passes it back to whatever application is listening on that port. Regarding that, What *I think* he explains is he taking the context of the correct firewall rule config. and it is furtherly explained well by HCP.Anyway, sometimes I have a hard time to wording and explaining a technical term, so I hope you bear with me :P Oh, yes, I got it. I will double check that my router config is forwarding the traffic to port 5001 and not 5000 by accident. One more question. This one is not exactly related to the configuration itself but I'll ask it anyway. You guys know what is the meaning of the following 2 lines in the configuration of my router? What means each one? Code: option dest_port '5001' and Code: option dest_dport '5001' What means, for instance, if I use 2 different ports there? Title: Re: LNBits won't start due to python error Post by: vv181 on November 01, 2021, 11:25:31 AM I guess you mean dest_port and src_dport. Either way, you can refer to the documentation below to see the details.
https://openwrt.org/docs/guide-user/firewall/firewall_configuration?s=destination&s=port#options4 (https://openwrt.org/docs/guide-user/firewall/firewall_configuration?s=destination&s=port#options4): Code: Name Type Required Default Description For the full documentation of what the firewall means, you can see: Redirect (https://openwrt.org/docs/guide-user/firewall/firewall_configuration?s=destination&s=port#redirects) and Port forwarding for IPv4 (DNAT) (https://openwrt.org/docs/guide-user/firewall/fw3_configurations/fw3_nat#port_forwarding_for_ipv4_dnat) Title: Re: LNBits won't start due to python error Post by: darkv0rt3x on November 01, 2021, 08:05:16 PM I guess you mean dest_port and src_dport. Either way, you can refer to the documentation below to see the details. https://openwrt.org/docs/guide-user/firewall/firewall_configuration?s=destination&s=port#options4 (https://openwrt.org/docs/guide-user/firewall/firewall_configuration?s=destination&s=port#options4): Code: Name Type Required Default Description For the full documentation of what the firewall means, you can see: Redirect (https://openwrt.org/docs/guide-user/firewall/firewall_configuration?s=destination&s=port#redirects) and Port forwarding for IPv4 (DNAT) (https://openwrt.org/docs/guide-user/firewall/fw3_configurations/fw3_nat#port_forwarding_for_ipv4_dnat) Ok, ignore... I was stupid! I'll call this thread solved. Thanks @DaveF, @vv181 and @HCF |