Updating CLN 23.08 -> 24.02: No module named 'grpc_tools'

[BJ2gCFtGbiVK]

I'm currently running Core Lightning v23.08 and attempting to upgrade. I pulled v24.02.1, but the 'make' command fails with the following error:

Code:

/usr/bin/python3: Error while finding module specification for 'grpc_tools.protoc' (ModuleNotFoundError: No module named 'grpc_tools')
make: *** [Makefile:390: contrib/pyln-grpc-proto/pyln/grpc/primitives_pb2.py] Error 1

Running 'python3 -m pip list' returns the following:

Code:

gpiozero 1.6.2
grpcio 1.62.1
grpcio-tools 1.62.1
idna 2.10

I don't think it should be necessary to do the following, but I tried 'python3 -m pip install grpc-tools'. That failed, returning the error:

Code:

Collecting grpc-tools
  Using cached grpc-tools-1.0.0.tar.gz (5.2 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-ys1uf5oz/grpc-tools_0c8d98a949ee48628a7bfd0821c2ba53/setup.py", line 33, in <module>
          raise RuntimeError(HINT)
      RuntimeError: Please install the official package with: pip install grpcio-tools
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

The hardware is a Pi 4B. Running 'cat /etc/os-release' returns the following:

Code:

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"

Apologies if this is a silly or duplicate question, but am I missing something obvious?

[ABCbits]

I don't use Core Lightning. But looking at their repository, i think your approach isn't right.
1. https://github.com/ElementsProject/lightning/blob/master/doc/getting-started/getting-started/installation.md state they use poetry rather than pip as a way to install python package.
2. https://github.com/ElementsProject/lightning/blob/master/poetry.lock doesn't mention anything about grpc-tools, so you probably tried to install non-relevant package.

Did you follow step-by-step build on their documentation?

[BJ2gCFtGbiVK]

I don't use Core Lightning. But looking at their repository, i think your approach isn't right.
1. https://github.com/ElementsProject/lightning/blob/master/doc/getting-started/getting-started/installation.md state they use poetry rather than pip as a way to install python package.
2. https://github.com/ElementsProject/lightning/blob/master/poetry.lock doesn't mention anything about grpc-tools, so you probably tried to install non-relevant package.

Did you follow step-by-step build on their documentation?

Thank you. I installed as per the RaspiBolt documentation here:
https://raspibolt.org/guide/bonus/lightning/cln.html

At the time I don't recall there being any conflict with the Elements documentation, but I understand the c-lightning-REST plugin was moved into the main codebase around 23.08. The issue may stem from how I upgraded to that version.

I installed poetry and though it didn't immediately solve the failed build issue, that gives me something to look into more closely and I'll update here if I make progress.

[BJ2gCFtGbiVK]

This issue seemed to be due to the user/group structure specific to the RaspiBolt design and my error. I had installed grpcio-tools with the --user flag as user 'admin', with the command:

Code:

python3 -m pip install --user grpcio-tools

The upgrading process, as user lightningd, didn't have access presumably for that reason. Running the same command as user lightningd solved that issue. Running lightning-cli --version then showed 24.02.

However, restarting the service using 'sudo systemctl restart lightningd.service' failed. The logs showed two messages:

Code:

lightningd: v24.02.1-93-gc4edec8
plugin-clnrest: Killing plugin: disabled itself: No module named 'gevent'

I installed gevent both as user 'admin' and as user 'lightningd'. On repeated failures to restart, I repeated the above for modules gunicorn, flask, flask_restx, flask_cors, flask_socketio and json5.

Finally, restarting failed due to missing module 'pyln'. It could not be installed, returning:

Code:

ERROR: Could not find a version that satisfies the requirement pyln
ERROR: No matching distribution found for pyln

Searching for this, I find only Elements and lightningd results:

https://github.com/ElementsProject/lightning/issues/3982
https://github.com/lightningd/plugins/issues/314

This may be relevant as well: https://pypi.org/search/?q=pyln

There seems to be an issue with the requirements. I don't know if I can go any farther, but I'll update if I make any progress.

[BJ2gCFtGbiVK]

I opened an Issue: https://github.com/ElementsProject/lightning/issues/7220