KiKit pip install broken on newer Linux distros (i.e. Ubuntu 24.04, etc.)

[dnotq]

Prerequisites

• I have read FAQ
• I have searched existing issues (including closed ones)
• I use KiKit at least version 1.6.0 (older version are not supported)

KiKit version

Install is broken

KiCAD version

8.0.5

Operating system

Kubuntu 24.04 kernel 6.8.0-40-generic (64-bit)

Description

I had a working kikit on my 22.04 LTS, but I updated to 24.04 LTS and now I get a kikit install warning any time I open a board in KiCad. I decided to redo the command line install part, but it looks like all things Python and pip have now changed in Linux-land and beyond. Lovely.

$ pip install kikit
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Of course kikit is not in the system package, and kikit is the ONLY reason I had to install pip at all, so yeah, deleting all of this now. There has got to be an easier way to install/run kikit... Isn't that the whole reason for using a scripting language?

Steps to Reproduce

Try to pip install KiKit on Ubuntu 24.04.

[set-soft]

You can find Debian packages here:
https://github.com/INTI-CMNB/KiKit/releases/tag/v1.6.0-2

This is not a KiKit issue, but a Python developers style of solving things (reinventing the wheel and applying the Windows style).

[dnotq]

Thanks for the reply and the link, I'll give it a shot. Yeah, I know this is not specifically a KiKit issue, Python is out of control IMO.

As an aside, it would be nice if KiKit would just "install" by unzipping the code into a directory of my choice and adding it to my path (or setting a KIKIT_DIR environment variable, etc.).

[set-soft]

As an aside, it would be nice if KiKit would just "install" by unzipping the code into a directory of my choice and adding it to my path (or setting a KIKIT_DIR environment variable, etc.).

I do this for all the scripts I fork. In the case of KiKit I didn't include the script yet. So now I committed it:
https://github.com/INTI-CMNB/KiKit/blob/master/src/kikit

Also note that I maintain a Debian repo that contains KiKit and its dependencies, here is explained: https://set-soft.github.io/debian/

[dnotq]

Thanks for the wrapper script. I have managed to piece together a solution that works for me. I like things simple, and unfortunately dependency hell is anything but simple (and getting worse, IMO). In case this helps anyone else, below is the process I used:

I found the list of dependencies from the kikit repo setup.py script:

install_requires=[
"numpy", # Required for MacOS
"pcbnewTransition >= 0.4.1, <=0.5",
"shapely>=2.0.3",
"click>=7.1",
"markdown2>=2.4",
"pybars3>=0.9",
"solidpython>=1.1.2",
"commentjson>=0.9"
]

Of these, Ubuntu 24.04 apt has all packages except for four:

• kikit
• pcbnewTransition
• pybars3
• pymeta3 (dependency of pybars3)

I found where my system python searches for packages by running:
$ python -m site

This yields a list of directories, of which there is a path in my home directory:
~/.local/lib/python3.12/site-packages

So I put cloned the missing repos there and copied the python code such that the modules would be picked up:

$ cd ~/.local/lib/python3.12/site-packages
$ git clone https://github.com/yaqwsx/KiKit.git kikit_git
$ git clone https://github.com/yaqwsx/pcbnewTransition.git pcbnewTransition_git
$ git clone https://github.com/wbond/pybars3.git pybars3_git
$ git clone https://github.com/wbond/pymeta3.git pymeta3_git
$ cp -R kikit_git/kikit .
$ cp -R pcbnewTransition_git/pcbnewTransition .
$ cp -R pybars3_git/pybars .
$ cp -R pymeta3_git/pymeta .

At this point KiCad is happy, it finds KiKit and all is well. I copied the kikit wrapper script that @set-soft posted above (thank you) and put it in a directory in my path. Now I can also run kikit from the command line.

No magic, no fuss, no pip, no other non-system packages to manage or track. Updating is a little clunky, but it just means updating the git repos above and recopying the module files to their respective dirs. I'm ok with this. But it also means it won't break unexpectedly on me like it did last time, which put me in this situation to begin with.

[set-soft]

Hi @dnotq !
Note that the Debian repo that I maintain has all the dependencies you need as Debian packages. So you don't really need to worry about KiKit dependencies. And you can have both: KiKit from the repo and a cloned git repo to run KiKit experimental changes (this is what I do). All the stuff needed to package KiKit is in my fork, so you can even create a Debian package with some KiKit change.

[snhobbs]

Hey @set-soft any idea on issue tracking for this?
Kikit isn't the first place I've had this problem. The only way I've been able to fix it is by adding pyproject.toml files for each of the packages. It would be useful to add this to kikit but it seems that several dependencies also have the issue.

[set-soft]

Hi @snhobbs !
I'm not sure what exactly is your problem. This issue is about the message you find in new (relative new, because it was already there on Debian 12, which is a slow release distro) distros. The message warns about using pip for a system that has its own package manager. This is actually true for all Linuxes and also macOS, so this message will pop-up on most systems, just not on Windows, where pip is king because this is the "kingdom of chaos".

The message is part of a PEP recommendation from Python, not an idea from Linux distros.

What the message says is that installing something with pip on a system with its own package manager could lead to undesirable problems. So the solution is to avoid pip for systems with a package manager. This is something already known, and this is the reason behind the Debian packages I maintain.

IMHO Python developer are wrong, the problem is that they like the Windows chaos and solve things in the "Windows way". They should change their mind and cooperate with the system packages. The problem is the complexity, but their solution isn't acceptable.

While playing with AI inference I saw how bad pip can behave. When packages can weight GBs it becomes impossible to do things in the way pip was designed. And the mechanism used to declare dependencies in Python is obsolete, it doesn't work. Package systems like the one used by Debian solved these problems years ago.

In the case of KiKit on Linux: if your system is based on Debian, just use the packages I provide, this solves all the dependency problems. In fact I hack some stuff to avoid ridiculous dependencies. On other Linux systems: use Python virtual environments, they are a waste of resources, but more efficient than docker images.