Make Jupyter AI a metapackage

[dlqqq]

Description

Implements #1495. See the pinned issue for more information.

Code changes

Makes Jupyter AI a metapackage, a blank package that exclusively lists other packages as dependencies which form a functional AI extension when installed together.

User-facing changes

None expected.

Backwards-incompatible changes

All developer APIs are now provided by subpackages. See the subpackages under jupyter-ai-contrib/ for more information.

[3coins]

@dlqqq
Thanks for working on this. Local build for me is failing at the moment python -m build, it could be related to the front end extension source missing, we will need to add an empty FE extension or figure how to make python only extensions work with releaser.

[dlqqq]

python -m build passes for me now. Will test with jupyter_releaser CLI after lunch.

[dlqqq]

Figured out how to get Jupyter Releaser to skip the NPM release. Needed to remove all references to the JupyterLab labextension in pyproject.toml and remove package.json. Check Release and python -m build both work now.

See these logs from the check-release workflow:

build-npm
--------------------------------------------------
Using default value for dist_dir: 'dist'
Using default value for package: '.'
Using default value for help: 'False'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Skipping build-npm since there is no package.json file
--------------------------------------------------
check-npm
--------------------------------------------------
Using default value for dist_dir: 'dist'
Using default value for npm_install_options: ''
Using env value for repo: 'jupyterlab/jupyter-ai'
Using default value for help: 'False'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Skipping check-npm since there is no package.json file

[dlqqq]

@3coins Thanks for the feedback. Removing the labextension-related portions of pyproject.toml was the right choice. After making those changes & removing workflows that are no longer needed, CI is passing on this PR. 👍

[dlqqq]

This PR is pretty much ready! Just verified that jupyter releaser build-python works locally, so it is possible to release this as jupyter_ai==3.0.0b8.

@Zsailer Right now, this PR lacks version range specifiers on the subpackages. I remember you had some thoughts on this topic. Should we just add version floors without ceilings, or should we use both floors & ceilings to ensure users always get a stable experience? cc @ellisonbg

[krassowski]

It would be nice to be able to test it out on Bidner.

[Zsailer]

@dlqqq I would recommend putting floors and no ceilings for now. Let's see how that works and adjust if issues arise.

[dlqqq]

Thanks, I've added version floors. Still waiting on some changes to jupyter_ai_jupyternaut and the first release of jupyter_ai_magic_commands.

As for Binder support, I'm not sure how to set that up, but I can make a note of this to explore after JupyterCon.