Adding a CLI

[Goldziher]

This has been an issue that is now closed #538 , because we didn't have a clear enough direction. The questions are:

1. what are the use cases for a CLI?
2. how compelling are they?
3. if we add a CLI, should it be a part of this repository or a separate package?
4. how to design the CLI and what packages to use.

[Alex-CodeLab]

0. What is a CLI?
   A CLI accepts commands in order to perform some (administrative) tasks, and to make it useful it should also

   • lists all available commands.
   • show info about the commands (--help)
1. • startproject - create a project structure. Not only useful to get started quickly, but also to somewhat standardize projects. A problem with flask/fastapi is that each project looks different because they all invent their own structure. Django does not have this issue, because most projects are created using the commands startproject and startapp.
   • listroutes - show available urls and related controller. (already in the example Add CLI #538 )
   • some things I find useful: runserver test test_db_connection clear_cache show_config
   • more ideas here https://docs.djangoproject.com/en/4.1/ref/django-admin/ and https://django-extensions.readthedocs.io/en/latest/command_extensions.html

3. once it all works well, include it. It would be nice to have some default commands. Commands like startproject could have a option to download a project-template from some separate repo/url/zip-file.

4. Cleo has nice colors, it is easy to add new commands and is already included in Poetry.

5. nice to have:

• tab-completion (this is possible when using Cleo).
• install the command using setup.py (scripts) during install.
• real commands like starlite startproject, instead of python main.py startproject

[Alex-CodeLab]

https://github.com/Alex-CodeLab/starlite/tree/cli4/commands

[Goldziher]

@Alex-CodeLab - I think a cli shouldn't require manual registration in main.py. I.e. this shouldn't be necessary

from starlite import Starlite, CLI

app = Starlite(...)

if __name__ == "main":
    cli = CLI(app=app)
    ...

One of the base features should be discovery of any Starlite application declared in the directories in the cwd.

I would also consider automatic finding of commands as a nice to have feature, but maybe that is a bit out there.

[Alex-CodeLab]

yes, but that all depends on the project-structure, because it needs to know where to look for the files.
In Django commands are always in folders like mymodule/management/commands.py

In Flask it looks for the project in a file called app.py (iirc).

I was thinking of something like this:

starlite.py

!#/bin/python

from main import app

if __name__ == "main":
    cli = CLI(app=app)
    cli.register(ListRoutes)
    cli.register(StartProject)
    ...

$ ./starlite.py mycommand

That requires that the entrypoint is always in main.py .

But, there is also a chicken&egg problem in case of a startproject, when the files do not yet exist

[Goldziher]

discovery can simply be done by finding invocations of Starlite() not inside tests.

[Alex-CodeLab]

Good. So in pseudo-code something like this:

# starlite.py  

app = discover_app()  #  in main.py, app.py, or similar
if not app:
   cli = CLI()
   cli.register(StartProject)  #  a empty project, so this the only available command 
else:
   cli = CLI(app=app)
   cli.register(ListRoutes)
   # cli.unregister(SomeExperiment)
cli.run()

I think that automatic discovery of commands is not very useful, because it takes away all control and introduces magic.
However, It would be nice if commands could also be registered from other modules, so, whenever I install some module, it automatically adds the commands for that module. (I'm not yet sure how to do this)

[provinzkraut]

I think automatic discovery is very useful, if be done right. I propose the following flow:

• If an argument is passed to the CLI for the app location, load it from there: starlette run --app "myapp/__main__.py:application"
• If an environment variable exists for the app location, load it from there: export STARLITE_APP=myapp/__main__.py:application
• Try autodiscovery by looking for canonical file names (asgi.py, app.py) in the cwd
  • Search for canonical application names within the files (app, application)
  • Search for references to Starlite

It would be nice if commands could also be registered from other modules, so, whenever I install some module, it automatically adds the commands for that module. (I'm not yet sure how to do this)

This is usually done with some sort of global registry that an external module could import from starlite and register the commands in. Think of something like SQLAlchemy's orm.registry

[provinzkraut]

I started work on the CLI and there a few open question:

What functionalities should the CLI include?

Currently I added:

1. Running the app (supports autodiscovery as well as setting it explicit via an env var)
2. Displaying a tree view of the routing map (with information about paths, handler type, sync/async mode, http methods)

Ideas:

• Request cache commands to clear/show size. Requires some modifications of the cache backens
• Server-side session commands (clear/delete/show active)
• "linter" command that could warn for unused route handlers and such.

How should new projects be generated?

This was a much requested feature, but I'm having doubts if this should actually go into the CLI.
For one, there is cookiecutter, which basically covers this and which is what I'd use internally anyway since there's no point in re-inventing the wheel here.

Another issue is that of dependencies. If you can generate a new project with starlite new, then you'd need to have Starlite already installed in your current environment, which means you'd need to setup the env before the project, which conflicts with most of the standard workflows. A fix to this could be to have the CLI be a separate package: starlite-cli.

Should the CLI be a standalone package

Pros:

• Can be installed globally and easily generate new projects

Cons:

• Higher maintenance burden

Should the CLI be included by default?

The CLI comes with extra dependencies not everyone needs. Whether or not we opt for a separate package,
should it be a base dependency or an extra (pip install starlite[cli])?

For me, the answer depends on the dependencies. If it's just click for example, I'd be okay with having it added by default. If there are more dependencies (e.g. cookiecutter for generating projects) it would be better to have it as an extra.

[Alex-CodeLab]

Using cookiecutter is fine, but that is a different project ( it could even be added without any new Cli functionality).
But cookiecutter isn't magic, it simply downloads a template in a zip or git-repo and does a search&replace (using Jinja, iirc).
The difficulty is in creating a useful tempate.

[provinzkraut]

it simply downloads a template in a zip or git-repo and does a search&replace

That's not really true. They're regular Jinja templates, so you can be quite expressive.

The difficulty is in creating a useful tempate.

Which we would need to do anyway?

No matter how we'll implement this, I don't see a better way than to do it with templates. So why not use something that's built for this exact purpose?

What do you envision that can't be done with cookiecutter + Jinja and how else would you solve it?

[Alex-CodeLab]

read again what I wrote please, because I'm not saying anything different. you're repeating what I wrote.
I'm just pointing out that they're jinja templates (in a zip or git repo).
So that could be processed with a few lines of code, or by using cookie-cutter, which does the same.

I would start with creating the templates, and install it with cookiecutter (this does not require any new CLI functionality, it is unrelated, so I don't understand why these discussions get mixed up all the time). Anyway, once it works, and the Cli framework is in place, perhaps create a command for it.
(I tried to start a discussion about the templates/folderstructure a few times, but never got a reply, so that is why I said it would be the difficult part.)

[provinzkraut]

Okay so we agree? I said that this doesn't necessarily need to go into the CLI and could just be done with cookiecutter alone.
I'm not sure what you're getting at if you agree that using cookiecutter is fine for this?

it is unrelated, so I don't understand why these discussions get mixed up all the time

Because people requested this feature for the CLI.