verdi presto: Fall back to ZMQ (aiidateam#7284)

Change `verdi presto` to always configure a broker: it tries RabbitMQ
first and falls back to ZMQ if unavailable. Add `--use-zmq` flag to
skip RabbitMQ detection and use ZMQ directly.

Previously, profiles created without RabbitMQ had no broker at all,
limiting functionality. Now every `verdi presto` profile gets a working
broker out of the box.

Regenerate the command-line reference during the rebase so the tracked
autodocs stay in sync with the updated CLI.

Author: agoscinski

docs/source/reference/command_line.rst

@@ -320,18 +320,21 @@ Below is a list with all available subcommands.
       * Set up the localhost as a `Computer` and configure it
       * Set a number of configuration options with sensible defaults
 
-      By default the command creates a profile that uses SQLite for the database. It
-      automatically checks for RabbitMQ running on the localhost, and, if it can connect,
-      configures that as the broker for the profile. Otherwise, the profile is created without
-      a broker, in which case some functionality will be unavailable, most notably running the
-      daemon and submitting processes to said daemon.
+      By default the command creates a profile that uses SQLite for the database. For the
+      message broker, it automatically checks for RabbitMQ running on localhost. If found, it
+      configures RabbitMQ as the broker. Otherwise, it falls back to the ZMQ broker, which
+      requires no external services and is started automatically with the daemon.
 
       When the `--use-postgres` flag is toggled, the command tries to connect to the
       PostgreSQL server with connection paramaters taken from the `--postgres-hostname`,
       `--postgres-port`, `--postgres-username` and `--postgres-password` options. It uses
       these credentials to try and automatically create a user and database. If successful,
       the newly created profile uses the new PostgreSQL database instead of SQLite.
 
+      When the `--use-zmq` flag is toggled, the command skips the RabbitMQ auto-detection and
+      directly configures the ZMQ broker. To switch to RabbitMQ later, use `verdi profile
+      configure-rabbitmq`.
+
     Options:
       -p, --profile-name TEXT         Name of the profile. By default, a unique name starting
                                       with `presto` is automatically generated.  [default:
@@ -343,6 +346,12 @@ Below is a list with all available subcommands.
                                       options. The command attempts to automatically create a
                                       user and database to use for the profile, but this can
                                       fail depending on the configuration of the server.
+      --use-zmq                       When toggled on, the profile uses the ZMQ broker, which
+                                      requires no external services and is started
+                                      automatically with the daemon. When not specified, the
+                                      command automatically tries RabbitMQ first and falls
+                                      back to ZMQ if unavailable. To switch to RabbitMQ later,
+                                      use `verdi profile configure-rabbitmq`.
       --postgres-hostname TEXT        The hostname of the PostgreSQL server.
       --postgres-port INTEGER         The port of the PostgreSQL server.
       --postgres-username TEXT        The username of the PostgreSQL user that is authorized

src/aiida/cmdline/commands/cmd_presto.py

@@ -126,6 +126,13 @@ def detect_postgres_config(
     'the PostgreSQL server can be configured with the relevant options. The command attempts to automatically create a '
     'user and database to use for the profile, but this can fail depending on the configuration of the server.',
 )
+@click.option(
+    '--use-zmq',
+    is_flag=True,
+    help='When toggled on, the profile uses the ZMQ broker, which requires no external services and is started '
+    'automatically with the daemon. When not specified, the command automatically tries RabbitMQ first and falls back '
+    'to ZMQ if unavailable. To switch to RabbitMQ later, use `verdi profile configure-rabbitmq`.',
+)
 @click.option('--postgres-hostname', type=str, default='localhost', help='The hostname of the PostgreSQL server.')
 @click.option('--postgres-port', type=int, default=5432, help='The port of the PostgreSQL server.')
 @click.option(
@@ -147,6 +154,7 @@ def verdi_presto(
     profile_name,
     email,
     use_postgres,
+    use_zmq,
     postgres_hostname,
     postgres_port,
     postgres_username,
@@ -169,18 +177,21 @@ def verdi_presto(
     * Set up the localhost as a `Computer` and configure it
     * Set a number of configuration options with sensible defaults
 
-    By default the command creates a profile that uses SQLite for the database. It automatically checks for RabbitMQ
-    running on the localhost, and, if it can connect, configures that as the broker for the profile. Otherwise, the
-    profile is created without a broker, in which case some functionality will be unavailable, most notably running the
-    daemon and submitting processes to said daemon.
+    By default the command creates a profile that uses SQLite for the database. For the message broker, it automatically
+    checks for RabbitMQ running on localhost. If found, it configures RabbitMQ as the broker. Otherwise, it falls back
+    to the ZMQ broker, which requires no external services and is started automatically with the daemon.
 
     When the `--use-postgres` flag is toggled, the command tries to connect to the PostgreSQL server with connection
     paramaters taken from the `--postgres-hostname`, `--postgres-port`, `--postgres-username` and `--postgres-password`
     options. It uses these credentials to try and automatically create a user and database. If successful, the newly
     created profile uses the new PostgreSQL database instead of SQLite.
+
+    When the `--use-zmq` flag is toggled, the command skips the RabbitMQ auto-detection and directly configures the ZMQ
+    broker. To switch to RabbitMQ later, use `verdi profile configure-rabbitmq`.
     """
     from aiida.brokers.rabbitmq.defaults import detect_rabbitmq_config
-    from aiida.common import docs, exceptions
+    from aiida.brokers.zmq.defaults import get_zmq_config
+    from aiida.common import exceptions
     from aiida.manage.configuration import create_profile, load_profile
     from aiida.orm import Computer
 
@@ -212,17 +223,20 @@ def verdi_presto(
     else:
         echo.echo_report('Option `--use-postgres` not enabled: configuring the profile to use SQLite.')
 
-    broker_backend = None
-    broker_config = None
-
-    try:
-        broker_config = detect_rabbitmq_config()
-    except ConnectionError as exception:
-        echo.echo_report(f'RabbitMQ server not found ({exception}): configuring the profile without a broker.')
-        echo.echo_report(f'See {docs.URL_NO_BROKER} for details on the limitations of running without a broker.')
+    if use_zmq:
+        echo.echo_report('`--use-zmq` enabled: configuring the profile with ZMQ broker.')
+        broker_backend = 'core.zmq'
+        broker_config = get_zmq_config()
     else:
-        echo.echo_report('RabbitMQ server detected: configuring the profile with a broker.')
-        broker_backend = 'core.rabbitmq'
+        try:
+            broker_config = detect_rabbitmq_config()
+        except ConnectionError:
+            echo.echo_report('RabbitMQ server not found: falling back to ZMQ broker.')
+            broker_backend = 'core.zmq'
+            broker_config = get_zmq_config()
+        else:
+            echo.echo_report('RabbitMQ server detected: configuring the profile with RabbitMQ broker.')
+            broker_backend = 'core.rabbitmq'
 
     try:
         profile = create_profile(

tests/cmdline/commands/test_presto.py

@@ -32,7 +32,7 @@ def get_profile_names(self):
 
 @pytest.mark.usefixtures('empty_config')
 def test_presto_without_rmq(pytestconfig, run_cli_command, monkeypatch):
-    """Test the ``verdi presto`` without RabbitMQ."""
+    """Test the ``verdi presto`` without RabbitMQ falls back to ZMQ."""
     from aiida.brokers.rabbitmq import defaults
 
     def detect_rabbitmq_config(**kwargs):
@@ -48,21 +48,22 @@ def detect_rabbitmq_config(**kwargs):
         assert profile.name == 'presto'
         localhost = Computer.collection.get(label='localhost')
         assert localhost.is_configured
-        assert profile.process_control_backend is None
+        # When RabbitMQ is not available, presto falls back to ZMQ broker
+        assert profile.process_control_backend == 'core.zmq'
 
 
-@pytest.mark.requires_rmq
 @pytest.mark.usefixtures('empty_config')
-def test_presto_with_rmq(pytestconfig, run_cli_command):
-    """Test the ``verdi presto``."""
+def test_presto(run_cli_command):
+    """Test that ``verdi presto`` configures a broker (RabbitMQ if available, otherwise ZMQ)."""
     result = run_cli_command(verdi_presto, ['--non-interactive'])
     assert 'Created new profile `presto`.' in result.output
 
     with profile_context('presto', allow_switch=True) as profile:
         assert profile.name == 'presto'
         localhost = Computer.collection.get(label='localhost')
         assert localhost.is_configured
-        assert profile.process_control_backend == 'core.rabbitmq'
+        # Presto auto-detects RabbitMQ, falls back to ZMQ if not available
+        assert profile.process_control_backend in ('core.rabbitmq', 'core.zmq')
 
 
 @pytest.mark.requires_psql