Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle TW_AGENT_WORK #169

Open
multimeric opened this issue Oct 9, 2024 · 0 comments
Open

Handle TW_AGENT_WORK #169

multimeric opened this issue Oct 9, 2024 · 0 comments

Comments

@multimeric
Copy link

Seqera Platform supports a kind of magic pseudo variable called $TW_AGENT_WORK, which is dynamically set to the working directory of the Tower Agent at the time of pipeline execution, and not at the time of compute environment creation. There is some discussion of it here, although it doesn't really seem to be documented anywhere.

However, seqera-kit doesn't seem to understand this, and instead tries to interpolate it from the environment at setup time. For example if I have a compute environment definition that has parameters a bit like this:

compute-envs:
  - type: slurm
    work-dir: "$TW_AGENT_WORK"
    launch-dir: "$TW_AGENT_WORK"
    ...

I get a traceback with:

Traceback (most recent call last):
  File "/Users/milton.m/Programming/TowerTemplate/venv/bin/seqerakit", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/milton.m/Programming/TowerTemplate/venv/lib/python3.12/site-packages/seqerakit/cli.py", line 198, in main
    block_manager.handle_block(
  File "/Users/milton.m/Programming/TowerTemplate/venv/lib/python3.12/site-packages/seqerakit/cli.py", line 146, in handle_block
    block_handler_map[block](self.sp, args["cmd_args"])
  File "/Users/milton.m/Programming/TowerTemplate/venv/lib/python3.12/site-packages/seqerakit/helper.py", line 364, in handle_compute_envs
    method("add", *args)
  File "/Users/milton.m/Programming/TowerTemplate/venv/lib/python3.12/site-packages/seqerakit/seqeraplatform.py", line 43, in __call__
    return self.tw_instance._tw_run(command, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/TowerTemplate/venv/lib/python3.12/site-packages/seqerakit/seqeraplatform.py", line 139, in _tw_run
    full_cmd = self._construct_command(cmd, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/TowerTemplate/venv/lib/python3.12/site-packages/seqerakit/seqeraplatform.py", line 74, in _construct_command
    return self._check_env_vars(command)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/milton.m/Programming/TowerTemplate/venv/lib/python3.12/site-packages/seqerakit/seqeraplatform.py", line 94, in _check_env_vars
    raise EnvironmentError(
OSError:  Environment variable $TW_AGENT_WORK not found!

The same thing happens when using the Python API:

client = SeqeraPlatform(cli_args=["--output", "json"])
client.compute_envs(
        "add", "slurm",
        "--work-dir", "$TW_AGENT_WORK",
        "--launch-dir", "$TW_AGENT_WORK",
        # Other parameters excluded for brevity
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant