BEE Release 0.1.10

Major Features:

Adds the capability to limit the number of jobs in active scheduling
Improves archiving failed and cancelled workflows
Improvements to API for generating CWL specifications
Adds SlurmRequirements for qos and reservation
Improved configuration
Checks if beeflow is running on other nodes of the system

BEE Release 0.1.9

Major features:

• Updated the Graph Database to use one instance of Neo4j for all workflows for a user vs. using one for each workflow.
• Added Python API for generating cwl specification of a workflow.
• Added coverage metrics for unit tests.
• Added resiliency to Task Manager to update tasks status when coming back up
• Enhanced pre/post script capabilities (added flux capability) and added checks for shell compatibility.
• Added capability to export dag of a workflow before and during execution

What's Changed

• Ensure failed tasks cause dependent tasks to fail by @jtronge in #806
• Update Neo4j to 5.x by @jtronge in #819
• Archive failed checkpoint restart workflows by @jtronge in #824
• Add pre/post script support to flux worker by @kchilleri in #818
• Add integration test for pre/post scripts (beeflow:ScriptRequirement) by @jtronge in #838
• Update Task Manager for task update resiliency by @jtronge in #827
• Restart GDB by @rstyd in #759
• Allow removal of a workflow with any Archive status by @pagrubel in #828
• Add Pytest coverage by @aquan9 in #843
• Test slurm commands worker in CI by @jtronge in #839
• Check for missing workflow inputs by @jtronge in #842
• Validate shell environment in pre/post script files by @kchilleri in #844
• Add Script Option to Pre/Post Script & Change MPI Call by @rstyd in #856
• Add MPI integration test code by @jtronge in #858
• Fix edge case in badge generation. by @aquan9 in #851
• Security concrete by @aquan9 in #852
• Remove enhanced client from github by @rstyd in #865
• Bump gunicorn from 20.1.0 to 22.0.0 by @dependabot in #862
• Bump urllib3 from 1.26.18 to 1.26.19 by @dependabot in #863
• Enable beeflow submit from working directory by @jtronge in #854
• Update work_dir path by @Leahh02 in #869
• Issue735/clear workflow by @kabir-vats in #872
• Pause running workflows when stopping beeflow with 'beeflow core stop' by @pagrubel in #830
• Bump certifi from 2024.6.2 to 2024.7.4 by @dependabot in #878
• Enable shell option on flux worker by @kchilleri in #866
• Issue871/delete_use_archive_config by @Leahh02 in #879
• Update documentation for the integration tests by @jtronge in #887
• Validate shell option in cwl file; ensure it matches shell interpreter in pre/post scripts by @kchilleri in #880
• Stop checking job state when it's completed by @Leahh02 in #892
• Add WorkerError exception handling by @jtronge in #888
• Add check for /tmp or /var/tmp by @Leahh02 in #885
• Generate CWL by @rstyd in #841
• Only Run One Neo4j Instance for a BEE Instance by @kabir-vats in #882
• Remove shebang lines from pre/post scripts by @kchilleri in #895
• Add "beeflow core info" by @aquan9 in #908
• Fix Reexecute by @kabir-vats in #902
• Update Poetry developer's guide by @Leahh02 in #899
• Update Git workflow documentation by @Leahh02 in #897
• Bump cryptography from 42.0.8 to 43.0.1 by @dependabot in #916
• Add Beeflow connect - A REST API for BEEflow itself. by @aquan9 in #837
• Add No Start for workflows submitted but not started by @Leahh02 in #914
• Add task state when creating task for checkpoint by @pagrubel in #923
• Export dag by @Leahh02 in #909
• Release 0.1.9 by @pagrubel in #925

Full Changelog: 0.1.8...0.1.9

BEE Release 0.1.8

Features: Fixes sphinx version to enable publishing documentation, now includes CI for testing documentation builds

• Update sphinx version, update actions and release docs (#812)
• Add separate action for testing docs
• Fix beeflow config new error

BEE release 0.1.7

Major features: adds the capability to include post- and pre-processing scripts to tasks, fixes the Checkpoint/Restart capability, increases logging, and adds some features to the client.

• Initial task manager resiliency and error handling (#789)
• Add pre/post script support (#788)
• Fix LOCALE error for systems where redis container failed to start
• Add logging to workflow interface (#764)
  • Enable logging in neo4j_cypher.py, neo4j_driver.py, and gdb_driver.py
• Add beeflow remove command to client
  • Enables removal of archived or cancelled workflows and associated artifacts
• Update minimum Charliecloud version to 0.36
• CI refactor to allow running jobs on runners other than github
• Add sumbit command options to workflow artifact for archive purposes
• Increase maximum version of python to 3.12
• Fix Checkpoint/Restart capability
• Add testing for Checkpoint/Restart
• Adds capability to reset the beeflow files (deletes all artifacts) especially useful for developers.

BEE release 0.1.6

Release 0.1.6 is mainly a clean up of processes, logs and directory space

• Eliminates extraneous Neo4j instances from cancelled/failed tasks
• Cleans up log entries for query
• Improves start time for celery
• Makes start time configurable
• Decreases the number of celery processes
• Fixes capability to specify a main cwl file and/or yaml file not in the CWL directory
• Parses CWL after packaging the directory
• Moves temporary files for unit tests out of $HOME

BEE release 0.1.5

• Combined beeflow, beeclient and beecfg commands. All commands now are invoked via beeflow.
• Fixed an obscure dependency issue between tasks
• Simplified config file, deleted duplications of bee_workdir
• CWL Parser was moved to the client
  • CwlParser is now instantiated in bee_client.py
  • CwlParser no longer invokes Workflow Interface, now returns Workflow and Task objects
  • Allows verification of CWL specification without running the workflow
• Added support for Flux scheduler

What's Changed

• Flux support by @jtronge in #696
• Move parser to client by @Boogie3D in #697
• Remove bee_workdir duplication in config file by @jtronge in #709
• Fix Dependencies Bug by @Boogie3D in #713
• Combine beeflow, beeclient, and beecfg by @jtronge in #710

Full Changelog: 0.1.4...0.1.5

BEE release 0.1.4

What's Changed

• Scheduler options added for time-limit, account and partitions as CWL extensions
• Fixes for MPI
• Jinja file no longer required
• Merge submit and start commands
• Improved usability of 'beecfg new'
• Combined gdbs
• Add restart code to beeflow
• Checkpoint restart fix
• Allow Absolute/Relative Paths for Main CWL and YAML Files
• Minimum version of Charliecloud required is now 0.32

Full Changelog: 0.1.3...0.1.4

BEE release 0.1.3

BEE now accepts stdout and stderr CWL specifications to direct those outputs for each task.