Allow backend lifecycle management to work with Agentic workflows

[mcruzdev]

This pull request introduces several important improvements to the workflow Dev UI's backend configuration, storage, and serialization. The main focus is on making backend persistence optional and more configurable, refactoring the MVStore instance store for better testability and clarity, and enhancing serialization to handle non-Jackson-friendly workflow models. Tests and configuration files are updated accordingly.

Backend configuration and conditional bean registration:

• Added a new FlowDevUIBackendConfig interface to allow enabling or disabling backend persistence for Dev UI workflow data via the quarkus.flow.devui.backend.enabled property. The configuration is now injected and checked in the processor and related beans. [1] [2] [3] [4] [5]

• Updated FlowDevUIProcessor to only register backend-related beans and JSON-RPC providers when backend persistence is enabled, improving startup performance and flexibility.

MVStore instance store refactoring and testability:

• Refactored MVStoreWorkflowInstanceStore to use constructor injection for the database path, removing direct dependency on the configuration object, and improved initialization/cleanup logic. This makes the store easier to test and more robust. [1] [2] [3] [4]

• Updated tests to use the new constructor and configuration style, and added a test to ensure serialization works with workflow models that are not Jackson-friendly. [1] [2]

Serialization improvements:

• Enhanced FlowInstanceSerializer to serialize workflow models as JSON-safe shapes, avoiding failures when the underlying Java objects are not compatible with Jackson. This ensures robust persistence and retrieval of workflow data. [1] [2]

Test and configuration updates:

• Updated and added test resources and properties files to reflect the new backend configuration and storage options. [1] [2] [3]

• Improved test assertions to verify correct behavior when retrieving workflow instances.

These changes collectively make backend persistence for workflow Dev UI optional, more configurable, and robust in both development and testing environments.

Closes #374

[github-actions]

🚀 PR Preview 73ab0fb has been successfully built and deployed to https://quarkiverse-flow-pr-375-preview.surge.sh

[mcruzdev]

Do not merge, I am getting a exception when saving Agentic state:

No serializer found for class dev.langchain4j.agentic.scope.DefaultAgenticScope and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)

Working to solve it.

[ricardozanini]

Before merging, please let's consider Javi's comment regarding the serialization.

[fjtirado]

This is still not going to work if the model is not jackson serializable (now it is working because it is)
So I think you should use WorkflowBufferFactory and MasrhallingUtils.getCustomMarshaller to write the model into a byte[] and write that byte[] using gen.writeBinary()
That will avoid future issues (we will likely have to change the way the agenticai model is serialized and using this code these changes will be missed here)

[fjtirado]

For WorkflowBufferFactory you can add a default that calls DefaultBeanFactory with lower priority to not collide with the Default bean that is added by persistence layer, see https://github.com/quarkiverse/quarkus-flow/blob/main/persistence/common/src/main/java/io/quarkiverse/flow/persistence/common/FlowFactoryProducer.java#L13
Once you have that, the rest should be straighforward using MarshallingUtils.getCustomMarshaller()
Note that you do not lose any human readability because mvstore already store everything as byte[]

[mcruzdev]

I am trying the issue in many ways today, without success, I think for now the better approach is to rely in JsonUtils (it is simple and to work). Remembering that this feature is not enabled by default, there is no frontend for this.

I will left the summary I got after to have a issue with serialization detail I missed.

For agentic workflows, the model is wrapped as AgenticAwareWorkflowModel in AgenticAwareModelFactory. During marshalling the AbstractOutputBuffer.writeObject(...) falls into CUSTOM and writes the actual class name (AgenticAwareWorkflowModel), no dedicated marshaller exists for AgenticAwareWorkflowModel so by default the marshaller is JacksonObjectMarshaller.

Now the key detail:

1. WorkflowModel defines default method isNull() (sdk-java/.../WorkflowModel.java:43-45).
2. Jackson treats isNull() as boolean bean getter => property name "null".
3. So serialized JSON for that object includes "null": ....
4. On deserialize, Jackson tries to construct AgenticAwareWorkflowModel (constructor params agenticScope, delegate) and sees unexpected property "null", causing:

Exception I am getting: UnrecognizedPropertyException: Unrecognized field "null".

[mcruzdev]

I am not sure but the solution is to provide a CustomObjectMarshaller<AgenticAwareWorkflowModel> in langchain4j. WDYT @ricardozanini @fjtirado

I tested here and it works as expected.

[mcruzdev]

After researching here we have a problem that lives on @JsonRpcDescription method handler, where is used a serializer/deserializer that does not use the FlowInstanceSerializer and FlowInstanceDeserializer.

Click here to see the serialized workflow instance

[
  {
    "instanceId": "01KMSK9D4DSVWAH89QX5W7YS6W",
    "workflowNamespace": "org.acme",
    "workflowName": "intelligent-newsletter",
    "workflowVersion": "0.0.1",
    "status": "COMPLETED",
    "startTime": 1774680595461,
    "lastUpdateTime": 1774680658588,
    "endTime": 1774680658588,
    "input": {
      "mood": "BULLISH",
      "topMovers": [
        "AAPL"
      ],
      "macroData": "Fed expected to cut rates",
      "tone": "FRIENDLY",
      "length": "MEDIUM"
    },
    "output": {
      "title": "Market Update: A Neutral Overview of Current Trends",
      "lead": "The market is experiencing a period of positivity, and we'll provide an in-depth analysis of the current trends and their potential implications.",
      "body": "As we begin the new week, it's essential to acknowledge the current market mood. Our top movers on radar are [AAPL], which has been making significant strides in the tech sector. Recent macro data suggests that the Fed is expected to cut rates, a move that could have far-reaching effects on the economy. While this news has investors feeling optimistic, it's crucial to consider potential risks associated with these market trends. In this issue, we'll delve into the latest insights and analysis to help you make informed investment decisions. We'll examine the potential impact of the Fed's rate cut on the economy and provide a balanced view of the current market landscape."
    },
    "history": [
      {
        "type": "workflow.started",
        "timestamp": 1774680595461
      },
      {
        "type": "workflow.completed",
        "timestamp": 1774680658588
      },
      {
        "type": "task.started",
        "taskName": "draftAgent",
        "timestamp": 1774680595466
      },
      {
        "type": "task.completed",
        "taskName": "draftAgent",
        "timestamp": 1774680610517
      },
      {
        "type": "task.started",
        "taskName": "draftReady",
        "timestamp": 1774680610520
      },
      {
        "type": "task.completed",
        "taskName": "draftReady",
        "timestamp": 1774680610552
      },
      {
        "type": "task.started",
        "taskName": "waitHumanReview",
        "timestamp": 1774680610555
      },
      {
        "type": "workflow.status.changed",
        "timestamp": 1774680610558,
        "details": "WAITING"
      },
      {
        "type": "task.completed",
        "taskName": "waitHumanReview",
        "timestamp": 1774680658557
      },
      {
        "type": "workflow.status.changed",
        "timestamp": 1774680658561,
        "details": "RUNNING"
      },
      {
        "type": "task.started",
        "taskName": "switch-3",
        "timestamp": 1774680658564
      },
      {
        "type": "task.completed",
        "taskName": "switch-3",
        "timestamp": 1774680658570
      },
      {
        "type": "task.started",
        "taskName": "sendNewsletter",
        "timestamp": 1774680658572
      },
      {
        "type": "task.completed",
        "taskName": "sendNewsletter",
        "timestamp": 1774680658578
      },
      {
        "type": "workflow.status.changed",
        "timestamp": 1774680658586,
        "details": "COMPLETED"
      },
      {
        "type": "workflow.completed",
        "timestamp": 1774680658588
      }
    ]
  }
]