Only 1 handoff getting called no matter what

[adiberk]

Please read this first

• Have you read the docs?Agents SDK docs: YES
• Have you searched for related issues? Others may have faced similar issues: YES - best answer I found is to use tool calls, but then what is the point of agent handoff? Also I want to maintain context, which I thought we only get in handoff

Describe the bug

I have an agent setup where I have an orchestrator that should call both handoff agents.
However, from what I can tell only the content generation agent is being called. Not only that but the final response being returned by the orchestrator agent is the response from the handoff agent

Here is my basic code

layout_builder_profile = Agent(
    name="ProfileLayoutAgent",
    model=AIModel.gpt_4_1_0425.value,
    instructions=f"""
You are an expert layout builder for team/contact slides.

Input:
- `content`
- `template` (use `get_template_for_layout('contact_or_team_slides')`)
- `broker_and_sponsor_info`

Rules:
- Only include team members, not companies
- Respect spacing/layout from example
- Break into multiple slides if necessary
- Use images only if available
- Return JSON slide list
{shared_important_rules}
""",
    tools=[get_template_for_layout],
    handoff_description="Used when layout_type_to_use is 'contact_or_team_slides'.",
)


layout_orchestrator_agent = Agent(
    name="Slide Layout Orchestrator",
    model=AIModel.gpt_4_1_0425.value,
    instructions=f"""
Given content and layout_type_to_use, route the section to the correct layout builder agent.
You do NOT build the slide yourself.
Use the layout_type_to_use to pick the handoff.
..........

"""
    + """
Structure the final result of slides as follows:
{
    "id": "unique_id_for_this_slide_deck",
    "footer": "Footer slide content"
    "slides": [....The slide data generated by the layout builder agent, which will be a list of JSON objects representing each slide]
}
""",
    tools=[get_deal_facts],
    handoff_description="Generates slides based on the content passed to it!",
    handoffs=[
        layout_builder_generic,
        layout_builder_statistical,
        layout_builder_profile,
        layout_builder_company,
        layout_builder_cover,
        layout_builder_footer,
    ],
)

# === Top-Level Document Generation Orchestrator ===
document_generation_orchestrator = Agent(
    name="Document Generation Orchestrator",
    model=AIModel.gpt_4_1_0425.value,
    instructions="""
You are the main controller
........
{
    "id": "unique_id_for_this_slide_deck",
    "footer": "Footer slide content"
    "slides": [... The slide data generated by the layout orchestrator agent, which will be a list of JSON objects representing each slide]
}
As you can see you will have to extract the footer slide from the layout orchestrator agent and put it in the top level of the output.

YOU MUST call both prompts in the following order:
1. First call the content generator passing in any instructions from the user,
2. Then call the layout orchestrator passing all the resulting content along with any importnat user instructions.
Finally: Once you get the layout orchestrator result, you will have to extract the footer slide from the layout orchestrator result and put it in the top level of the output.
Return the final output in the structure above.

IF YOU DON"T CALL BOTH prompts
THen return an error message saying WHY YOU DIDN'T CALL BOTH prompts
""",
    handoffs=[content_generator_debt_brokerage, layout_orchestrator_agent],
)

I know people have suggested converting them to tool calls. But why? Does that not defeat the point of handoff?

LASTLY:
Am I not allowed to use dict[str, Any] as part of an "output_type" field?
class TestOutput(BaseModel):
slides: list[dict[str, Any]]

^^I get an error when I do this.

Debug information

• Agents SDK version: 0.0.7
• Python version: 3.11

Repro steps

I think you can reproduce using the above example

Expected behavior

I would expect with all of my prompting that both agents get called. But I am just getting the output of the content generation agent and it is not handing off to anyone else. The output is also incorrect. And when I set the output type I get an error because I can't give an ambiguous object dict[str, Any]

[T-Zaid]

I have also faced similar while creating a demo use case for shoe store and have some observations regarding this. I have attached the code snippet of the main/orchestrator agent where all the handoffs are occurring. The ShoeStoreAgent is the main bridge between all other specialist agent and is the one being called by Runner.run method. Now, based on the tasks, it is supposed to perform hand-offs between different agents. The specialist agents would handoff to the main agent and then main agent would handoff to the next appropriate agent based on the user query.

Main/Orchestrator Agent:

ShoeStoreAgent = Agent[UserContext](
    name="ShoeStoreAgent",
    handoff_description="Main agent for Shoe Store, coordinating between order, product, and cart agents.",
    instructions=f"""
        {RECOMMENDED_PROMPT_PREFIX}

        You are an assistant for Shoe Store named Freddie.
        You help customers with following:
        1. For checking order status, handoff to the Order Agent.
        2. For providing product information, handoff to the Product Agent.
        3. handoff to Cart Agent for adding products to cart
        4. handoff to Cart Agent for viewing cart contents
        5. handoff to Cart Agent for generating receipts

        Do not use your own knowledge or make assumptions about the store's policies or products, hand off to the appropriate agent.
        Do not mention that you are transferring the user to another agent.
        Use a friendly, helpful tone. If you don't understand a request or if it's for products we don't carry, politely explain what we do offer.
    """,
    handoffs=[order_agent, product_agent, cart_agent],
    model=model
)

order_agent.handoffs.append(ShoeStoreAgent)
product_agent.handoffs.append(ShoeStoreAgent)
cart_agent.handoffs.append(ShoeStoreAgent)

Main function to debug actions performed:

// logs every action performing by the agents.
async def main():
    current_agent: Agent[UserContext] = ShoeStoreAgent
    input_items: list[TResponseInputItem] = []
    context = UserContext(user_id="abc")

    while True:
        user_input = input("Enter your message: ")
        input_items.append({"content": user_input, "role": "user"})
        result = await Runner.run(current_agent, input_items, context=context)

        for new_item in result.new_items:
            agent_name = new_item.agent.name
            if isinstance(new_item, MessageOutputItem):
                print(f"{agent_name}: {ItemHelpers.text_message_output(new_item)}")
            elif isinstance(new_item, HandoffOutputItem):
                print(f"Handed off from {new_item.source_agent.name} to {new_item.target_agent.name}")
            elif isinstance(new_item, ToolCallItem):
                print(f"{agent_name}: Calling a tool")
            elif isinstance(new_item, ToolCallOutputItem):
                print(f"{agent_name}: Tool call output: {new_item.output}")
            else:
                print(f"{agent_name}: Skipping item: {new_item.__class__.__name__}")

        input_items = result.to_input_list()
        current_agent = result.last_agent

I have observed the following during my implementation and then had to switch to agent_as_tool for expected results temporarily.

• I was able to achieve these complex multiple hand-offs with the above setup, but it is not at all consistent. On the first run, it worked as expected, then immediately on the second run, it just forgets that it can do hand-offs.
• Using GPT-3.5-turbo, it ends prematurely. For example, if I asks for product status, it successfully hands-off to Order Agent, and it ends with 'I have handed off the Order Agent, it will help you shortly' without calling the necessary tools provided to the specialist agent, even if the product_id is mentioned in the query. In contrast, when using GPT-4o-mini, it generates the same message, but then it goes ahead and calls the tools in Order Agent and come back with the results.
• I am not sure if it is possible, but I have observed that, If I am testing the workflow between 4.30 PM UTC, it results in more expected runs. While testing the same workflow between 5 AM UTC, it rarely does hand-offs and produces expected results.

I will edit this comment more if I found any other notable observations worthwhile to mention.