Why Archytas over LangChain

Langchain's ReAct process does not work reliably with GPT-4

langchain defaults to GPT-3 (Davinci), but GPT-3 has limited planning capacity and frequently fails to accomplish mildly complex tasks. GPT-4 is much more capable, and quite necessary to have a chance at doing most of the things that are useful but complex

langchain supports the GPT-3.5/GPT-4 chat-style API interface, but the prompting is not good enough. Frequently both will fail to follow the expected langchain format (which would be parsed into thoughts/actions/input/etc.), causing langchain to crash.

Archytas is purpose built around GPT-4, and properly prompts it to generate output in the desired (json) format. On the rare occasions GPT-4 fails to follow the format, Archytas feeds the LLM sensible error messages that allow the agent to recover and continue using the correct format.

GPT-3.5-turbo support

unfortunately it seems GPT-3.5-turbo is a lost cause in any sort of ReAct loop:

No amount of prompting can convince it to output a valid json without any extra text outside the json body, though it will generate valid json's that Archytas can use.
handling the spurious text (e.g. via dirtyjson) is possible, but it frequently gets stuck in infinite loops of repeating the same thing over and over, rather than continuing the ReAct process until the task is solved

Archytas has better tool ergonomics

Tools created in langchain must be a function that take a single string as input, and then return a single string as output. If the tool wants to use a different type or have multiple inputs, it must explicitly handle parsing the input string into the desired shape. And same for any output which must be manually converted to a string format so the LLM can understand it. Additionally, it is up to the tool developer to explain to the LLM any expectations for the input so that it can be parsed accordingly. This is pretty ad-hoc if you're doing anything more complex than string in, string out.

Archytas has a consistent interface for all tools it presents to the LLM: JSON values. Tools can take any number of arguments, that may be any valid python-equivalent of JSON objects

dict
list
int/float
bool
str
None

Additionally, support has been added to allow for arbitrary structured types as tool arguments:

dataclass
BaseModel from pydantic

When the LLM is interacting with tools, Archytas will automatically parse/convert the LLM string into the correct types corresponding to the tool's function signature. Then when the tool returns it's response (if any), Archytas will then convert it back into a string the LLM can understand. This means that your tool functions can look much more like regular functions with multiple arguments, typing, etc. and aren't forced to conform to the def mytool(s:str) -> str interface required by langchain

Archytas effortlessly works with stateful tools and sets of related tools

If your python-fu is strong then yes you can certainly create a langchain tool that keeps internal state (e.g. by making a class instance, and then pulling one of the bound methods out to be the tool). However it's a rather cumbersome process.

Additionally, if you want to have a set of related tools that are conceptually bundled together, there's not really a canonical approach for doing this in langchain. To do so, it's ultimately a matter of giving your tools good names, and then explaining very well in all of the tool descriptions how they interact, and work together.

toolsets

In Archytas, the @tool decorator can simply be applied to class methods, creating a toolset. This easily solves both handling tools with state, as well as bundled toolsets that have many related tools. The class interface provides a solid conceptual interface for bundling related tools, as well as for managing stateful tools

e.g. Let's say you were making a bunch of math tools, you could have Math.sin, Math.abs, Math.sqrt, Math.pi, etc. all be grouped together under a single math toolset. Archytas automatically generates a prompt for the LLM explaining how to access each of the subtools in the toolset, and manages delegating calls to the correct methods in the class.
For stateful tools, Archytas maintains an instance of the class toolset when running the ReAct loop, making it trivial to maintain and update the state whenever any of the tool methods are called.

Graceful error handling

LangChain does not catch exceptions generated inside of tools. Instead if you want to indicate that an error has occurred, your tool must return a string that says so. This complicates the development of the tools as it muddies the distinction between valid tool results and invalid tool errors. The programmer needs to be careful that all string returned that were because of an error are obviously erroneous to the LLM.

In Archytas, errors can be indicated by raising an exception. Tools can raise an exception at any point, which then get caught and shown to the LLM in a consistent manner indicating the error state. In the Archytas ReAct loop, the LLM can then seamlessly recover from errors encountered while using tools, based on the exceptions. It makes it very easy for developers to indicate errors to the LLM agent when a tool gets misused. Sometimes, the LLM is even able to handle deeper exceptions that the developer didn't anticipate (e.g. due to bugs in the tool), and try alternative inputs that don't trigger the error.

Archytas generates better prompts

In langchain, when making a tool or set of tools, it's really up to you to provide correct/compelling descriptions of your tools and how they should be used. Unfortunately you can't explain using a python context, since the LLM doesn't call tools like python function calls--you have to explain according to how LangChain has prompted the LLM to respond. It's a bit difficult to dig into langchain to find the exact format that gets used to prompt the LLM, but it looks like this:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [<list of tool names you gave the llm>]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

so any descriptions you make for your tools need to keep this pattern in mind. Additionally, since it's freeform text, you're really on the hook for explaining to the LLM how you expect input, which is especially tricky if your tool needs multiple arguments. All this is to say, writing good prompts in langchain is a lot of work.

Archytas makes this process trivial: if your tool has type annotations, and a matching docstring, Archytas will automatically generate the entire prompt for you. All you have to worry about is describing how your tool works, and ensuring that the docstring is up to date. Say you have a tool with a complicated interface:

@tool
def example_tool(arg1:int, arg2:str='', arg3:dict=None) -> int:
    """
    Simple 1 sentence description of the tool

    More detailed description of the tool. This can be multiple lines.
    Explain more what the tool does, and what it is used for.

    Args:
        arg1 (int): Description of the first argument.
        arg2 (str): Description of the second argument. Defaults to ''.
        arg3 (dict): Description of the third argument. Defaults to {}.

    Returns:
        int: Description of the return value
    """
    # do tool work here. return an int

This docstring is sufficient to facilitate creating a prompt that will allow the agent to call this tool. The developer doesn't need to worry about ensuring that the prompt accurately describes the mechanics of calling tools, just that the tool has good enough (well formatted) documentation that a human could understand.

Archytas is simpler & easier to peek inside

Under the hood, Archytas does a lot of automatic prompt generation for you. But it is very easy to peek in and see the raw values being generated/sent to the OpenAI API:

# say you have an agent with some set of tools
agent = ReActAgent(tools=tools, verbose=True)

# you can trivially print out the initial prompt given to to the agent
print(agent.prompt)

# you can also easily see the list of messages (minus the initial prompt) that gets sent to the OpenAI API
print(agent.messages) #this starts empty

While True:
    query = input('>>> ')
    answer = agent.react(query)
    print(agent.messages) # now messages contains your query and the raw LLM response