Dexter V4 (#52)

* Add build step before typecheck in CI workflow

* Catch tokenizer error without console output

* Update `openai-fetch` to 3.3.1 and handle refusals

This commit updates the `openai-fetch` dependency to version 3.3.1.
It also introduces handling for message refusals by throwing a
`RefusalError` when a refusal is returned. This includes changes in
`chat.ts`, test enhancements, and updates to message types and checks.

* Delete documentation website

* Remove Datastore classes & add Knip to clean up

* Remove unused code and exports

* Refactor to remove top-level exports

* Remove remnants of deleted Datastore class

* Refactor telemetry file structure and imports

* Refactor: Move utils to `model/utils` directory

- Updated import paths for helper functions from `utils` to `model/utils`
- Removed `./` export paths in `package.json` for `utils`
- Deleted `src/utils/index.ts` file as it's no longer needed

* Refactor error handling into `model/utils/errors.ts`

Moved error-related functions and classes from `prompt` to the
`model/utils/errors.ts` file for better organization and reuse. Updated
imports to reflect this change.

* Refactor `Msg` to `MsgUtil` and update related imports

This commit refactors the `Msg` utility to `MsgUtil` and updates all
related imports and references throughout the codebase. Additionally,
it includes the unification of message type definitions under the
`Model` namespace and removes the duplicated type definitions.

* Refactor `prompt` directory to `ai-function`

This change restructures the codebase by renaming and relocating
files from the `prompt` directory to the `ai-function` directory.
This includes moving functions like `createAIFunction` and
`createAIRunner`, along with their corresponding types and tests.

* Move `@dexaai/dexter/model` exports to package root

* Update readme

* Remove unused exports

* Add `createExtractFunction` for data extraction

Introduce `createExtractFunction` using OpenAI's structured output
feature, leveraging a Zod schema for data validation and extraction.

This is a better and more modern alternative to the `createAIExtractFunction`
utility, which is now deprecated.

* Add rough Swarm implementation

This commit introduces a new `Swarm` module that includes REPL
functionality and tools for handling functions. Additional changes
include the integration of the `chalk` library for colorful terminal
output. Modifications are made to `package.json` to reflect
these new dependencies and module structures.

* 4.0.0-0

* Rename `Runner` to `AIRunner` for consistency

...with AIFunction.

* Add optional strict validation to extract function

* Replace legacy `Model.Message` type with `Msg`

* Enhance `readme.md` based on feedback

* Upgrade `openai-zod-to-json-schema` to v1.0.3

* Refactor imports and simplify module exports

* 4.0.0-1

Author: rileytomasek

.env.example

@@ -7,9 +7,5 @@
 
 OPENAI_API_KEY=
 
-# Most examples will work with a free-tier index, but you'll need a paid hybrid-compatible index to run the chatbot example
-PINECONE_API_KEY=
-PINECONE_BASE_URL=
-
-# Optional; only needed if using a hybrid datastore
+# Optional; only needed if using SPLADE embeddings
 #SPLADE_SERVICE_URL=

.github/workflows/main.yml

@@ -13,7 +13,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - uses: ./.github/actions/install-node-pnpm
-    - run: pnpm run typecheck
+    - run: pnpm run build && pnpm run typecheck
 
   lint:
     name: Lint

contributing.md

@@ -41,26 +41,3 @@ For example:
 ```bash
 npx tsx examples/basic.ts
 ```
-
-## Docs
-
-The `./docs/pages/docs` directory is autogenerated via [typedoc](https://typedoc.org) and [typedoc-plugin-markdown](https://github.com/tgreyuk/typedoc-plugin-markdown/tree/next/packages/typedoc-plugin-markdown).
-
-To rebuild the docs, run:
-
-```bash
-pnpm run docs
-```
-
-Please don't run this command when submitting PRs to keep them uncluttered, unless you're making changes to the docs themselves.
-
-The docs folder is a normal [Next.js](https://nextjs.org) pages app built using [Nextra](https://nextra.site/).
-
-You can run the docs dev server to preview your changes via:
-
-```bash
-cd docs
-pnpm dev
-```
-
-The docs are automatically deployed to [Vercel](https://vercel.com).