Dexter V4 (#52)

* Add build step before typecheck in CI workflow * Catch tokenizer error without console output * Update `openai-fetch` to 3.3.1 and handle refusals This commit updates the `openai-fetch` dependency to version 3.3.1. It also introduces handling for message refusals by throwing a `RefusalError` when a refusal is returned. This includes changes in `chat.ts`, test enhancements, and updates to message types and checks. * Delete documentation website * Remove Datastore classes & add Knip to clean up * Remove unused code and exports * Refactor to remove top-level exports * Remove remnants of deleted Datastore class * Refactor telemetry file structure and imports * Refactor: Move utils to `model/utils` directory - Updated import paths for helper functions from `utils` to `model/utils` - Removed `./` export paths in `package.json` for `utils` - Deleted `src/utils/index.ts` file as it's no longer needed * Refactor error handling into `model/utils/errors.ts` Moved error-related functions and classes from `prompt` to the `model/utils/errors.ts` file for better organization and reuse. Updated imports to reflect this change. * Refactor `Msg` to `MsgUtil` and update related imports This commit refactors the `Msg` utility to `MsgUtil` and updates all related imports and references throughout the codebase. Additionally, it includes the unification of message type definitions under the `Model` namespace and removes the duplicated type definitions. * Refactor `prompt` directory to `ai-function` This change restructures the codebase by renaming and relocating files from the `prompt` directory to the `ai-function` directory. This includes moving functions like `createAIFunction` and `createAIRunner`, along with their corresponding types and tests. * Move `@dexaai/dexter/model` exports to package root * Update readme * Remove unused exports * Add `createExtractFunction` for data extraction Introduce `createExtractFunction` using OpenAI's structured output feature, leveraging a Zod schema for data validation and extraction. This is a better and more modern alternative to the `createAIExtractFunction` utility, which is now deprecated. * Add rough Swarm implementation This commit introduces a new `Swarm` module that includes REPL functionality and tools for handling functions. Additional changes include the integration of the `chalk` library for colorful terminal output. Modifications are made to `package.json` to reflect these new dependencies and module structures. * 4.0.0-0 * Rename `Runner` to `AIRunner` for consistency ...with AIFunction. * Add optional strict validation to extract function * Replace legacy `Model.Message` type with `Msg` * Enhance `readme.md` based on feedback * Upgrade `openai-zod-to-json-schema` to v1.0.3 * Refactor imports and simplify module exports * 4.0.0-1
dexaai · Oct 22, 2024 · bb4f36e · bb4f36e
1 parent 3f88f13
commit bb4f36e
Show file tree

Hide file tree

Showing 249 changed files with 2,112 additions and 45,065 deletions.
diff --git a/.env.example b/.env.example
@@ -7,9 +7,5 @@
 
 OPENAI_API_KEY=
 
-# Most examples will work with a free-tier index, but you'll need a paid hybrid-compatible index to run the chatbot example
-PINECONE_API_KEY=
-PINECONE_BASE_URL=
-
-# Optional; only needed if using a hybrid datastore
+# Optional; only needed if using SPLADE embeddings
 #SPLADE_SERVICE_URL=
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -13,7 +13,7 @@ jobs:
     steps:
     - uses: actions/checkout@v4
     - uses: ./.github/actions/install-node-pnpm
-    - run: pnpm run typecheck
+    - run: pnpm run build && pnpm run typecheck
 
   lint:
     name: Lint

diff --git a/contributing.md b/contributing.md
@@ -41,26 +41,3 @@ For example:
 ```bash
 npx tsx examples/basic.ts
 ```
-
-## Docs
-
-The `./docs/pages/docs` directory is autogenerated via [typedoc](https://typedoc.org) and [typedoc-plugin-markdown](https://github.com/tgreyuk/typedoc-plugin-markdown/tree/next/packages/typedoc-plugin-markdown).
-
-To rebuild the docs, run:
-
-```bash
-pnpm run docs
-```
-
-Please don't run this command when submitting PRs to keep them uncluttered, unless you're making changes to the docs themselves.
-
-The docs folder is a normal [Next.js](https://nextjs.org) pages app built using [Nextra](https://nextra.site/).
-
-You can run the docs dev server to preview your changes via:
-
-```bash
-cd docs
-pnpm dev
-```
-
-The docs are automatically deployed to [Vercel](https://vercel.com).
diff --git a/docs/bin/transform-docs.ts b/docs/bin/transform-docs.ts
diff --git a/docs/next-env.d.ts b/docs/next-env.d.ts
diff --git a/docs/next.config.mjs b/docs/next.config.mjs
diff --git a/docs/package.json b/docs/package.json
diff --git a/docs/pages/_app.mdx b/docs/pages/_app.mdx
diff --git a/docs/pages/_meta.json b/docs/pages/_meta.json
diff --git a/docs/pages/docs/.nojekyll b/docs/pages/docs/.nojekyll