Skip to content

vincentkoc/tokenjuice

Repository files navigation

tokenjuice banner

tokenjuice πŸ§ƒ

lean output compaction for terminal-heavy agent workflows.

what is tokenjuice?

tokenjuice is a deterministic output compactor for terminal-heavy agent workflows. agents and harnesses run noisy commands like git status, pnpm test, docker build, rg, or pnpm --help; tokenjuice keeps the command semantics untouched, observes the output after execution, and returns a smaller payload built from rule-driven reducers instead of dumping the whole wall of terminal text back into context.

the point is leverage: less transcript waste, fewer useless reruns, and cleaner handoff between tools without making the shell magical. raw output stays available only when you explicitly ask for it through --raw / --full or opt-in artifact storage, rules stay inspectable JSON instead of LLM vibes, and host integrations stay thin wrappers around the same core reducer instead of becoming one-off adapter logic.

host integrations

supported integrations:

Logo Client Install Hook file
Claude Claude Code tokenjuice install claude-code ~/.claude/settings.json
CodeBuddy CodeBuddy tokenjuice install codebuddy ~/.codebuddy/settings.json
Codex Codex CLI tokenjuice install codex ~/.codex/hooks.json
Cursor Cursor tokenjuice install cursor ~/.cursor/hooks.json
Droid Droid (Factory CLI) tokenjuice install droid ~/.factory/settings.json
GitHub Copilot CLI GitHub Copilot CLI tokenjuice install copilot-cli ~/.copilot/hooks/tokenjuice-cli.json
OpenClaw OpenClaw openclaw config set plugins.entries.tokenjuice.enabled true ~/.openclaw/openclaw.json
OpenCode OpenCode tokenjuice install opencode ~/.config/opencode/plugins/tokenjuice.js
pi pi tokenjuice install pi ~/.pi/agent/extensions/tokenjuice.js
VS Code Copilot VS Code Copilot Chat tokenjuice install vscode-copilot ~/.copilot/hooks/tokenjuice-vscode.json

beta integrations:

Logo Client Install Hook file
AdaL CLI AdaL CLI tokenjuice install adal AGENTS.md
Aether Aether tokenjuice install aether .aether/tokenjuice.md / .aether/settings.json
aictl aictl tokenjuice install aictl AICTL.md
AI Memory Protocol AI Memory Protocol tokenjuice install ai-memory-protocol .memories/memory/preferences.rst
Aider Aider tokenjuice install aider CONVENTIONS.tokenjuice.md
Agent Layer Agent Layer tokenjuice install agent-layer .agent-layer/instructions/tokenjuice.md; run al sync after install or uninstall
AgentInit AgentInit tokenjuice install agentinit AGENTS.md; run agentinit sync after install or uninstall
Agentlink Agentlink tokenjuice install agentlink AGENTS.md; run agentlink sync after install or uninstall
Agentloom Agentloom tokenjuice install agentloom .agents/rules/tokenjuice-agentloom.md; run agentloom sync after install or uninstall
agents-cli agents-cli tokenjuice install agents-cli ~/.agents/memory/AGENTS.md; run agents sync after install or uninstall
AGENTS.md AGENTS.md tokenjuice install agents-md AGENTS.md
agents.ge agents.ge tokenjuice install agentsge .agents/rules/tokenjuice-agentsge.md
AgentsMesh AgentsMesh tokenjuice install agentsmesh .agentsmesh/rules/tokenjuice.md; run agentsmesh generate after install or uninstall
Amazon Q Amazon Q Developer CLI / Kiro compatibility tokenjuice install amazon-q .amazonq/rules/tokenjuice.md
Amp Amp tokenjuice install amp AGENTS.md / AGENT.md / CLAUDE.md
Antigravity Google Antigravity tokenjuice install antigravity .agents/rules/tokenjuice.md
anywhere-agents anywhere-agents tokenjuice install anywhere-agents AGENTS.local.md; run anywhere-agents after install or uninstall
Augment Augment tokenjuice install augment .augment/rules/tokenjuice.md
Avante Avante.nvim tokenjuice install avante avante.md
Baz Baz tokenjuice install baz .baz/skills/tokenjuice/SKILL.md
Bito Bito tokenjuice install bito .bito.yaml / .bito/tokenjuice.md
Blackbox Blackbox CLI tokenjuice install blackbox .blackbox/skills/tokenjuice/SKILL.md
Blocks Blocks tokenjuice install blocks .agents/skills/tokenjuice-blocks/SKILL.md
Clawdbot Clawdbot tokenjuice install clawdbot skills/tokenjuice/SKILL.md
IBM Bob IBM Bob Shell tokenjuice install bob AGENTS.md
Builder Builder tokenjuice install builder .builder/rules/tokenjuice.mdc
Charlie Charlie tokenjuice install charlie AGENTS.md
Cline Cline tokenjuice install cline ~/Documents/Cline/Hooks/tokenjuice-post-tool-use
CodeAnt CodeAnt tokenjuice install codeant .codeant/instructions.json
Codebuff Codebuff tokenjuice install codebuff AGENTS.md
Codegen Codegen tokenjuice install codegen AGENTS.md
Coder Agents Coder Agents tokenjuice install coder-agents .agents/skills/tokenjuice/SKILL.md
CodeRabbit CodeRabbit tokenjuice install coderabbit .coderabbit.yaml
Command Code Command Code tokenjuice install command-code ~/.commandcode/settings.json / .commandcode/settings.json
Continue Continue tokenjuice install continue .continue/rules/tokenjuice.md
Crush Crush tokenjuice install crush .crush/skills/tokenjuice/SKILL.md
Deep Agents Code Deep Agents Code tokenjuice install deepagents .deepagents/AGENTS.md
Devin Devin for Terminal tokenjuice install devin .devin/hooks.v1.json
dot-agents dot-agents tokenjuice install dot-agents ~/.agents/rules/global/rules.mdc; run dot-agents sync after install or uninstall
Docker Agent Docker Agent tokenjuice install docker-agent .docker-agent/tokenjuice.md
ECA ECA tokenjuice install eca .eca/skills/tokenjuice/SKILL.md
Elyra Elyra tokenjuice install elyra .elyra/skills/tokenjuice/SKILL.md
Firebase Studio Firebase Studio tokenjuice install firebase-studio .idx/airules.md
ForgeCode ForgeCode tokenjuice install forgecode AGENTS.md
Gemini Gemini CLI tokenjuice install gemini-cli ~/.gemini/settings.json
GitLab Duo GitLab Duo Agent Platform tokenjuice install gitlab-duo .gitlab/duo/chat-rules.md
Goose Goose tokenjuice install goose .goosehints
Greptile Greptile tokenjuice install greptile .greptile/rules.md
Grok Build Grok Build tokenjuice install grok-build AGENTS.md
Grok CLI Grok CLI tokenjuice install grok-cli ~/.grok/user-settings.json
gptme gptme tokenjuice install gptme AGENTS.md
GitHub Copilot coding agent GitHub Copilot coding agent tokenjuice install copilot-agent .github/hooks/tokenjuice-agent.json
Jean2 Jean2 tokenjuice install jean2 AGENTS.md
JetBrains AI Assistant JetBrains AI Assistant tokenjuice install jetbrains-ai .aiassistant/rules/tokenjuice.md
Junie Junie tokenjuice install junie .junie/AGENTS.md
Jules Jules tokenjuice install jules AGENTS.md
LeanCTL LeanCTL tokenjuice install leanctl .leanctl/instructions.md
Kimi Kimi Code CLI tokenjuice install kimi ~/.kimi/config.toml
Kiro Kiro tokenjuice install kiro .kiro/steering/tokenjuice.md
Kilo Code Kilo Code tokenjuice install kilo kilo.jsonc or .kilo/kilo.jsonc + .kilo/rules/tokenjuice.md
LocalCode LocalCode tokenjuice install localcode ~/.localcode/plugins/tokenjuice/
mcp-agent mcp-agent tokenjuice install mcp-agent .mcp-agent/agents/tokenjuice.md
mini-SWE-agent mini-SWE-agent tokenjuice install mini-swe-agent .mini-swe-agent/tokenjuice.yaml
SWE-agent SWE-agent tokenjuice install swe-agent .swe-agent/tokenjuice.yaml
Stagewise Stagewise tokenjuice install stagewise .stagewise/skills/tokenjuice/SKILL.md
Mistral Vibe Mistral Vibe tokenjuice install mistral-vibe AGENTS.md
Mux Mux tokenjuice install mux .mux/tool_post
NovaKit NovaKit CLI tokenjuice install novakit NOVAKIT.md
Knowns Knowns tokenjuice install knowns KNOWNS.md
Ona Ona Agent tokenjuice install ona .ona/skills/tokenjuice/SKILL.md
OpenHands OpenHands tokenjuice install openhands .openhands/hooks.json
Open Interpreter Open Interpreter tokenjuice install open-interpreter AGENTS.md
Open WebUI Open WebUI tokenjuice install openwebui .openwebui/tools/tokenjuice_compact.py
pi-go pi-go tokenjuice install pi-go .pi/skills/tokenjuice/SKILL.md
Plandex Plandex tokenjuice install plandex PLANDEX.tokenjuice.md
Qodo Qodo Code Review tokenjuice install qodo .pr_agent.toml
Qoder Qoder CLI tokenjuice install qoder AGENTS.md
Qwen Code Qwen Code tokenjuice install qwen-code .qwen/settings.json
Replit Replit Agent tokenjuice install replit replit.md
Roo Code Roo Code tokenjuice install roo .roo/rules/tokenjuice.md
Rovo Rovo Dev CLI tokenjuice install rovo AGENTS.md
Ruler Ruler tokenjuice install ruler .ruler/tokenjuice.md
Tabby Tabby tokenjuice install tabby ~/.tabby/config.toml
Tabnine Tabnine CLI tokenjuice install tabnine TABNINE.md
Trae Trae tokenjuice install trae .trae/rules/project_rules.md
UiPath UiPath for Coding Agents tokenjuice install uipath AGENTS.md
Warp Warp tokenjuice install warp AGENTS.md / WARP.md
Windsurf Windsurf tokenjuice install windsurf .windsurf/rules/tokenjuice.md
Zed Zed tokenjuice install zed .rules
Zencoder Zencoder tokenjuice install zencoder .zencoder/rules/tokenjuice.md

install

npm install -g tokenjuice
# or
pnpm add -g tokenjuice
# or
yarn global add tokenjuice
# or
brew tap vincentkoc/tap
brew install tokenjuice

then:

tokenjuice --help
tokenjuice --version
tokenjuice install [adal|aether|aictl|ai-memory-protocol|aider|agent-layer|agentinit|agentlink|agentloom|agents-cli|agents-md|agentsge|agentsmesh|amazon-q|amp|antigravity|anywhere-agents|augment|avante|baz|bito|blackbox|blocks|clawdbot|bob|builder|charlie|codex|claude-code|cline|codeant|codebuff|codegen|coder-agents|coderabbit|codebuddy|command-code|continue|copilot-agent|crush|cursor|deepagents|devin|dot-agents|docker-agent|droid|eca|elyra|firebase-studio|forgecode|gemini-cli|gitlab-duo|goose|greptile|grok-build|grok-cli|gptme|jean2|jetbrains-ai|junie|jules|leanctl|kimi|kiro|kilo|localcode|mcp-agent|mini-swe-agent|swe-agent|stagewise|mistral-vibe|mux|novakit|knowns|ona|openhands|open-interpreter|openwebui|pi|pi-go|opencode|plandex|qodo|qoder|qwen-code|replit|roo|rovo|ruler|tabby|tabnine|trae|uipath|vscode-copilot|warp|windsurf|copilot-cli|zed|zencoder]
tokenjuice uninstall [adal|aether|aictl|ai-memory-protocol|aider|agent-layer|agentinit|agentlink|agentloom|agents-cli|agents-md|agentsge|agentsmesh|amazon-q|amp|antigravity|anywhere-agents|augment|avante|baz|bito|blackbox|blocks|clawdbot|bob|builder|charlie|codex|cline|codeant|codebuff|codegen|coder-agents|coderabbit|command-code|continue|copilot-agent|crush|deepagents|devin|dot-agents|docker-agent|droid|eca|elyra|firebase-studio|forgecode|gemini-cli|gitlab-duo|goose|greptile|grok-build|grok-cli|gptme|jean2|jetbrains-ai|junie|jules|leanctl|kimi|kiro|kilo|localcode|mcp-agent|mini-swe-agent|swe-agent|stagewise|mistral-vibe|mux|novakit|knowns|ona|openhands|open-interpreter|openwebui|pi-go|opencode|plandex|qodo|qoder|qwen-code|replit|roo|rovo|ruler|tabby|tabnine|trae|uipath|vscode-copilot|warp|windsurf|copilot-cli|zed|zencoder]

OpenClaw support is bundled on the OpenClaw side. Do not run tokenjuice install openclaw; enable the bundled plugin instead:

openclaw config set plugins.entries.tokenjuice.enabled true

this requires OpenClaw 2026.4.22 or newer.

commands

tokenjuice --help
tokenjuice --version
tokenjuice reduce [file]
tokenjuice reduce-json [file]
tokenjuice wrap -- <command> [args...]
tokenjuice wrap --raw -- <command> [args...]
tokenjuice wrap --store -- <command> [args...]
tokenjuice install [adal|aether|aictl|ai-memory-protocol|aider|agent-layer|agentinit|agentlink|agentloom|agents-cli|agents-md|agentsge|agentsmesh|amazon-q|amp|antigravity|anywhere-agents|augment|avante|baz|bito|blackbox|blocks|clawdbot|bob|builder|charlie|codex|claude-code|cline|codeant|codebuff|codegen|coder-agents|coderabbit|codebuddy|command-code|continue|copilot-agent|crush|cursor|deepagents|devin|dot-agents|docker-agent|droid|eca|elyra|firebase-studio|forgecode|gemini-cli|gitlab-duo|goose|greptile|grok-build|grok-cli|gptme|jean2|jetbrains-ai|junie|jules|leanctl|kimi|kiro|kilo|localcode|mcp-agent|mini-swe-agent|swe-agent|stagewise|mistral-vibe|mux|novakit|knowns|ona|openhands|open-interpreter|openwebui|pi|pi-go|opencode|plandex|qodo|qoder|qwen-code|replit|roo|rovo|ruler|tabby|tabnine|trae|uipath|vscode-copilot|warp|windsurf|copilot-cli|zed|zencoder]
tokenjuice install [adal|aether|aictl|ai-memory-protocol|aider|agent-layer|agentinit|agentlink|agentloom|agents-cli|agents-md|agentsge|agentsmesh|amazon-q|amp|antigravity|anywhere-agents|augment|avante|baz|bito|blackbox|blocks|clawdbot|bob|builder|charlie|codex|claude-code|cline|codeant|codebuff|codegen|coder-agents|coderabbit|codebuddy|command-code|continue|copilot-agent|crush|cursor|deepagents|devin|dot-agents|docker-agent|droid|eca|elyra|firebase-studio|forgecode|gemini-cli|gitlab-duo|goose|greptile|grok-build|grok-cli|gptme|jean2|jetbrains-ai|junie|jules|leanctl|kimi|kiro|kilo|localcode|mcp-agent|mini-swe-agent|swe-agent|stagewise|mistral-vibe|mux|novakit|knowns|ona|openhands|open-interpreter|openwebui|pi|pi-go|opencode|plandex|qodo|qoder|qwen-code|replit|roo|rovo|ruler|tabby|tabnine|trae|uipath|vscode-copilot|warp|windsurf|copilot-cli|zed|zencoder] --local
tokenjuice uninstall [adal|aether|aictl|ai-memory-protocol|aider|agent-layer|agentinit|agentlink|agentloom|agents-cli|agents-md|agentsge|agentsmesh|amazon-q|amp|antigravity|anywhere-agents|augment|avante|baz|bito|blackbox|blocks|clawdbot|bob|builder|charlie|codex|cline|codeant|codebuff|codegen|coder-agents|coderabbit|command-code|continue|copilot-agent|crush|deepagents|devin|dot-agents|docker-agent|droid|eca|elyra|firebase-studio|forgecode|gemini-cli|gitlab-duo|goose|greptile|grok-build|grok-cli|gptme|jean2|jetbrains-ai|junie|jules|leanctl|kimi|kiro|kilo|localcode|mcp-agent|mini-swe-agent|swe-agent|stagewise|mistral-vibe|mux|novakit|knowns|ona|openhands|open-interpreter|openwebui|pi-go|opencode|plandex|qodo|qoder|qwen-code|replit|roo|rovo|ruler|tabby|tabnine|trae|uipath|vscode-copilot|warp|windsurf|copilot-cli|zed|zencoder]
tokenjuice ls
tokenjuice cat <artifact-id>
tokenjuice verify
tokenjuice discover
tokenjuice doctor
tokenjuice doctor hooks
tokenjuice doctor pi
tokenjuice doctor opencode
tokenjuice stats
tokenjuice stats --timezone utc

overview

tokenjuice has three surfaces. reduce compacts text that already exists, wrap runs a command and compacts the observed output, and reduce-json gives host adapters a stable machine protocol. host integrations are intentionally thin: they install a hook, extension, rule, or guidance file; call the shared compactor; and return compacted context through the host's native surface. use tokenjuice doctor hooks to check installed wiring, tokenjuice doctor <host> for one integration, and tokenjuice install <host> --local when validating the current repo build before release.

the reduction engine is rule-driven. built-in JSON rules live in src/rules, user overrides live in ~/.config/tokenjuice/rules, and project overrides live in .tokenjuice/rules; later layers override earlier ones by rule id. rules classify command output, normalize lines, keep or drop patterns, count facts, and retain deterministic head/tail slices. host adapters also apply a narrow safe-inventory policy: exact file-content reads stay raw, standalone repository inventory commands can compact, and unsafe mixed command sequences stay raw.

when a reducer gets it wrong or the task needs untouched bytes, use the explicit bypass:

tokenjuice wrap --raw -- pnpm --help
tokenjuice wrap --full -- git status

useful maintenance commands:

tokenjuice verify --fixtures
tokenjuice discover
tokenjuice doctor hooks
tokenjuice stats --timezone utc

adapter JSON

reduce-json is the machine-facing adapter command. it reads JSON from stdin or a file and always writes JSON to stdout; see the spec for envelope options and adapter behavior.

direct payload:

{
  "toolName": "exec",
  "command": "pnpm test",
  "argv": ["pnpm", "test"],
  "combinedText": "RUN  v3.2.4 /repo\n...",
  "exitCode": 1
}

docs

status

usable foundation for token reduction with diagnostics and a growing reducer set, now focused on deeper coverage and tuning.

πŸ’™ built by Vincent Koc.

About

πŸ§ƒ Token weight loss. Lean output compaction for terminal-heavy agent workflows. Works as a native CLI tool or as an extension to popular coding and agent frameworks.

Topics

Resources

License

Security policy

Stars

Watchers

Forks

Sponsor this project

 

Packages

 
 
 

Contributors