Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 2, 2025

Windows users reported two critical failures when running eval experiments: ESM loader rejecting absolute paths (ERR_UNSUPPORTED_ESM_URL_SCHEME: Received protocol 'c:') and Claude Code CLI only receiving "1" instead of the full prompt.

Changes

File URL conversion for dynamic imports

  • Added pathToFileURL to convert absolute paths to file:// URLs before dynamic import()
  • Affected 6 files with JSON imports: eval.ts, collect-args.ts, prepare-experiment.ts, prepare-evaluations.ts, test-stories.ts, chromatic.ts
// Before
await import(path.join(evalPath, 'hooks.ts'))

// After  
import { pathToFileURL } from 'node:url';
await import(pathToFileURL(path.join(evalPath, 'hooks.ts')).href)

Claude Code CLI prompt delivery

  • Moved prompt from command args to stdin (claude-code-cli.ts)
  • Ensures complete prompt transmission on Windows
// Before
const args = ['--print', '--verbose', prompt];

// After
const args = ['--print', '--verbose'];
claudeProcess.stdin.write(prompt);

Cross-platform compatible: pathToFileURL handles OS-specific file URL formatting automatically.

View original Slack conversation


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@changeset-bot
Copy link

changeset-bot bot commented Dec 2, 2025

⚠️ No Changeset found

Latest commit: 4418f1b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

💥 An error occurred when fetching the changed packages and changesets in this PR
Some errors occurred when validating the changesets config:
The package or glob expression "@storybook/mcp-eval*" is specified in the `ignore` option but it is not found in the project. You may have misspelled the package name or provided an invalid glob expression. Note that glob expressions must be defined according to https://www.npmjs.com/package/micromatch.

Copilot AI and others added 2 commits December 2, 2025 12:38
- Convert absolute paths to file:// URLs using pathToFileURL for dynamic imports
- Send Claude Code CLI prompt through stdin instead of as argument
- Fixes error: Only URLs with a scheme in: file, data, and node are supported
- Ensures prompt is fully received by Claude on Windows

Co-authored-by: JReinhold <[email protected]>
Copilot AI changed the title [WIP] Document automated evals process and contribution guidelines Fix Windows compatibility in eval framework: file URL scheme and stdin prompt delivery Dec 2, 2025
Copilot AI requested a review from JReinhold December 2, 2025 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants