Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pipeStdout()-related methods #531

Merged
merged 9 commits into from Mar 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
49 changes: 40 additions & 9 deletions index.d.ts
@@ -1,6 +1,6 @@
import {type Buffer} from 'node:buffer';
import {type ChildProcess} from 'node:child_process';
import {type Stream, type Readable as ReadableStream} from 'node:stream';
import {type Stream, type Readable as ReadableStream, type Writable as WritableStream} from 'node:stream';

export type StdioOption =
| 'pipe'
Expand Down Expand Up @@ -285,7 +285,9 @@ export type NodeOptions<EncodingType = string> = {
readonly nodeOptions?: string[];
} & Options<EncodingType>;

export type ExecaReturnBase<StdoutStderrType> = {
type StdoutStderrAll = string | Buffer | undefined;

export type ExecaReturnBase<StdoutStderrType extends StdoutStderrAll> = {
/**
The file and arguments that were run, for logging purposes.

Expand Down Expand Up @@ -346,7 +348,7 @@ export type ExecaReturnBase<StdoutStderrType> = {
signalDescription?: string;
};

export type ExecaSyncReturnValue<StdoutStderrType = string> = {
export type ExecaSyncReturnValue<StdoutStderrType extends StdoutStderrAll = string> = {
} & ExecaReturnBase<StdoutStderrType>;

/**
Expand All @@ -359,7 +361,7 @@ The child process fails when:
- being canceled
- there's not enough memory or there are already too many child processes
*/
export type ExecaReturnValue<StdoutStderrType = string> = {
export type ExecaReturnValue<StdoutStderrType extends StdoutStderrAll = string> = {
/**
The output of the process with `stdout` and `stderr` interleaved.

Expand All @@ -377,7 +379,7 @@ export type ExecaReturnValue<StdoutStderrType = string> = {
isCanceled: boolean;
} & ExecaSyncReturnValue<StdoutStderrType>;

export type ExecaSyncError<StdoutStderrType = string> = {
export type ExecaSyncError<StdoutStderrType extends StdoutStderrAll = string> = {
/**
Error message when the child process failed to run. In addition to the underlying error message, it also contains some information related to why the child process errored.

Expand All @@ -398,7 +400,7 @@ export type ExecaSyncError<StdoutStderrType = string> = {
originalMessage?: string;
} & Error & ExecaReturnBase<StdoutStderrType>;

export type ExecaError<StdoutStderrType = string> = {
export type ExecaError<StdoutStderrType extends StdoutStderrAll = string> = {
/**
The output of the process with `stdout` and `stderr` interleaved.

Expand All @@ -425,7 +427,7 @@ export type KillOptions = {
forceKillAfterTimeout?: number | false;
};

export type ExecaChildPromise<StdoutStderrType> = {
export type ExecaChildPromise<StdoutStderrType extends StdoutStderrAll> = {
/**
Stream combining/interleaving [`stdout`](https://nodejs.org/api/child_process.html#child_process_subprocess_stdout) and [`stderr`](https://nodejs.org/api/child_process.html#child_process_subprocess_stderr).

Expand All @@ -448,9 +450,38 @@ export type ExecaChildPromise<StdoutStderrType> = {
Similar to [`childProcess.kill()`](https://nodejs.org/api/child_process.html#child_process_subprocess_kill_signal). This used to be preferred when cancelling the child process execution as the error is more descriptive and [`childProcessResult.isCanceled`](#iscanceled) is set to `true`. But now this is deprecated and you should either use `.kill()` or the `signal` option when creating the child process.
*/
cancel(): void;

/**
[Pipe](https://nodejs.org/api/stream.html#readablepipedestination-options) the child process's `stdout` to `target`, which can be:
- Another `execa()` return value
- A writable stream
- A file path string

If the `target` is another `execa()` return value, it is returned. Otherwise, the original `execa()` return value is returned. This allows chaining `pipeStdout()` then `await`ing the final result.

The `stdout` option] must be kept as `pipe`, its default value.
*/
pipeStdout?<Target extends ExecaChildPromise<StdoutStderrAll>>(target: Target): Target;
pipeStdout?(target: WritableStream | string): ExecaChildProcess<StdoutStderrType>;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those methods must be marked as optional since they depend on the value of the stdout, stderr, stdio, all and encoding options.

In the future, we might want to improve the types to infer the value of those options so they can be used to determine those methods' optionality. However, this would complicate the types quite a bit, so I'm leaving it off for this PR.


/**
Like `pipeStdout()` but piping the child process's `stderr` instead.

The `stderr` option must be kept as `pipe`, its default value.
*/
pipeStderr?<Target extends ExecaChildPromise<StdoutStderrAll>>(target: Target): Target;
pipeStderr?(target: WritableStream | string): ExecaChildProcess<StdoutStderrType>;

/**
Combines both `pipeStdout()` and `pipeStderr()`.

Either the `stdout` option or the `stderr` option must be kept as `pipe`, their default value. Also, the `all` option must be set to `true`.
*/
pipeAll?<Target extends ExecaChildPromise<StdoutStderrAll>>(target: Target): Target;
pipeAll?(target: WritableStream | string): ExecaChildProcess<StdoutStderrType>;
};

export type ExecaChildProcess<StdoutStderrType = string> = ChildProcess &
export type ExecaChildProcess<StdoutStderrType extends StdoutStderrAll = string> = ChildProcess &
ExecaChildPromise<StdoutStderrType> &
Promise<ExecaReturnValue<StdoutStderrType>>;

Expand Down Expand Up @@ -557,7 +588,7 @@ type TemplateExpression =
| ExecaSyncReturnValue<string | Buffer>
| Array<string | number | ExecaReturnValue<string | Buffer> | ExecaSyncReturnValue<string | Buffer>>;

type Execa$<StdoutStderrType = string> = {
type Execa$<StdoutStderrType extends StdoutStderrAll = string> = {
/**
Same as `execa()` except both file and arguments are specified in a single tagged template string. For example, `` $`echo unicorns` `` is the same as `execa('echo', ['unicorns'])`.

Expand Down
8 changes: 6 additions & 2 deletions index.js
Expand Up @@ -9,6 +9,7 @@ import onetime from 'onetime';
import {makeError} from './lib/error.js';
import {normalizeStdio, normalizeStdioNode} from './lib/stdio.js';
import {spawnedKill, spawnedCancel, setupTimeout, validateTimeout, setExitHandler} from './lib/kill.js';
import {addPipeMethods} from './lib/pipe.js';
import {handleInput, getSpawnedResult, makeAllStream, validateInputSync} from './lib/stream.js';
import {mergePromise, getSpawnedPromise} from './lib/promise.js';
import {joinCommand, parseCommand, parseTemplates, getEscapedCommand} from './lib/command.js';
Expand Down Expand Up @@ -100,7 +101,8 @@ export function execa(file, args, options) {
isCanceled: false,
killed: false,
}));
return mergePromise(dummySpawned, errorPromise);
mergePromise(dummySpawned, errorPromise);
return dummySpawned;
}

const spawnedPromise = getSpawnedPromise(spawned);
Expand Down Expand Up @@ -161,7 +163,9 @@ export function execa(file, args, options) {

spawned.all = makeAllStream(spawned, parsed.options);

return mergePromise(spawned, handlePromiseOnce);
addPipeMethods(spawned);
mergePromise(spawned, handlePromiseOnce);
ehmicky marked this conversation as resolved.
Show resolved Hide resolved
return spawned;
}

export function execaSync(file, args, options) {
Expand Down
39 changes: 38 additions & 1 deletion index.test-d.ts
Expand Up @@ -4,7 +4,8 @@ import {Buffer} from 'node:buffer';
// to get treated as `any` by `@typescript-eslint/no-unsafe-assignment`.
import * as process from 'node:process';
import {type Readable as ReadableStream} from 'node:stream';
import {expectType, expectError} from 'tsd';
import {createWriteStream} from 'node:fs';
import {expectType, expectError, expectAssignable} from 'tsd';
import {
$,
execa,
Expand All @@ -24,6 +25,39 @@ try {
execaPromise.cancel();
expectType<ReadableStream | undefined>(execaPromise.all);

const execaBufferPromise = execa('unicorns', {encoding: null});
const writeStream = createWriteStream('output.txt');

expectAssignable<Function | undefined>(execaPromise.pipeStdout);
expectType<ExecaChildProcess>(execaPromise.pipeStdout!('file.txt'));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeStdout!('file.txt'));
expectType<ExecaChildProcess>(execaPromise.pipeStdout!(writeStream));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeStdout!(writeStream));
expectType<ExecaChildProcess>(execaPromise.pipeStdout!(execaPromise));
expectType<ExecaChildProcess<Buffer>>(execaPromise.pipeStdout!(execaBufferPromise));
expectType<ExecaChildProcess>(execaBufferPromise.pipeStdout!(execaPromise));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeStdout!(execaBufferPromise));

expectAssignable<Function | undefined>(execaPromise.pipeStderr);
expectType<ExecaChildProcess>(execaPromise.pipeStderr!('file.txt'));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeStderr!('file.txt'));
expectType<ExecaChildProcess>(execaPromise.pipeStderr!(writeStream));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeStderr!(writeStream));
expectType<ExecaChildProcess>(execaPromise.pipeStderr!(execaPromise));
expectType<ExecaChildProcess<Buffer>>(execaPromise.pipeStderr!(execaBufferPromise));
expectType<ExecaChildProcess>(execaBufferPromise.pipeStderr!(execaPromise));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeStderr!(execaBufferPromise));

expectAssignable<Function | undefined>(execaPromise.pipeAll);
expectType<ExecaChildProcess>(execaPromise.pipeAll!('file.txt'));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeAll!('file.txt'));
expectType<ExecaChildProcess>(execaPromise.pipeAll!(writeStream));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeAll!(writeStream));
expectType<ExecaChildProcess>(execaPromise.pipeAll!(execaPromise));
expectType<ExecaChildProcess<Buffer>>(execaPromise.pipeAll!(execaBufferPromise));
expectType<ExecaChildProcess>(execaBufferPromise.pipeAll!(execaPromise));
expectType<ExecaChildProcess<Buffer>>(execaBufferPromise.pipeAll!(execaBufferPromise));

const unicornsResult = await execaPromise;
expectType<string>(unicornsResult.command);
expectType<string>(unicornsResult.escapedCommand);
Expand Down Expand Up @@ -63,6 +97,9 @@ try {
expectType<string>(unicornsResult.stdout);
expectType<string>(unicornsResult.stderr);
expectError(unicornsResult.all);
expectError(unicornsResult.pipeStdout);
expectError(unicornsResult.pipeStderr);
expectError(unicornsResult.pipeAll);
expectType<boolean>(unicornsResult.failed);
expectType<boolean>(unicornsResult.timedOut);
expectError(unicornsResult.isCanceled);
Expand Down
42 changes: 42 additions & 0 deletions lib/pipe.js
@@ -0,0 +1,42 @@
import {createWriteStream} from 'node:fs';
import {ChildProcess} from 'node:child_process';
import {isWritableStream} from 'is-stream';

const isExecaChildProcess = target => target instanceof ChildProcess && typeof target.then === 'function';

const pipeToTarget = (spawned, streamName, target) => {
if (typeof target === 'string') {
spawned[streamName].pipe(createWriteStream(target));
return spawned;
}

if (isWritableStream(target)) {
spawned[streamName].pipe(target);
return spawned;
}

if (!isExecaChildProcess(target)) {
throw new TypeError('The second argument must be a string, a stream or an Execa child process.');
}

if (!isWritableStream(target.stdin)) {
throw new TypeError('The target child process\'s stdin must be available.');
}

spawned[streamName].pipe(target.stdin);
return target;
};

export const addPipeMethods = spawned => {
if (spawned.stdout !== null) {
spawned.pipeStdout = pipeToTarget.bind(undefined, spawned, 'stdout');
}

if (spawned.stderr !== null) {
spawned.pipeStderr = pipeToTarget.bind(undefined, spawned, 'stderr');
}

if (spawned.all !== undefined) {
spawned.pipeAll = pipeToTarget.bind(undefined, spawned, 'all');
}
};
2 changes: 0 additions & 2 deletions lib/promise.js
Expand Up @@ -16,8 +16,6 @@ export const mergePromise = (spawned, promise) => {

Reflect.defineProperty(spawned, property, {...descriptor, value});
}

return spawned;
};

// Use promises instead of `child_process` events
Expand Down
80 changes: 55 additions & 25 deletions readme.md
Expand Up @@ -20,6 +20,7 @@ This package improves [`child_process`](https://nodejs.org/api/child_process.htm
- [Executes locally installed binaries by name.](#preferlocal)
- [Cleans up spawned processes when the parent process dies.](#cleanup)
- [Get interleaved output](#all) from `stdout` and `stderr` similar to what is printed on the terminal. [*(Async only)*](#execasyncfile-arguments-options)
- Convenience methods to [pipe processes' output](#redirect-output-to-a-file)
- [Can specify file and arguments as a single string without a shell](#execacommandcommand-options)
- More descriptive errors.

Expand Down Expand Up @@ -115,12 +116,41 @@ unicorns
rainbows
```

### Pipe the child process stdout to the parent
### Redirect output to a file

```js
import {execa} from 'execa';

// Similar to `echo unicorns > stdout.txt` in Bash
await execa('echo', ['unicorns']).pipeStdout('stdout.txt')

// Similar to `echo unicorns 2> stdout.txt` in Bash
await execa('echo', ['unicorns']).pipeStderr('stderr.txt')

// Similar to `echo unicorns &> stdout.txt` in Bash
await execa('echo', ['unicorns'], {all:true}).pipeAll('all.txt')
```

### Save and pipe output from a child process

```js
import {execa} from 'execa';

execa('echo', ['unicorns']).stdout.pipe(process.stdout);
const {stdout} = await execa('echo', ['unicorns']).pipeStdout(process.stdout);
// Prints `unicorns`
console.log(stdout);
// Also returns 'unicorns'
```

### Pipe multiple processes

```js
import {execa} from 'execa';

// Similar to `echo unicorns | cat` in Bash
const {stdout} = await execa('echo', ['unicorns']).pipeStdout(execa('cat'));
console.log(stdout);
//=> 'unicorns'
```

### Handling Errors
Expand Down Expand Up @@ -261,6 +291,29 @@ This is `undefined` if either:
- the [`all` option](#all-2) is `false` (the default value)
- both [`stdout`](#stdout-1) and [`stderr`](#stderr-1) options are set to [`'inherit'`, `'ipc'`, `Stream` or `integer`](https://nodejs.org/dist/latest-v6.x/docs/api/child_process.html#child_process_options_stdio)

#### pipeStdout(target)

[Pipe](https://nodejs.org/api/stream.html#readablepipedestination-options) the child process's `stdout` to `target`, which can be:
- Another [`execa()` return value](#pipe-multiple-processes)
- A [writable stream](#save-and-pipe-output-from-a-child-process)
- A [file path string](#redirect-output-to-a-file)

If the `target` is another [`execa()` return value](#execacommandcommand-options), it is returned. Otherwise, the original `execa()` return value is returned. This allows chaining `pipeStdout()` then `await`ing the [final result](#childprocessresult).

The [`stdout` option](#stdout-1) must be kept as `pipe`, its default value.

#### pipeStderr(target)

Like [`pipeStdout()`](#pipestdouttarget) but piping the child process's `stderr` instead.

The [`stderr` option](#stderr-1) must be kept as `pipe`, its default value.

#### pipeAll(target)

Combines both [`pipeStdout()`](#pipestdouttarget) and [`pipeStderr()`](#pipestderrtarget).

Either the [`stdout` option](#stdout-1) or the [`stderr` option](#stderr-1) must be kept as `pipe`, their default value. Also, the [`all` option](#all-2) must be set to `true`.

### execaSync(file, arguments?, options?)

Execute a file synchronously.
Expand Down Expand Up @@ -706,29 +759,6 @@ const run = async () => {
console.log(await pRetry(run, {retries: 5}));
```

### Save and pipe output from a child process

Let's say you want to show the output of a child process in real-time while also saving it to a variable.

```js
import {execa} from 'execa';

const subprocess = execa('echo', ['foo']);
subprocess.stdout.pipe(process.stdout);

const {stdout} = await subprocess;
console.log('child output:', stdout);
```

### Redirect output to a file

```js
import {execa} from 'execa';

const subprocess = execa('echo', ['foo'])
subprocess.stdout.pipe(fs.createWriteStream('stdout.txt'))
```

### Redirect input from a file

```js
Expand Down