feat: add support for piped input

[stefanbuck]

Resolves #65

To make it easier to operate on hundreds of repositories, this pull request allows you to pipe input to octoherd.

Example

list.txt

octoherd/script-hello-world
octoherd/script-rename-master-branch-to-main
// octokit/octokit.js
# sindresorhus/get-stdin

cat list.txt | octoherd run -S path/to/script.js -T $TOKEN

When piping a list of repositories, lines starting with a comment delimiter such as # or // will be ignored.

[stefanbuck]

With the change made in https://github.com/octoherd/cli/pull/66/files#diff-3d406faa7d382dc8468988af3792bf78dfd057c449283aae37ad03a68d6b5142R27 octoherdRepos will be at least an empty array

[stefanbuck]

get-stdin is responsible to get the piped input and returns a string

[stefanbuck]

I just did another test and looks like piping into octoherd breaks the interactive mode somehow. IT just exists. I'll investigate

[stefanbuck]

Just a quick update. I made some progress (at least I understand the problem) a little better. However piping is a little more complicated than I expected.

The problem

Let's start simple with the following enquirer example

import enquirer from 'enquirer';

const prompt = new enquirer.Select({
  name: 'color',
  message: 'Pick a flavor',
  choices: ['apple', 'grape', 'watermelon', 'cherry', 'orange'],
});

prompt
  .run()
  .then((answer) => console.log('Answer:', answer))
  .catch(console.error);

When running node demo-1.js in the terminal, the user is able to select a flavour using the arrow keys.

Now if we pipe some random value echo "hi" | node demo-1.mjs in the terminal, enquirer by passes the interactive mode and selects the first value. This is what is currently happening with this pull request.

Research

ipt (Interactive Pipe To) is a cli written in Node.js and the piping works as expected. The project is using inquirer, and first I though inquirer is handling this piping case, but it is not. After reading through the source code I found https://github.com/ruyadorno/ipt/blob/8a9e6791ec7af3e471247192c42803c74c657c3b/src/cli.js#L106-L138 which seems to be the magic trick. I was able to simplify as follow:

import { promisify } from "util";
import enquirer from "enquirer";
import reopenTTY from 'reopen-tty';

function end() {
    process.exit(0);
}

function error(err) {
    console.error(err);
    process.exit(1);
}

Promise.all([promisify(reopenTTY.stdin)(), promisify(reopenTTY.stdout)()]).then(
    (stdio) => {
        const [ttyStdin, ttyStdout] = stdio;
        const stdin = ttyStdin;
        const stdout = ttyStdout;

        const prompt = new enquirer.Select({
            name: 'color',
            message: 'Pick a flavor',
            choices: ['apple', 'grape', 'watermelon', 'cherry', 'orange'],
            stdout,
            stdin,
        });

        prompt
            .run()
            .then((answer) => console.log('Answer:', answer))
            .then(end)
            .catch(error);
    }
);

Disclaimer, I'm new to this as well and this comment describes my current understanding (which might be wrong or partial incomplete).

From the TTY documentation page

When Node.js detects that it is being run with a text terminal ("TTY") attached, process.stdin will, by default, be initialized as an instance of tty.ReadStream and both process.stdout and process.stderr will, by default, be instances of tty.WriteStream.

When piping something to a Node.js process, TTY is undefined and process.stdin is initialized with an instance of Socket. Therefore there is not connection between the active terminal session and the running Node.js process. We need to recreate such a TTY stream by using reopen-tty and then pass the new stdin and stdout stream to enquirer.

This is how I understood how it after reading and try and error the code snippet. How to integrate this into Octoherd is something for later this week.

[gr2m]

Thanks for doing all the research, this is great!

Maybe we could detect piping mode, and in that case require all arguments to be set by CLI flags? And if one is missing, throw an error. That way we would never get into an interactive mode. I think we also thought about adding an explicit flag to prevent interactive mode, which is useful e.g. scripts run in CI, so we would need to create that anyway. If it would help, maybe we do that first, as a preparation for piping support, as it seems to unblock it? What do you think @stefanbuck @oscard0m

[stefanbuck]

Maybe we could detect piping mode, and in that case require all arguments to be set by CLI flags?

From my understanding we can use process.stdin.isTTY for that. It returns undefined in piping mode otherwise true (when calling the script straight away)

I think we also thought about adding an explicit flag to prevent interactive mode

So in comparison to --octoherd-bypass-confirms, this new flag will prevent the script from executing?

[gr2m]

this new flag will prevent the script from executing?

only if a required argument was not set. The new flag would set --octoherd-bypass-confirms implicitly.

[oscard0m]

I just did another test and looks like piping into octoherd breaks the interactive mode somehow. IT just exists. I'll investigate

@stefanbuck be careful with publishing a screenshot with your GitHub Token. Make sure you expire/delete it and create a new one.

When piping something to a Node.js process, TTY is undefined and process.stdin is initialized with an instance of Socket. Therefore there is not connection between the active terminal session and the running Node.js process. We need to recreate such a TTY stream by using reopen-tty and then pass the new stdin and stdout stream to enquirer.

I don't have a deep experience with CLI's in Node.js and enquirer.
Searching a little bit, I come up with this nice CLI Best Practices Repo which suggests this for reading from stdin:

const readline = require("readline");

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

rl.question("What do you think of Node.js? ", (answer) => {
  // TODO: Log the answer in a database
  console.log(`Thank you for your valuable feedback: ${answer}`);

  rl.close();
});

Is not enquirer doing something similar?

• I see they set process.stdin and process.stdout as default options.
• Maybe we should check if this is a bug from enquirer or bad usage of the library? (from @stefanbuck I assume is a bug of the library?)

---

Maybe we could detect piping mode, and in that case, require all arguments to be set by CLI flags?

I think this is the user's responsibility. When piping something as input for the command, it should be read, but still, be in interactive mode for the rest of the flags (unless the rest of the flags were provided too).

I think we also thought about adding an explicit flag to prevent interactive mode, which is useful e.g. scripts run in CI

For CI environments I come up with the following utility library. I think we should open a different discussion for CI mode (I can create it if you are happy with it) and talk about piped input here (and link comments if they are interconnected of course).