[GNU Make Jobserver] Add newline when printing recipes when Make jobserver is active

[carterols]

When Ninja is invoked from Make, the stdout gets jumbled because Ninja recipes don't insert a newline at the end of a recipe.

When invoking ninja directly, I like the behavior of overwriting the current line with the current recipe, but when Ninja is subinvoked, I get output like:

[1/10] Building CXX Object test.c.....Compiling abc.c

Where the ..Compiling abc.c is output from a Make recipe.

Expected Behavior: When ninja is run and detects a make jobserver, it should append a newline to the status.

[digit-google]

That's a problem with GNU Make really, have you tried the --output-sync=line option?

See https://www.gnu.org/software/make/manual/make.html#Parallel-Output

[carterols]

That's a problem with GNU Make really, have you tried the --output-sync=line option?

See https://www.gnu.org/software/make/manual/make.html#Parallel-Output

This doesn't solve the issue. Instead, this waits to print out all the ninja output until the ninja build is completely finished. The --output-sync=line only prints output from a recipe when it finds a newline in the output from the recipe.

[digit-google]

Interesting, thanks for checking.

Ninja only does the overwriting status when it detects that stdout is a tty (interactive terminal). You can disable this in several ways, e.g.:

• Pipe the result of Ninja through cat, e.g. ninja <args> 2>&1 | cat.
• Set TERM=dumb in your environment before calling Ninja, e.g.: TERM=dumb ninja <args>.

Both will also disable ANSI colored output from Ninja itself, but will get you a cleaner output from GNU Make when running other unrelated recipes. However, if everything runs in parallel and write to the same stdout without coordination, expect some accidental overlaps to still occur randomly (this is why Ninja buffers all command outputs, except when they are in the console pool).

There is no reliable way to detect a GNU Make specific jobserver, as far as I know.

[carterols]

As you mentioned, the issue with setting TERM=dumb and piping through cat is that you lose colorization. I want colorization. Also, it definitely is possible to detect a jobserver. If ninja detects a make jobserver, it prints it out in one of the first lines:

ninja: Jobserver mode detected: r -j64 -l128 --jobserver-auth=fifo:/tmp/GMfifo45

Listen, I don't want to be invoking Ninja this way. I would always prefer to just invoke ninja via CMake and be done with it. However, I am in the process of converting my company's build system from Make -> CMake and we are doing so incrementally by currently subinvoking ninja inside of our make build.

[jhasse]

Use CLICOLOR_FORCE=1 ninja <args> 2>&1 | cat to not lose colorization.

[digit-google]

As you mentioned, the issue with setting TERM=dumb and piping through cat is that you lose colorization. I want colorization. Also, it definitely is possible to detect a jobserver. If ninja detects a make jobserver, it prints it out in one of the first lines:

ninja: Jobserver mode detected: r -j64 -l128 --jobserver-auth=fifo:/tmp/GMfifo45

Just for clarity. That detects a jobserver being setup, but this doesn't mean it is implemented by GNU Make (which is the problematic program here). There are several other implementations, and there is even PR #2634 to make Ninja act as one.

Apart from that, I agree with @jhasse here that CLICOLOR_FORCE=1 should be enough for your use case.

[carterols]

Ok that is an acceptable solution. I will go ahead and try that out.

[jhasse]

In general there should be more control over the way status is printed. Also see issues as #2475.

My idea would be to improve the command arg parsing and then add the options as new args (for example an option to control if the status should only be one line).

[carterols]

Use CLICOLOR_FORCE=1 ninja <args> 2>&1 | cat to not lose colorization.

Actually I take back what I said. This is not great because now I lose the error code when subinvoking ninja from make. So if I encounter a compile error, Make thinks everything is ok. These "solutions" seem more like "workarounds" for Ninja's lack of output control, as @jhasse mentioned. There needs to be a way for ninja to output a newline from an argument or env variable. This shouldn't be that hard, right?