fix(libutil/terminal): require both stderr/stdout to be tty to output color

[xokdvium]

Motivation

Fixes #11583
Fixes #4848

This is a bit more conservative and safe for cli composability. A common use-case for nix cli is to filter/transform output with grep/ripgrep/jq from stdout. Those tools usually have their own colored output. It's best not to step on their toes.

Once example of such command is: nix registry list | rg nixpkgs

Note that this is more of a band-aid solution. Nix generally handles ANSI colors in an ad-hoc way and there might be some other unexpected bugs related to control characters.

Context

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[xokdvium]

My linux-jitsu is too weak to implement regression tests for this. Is there some way to redirect output to a file but pretending that's a tty? Not sure if using strace would be a great and portable approach either.

[xokdvium]

@Ericson2314, I'm not very familiar with portability for windows. Are we okay with generally using POSIX isatty with STDOUT_FILENO? Should this be cleaned up or is it fine as-is?

[Mic92]

My linux-jitsu is too weak to implement regression tests for this. Is there some way to redirect output to a file but pretending that's a tty? Not sure if using strace would be a great and portable approach either.

You can use the script command for this.

 faketty () {
     script -qefc "$(printf "%q " "$@")" /dev/null
 }

[Mic92]

Only caveat I see just now is that we have tooling like cachix (maybe also attic?), will no have less fancy builds logs:

nix-build | cachix push

Also was nix-build output maybe not relying on this.

[Mic92]

Do the said commands actually profit from colors much?
We could just unconditionally apply filterANSIEscapes and call it a day.

[edolstra]

I'm not sure about this. It's pretty reasonable to do something like nix some-command --json > out and still expect to get the progress bar and colors in interactive terminals.

[roberth]

Linux has atomic write behavior for single syscalls up to 4k or page size or something, so if both commands use a single write syscall per line, then the result would be mostly not scrambled, especially if Nix keeps an empty line above the progress bar, and puts the cursor there after its refresh (otherwise the progress bar sort of becomes a background).
I know it's not a complete solution, but it might help for some of the cases where it works, or where some mild scrambling isn't the end of the world.

I'm not looking forward to bug reports like "nix some-command --json >out hangs" where it turns out to be a slow network or something, so I'm leaning towards @edolstra's skepticism here.

[Mic92]

So the question is, how can we make stdout not containing colour when the output is not a tty?

[edolstra]

So the question is, how can we make stdout not containing colour when the output is not a tty?

Our stdout output doesn't contain color AFAIK. For example, the submitter's example (nix registry list) doesn't have color.

I think the real issue is that the progress bar doesn't play nice with other commands when piped into them because it does raw terminal I/O (like moving the cursor).

[Mic92]

Than is this issue invalid? #11583 I was also trying to reproduce this issue locally but couldn't find any colors in the output. I haven't looked for non-visible ansi characters.

[roberth]

raw terminal I/O

Doesn't have to be "raw". Basically everything can be ANSI bytes, and those don't have to be chunked in any particular way.

Than is this issue invalid? #11583

Probably valid. The progress bar is happy to appear, disappear, and leave the cursor at the start of the line.

Value printing in the repl is supposed to be streaming, but apparently also "permits" progress bar updates or something while it's doing that.
To fix that without compromising the streaming (laziness) property, the repl needs to become a proper TUI. I hope C++ has a good library for combining TUI-like behaviors and log streaming, because that interaction is not at all trivial to get right.

[edolstra]

Value printing in the repl is supposed to be streaming, but apparently also "permits" progress bar updates or something while it's doing that.

That's because it needs to call logger->pause() before starting printValue().