llnl.util.tty package

class llnl.util.tty.SuppressOutput(msg_enabled=True, warn_enabled=True, error_enabled=True)

Bases: object

Class for disabling output in a scope using ‘with’ keyword

llnl.util.tty.debug(message, *args, **kwargs)

llnl.util.tty.debug_level()

llnl.util.tty.die(message, *args, **kwargs) → NoReturn

llnl.util.tty.error(message, *args, **kwargs)

llnl.util.tty.error_enabled()

llnl.util.tty.get_number(prompt, **kwargs)

llnl.util.tty.get_timestamp(force=False)

Get a string timestamp

llnl.util.tty.get_yes_or_no(prompt, **kwargs)

llnl.util.tty.hline(label=None, **kwargs)

Draw a labeled horizontal line.

Keyword Arguments:

• char (str) – Char to draw the line with. Default ‘-’

• max_width (int) – Maximum width of the line. Default is 64 chars.

llnl.util.tty.info(message, *args, **kwargs)

llnl.util.tty.is_debug(level=1)

llnl.util.tty.is_stacktrace()

llnl.util.tty.is_verbose()

llnl.util.tty.msg(message, *args, **kwargs)

llnl.util.tty.msg_enabled()

llnl.util.tty.output_filter(filter_fn)

Context manager that applies a filter to all output.

llnl.util.tty.process_stacktrace(countback)

Gives file and line frame ‘countback’ frames from the bottom

llnl.util.tty.set_debug(level=0)

llnl.util.tty.set_error_enabled(flag)

llnl.util.tty.set_msg_enabled(flag)

llnl.util.tty.set_stacktrace(flag)

llnl.util.tty.set_timestamp(flag)

llnl.util.tty.set_verbose(flag)

llnl.util.tty.set_warn_enabled(flag)

llnl.util.tty.show_pid()

llnl.util.tty.terminal_size()

Gets the dimensions of the console: (rows, cols).

llnl.util.tty.verbose(message, *args, **kwargs)

llnl.util.tty.warn(message, *args, **kwargs)

llnl.util.tty.warn_enabled()

Submodules

llnl.util.tty.colify module

Routines for printing columnar output. See colify() for more information.

class llnl.util.tty.colify.ColumnConfig(cols)

Bases: object

llnl.util.tty.colify.colified(elts: List[Any], cols: int = 0, output: IO | None = None, indent: int = 0, padding: int = 2, tty: bool | None = None, method: str = 'variable', console_cols: int | None = None)

Invokes the colify() function but returns the result as a string instead of writing it to an output string.

llnl.util.tty.colify.colify(elts: List[Any], cols: int = 0, output: IO | None = None, indent: int = 0, padding: int = 2, tty: bool | None = None, method: str = 'variable', console_cols: int | None = None)

Takes a list of elements as input and finds a good columnization of them, similar to how gnu ls does. This supports both uniform-width and variable-width (tighter) columns.

If elts is not a list of strings, each element is first conveted using str().

Keyword Arguments:

• output – A file object to write to. Default is sys.stdout

• indent – Optionally indent all columns by some number of spaces

• padding – Spaces between columns. Default is 2

• width – Width of the output. Default is 80 if tty not detected

• cols – Force number of columns. Default is to size to terminal, or single-column if no tty

• tty – Whether to attempt to write to a tty. Default is to autodetect a tty. Set to False to force single-column output

• method – Method to use to fit columns. Options are variable or uniform. Variable-width columns are tighter, uniform columns are all the same width and fit less data on the screen

• console_cols – number of columns on this console (default: autodetect)

llnl.util.tty.colify.colify_table(table: List[List[Any]], output: IO | None = None, indent: int = 0, padding: int = 2, console_cols: int | None = None)

Version of colify() for data expressed in rows, (list of lists).

Same as regular colify but:

1. This takes a list of lists, where each sub-list must be the same length, and each is interpreted as a row in a table. Regular colify displays a sequential list of values in columns.

2. Regular colify will always print with 1 column when the output is not a tty. This will always print with same dimensions of the table argument.

llnl.util.tty.colify.config_uniform_cols(elts, console_width, padding, cols=0)

Uniform-width column fitting algorithm.

Determines the longest element in the list, and determines how many columns of that width will fit on screen. Returns a corresponding column config.

llnl.util.tty.colify.config_variable_cols(elts, console_width, padding, cols=0)

Variable-width column fitting algorithm.

This function determines the most columns that can fit in the screen width. Unlike uniform fitting, where all columns take the width of the longest element in the list, each column takes the width of its own longest element. This packs elements more efficiently on screen.

If cols is nonzero, force

llnl.util.tty.color module

This file implements an expression syntax, similar to printf, for adding ANSI colors to text.

See colorize(), cwrite(), and cprint() for routines that can generate colored output.

colorize will take a string and replace all color expressions with ANSI control codes. If the isatty keyword arg is set to False, then the color expressions will be converted to null strings, and the returned string will have no color.

cwrite and cprint are equivalent to write() and print() calls in python, but they colorize their output. If the stream argument is not supplied, they write to sys.stdout.

Here are some example color expressions:

Expression	Meaning
@r	Turn on red coloring
@R	Turn on bright red coloring
@*{foo}	Bold foo, but don’t change text color
@_{bar}	Underline bar, but don’t change text color
@*b	Turn on bold, blue text
@_B	Turn on bright blue text with an underline
@.	Revert to plain formatting
@*g{green}	Print out ‘green’ in bold, green text, then reset to plain.
@*ggreen@.	Print out ‘green’ in bold, green text, then reset to plain.

The syntax consists of:

color-expr	‘@’ [style] color-code ‘{’ text ‘}’ | ‘@.’ | ‘@@’
style	‘*’ | ‘_’
color-code	[krgybmcwKRGYBMCW]
text	.*

‘@’ indicates the start of a color expression. It can be followed by an optional * or _ that indicates whether the font should be bold or underlined. If * or _ is not provided, the text will be plain. Then an optional color code is supplied. This can be [krgybmcw] or [KRGYBMCW], where the letters map to black(k), red(r), green(g), yellow(y), blue(b), magenta(m), cyan(c), and white(w). Lowercase letters denote normal ANSI colors and capital letters denote bright ANSI colors.

Finally, the color expression can be followed by text enclosed in {}. If braces are present, only the text in braces is colored. If the braces are NOT present, then just the control codes to enable the color will be output. The console can be reset later to plain text with ‘@.’.

To output an @, use ‘@@’. To output a } inside braces, use ‘}}’.

exception llnl.util.tty.color.ColorParseError(message)

Bases: Exception

Raised when a color format fails to parse.

class llnl.util.tty.color.ColorStream(stream, color=None)

Bases: object

write(string, **kwargs)

writelines(sequence, **kwargs)

llnl.util.tty.color.cescape(string)

Escapes special characters needed for color codes.

Replaces the following symbols with their equivalent literal forms:

@	@@
}	}}

Parameters:

string (str) – the string to escape

Returns:

the string with color codes escaped

Return type:

(str)

llnl.util.tty.color.cextra(string)

Length of extra color characters in a string

llnl.util.tty.color.clen(string)

Return the length of a string, excluding ansi color sequences.

llnl.util.tty.color.color_when(value)

Context manager to temporarily use a particular color setting.

llnl.util.tty.color.colorize(string, **kwargs)

Replace all color expressions in a string with ANSI control codes.

Parameters:

string (str) – The string to replace

Returns:

The filtered string

Return type:

str

Keyword Arguments:

• color (bool) – If False, output will be plain text without control codes, for output to non-console devices.

• enclose (bool) – If True, enclose ansi color sequences with square brackets to prevent misestimation of terminal width.

llnl.util.tty.color.cprint(string, stream=None, color=None)

Same as cwrite, but writes a trailing newline to the stream.

llnl.util.tty.color.cwrite(string, stream=None, color=None)

Replace all color expressions in string with ANSI control codes and write the result to the stream. If color is False, this will write plain text with no color. If True, then it will always write colored output. If not supplied, then it will be set based on stream.isatty().

llnl.util.tty.color.get_color_when()

Return whether commands should print color or not.

class llnl.util.tty.color.match_to_ansi(color=True, enclose=False)

Bases: object

escape(s)

Returns a TTY escape sequence for a color

llnl.util.tty.color.set_color_when(when)

Set when color should be applied. Options are:

• True or ‘always’: always print color

• False or ‘never’: never print color

• None or ‘auto’: only print color if sys.stdout is a tty.

llnl.util.tty.color.try_enable_terminal_color_on_windows()

Turns coloring in Windows terminal by enabling VTP in Windows consoles (CMD/PWSH/CONHOST) Method based on the link below https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences#example-of-enabling-virtual-terminal-processing

Note: No-op on non windows platforms

llnl.util.tty.log module

Utility classes for logging the output of blocks of code.

class llnl.util.tty.log.FileWrapper(file_like)

Bases: object

Represents a file. Can be an open stream, a path to a file (not opened yet), or neither. When unwrapped, it returns an open file (or file-like) object.

close()

unwrap()

class llnl.util.tty.log.MultiProcessFd(fd)

Bases: object

Return an object which stores a file descriptor and can be passed as an argument to a function run with multiprocessing.Process, such that the file descriptor is available in the subprocess.

close()

property fd

class llnl.util.tty.log.StreamWrapper(sys_attr)

Bases: object

Wrapper class to handle redirection of io streams

close()

Redirect back to the original system stream, and close stream

flush()

redirect_stream(to_fd)

Redirect stdout to the given file descriptor.

class llnl.util.tty.log.Unbuffered(stream)

Bases: object

Wrapper for Python streams that forces them to be unbuffered.

This is implemented by forcing a flush after each write.

write(data)

writelines(datas)

llnl.util.tty.log.close_connection_and_file(multiprocess_fd, file)

llnl.util.tty.log.ignore_signal(signum)

Context manager to temporarily ignore a signal.

class llnl.util.tty.log.keyboard_input(stream)

Bases: object

Context manager to disable line editing and echoing.

Use this with sys.stdin for keyboard input, e.g.:

with keyboard_input(sys.stdin) as kb:
    while True:
        kb.check_fg_bg()
        r, w, x = select.select([sys.stdin], [], [])
        # ... do something with keypresses ...

The keyboard_input context manager disables canonical (line-based) input and echoing, so that keypresses are available on the stream immediately, and they are not printed to the terminal. Typically, standard input is line-buffered, which means keypresses won’t be sent until the user hits return. In this mode, a user can hit, e.g., ‘v’, and it will be read on the other end of the pipe immediately but not printed.

The handler takes care to ensure that terminal changes only take effect when the calling process is in the foreground. If the process is backgrounded, canonical mode and echo are re-enabled. They are disabled again when the calling process comes back to the foreground.

This context manager works through a single signal handler for SIGTSTP, along with a poolling routine called check_fg_bg(). Here are the relevant states, transitions, and POSIX signals:

[Running] -------- Ctrl-Z sends SIGTSTP ------------.
[ in FG ] <------- fg sends SIGCONT --------------. |
   ^                                              | |
   | fg (no signal)                               | |
   |                                              | v
[Running] <------- bg sends SIGCONT ---------- [Stopped]
[ in BG ]                                      [ in BG ]

We handle all transitions exept for SIGTSTP generated by Ctrl-Z by periodically calling check_fg_bg(). This routine notices if we are in the background with canonical mode or echo disabled, or if we are in the foreground without canonical disabled and echo enabled, and it fixes the terminal settings in response.

check_fg_bg() works except for when the process is stopped with SIGTSTP. We cannot rely on a periodic timer in this case, as it may not rrun before the process stops. We therefore restore terminal settings in the SIGTSTP handler.

Additional notes:

• We mostly use polling here instead of a SIGARLM timer or a thread. This is to avoid the complexities of many interrupts, which seem to make system calls (like I/O) unreliable in older Python versions (2.6 and 2.7). See these issues for details:

  1. https://www.python.org/dev/peps/pep-0475/

  2. https://bugs.python.org/issue8354

  There are essentially too many ways for asynchronous signals to go wrong if we also have to support older Python versions, so we opt not to use them.

• SIGSTOP can stop a process (in the foreground or background), but it can’t be caught. Because of this, we can’t fix any terminal settings on SIGSTOP, and the terminal will be left with ICANON and ECHO disabled until it is resumes execution.

• Technically, a process could be sent SIGTSTP while running in the foreground, without the shell backgrounding that process. This doesn’t happen in practice, and we assume that SIGTSTP always means that defaults should be restored.

• We rely on termios support. Without it, or if the stream isn’t a TTY, keyboard_input has no effect.

check_fg_bg()

llnl.util.tty.log.log_output(*args, **kwargs)

Context manager that logs its output to a file.

In the simplest case, the usage looks like this:

with log_output('logfile.txt'):
    # do things ... output will be logged

Any output from the with block will be redirected to logfile.txt. If you also want the output to be echoed to stdout, use the echo parameter:

with log_output('logfile.txt', echo=True):
    # do things ... output will be logged and printed out

The following is available on Unix only. No-op on Windows. And, if you just want to echo some stuff from the parent, use force_echo:

with log_output('logfile.txt', echo=False) as logger:
    # do things ... output will be logged

    with logger.force_echo():
        # things here will be echoed *and* logged

See individual log classes for more information.

This method is actually a factory serving a per platform (unix vs windows) log_output class

class llnl.util.tty.log.nixlog(file_like=None, echo=False, debug=0, buffer=False, env=None, filter_fn=None)

Bases: object

Under the hood, we spawn a daemon and set up a pipe between this process and the daemon. The daemon writes our output to both the file and to stdout (if echoing). The parent process can communicate with the daemon to tell it when and when not to echo; this is what force_echo does. You can also enable/disable echoing by typing ‘v’.

We try to use OS-level file descriptors to do the redirection, but if stdout or stderr has been set to some Python-level file object, we use Python-level redirection instead. This allows the redirection to work within test frameworks like nose and pytest.

force_echo()

Context manager to force local echo, even if echo is off.

llnl.util.tty.log.replace_environment(env)

Replace the current environment (os.environ) with env.

If env is empty (or None), this unsets all current environment variables.

class llnl.util.tty.log.winlog(file_like=None, echo=False, debug=0, buffer=False, env=None, filter_fn=None)

Bases: object

Similar to nixlog, with underlying functionality ported to support Windows.

Does not support the use of ‘v’ toggling as nixlog does.

force_echo()

Context manager to force local echo, even if echo is off.

llnl.util.tty.pty module

The pty module handles pseudo-terminals.

Currently, the infrastructure here is only used to test llnl.util.tty.log.

If this is used outside a testing environment, we will want to reconsider things like timeouts in ProcessController.wait(), which are set to get tests done quickly, not to avoid high CPU usage.

Note: The functionality in this module is unsupported on Windows

class llnl.util.tty.pty.ProcessController(pid, controller_fd, timeout=1, sleep_time=0.1, debug=False)

Bases: object

Wrapper around some fundamental process control operations.

This allows one process (the controller) to drive another (the minion) similar to the way a shell would, by sending signals and I/O.

background()

True if pgid is in a background pgroup of controller_fd’s tty.

bg()

cont()

fg()

get_canon_echo_attrs()

Get echo and canon attributes of the terminal of controller_fd.

horizontal_line(name)

Labled horizontal line for debugging.

input_on()

True if keyboard input is enabled on the controller_fd pty.

proc_status()

status()

Print debug message with status info for the minion.

tstp()

Send SIGTSTP to the controlled process.

wait(condition)

wait_disabled()

wait_disabled_fg()

wait_enabled()

wait_running()

wait_stopped()

write(byte_string)

class llnl.util.tty.pty.PseudoShell(controller_function, minion_function)

Bases: object

Sets up controller and minion processes with a PTY.

You can create a PseudoShell if you want to test how some function responds to terminal input. This is a pseudo-shell from a job control perspective; controller_function and minion_function are set up with a pseudoterminal (pty) so that the controller can drive the minion through process control signals and I/O.

The two functions should have signatures like this:

def controller_function(proc, ctl, **kwargs)
def minion_function(**kwargs)

controller_function is spawned in its own process and passed three arguments:

proc

the multiprocessing.Process object representing the minion

ctl

a ProcessController object tied to the minion

kwargs

keyword arguments passed from PseudoShell.start().

minion_function is only passed kwargs delegated from PseudoShell.start().

The ctl.controller_fd will have its controller_fd connected to sys.stdin in the minion process. Both processes will share the same sys.stdout and sys.stderr as the process instantiating PseudoShell.

Here are the relationships between processes created:

._________________________________________________________.
| Minion Process                                          | pid     2
| - runs minion_function                                  | pgroup  2
|_________________________________________________________| session 1
    ^
    | create process with controller_fd connected to stdin
    | stdout, stderr are the same as caller
._________________________________________________________.
| Controller Process                                      | pid     1
| - runs controller_function                              | pgroup  1
| - uses ProcessController and controller_fd to           | session 1
|   control minion                                        |
|_________________________________________________________|
    ^
    | create process
    | stdin, stdout, stderr are the same as caller
._________________________________________________________.
| Caller                                                  |  pid     0
| - Constructs, starts, joins PseudoShell                 |  pgroup  0
| - provides controller_function, minion_function         |  session 0
|_________________________________________________________|

join()

Wait for the minion process to finish, and return its exit code.

start(**kwargs)

Start the controller and minion processes.

Parameters:

kwargs (dict) – arbitrary keyword arguments that will be passed to controller and minion functions

The controller process will create the minion, then call controller_function. The minion process will call minion_function.