Skip to content

proc

This section contains functions related to handling co-processes. The Bash builtin coproc is missing features, in particular there may be only one coproc open at any time.

This library comes with L_proc_popen which allows to open any number of child processes for writing and reading.

Usage Guide

The core function is L_proc_popen. It launches a command asynchronously (like coproc) but gives you a handle to manage it. This handle is the PID of the process, stored in the variable you provide.

Basic Interaction

The most common use case is opening a process, writing to it, and reading from it.

# Open a process (cat) that reads from stdin and writes to stdout
# -I pipe: We want to write to its stdin
# -O pipe: We want to read from its stdout
L_proc_popen -I pipe -O pipe proc cat

# Write to the process's stdin
L_proc_printf "$proc" "Hello World\n"

# Close stdin to signal EOF to the process (cat will then finish and exit)
L_proc_close_stdin "$proc"

# Read the process's output
L_proc_read "$proc" line
echo "Got: $line"

# Wait for the process to clean up
L_proc_wait "$proc"

Input/Output Modes

L_proc_popen is powerful because it allows precise control over standard input (-I), output (-O), and error (-E) streams. You can mix and match these modes.

Pipe (Default for interaction)

Use pipe when you want to interact with the stream using L_proc_printf, L_proc_read, or L_proc_communicate.

L_proc_popen -I pipe -O pipe -E pipe proc my_command
# Now you can write to proc stdin, and read from proc stdout and stderr

Fixed Input String

Use -I input with -i "string" to pass a fixed string to the process's stdin immediately. This is useful when you don't need to interactively write to the process.

# Pass "hello" to grep's stdin
L_proc_popen -I input -i "hello" -O pipe proc grep "h"
L_proc_communicate -o output "$proc"
echo "$output" # hello

File Redirection

Redirect streams directly to or from files using file. This avoids the overhead of reading/writing in the shell.

# Write stdout directly to a file
L_proc_popen -O file -o "/tmp/output.log" proc echo "log entry"
L_proc_wait "$proc"
cat /tmp/output.log

# Read stdin directly from a file
echo "data" > /tmp/input.txt
L_proc_popen -I file -i "/tmp/input.txt" -O pipe proc cat
L_proc_communicate -o output "$proc"

Discarding Output (Null)

Use null to redirect a stream to /dev/null.

# Ignore stderr
L_proc_popen -O pipe -E null proc command_with_noisy_stderr

Closing Streams

Use close to completely close the file descriptor for the process.

# Process will receive "Bad file descriptor" if it tries to write to stdout
L_proc_popen -O close proc echo "this will fail"

Existing File Descriptors

Use fd to connect streams to existing file descriptors open in your current shell.

# Connect process stderr to the current shell's stderr (pass-through)
L_proc_popen -E fd -e 2 proc my_command

Automatic Cleanup

You can register a "finally" trap to ensure the process is waited upon when your function returns, preventing zombie processes.

my_func() {
    # -W 0 registers a cleanup handler for the current stack level
    L_proc_popen -W 0 -I pipe -O pipe proc sleep 10
    # Even if we return early or error out here, the process will be cleaned up
}

Managing Multiple Processes

Since L_proc_popen uses variables to store handles, you can manage multiple processes easily using arrays.

#!/usr/bin/env bash
. $(dirname "$0")/../L_lib.sh
L_finally wait
L_finally L_kill_all_childs
childs=()
for script in 'sleep 1 && exit 1' 'sleep 2; exit 2' 'sleep 3; exit 3'; do
  L_proc_popen tmp bash -c "$script"
  childs+=("$tmp")
done
for i in "${childs[@]}"; do
  L_proc_wait i
  echo "Process [$(L_proc_get_cmd i)] pid $(L_proc_get_pid i) exited with $(L_proc_get_exitcode i)"
done

API Reference

proc

Processes and jobs related functions.

L_bashpid_to

Get bashpid in a way compatible with Bash before 4.0.

Argument: $1 Variable to store the result to.

L_raise

Send signal to itself.

Argument: $@ Kill arguments. See kill --help.

L_kill_all_jobs

L_wait_all_jobs

L_get_all_childs

Get all pids of all child processes including grandchildren recursively.

Option: -v <var>

Argument: [$1] Pid of the parent. Default: $BASHPID.

See: https://stackoverflow.com/a/52544126/9072753

L_get_all_childs_v

L_kill_all_childs

Kills all childs of the pid.

Arguments:

  • -sigspec Signal to use.
  • [$1] Pid of the process to kill all childs of. Defualt: $BASHPID

L_is_fd_open

Check if file descriptor is open.

Argument: $1 file descriptor

Shellcheck disable= SC2188

L_get_free_fd_to

Get free file descriptors

Argument: $@ variables to assign with the file descriptor numbers

L_pipe

Open two connected file descriptors.

This internally creates a temporary file with mkfifo The result variable is assigned an array that: - [0] element is the input to the pipe, - [1] element is the output from the pipe. This is meant to mimic the pipe() C function.

Example

L_pipe tmp
L_array_extract tmp in out
echo 123 >&"$in"
exec "$in">&-
cat <&"$out"
exec "$out"<&-

Arguments:

  • <var> variable name to assign result to
  • [str] template temporary filename, default: ${TMPDIR:/tmp}/L_pipe_XXXXXXXXXX

L_mkstemp

Open three file descriptors read-write connected to a deleted temporary file.

This internally creates a temporary file and immidately removes it. Why three FD? Because it is not possible to rewind the file descriptor in shell, so you get extra spare ones to read from the file.

Arguments:

  • <var> variable name to assign result to
  • [str] template temporary filename, default: ${TMPDIR:/tmp}/L_mkstemp_XXXXXXXXXX

L_proc_popen

Process open. Coproc replacement.

The input/output options are in three groups:

  • -I and -i for stdin,
  • -O and -o for stdout,
  • -E and -e for stderr.

Uppercase letter option specifies the mode for the file descriptor.

There are following modes available that you can give to uppercase options -I -O and -E:

  • null - redirect to or from /dev/null
  • close - close the file descriptor >&-
  • input - -i specifies the string to forward to stdin. Only allowed for -I.
  • stdout - connect file descriptor to stdout. -o or -e value are ignored.
  • stderr - connect file descriptor to stderr. -o or -e value are ignored.
  • pipe - create a fifo and connect file descriptor to it. -i -o or -e option specifies part of the temporary filename.
  • file - connect file descriptor to file specified by -i -o or -e option
  • fd - connect file descriptor to another file descriptor specified by -i -o or -e option

There first argument specifies an output variable that will be assigned the PID.

Several global array variables hold additional information about the process:

  • _L_PROC_EXIT[PID] - Exitcode or empty if not yet finished.
  • _L_PROC_FD0[PID] - If -Ipipe the file descriptor connected to stdin of the program, otherwise empty.
  • _L_PROC_FD1[PID] - If -Opipe the file descriptor connected to stdout of the program, otherwise empty.
  • _L_PROC_FD2[PID] - If -Epipe the file descriptor connected to stderr of the program, otherwise empty.
  • _L_PROC_CMD[PID] - The %q escaped command that was executed.

You should use getters L_proc_get_* to extract the data from them variable.

Example

L_proc_popen -Ipipe -Opipe -Estdout proc sed 's/w/W/g'
L_proc_printf proc "%s\n" "Hello world"
L_proc_read proc line
L_proc_wait -c -v exitcode proc
echo "$line"
echo "$exitcode"

Options:

  • -I <mode> stdin mode
  • -i <param> string for -Iinput, file for -Ifile, fd for -Ifd
  • -O <mode> stdout mode
  • -o <param> file for -Ifile, fd for -Ifd
  • -E <mode> stderr mode
  • -e <param> file for -Efile, fd for -Efd
  • -n Dryrun mode. Do not execute the generated command. Instead print it to stdout.
  • -W <int> Register with L_finally a return trap on stacklevel that will wait for the popen to finish. Typically -W 0
  • -h Print this help and return 0.

Arguments:

  • $1 variable name to store the result to.
  • $@ command to execute.

L_proc_get_exitcode

Get exitcode of L_proc.

Option: -v <var> Store the output in variable instead of printing it.

Argument: $1 PID from L_proc_popen

L_proc_get_exitcode_v

L_proc_get_pid

Get PID from L_proc_popen of L_proc.

Option: -v <var> Store the output in variable instead of printing it.

Argument: $1 PID from L_proc_popen

L_proc_get_pid_v

L_proc_get_stdin

Get file descriptor for stdin of L_proc.

Option: -v <var> Store the output in variable instead of printing it.

Argument: $1 PID from L_proc_popen

L_proc_get_stdin_v

L_proc_get_stdout

Get file descriptor for stdout of L_proc.

Option: -v <var> Store the output in variable instead of printing it.

Argument: $1 PID from L_proc_popen

L_proc_get_stdout_v

L_proc_get_stderr

Get file descriptor for stderr of L_proc.

Option: -v <var> Store the output in variable instead of printing it.

Argument: $1 PID from L_proc_popen

L_proc_get_stderr_v

L_proc_get_cmd

Get command of L_proc.

Option: -v <var> Store the output in the array variable instead of printing it.

Argument: $1 PID from L_proc_popen

L_proc_get_cmd_v

L_proc_free

L_proc_popen_finally

Handler that can be closed from L_finally or a signal handler.

Closes file descriptors. If L_SIGNAL is a signal, forwards it to the process. Then wait for the process termination.

Example

L_finally proc -W sleep infinity

Example

L_finally proc sleep infinity
L_finally L_proc_popen_finally "$proc"

# or
L_finally proc sleep infinity
L_finally L_eval 'L_proc_popen_finally "${!1}"' proc

Argument: $1

L_proc value. This is because the variable may go out of scope.

This is bad. when closing a file descriptor, we "remember" that the file descriptor was closed in the variable. This is less then ideal, but I do not know at this time how to fix it better. Potentially, this can cause unrelated file descriptors to get closed. This might change in the future.

L_proc_printf

Write printf formatted string to coproc.

Arguments:

  • $1 PID from L_proc_popen
  • $@ any printf arguments

L_proc_read

Exec read bultin with -u file descriptor of stdout of coproc.

Arguments:

  • $1 PID from L_proc_popen
  • $@ any builtin read options

L_proc_read_stderr

Exec read bultin with -u file descriptor of stderr of coproc.

Arguments:

  • $1 PID from L_proc_popen
  • $@ any builtin read options

See: L_proc_read

L_proc_close

Close stdin, stdout and stderr of L_proc

Argument: $1 PID from L_proc_popen

L_proc_close_stdin

Close stdin of L_proc.

Does nothing if already closed or not started with -Opipe.

Argument: $1 PID from L_proc_popen

L_proc_close_stdout

Close stdout of L_proc.

Does nothing if already closed or not started with -Opipe

Argument: $1 PID from L_proc_popen

L_proc_close_stderr

Close stderr of L_proc.

Does nothing if already closed or not started with -Epipe.

Argument: $1 PID from L_proc_popen

L_proc_poll

Check if L_proc is finished.

Argument: $1 PID from L_proc_popen

Exit: 0 if L_proc is running, 1 otherwise

L_wait

Wait for pids to be finished with a timeout and capture exit codes.

Tries to use waitpid or tail --pid or a busy loop for best performance.

The waiting is uninterruptible by signals.

Note: builtin kill with multiple pids has different exit code depending on posix mode.

Options:

  • -t <timeout> Wait for this long. Timeout 0 results in just collecting all pids.
  • -v <var> Exit code of PIDs will be assigned to . The elements of -v and -p arrays are pairs.
  • -p <var> PIDs that exited will be assigned to array variable .
  • -l <var> Left running PIDs will be assigned to array variable .
  • -P <polltime> The time to poll processes when not possible to use waitpid or tail. Default: 0.1
  • -n Return 0 when at least one of the pids is finished.
  • -b Bash only. Do not use waitpid or tail.
  • -h Print this help and exit.

Argument: $@ pids to wait on

Return:

0 on success

2 usage error 124 timeout

L_proc_wait

Wait for L_proc to finish.

If L_proc has already finished execution, will only evaluate -v option.

Options:

  • -t <int> Timeout in seconds. Will try to use waitpid, tail --pid or busy loop with sleep.
  • -v <var> Store the L_proc exit code in the variable.
  • -c Close L_proc file descriptors before waiting.
  • -h Print this help and return 0.

Argument: $1 PID from L_proc_popen

Exit:

0 if L_proc has finished

124 if timeout expired

L_read_fds

Read from multiple file descriptors at the same time.

Note

The minimum read -t argument in Bash3.2 is 1 second. It is not possible

to set it lower or to a fraction. This does not work great for Bash3.2 for short timeout, as one read takes at least 1 second to execute.

Example

exec 10< <(for ((i=0;i<5;++i)); do echo $i; sleep 1; done)
exec 11< <(for ((i=0;i<5;++i)); do echo $i; sleep 2; done)
L_read_fds 10 a 11 b
echo "read from 10 fd text: $a"
echo "read from 11 fd text: $b"

Options:

  • -t <timeout> Timeout in seconds.
  • -p <timeout> Poll timeout. The read -t argument value. Default: 0.05 or 1 in Bash3.2
  • -h Print this help and return 0.
  • -v <var> After first fd EOF or error, assign the fd number to and return 0.
  • -i <var> After first fd EOF or error, assign the argument fd number to and return 0.
  • -d <delim> Read -d argument. Default: ''
  • -n Run the reading loop possible once.

Arguments:

  • $1 File descriptor to read from.
  • $2 Variable to assign the output of $1.
  • $@

    Continued pairs of file descriptor and variable names.

    I do not like this. This should just read from two arrays and return array index.

Return:

0 on success

124 on timeout

L_proc_communicate

Communicate with L_proc.

Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.

Options:

  • -i <str>

    Send string to stdin.

    Note that if you want to send data to the process’s stdin, you need to create the L_proc object with -I PIPE.

  • -o <var>

    Append stdout data to this variable. Variable is not cleared.

    To get anything, you need to create L_proc object with -O PIPE.

  • -e <var>

    Append stderr to this variable. Variable is not cleared.

    To get anything, you need to create L_proc object with -E PIPE.

  • -t <int> Timeout in seconds.
  • -k Kill L_proc after communication.
  • -v <var> Store the output in variable instead of printing it.
  • -h Print this help and return 0.

Argument: $1 PID from L_proc_popen

Exit: 0 on success. 2 on usage error. 124 on timeout.

L_proc_send_signal

Send signal to L_proc.

Arguments:

  • $1 PID from L_proc_popen
  • $2 signal to send

L_proc_terminate

Terminate L_proc.

Argument: $1 PID from L_proc_popen

L_proc_kill

Kill L_proc.

Argument: $1 PID from L_proc_popen