Xargs¶

L_xargs is a high-performance, pure-Bash implementation of the xargs utility, designed for seamless integration with local shell environments.

User Guide¶

Unlike the standard GNU xargs which is a compiled binary, L_xargs runs within the current shell context. This allows it to directly execute Bash functions, use aliases, and access shell variables without needing to export them. It is a powerful tool for building complex data-processing pipelines directly in Bash.

The Processing Pipeline: Records and Atoms¶

L_xargs operates on two levels of input units:

Records: These are the primary chunks of input, separated by a delimiter. By default, the delimiter is a newline character (\n), so each line of input is one record. You can change this with the -d (delimiter) or -0 (null character) options.
Atoms: These are the final arguments that are passed to the command being executed. By default, L_xargs splits each Record into Atoms using shell-like quoting rules (split mode). Use -Z to treat each Record as a single, solid Atom.

The command is executed when either the number of accumulated Atoms reaches the limit set by -n, or the number of Records reaches the limit set by -L.

Key Features and Differences from GNU xargs¶

Shell Integration: The most significant advantage. L_xargs can call shell functions and aliases directly, which is impossible with standard xargs without using export -f.
Advanced Input Sources: L_xargs can read items from a Bash array (-A <array_name>) or a custom callback function (-C <callback_eval_string>), in addition to stdin.
Performance: While highly optimized for shell environments, L_xargs is a pure Bash implementation and will generally be slower than the native C-based GNU xargs. For most scripting tasks, its flexibility and integration are more valuable.
Granular Return Codes: Provides specific return codes (123, 124, 125, etc.) to indicate different failure modes, allowing for more robust error handling.

Basic Usage¶

# Reads newline-separated items from stdin and passes them as arguments to echo
printf "item1\nitem2\nitem3" | L_xargs
# Output: item1 item2 item3

Default command: echo. Use an explicit command like L_quote_printf if you want shell-quoted output.

Options and Examples¶

Executing a Shell Function¶

This is a primary use case for L_xargs. The function does not need to be exported. Because L_xargs operates within the same shell, the function can also access any variables or other functions from your script.

By default, L_xargs executes commands in a subshell (forked process). To run the function in the current shell execution environment (so modifications to variables persist), use the -F (foreground) option.

#!/usr/bin/env bash
. L_lib.sh -s

my_prefix="Item"
counter=0

# This function can access and modify variables from the script
process_item() {
  echo "Processing $my_prefix: $1"
  (( counter++ ))
}

# Use -F to run in current shell so counter persists
L_xargs -F -n 1 process_item <<<$'A\nB\nC'

echo "Total items processed: $counter"
# Output:
# Processing Item: A
# Processing Item: B
# Processing Item: C
# Total items processed: 3

Input from an Array (-A)¶

Use the -A option to read input directly from a Bash array.

my_items=("First item" "Second item" "Third item")
# -Z ensures each element is a single argument
L_xargs -Z -A my_items -n 1 echo
# Output:
# First item
# Second item
# Third item

Input from a Callback Function (-C)¶

The -C option allows you to provide a string that will be evaled to generate input Records. The evaluated string must populate the L_RET variable (as an array) and return 0 for success. A non-zero return code signals the end of input.

i=0
generate_items() {
    if (( i < 3 )); then
        L_RET="item_$((++i))"
        return 0
    fi
    return 1
}

L_xargs -n 1 -C 'generate_items' echo
# Output:
# item_1
# item_2
# item_3

Delimiter and Record Handling (-d, -0)¶

By default, L_xargs uses a newline to separate records. -d changes the delimiter. -0 is a shorthand for -d '', using the null character, which is useful for working with find -print0.

By default, L_xargs splits each record into atoms using shell-like quoting rules (split mode). Use -Z to treat each record as a single, solid atom.

# Default behavior (Split mode)
printf "A B\nC" | L_xargs -n 1 echo
# Output:
# A
# B
# C

# Solid mode
printf "A B\nC" | L_xargs -Z -n 1 echo
# Output:
# A B
# C

Parallel Execution (-P)¶

Use -P to run commands in parallel. -P nproc is a convenient shortcut to use all available CPU cores.

# Run up to 4 sleep commands in parallel
printf "1\n2\n3\n4" | L_xargs -P 4 -n 1 sleep

Ordered Parallel Output (-O)¶

When running in parallel with -P, output from different commands can be interleaved. The -O option ensures that the output from each command is buffered and printed atomically once the command completes. This prevents interleaving but may result in output order not matching the input order.

# Without -O, output can be mixed.
printf "A\nB" | L_xargs -P 2 -n 1 -- bash -c 'echo "start $1"; sleep 0.1; echo "end $1"' --
# Output:
# start A
# start B
# end B
# end A
# With -O, each command's output is grouped.
printf "A\nB" | L_xargs -O -P 2 -n 1 -- bash -c 'echo "start $1"; sleep 0.1; echo "end $1"' --
# Output:
# start A
# end A
# start B
# end B

Controlling Command Execution (-n, -L)¶

-n (max-atoms) and -L (max-records) control how many items are processed before the command is executed. The command is triggered as soon as either limit is reached.

-n 2: Executes the command for every 2 atoms collected.
-L 2: Executes the command for every 2 records read.
-n 2 -L 3: If 2 atoms are collected before 3 records are read, the command runs. If 3 records are read before 2 atoms are collected, the command runs.

Example:

# -z splits "A B" into two atoms. The -n 2 limit is hit after the first line.
# The command runs, and the limits are reset. Then "C" is processed.
printf "A B\nC" | L_xargs -z -n 2 -L 3 echo
# Output:
# A B
# C

Prefixing Output (-^)¶

The -^ option prepends the arguments used for the command, followed by a colon, to each line of the command's output.

printf "A\nB" | L_xargs -n 1 -^ -- L_eval 'echo "Line 1 of $1"; echo "Line 2 of $1"'
# Output:
# A: Line 1 of A
# A: Line 2 of A
# B: Line 1 of B
# B: Line 2 of B

API Reference¶

xargs ¶

L_nproc ¶

Returns the number of CPU cores.

Options:

-v <var>
-h

L_nproc_vL_RET ¶

L_sleep ¶

Pause for a specified duration using the best available sleep method.

Argument: $1 Duration in floating point seconds.

L_xargs ¶

Bash implementation of the xargs utility designed for seamless

integration with local shell environments. Unlike binary xargs, L_xargs executes within the current shell context, enabling the direct use of unexported Bash functions, aliases, and variables without requiring export or export -f.

The tool operates on a dual-unit architecture: 1. Records: Discrete segments of input defined by a delimiter (default: \n). 2. Atoms: The individual arguments passed to the command.

By default, L_xargs operates in -s -0 mode. If -d -0 -a options are specified without -z -Z, -Z is implied.

Execution follows a first-to-threshold trigger system: the command is dispatched as soon as either the Atom limit (-n) or the Record limit (-L) is reached. If no limits are specified, the command executes exactly once upon reaching EOF.

Options:

-0 Use the null character (\0) as the Record separator.
-a <file> Read Records from the specified file.
-A <var> Read Records from the specified Bash array variable.
-C <callback> Execute an eval string to fetch the next Record. Must populate L_RET=() and return 0.
-d <delimiter> Set the Record separator to the specified character.
-s <max-chars> Use at most max-chars characters per command line.
-m <task-max-time> If a task is running longer then specified time, it is killed.
-M <global-max-time> If xargs is runnig longer then specified time, tasks are getting killed and xargs returns.
-z Split Mode: Parse internal Records into multiple Atoms using L_unquote.
-Z Solid Mode: Treat the entire delimited Record as a single literal Atom (Default).
-u <fd> Read the input stream from the specified file descriptor.
-I <replace-str> Replace occurrences of replace-str in the command. Sets -n 1.
-i Shorthand for -I{}.
-L <max-records> Trigger execution once have been accumulated.
-l Shorthand for -L1.
-n <max-atoms> Trigger execution once have been accumulated.
-r If the input does not contain any atoms, do not run the command. Normally, the command is run once even if there is no input.
-P <max-procs> Concurrent process limit. Supports an integer or 'nproc' for CPU count.
-O Separate output of each command by using pipes. Use twice to keep the output of pipes in order.
-t Verbose: Print each command to STDERR before execution.
-^ Prefix Mode: Prepends the command arguments and a colon to each line of output.
-q Be quiet.
-v <var> Assign array variable the exit statuses of commands. Do not exit with 123-127 exit codes.
-E <eof-str> Set the end of file string to eof-str. If the end of file string occurs as a line of input, the rest of the input is ignored.
-e <eof-str> Like -E, compatibility wtih GNU xargs, use -E.
-F Run the command in current shell execution context. Do not fork.
-h Display this help documentation and exit.

Argument: $@ Command to execute. Default: L_quote_printf.

Uses environment variable: L_XARGS_INDEX The index of the job being executed.

Return:

0 on success

1 on some other error 64 ($L_EX_USAGE) on invalid usage 123 if any invocation of the command exited with status 1-125 and 192-254 124 if the command exited with status 255 125 if the command exited with the status 128-192 126 if the command cannot be run 127 if the command is not found