unix-tools
ls, find, sh, nc, make and a dozen more classic Unix tools, rewritten from scratch in Ridgun
| 1 | # unix-tools |
| 2 | |
| 3 | A collection of **17 classic BSD and Unix userland utilities** written in 100% pure **Ridgun** (`v0.9.17` PAL) and compiled directly to standalone native Linux `x86_64` ELF64 executables with zero C, Rust, or `libc` runtime dependencies. |
| 4 | |
| 5 | All utilities operate over POSIX file descriptors and system calls provided by `~/ridgun/libs` (`posix.rglib`, `io.rglib`, `crypto.rglib`, and `socket.rglib`). |
| 6 | |
| 7 | --- |
| 8 | |
| 9 | ## Included Utilities (17 Tools) |
| 10 | |
| 11 | | Domain | Utility | Description | Source | |
| 12 | | :--- | :--- | :--- | :--- | |
| 13 | | **FILTER** | `jot(1)` | Sequential, repetitive, character, and pseudo-random data generator | `src/main/ridgun/jot/main.rg` | |
| 14 | | **FILTER** | `rs(1)` | Array and stream reshaping, transposition, and column formatting filter | `src/main/ridgun/rs/main.rg` | |
| 15 | | **FILTER** | `lam(1)` | Multi-file line lamination tool with per-stream padding and separators | `src/main/ridgun/lam/main.rg` | |
| 16 | | **FILTER** | `vis(1)` | Lossless visual encoding of non-printable and binary characters | `src/main/ridgun/vis/main.rg` | |
| 17 | | **FILTER** | `unvis(1)` | Lossless decoder restoring `vis(1)`-encoded streams to raw bytes | `src/main/ridgun/unvis/main.rg` | |
| 18 | | **FILTER** | `mdrr(1)` | Terminal Markdown formatter and reader with GFM tables, alerts, and TOC filter | `src/main/ridgun/mdrr/main.rg` | |
| 19 | | **FS** | `ls(1)` | Directory contents listing with classification, sorting, and ANSI color | `src/main/ridgun/ls/main.rg` | |
| 20 | | **FS** | `find(1)` | Recursive directory hierarchy walker and boolean expression evaluator | `src/main/ridgun/find/main.rg` | |
| 21 | | **FS** | `mtree(8)` | Directory hierarchy specification generator, SHA-256 auditor, and healer | `src/main/ridgun/mtree/main.rg` | |
| 22 | | **FS** | `mktemp(1)` | Atomic temporary file (`0600`) and directory (`0700`) generator | `src/main/ridgun/mktemp/main.rg` | |
| 23 | | **PROC** | `apply(1)` | Fixed-arity argument batching and command runner | `src/main/ridgun/apply/main.rg` | |
| 24 | | **PROC** | `make(1)` | Directed Acyclic Graph (DAG) build tool with topological sort and cycle checks | `src/main/ridgun/make/main.rg` | |
| 25 | | **PROC** | `sh(1)` | POSIX/BSD command interpreter (pipelines, redirections, control flow, built-ins) | `src/main/ridgun/sh/main.rg` | |
| 26 | | **PROC** | `timeout(1)` | Time-bounded command execution with signal escalation | `src/main/ridgun/timeout/main.rg` | |
| 27 | | **PROC** | `nc(1)` | TCP, UDP, and Unix-domain socket Swiss army knife and port scanner | `src/main/ridgun/nc/main.rg` | |
| 28 | | **SEC** | `doas(1)` | OpenBSD rule-based privilege escalation utility and `doas.conf(5)` auditor | `src/main/ridgun/doas/main.rg` | |
| 29 | | **SEC** | `sha256(1)` | BSD SHA-256 digest, HMAC-SHA256 (`-k`), and Argon2id (`--argon2id`) utility | `src/main/ridgun/sha256/main.rg` | |
| 30 | |
| 31 | --- |
| 32 | |
| 33 | ## Detailed Utility Reference |
| 34 | |
| 35 | ### Tier 1 — BSD Text, Table & Markdown Filters (`FILTER`) |
| 36 | |
| 37 | #### 1. `jot(1)` — Sequential & Random Data Generator |
| 38 | Generates sequences of numbers, repeated strings, ASCII characters, or deterministic pseudo-random numbers. |
| 39 | - **Flags**: |
| 40 | - `-r`: Generate pseudo-random values (`-S seed` for deterministic seeding). |
| 41 | - `-b word`: Repeat `word` verbatim. |
| 42 | - `-w word`: Format output using `printf`-style specifiers (`%d`, `%i`, `%04d`, `%-4d`, `%x`, `%X`, `%o`, `%c`, `%.2f`, `%%`). |
| 43 | - `-s string`: Join items with `string` instead of newline. |
| 44 | - `-n`: Omit the final trailing newline. |
| 45 | - `-p precision`: Set fixed-point decimal precision. |
| 46 | - `-c`: Character output mode. |
| 47 | - **Examples**: |
| 48 | ```sh |
| 49 | ./target/jot 5 10 50 # 10, 20, 30, 40, 50 |
| 50 | ./target/jot -n -s , 4 1 4 # 1,2,3,4 |
| 51 | ./target/jot -w '0x%02x' 4 0 3 # 0x00, 0x01, 0x02, 0x03 |
| 52 | ``` |
| 53 | |
| 54 | #### 2. `rs(1)` — Reshape Data Array |
| 55 | Reads whitespace- or delimiter-separated items from files or descriptor `0` (`stdin`) and arranges them into a 2D matrix. |
| 56 | - **Flags**: |
| 57 | - `[rows [cols]]`: Target dimensions (`0` auto-infers from input length). |
| 58 | - `-t`: Transpose rows and columns. |
| 59 | - `-c<ch>` / `-C<ch>`: Custom input / output column delimiter. |
| 60 | - `-g<n>`: Set inter-column gutter width. |
| 61 | - `-j`: Right-align columns within their computed widths. |
| 62 | - `-w<width>`: Auto-calculate column count to fit display width. |
| 63 | - `-e`: Treat each input line as a single entry. |
| 64 | - `-k<n>`: Ignore the first `n` lines of input. |
| 65 | - `-y`: Recycle elements when the last row is incomplete. |
| 66 | - `-h`: Print matrix dimensions (`rows by cols`) only. |
| 67 | - `-z`: Squeeze column widths to exact item lengths. |
| 68 | - **Example**: |
| 69 | ```sh |
| 70 | ./target/jot 6 1 6 | ./target/rs -t 2 3 |
| 71 | # 1 3 5 |
| 72 | # 2 4 6 |
| 73 | ``` |
| 74 | |
| 75 | #### 3. `lam(1)` — Laminate Multiple Files Side-by-Side |
| 76 | Concatenates corresponding lines from multiple input files or `-` (`stdin`) simultaneously, supporting both per-file lowercase flags and persistent uppercase flags. |
| 77 | - **Flags**: |
| 78 | - `-s sep` / `-S sep`: Print separator string before the next file column (reset per file vs. persistent). |
| 79 | - `-f min.max` / `-F min.max`: Pad or truncate column width (`-min.max` left-aligns; `min.max` right-aligns). |
| 80 | - `-p min.max` / `-P min.max`: Continue padding when an input file reaches EOF early. |
| 81 | - `-t char`: Use `char` instead of newline as the input line terminator. |
| 82 | - **Example**: |
| 83 | ```sh |
| 84 | ./target/lam users.txt -s ' -> ' roles.txt |
| 85 | ``` |
| 86 | |
| 87 | #### 4. `vis(1)` & `unvis(1)` — Visual Byte Encoding & Lossless Decoding |
| 88 | Encodes non-printable, control, whitespace, and arbitrary binary bytes (`0x00..0xFF`) into unambiguous printable ASCII sequences (`\^A`, `\M-x`, `\040`, `\n`, `\t`, `\\`), and restores them byte-for-byte with `unvis(1)`. |
| 89 | - **Flags (`vis`)**: |
| 90 | - `-c`: Backslash-escape C-style control sequences (`\n`, `\t`, `\r`, `\b`, `\a`, `\v`, `\f`). |
| 91 | - `-o`: Encode all non-printable bytes in 3-digit octal (`\ddd`). |
| 92 | - `-w`: Encode spaces, tabs, and newlines visually. |
| 93 | - `-t`: Encode tabs and newlines visually. |
| 94 | - `-l`: Mark newlines with `\$` before the line break. |
| 95 | - `-F width`: Fold output lines at `width` columns using soft `\` line continuations. |
| 96 | - `-n`: Encode only backslashes. |
| 97 | - `-b`: Suppress doubling of backslashes. |
| 98 | - **Example**: |
| 99 | ```sh |
| 100 | ./target/vis -tw binary.dat | ./target/unvis > restored.dat |
| 101 | ``` |
| 102 | |
| 103 | #### 5. `mdrr(1)` — Terminal Markdown Formatter & Reader |
| 104 | Parses and renders Markdown documents for terminal viewing with ANSI colors, Unicode box-drawing tables, GitHub alerts (`[!NOTE]`, `[!TIP]`, `[!IMPORTANT]`, `[!WARNING]`, `[!CAUTION]`), task lists (`☐`/`☑`), and fenced code blocks. |
| 105 | - **Flags**: |
| 106 | - `-w width`: Wrap paragraphs and blocks to `width` visible columns (ignoring zero-width ANSI SGR codes). |
| 107 | - `-p`: Plain mode (strip ANSI color and style sequences). |
| 108 | - `-a`: ASCII-only borders and checkboxes (`+---+`, `[ ]`, `[x]`). |
| 109 | - `-t`: Extract and print an indented Table of Contents from document headings. |
| 110 | - `-s heading`: Filter and render only the section matching `heading` (and its nested subsections). |
| 111 | - `-n`: Display line numbers inside fenced code blocks. |
| 112 | - `-H`: Hide YAML front-matter blocks. |
| 113 | - **Example**: |
| 114 | ```sh |
| 115 | ./target/mdrr -n README.md |
| 116 | ./target/mdrr -p -a -t README.md |
| 117 | ``` |
| 118 | |
| 119 | --- |
| 120 | |
| 121 | ### Tier 2 — BSD Filesystem & Hierarchy Tools (`FS`) |
| 122 | |
| 123 | #### 6. `ls(1)` — List Directory Contents |
| 124 | Lists files and directories with BSD mode formatting (`drwxr-xr-x`, sticky `t`/`T`, setuid/setgid `s`/`S`), type classification, and sorting. |
| 125 | - **Flags**: |
| 126 | - `-l`: Long listing format (permissions, link count, owner, group, size, timestamp, symlink target). |
| 127 | - `-a` / `-A`: Include dot-entries (`.` and `..` included with `-a`, excluded with `-A`). |
| 128 | - `-F`: Append file type indicator (`/` dir, `*` executable, `@` symlink, `|` FIFO, `=` socket). |
| 129 | - `-G`: Enable ANSI colorized output by file type. |
| 130 | - `-h`: Format sizes with human-readable binary suffixes (`B`, `K`, `M`, `G`). |
| 131 | - `-R`: Recursively list subdirectories. |
| 132 | - `-S` / `-t` / `-r`: Sort by descending size (`-S`), modification time (`-t`), or reverse order (`-r`). |
| 133 | - `-1` / `-d`: Force single-column output (`-1`) or list directory entries themselves (`-d`). |
| 134 | |
| 135 | #### 7. `find(1)` — Walk a File Hierarchy |
| 136 | Recursively traverses one or more root paths and evaluates compound boolean expressions on every entry. |
| 137 | - **Primaries & Operators**: |
| 138 | - `-name pattern` / `-iname pattern` / `-path pattern`: Glob matching (`*`, `?`, `[a-z]`, `[!a-z]`). |
| 139 | - `-type [f|d|l|p|s|b|c]`: Match file type. |
| 140 | - `-size [+|-]N[c|k|M|G]`: Match file size (greater than, less than, or exact). |
| 141 | - `-empty`: Match 0-byte regular files or empty directories. |
| 142 | - `-maxdepth d` / `-mindepth d`: Constrain traversal depth. |
| 143 | - `! expr`, `expr -a expr`, `expr -o expr`, `( expr )`: Full boolean precedence. |
| 144 | - `-print0`: Delimit results with NUL (`\0`) bytes. |
| 145 | - `-exec prog args ... {} ;`: Execute a command per matched entry. |
| 146 | - **Example**: |
| 147 | ```sh |
| 148 | ./target/find src -type f -name '*.rg' ! -empty |
| 149 | ``` |
| 150 | |
| 151 | #### 8. `mtree(8)` — Map & Audit Directory Hierarchies |
| 152 | Generates declarative hierarchy specifications (`type`, `mode`, `size`, `sha256`, `link`) and audits live directory trees against saved specifications. |
| 153 | - **Flags**: |
| 154 | - `-c`: Generate a specification from the target root directory. |
| 155 | - `-f specfile`: Verify the live tree against `specfile` (exits `0` if clean, `2` on mismatch). |
| 156 | - `-p path`: Root path to generate or verify (default `.`). |
| 157 | - `-u`: Heal missing directories and restore mismatched octal file permissions automatically. |
| 158 | - `-K keywords`: Add comma-separated keywords to the specification. |
| 159 | - **Example**: |
| 160 | ```sh |
| 161 | ./target/mtree -c -p src > src.mtree |
| 162 | ./target/mtree -f src.mtree -p src |
| 163 | ``` |
| 164 | |
| 165 | #### 9. `mktemp(1)` — Make Temporary File or Directory |
| 166 | Safely creates unique temporary files (`0600`) or directories (`0700`) using `O_CREAT | O_EXCL` and cryptographic randomness. |
| 167 | - **Flags**: |
| 168 | - `-d`: Create a directory instead of a regular file. |
| 169 | - `-u`: Dry-run ("unsafe") mode — print the generated unique path without creating it on disk. |
| 170 | - `-q`: Suppress diagnostic output on failure. |
| 171 | - `-p dir`: Base directory for `-t` templates (defaults to `/tmp`). |
| 172 | - `-t prefix`: Construct template `<dir>/<prefix>.XXXXXXXX`. |
| 173 | - **Example**: |
| 174 | ```sh |
| 175 | TMP_DIR=$(./target/mktemp -d /tmp/build.XXXXXX) |
| 176 | ``` |
| 177 | |
| 178 | --- |
| 179 | |
| 180 | ### Tier 3 — BSD Process, Build, Shell & Network Tools (`PROC`) |
| 181 | |
| 182 | #### 10. `apply(1)` — Apply a Command to Groups of Arguments |
| 183 | Partitions a flat argument list into fixed-size tuples and executes a command template for each group. |
| 184 | - **Flags**: |
| 185 | - `-N`: Consume `N` arguments per invocation (`-0` runs the command once per argument without appending it). |
| 186 | - `%1..%9`: Positional placeholders inside the command string (auto-infers `N` from the highest `%k`). |
| 187 | - `-a char`: Change the placeholder magic character from `%` to `char`. |
| 188 | - `-d`: Dry-run mode — print constructed commands without executing them. |
| 189 | - **Example**: |
| 190 | ```sh |
| 191 | ./target/apply -2 'cmp -s %1 %2' a.old a.new b.old b.new |
| 192 | ``` |
| 193 | |
| 194 | #### 11. `make(1)` — Maintain Program Dependencies |
| 195 | Parses a `Makefile`, resolves macro definitions (`=`, `:=`, `?=`) and CLI overrides (`VAR=val`), checks topological DAG dependencies, detects dependency cycles, and runs target recipes. |
| 196 | - **Flags**: |
| 197 | - `-f makefile`: Read rules from `makefile`. |
| 198 | - `-n`: Dry-run — print recipe commands in topological order without executing them. |
| 199 | - `-B`: Unconditionally build all targets. |
| 200 | - `-j jobs`: Specify concurrency level. |
| 201 | |
| 202 | #### 12. `sh(1)` — Command Interpreter (Shell) |
| 203 | Executes POSIX/BSD shell scripts or `-c` command strings. |
| 204 | - **Supported Grammar & Features**: |
| 205 | - Pipelines (`cmd1 | cmd2`), sequential lists (`;`), and short-circuit boolean lists (`&&`, `||`). |
| 206 | - Redirections (`>`, `>>`, `<`, `2>&1`). |
| 207 | - Control structures: `if ...; then ...; elif ...; then ...; else ...; fi`, `for var in ...; do ...; done`, `while ...; do ...; done`, and isolated subshells `( ... )`. |
| 208 | - Quoting & Expansion: single quotes `'...'`, double quotes `"..."`, `$VAR`, `${VAR}`, `${VAR:-default}`, `$#`, `$?`, `$1..$9`. |
| 209 | - Built-ins: `cd`, `export`, `unset`, `set`, `echo`, `exit`, and `test` / `[ ... ]` (`=`, `!=`, `-z`, `-n`, `-eq`, `-ne`, `-lt`, `-le`, `-gt`, `-ge`, `-f`, `-d`, `-e`). |
| 210 | |
| 211 | #### 13. `timeout(1)` — Run a Command with a Time Limit |
| 212 | Executes a utility under a strict deadline and terminates it if the time limit expires. |
| 213 | - **Flags**: |
| 214 | - `DURATION`: Supports `ms`, `s`, `m`, `h`, and `d` suffixes (e.g., `250ms`, `5s`, `1m`). |
| 215 | - `-s SIGNAL`: Signal to send on timeout (default `TERM`). |
| 216 | - `-k DURATION`: Send `KILL` if the process is still running `DURATION` after the initial signal. |
| 217 | - `--preserve-status`: Exit with the child's signal status instead of `124` on timeout. |
| 218 | - **Example**: |
| 219 | ```sh |
| 220 | ./target/timeout -k 100ms 2s ./target/sh -c 'echo ok' |
| 221 | ``` |
| 222 | |
| 223 | #### 14. `nc(1)` — Arbitrary TCP, UDP & Unix Socket Connections |
| 224 | Establishes outbound connections, listens for inbound connections, transfers data streams, and probes open ports via `socket.rglib`. |
| 225 | - **Flags**: |
| 226 | - `-l`: Listen for an incoming connection on the specified port or Unix socket path. |
| 227 | - `-z`: Zero-I/O port scanning mode (check connectivity and close immediately). |
| 228 | - `-v`: Verbose connection diagnostics. |
| 229 | - `-w sec`: Connection timeout in seconds. |
| 230 | - `-u`: Use UDP sockets instead of TCP. |
| 231 | - `-U`: Use Unix-domain stream sockets (`AF_UNIX`). |
| 232 | - **Example**: |
| 233 | ```sh |
| 234 | ./target/nc -zv -w 1 127.0.0.1 8080 |
| 235 | ``` |
| 236 | |
| 237 | --- |
| 238 | |
| 239 | ### Tier 4 — BSD Security, Privilege & Cryptography Tools (`SEC`) |
| 240 | |
| 241 | #### 15. `doas(1)` — Execute Commands as Another User |
| 242 | Implements the OpenBSD `doas.conf(5)` parser, "last matching rule wins" policy engine, environment sanitization (`keepenv`, `setenv`), and configuration auditing. |
| 243 | - **Flags**: |
| 244 | - `-C config [cmd args...]`: Parse and validate `config`; if a command is supplied, print the policy verdict (`permit`, `permit nopass`, or `deny`) without executing the command. |
| 245 | - `-u user`: Target user account (default `root`). |
| 246 | - `-n`: Non-interactive mode — fail immediately if a rule requires a password. |
| 247 | - `-s`: Execute the user's shell. |
| 248 | - **Example**: |
| 249 | ```sh |
| 250 | ./target/doas -C /etc/doas.conf -u root /sbin/reboot |
| 251 | ``` |
| 252 | |
| 253 | #### 16. `sha256(1)` — Cryptographic SHA-256, HMAC-SHA256 & Argon2id |
| 254 | Computes FIPS 180-4 SHA-256 digests, RFC 2104 HMAC-SHA256 authentication tags, and RFC 9106 Argon2id password hashes via `crypto.rglib`. |
| 255 | - **Flags**: |
| 256 | - `-s string`: Digest the literal `string`. |
| 257 | - `-q`: Quiet mode — output only the raw hex digest or PHC string. |
| 258 | - `-r`: Reverse output format (`<digest> <filename>`). |
| 259 | - `-k key`: Compute `HMAC-SHA256` using `key`. |
| 260 | - `--argon2id salt`: Derive an Argon2id PHC string (`$argon2id$v=19$m=64,t=2,p=1$...`). |
| 261 | - `--verify phc`: Verify input password against an Argon2id `phc` string in constant time (exits `0` on match, `1` on mismatch). |
| 262 | - **Examples**: |
| 263 | ```sh |
| 264 | ./target/sha256 -s "abc" |
| 265 | # SHA256 ("abc") = ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad |
| 266 | |
| 267 | ./target/sha256 -q -k "secret" -s "message" |
| 268 | # HMAC-SHA256 hex digest |
| 269 | |
| 270 | PHC=$(./target/sha256 -q --argon2id "saltsalt" -s "correcthorse") |
| 271 | ./target/sha256 --verify "$PHC" -s "correcthorse" |
| 272 | ``` |
| 273 | |
| 274 | --- |
| 275 | |
| 276 | ## Building & Testing with `stable` |
| 277 | |
| 278 | This project is managed by **`stable`** (`build.rg`), Ridgun's self-hosted build orchestrator, dependency manager, parallel type-checker, canonical tabular formatter, and test runner. |
| 279 | |
| 280 | ### Prerequisites |
| 281 | - `stable` and `ridgunc` (`v0.9.17` PAL) in `PATH`. |
| 282 | - Ridgun standard libraries available at `~/ridgun/libs`. |
| 283 | |
| 284 | ### 1. Inspect Project & Custom Tasks |
| 285 | ```sh |
| 286 | stable info |
| 287 | stable tasks |
| 288 | ``` |
| 289 | |
| 290 | ### 2. Sync `.rglib` Dependencies (`posix`, `io`, `crypto`, `socket`) |
| 291 | Run the custom `sync libs` task defined in `build.rg` to compile and install the `.rglib` bundles into `.stable/deps/` and `target/`: |
| 292 | ```sh |
| 293 | stable sync libs |
| 294 | ``` |
| 295 | |
| 296 | ### 3. Compile All 17 Native ELF64 Binaries (Strict Mode) |
| 297 | Build every discovered utility in `src/main/ridgun/<tool>/main.rg` into `target/` using `ridgunc build --strict` via the custom `strict binaries` task (or use `stable built` / `stable built mktemp`): |
| 298 | ```sh |
| 299 | stable strict binaries |
| 300 | stable built |
| 301 | ``` |
| 302 | |
| 303 | ### 4. Type-Check, Format & Run the Automated Test Suites (338 Assertions) |
| 304 | `stable` executes type-checking, formatting, and all 5 test suites (`test_filter.rg`, `test_md.rg`, `test_fs.rg`, `test_proc.rg`, `test_sec.rg`) concurrently across cores via Ridgun's `||map`: |
| 305 | ```sh |
| 306 | stable check |
| 307 | stable format |
| 308 | stable test |
| 309 | ``` |
| 310 | |
| 311 | ### 5. Install Executables Locally (Optional) |
| 312 | Build and install the ELF64 binaries into `~/.local/bin/` and `./bin/`: |
| 313 | ```sh |
| 314 | stable install |
| 315 | ``` |