Skip to content
Ridgun
English
EnglishEspañolCatalàEuskaraGalegoPortuguêsSvenskaItaliano

Ridgun

The language itself: a compiler that compiles itself, the stable project tool, the specification and the book

README.md278 lines · 20.8 KBRaw
1# Ridgun
2
3**A tabular, pure-by-default functional & systems programming language built for massive parallelism, native isolated agents, symbolic tensor algebra, and deterministic zero-GC memory management.**
4
5Ridgun combines the declarative clarity of tabular functional programming with native systems performance. Its self-hosted compiler (**`ridgunc`**) and project manager/formatter (**`stable`**) are written **100% in pure Ridgun** (`src/main/ridgun/`, zero Rust/C code or external compiler dependencies) and emit standalone, statically linked Linux ELF64 executables (`x86_64` and `ARM64` / Raspberry Pi) directly to disk with zero Garbage Collector pauses.
6
7---
8
9## 📖 The Ridgun Book — Complete Guide in 8 Languages
10
11Learn Ridgun from scratch—from your first tabular function to parallel DAGs, the Native Agent Model (`agent`, `send ... to`, `receive from`), symbolic tensors, and `RMM1`/`RMM2` deterministic memory management—in your preferred language:
12
13| Language | Online Interactive Book (`HTML`) | Markdown Chapters (`00`–`08`) |
14|---|---|---|
15| 🇪🇸 **Español** | [**El Libro de Ridgun (`es/index.html`)**](docs/book/es/index.html) | [`docs/book/es/INDEX.md`](docs/book/es/INDEX.md) — [Cap. 00](docs/book/es/00-por-que-necesito-ridgun.md) · [01](docs/book/es/01-bienvenido-a-ridgun.md) · [02](docs/book/es/02-primeros-pasos-y-herramientas.md) · [03](docs/book/es/03-funciones-tipos-y-alineacion.md) · [04](docs/book/es/04-pattern-matching-y-guardas.md) · [05](docs/book/es/05-listas-tuplas-y-secuencias.md) · [06](docs/book/es/06-tipos-algebraicos-y-polimorfismo.md) · [07](docs/book/es/07-orden-superior-y-paralelismo-dag.md) · [08](docs/book/es/08-tensores-regiones-y-sistemas.md) |
16| 🇬🇧 **English** | [**The Ridgun Book (`en/index.html`)**](docs/book/en/index.html) | [`docs/book/en/INDEX.md`](docs/book/en/INDEX.md) — [Ch. 00](docs/book/en/00-why-do-i-need-ridgun.md) · [01](docs/book/en/01-welcome-to-ridgun.md) · [02](docs/book/en/02-getting-started-and-tooling.md) · [03](docs/book/en/03-functions-types-and-alignment.md) · [04](docs/book/en/04-pattern-matching-and-guards.md) · [05](docs/book/en/05-lists-tuples-and-sequences.md) · [06](docs/book/en/06-algebraic-data-types-and-polymorphism.md) · [07](docs/book/en/07-higher-order-functions-and-dag-parallelism.md) · [08](docs/book/en/08-tensors-rmm-regions-and-systems.md) |
17| 🇮🇹 **Italiano** | [**Il Libro di Ridgun (`it/index.html`)**](docs/book/it/index.html) | [`docs/book/it/INDEX.md`](docs/book/it/INDEX.md) — [Cap. 00](docs/book/it/00-perche-ho-bisogno-di-ridgun.md) · [01](docs/book/it/01-benvenuto-in-ridgun.md) · [02](docs/book/it/02-primi-passi-e-strumenti.md) · [03](docs/book/it/03-funzioni-tipi-e-allineamento.md) · [04](docs/book/it/04-pattern-matching-e-guardie.md) · [05](docs/book/it/05-liste-tuple-e-sequenze.md) · [06](docs/book/it/06-tipi-algebrici-e-polimorfismo.md) · [07](docs/book/it/07-ordine-superiore-e-parallelismo-dag.md) · [08](docs/book/it/08-tensori-rmm-regioni-e-sistemi.md) |
18| 🇵🇹 **Português** | [**O Livro do Ridgun (`pt/index.html`)**](docs/book/pt/index.html) | [`docs/book/pt/INDEX.md`](docs/book/pt/INDEX.md) — [Cap. 00](docs/book/pt/00-por-que-preciso-do-ridgun.md) · [01](docs/book/pt/01-bem-vindo-ao-ridgun.md) · [02](docs/book/pt/02-primeiros-passos-e-ferramentas.md) · [03](docs/book/pt/03-funcoes-tipos-e-alinhamento.md) · [04](docs/book/pt/04-pattern-matching-e-guardas.md) · [05](docs/book/pt/05-listas-tuplas-e-sequencias.md) · [06](docs/book/pt/06-tipos-algebricos-e-polimorfismo.md) · [07](docs/book/pt/07-ordem-superior-e-paralelismo-dag.md) · [08](docs/book/pt/08-tensores-rmm-regioes-e-sistemas.md) |
19| 🇸🇪 **Svenska** | [**Boken om Ridgun (`sv/index.html`)**](docs/book/sv/index.html) | [`docs/book/sv/INDEX.md`](docs/book/sv/INDEX.md) — [Kap. 00](docs/book/sv/00-varfor-behover-jag-ridgun.md) · [01](docs/book/sv/01-valkommen-till-ridgun.md) · [02](docs/book/sv/02-forsta-stegen-och-verktyg.md) · [03](docs/book/sv/03-funktioner-typer-och-justering.md) · [04](docs/book/sv/04-monstermatchning-och-vakter.md) · [05](docs/book/sv/05-listor-tupler-och-sekvenser.md) · [06](docs/book/sv/06-algebraiska-datatyper-och-polymorfism.md) · [07](docs/book/sv/07-hogre-ordningen-och-dag-parallellism.md) · [08](docs/book/sv/08-tensorer-rmm-regioner-och-system.md) |
20| 🏴 **Català** | [**El Llibre de Ridgun (`ca/index.html`)**](docs/book/ca/index.html) | [`docs/book/ca/INDEX.md`](docs/book/ca/INDEX.md) — [Cap. 00](docs/book/ca/00-per-que-necessito-ridgun.md) · [01](docs/book/ca/01-benvingut-a-ridgun.md) · [02](docs/book/ca/02-primers-passos-i-eines.md) · [03](docs/book/ca/03-funcions-tipus-i-alineacio.md) · [04](docs/book/ca/04-pattern-matching-i-guardes.md) · [05](docs/book/ca/05-llistes-tuples-i-sequencies.md) · [06](docs/book/ca/06-tipus-algebraics-i-polimorfisme.md) · [07](docs/book/ca/07-ordre-superior-i-parallelisme-dag.md) · [08](docs/book/ca/08-tensors-rmm-regions-i-sistemes.md) |
21| 🏴 **Galego** | [**O Libro de Ridgun (`gl/index.html`)**](docs/book/gl/index.html) | [`docs/book/gl/INDEX.md`](docs/book/gl/INDEX.md) — [Cap. 00](docs/book/gl/00-por-que-preciso-ridgun.md) · [01](docs/book/gl/01-benvido-a-ridgun.md) · [02](docs/book/gl/02-primeiros-pasos-e-ferramentas.md) · [03](docs/book/gl/03-funcions-tipos-e-alinamento.md) · [04](docs/book/gl/04-pattern-matching-e-gardas.md) · [05](docs/book/gl/05-listas-tuplas-e-secuencias.md) · [06](docs/book/gl/06-tipos-alxebricos-e-polimorfismo.md) · [07](docs/book/gl/07-orde-superior-e-paralelismo-dag.md) · [08](docs/book/gl/08-tensores-rmm-rexions-e-sistemas.md) |
22| 🏴 **Euskara** | [**Ridgun Liburua (`eu/index.html`)**](docs/book/eu/index.html) | [`docs/book/eu/INDEX.md`](docs/book/eu/INDEX.md) — [00. kap.](docs/book/eu/00-zergatik-behar-dut-ridgun.md) · [01](docs/book/eu/01-ongi-etorri-ridgunera.md) · [02](docs/book/eu/02-lehen-urratsak-eta-tresnak.md) · [03](docs/book/eu/03-funtzioak-motak-eta-lerrokatzea.md) · [04](docs/book/eu/04-eredu-bat-etortzea-eta-zainak.md) · [05](docs/book/eu/05-zerrendak-tuplak-eta-sekuentziak.md) · [06](docs/book/eu/06-datu-mota-aljebraikoak-eta-polimorfismoa.md) · [07](docs/book/eu/07-goi-mailako-funtzioak-eta-dag-paralelismoa.md) · [08](docs/book/eu/08-tentsoreak-rmm-eskualdeak-eta-sistemak.md) |
23
24---
25
26## Quick Start
27
28 Ridgun ships with two self-hosted tools—**`ridgunc`** (the single-file compiler & runner) and **`stable`** (the project manager, typechecker, test runner, and canonical tabular formatter)—configured via a pure Ridgun [`build.rg`](build.rg) file (no YAML, JSON, or TOML):
29
30```bash
31# Type-check all project modules in src/main/ridgun/
32./target/stable check
33
34# Compile & run all test suites in src/test/ridgun/
35./target/stable test
36
37# Format all .rg source files into canonical tabular alignment
38./target/stable format
39
40# Build all project targets into target/ (or install into ~/.local/bin/)
41./target/stable built
42./target/stable built lib
43./target/stable install
44
45# Scaffold a new app or library project, list tasks/plugins, or sync dependencies
46./target/stable new my_app
47./target/stable new --lib my_lib
48./target/stable tasks
49./target/stable plugins
50./target/stable deps
51
52# Compile or run a single .rg file or library directly with ridgunc (x86_64 or ARM64 Raspberry Pi)
53./target/ridgunc check my_program.rg
54./target/ridgunc run my_program.rg
55./target/ridgunc build my_program.rg -o my_program
56./target/ridgunc build --lib my_lib.rg -o my_lib.rglib
57./target/ridgunc build --so my_lib.rg -o libmy_lib.so
58./target/ridgunc build --target arm64-linux my_program.rg -o my_program_arm64
59```
60
61### Release Verification (`minisign`)
62
63Official Ridgun release downloads (`SHA256SUMS`) are signed with `minisign` using the following Ed25519 public key:
64
65```text
66RWQuKPmgklF+8SxAc6VK4RdRg/vw1/jBXvq5E0YcBN5QcFjezotHHWvq
67```
68
69Verify a downloaded release before unpacking:
70
71```bash
72minisign -Vm SHA256SUMS -P RWQuKPmgklF+8SxAc6VK4RdRg/vw1/jBXvq5E0YcBN5QcFjezotHHWvq
73sha256sum -c SHA256SUMS
74```
75
76---
77
78## What Makes Ridgun Different?
79
801. **Code as a 2D Table (`=>`, `=`, `->`)**: Type signatures act as column headers. Pattern-matching clauses align each argument beneath its type and align `=` beneath `=>`. `stable format` (and GNU Emacs `ridgun-mode`) formats your code automatically.
812. **Unified `=` Block Grammar (`(none)`, `|`, `||`) & Compact Nested Closers (`|.` / `||.`)**:
82 - Every `=` (in functions, operators, local bindings `x =`, or guard branches) accepts sequential lines, conditional guard blocks (`|`), or parallel DAG blocks (`||`).
83 - Guard branches can nest 3–4+ levels deep to form rich decision trees, using the compact inline **`|.`** closer (`| otherwise = val |.`) only when returning to an outer branch level below.
843. **Parallel DAG Blocks (`|| expr -> var`), Multicore Combinators (`||map`, `||filter`), and the Native Agent Model (`agent`, `send ... to`, `receive from`)**:
85 - Parallel dataflow dependencies are declared directly with `|| expr -> var` (no `where` keyword required). The static analyzer verifies cycle freedom at compile time and schedules independent branches in parallel.
86 - `map`, `filter`, and `fold` execute sequentially left-to-right, while **`||map`** and **`||filter`** distribute list elements across multicore worker threads using Ridgun's `||` parallel prefix.
87 - **Zero-boilerplate Native Agent Model & Smart Compile-Time Strategy Selection (`FR-COMP-009`)**: Any standard function or operator (`fn ::` or `op ::`) can be wrapped into an isolated concurrent agent in one line (`ag = agent (my fn)` for stateless agents, or `ag = agent (step fn) init_state` for stateful/session agents `S M => S` and `S M => S R`). Communicate via statically typed `send msg to ag` (or multi-argument `send a b to ag`), blocking `receive from ag`, non-blocking `try receive from ag` (`Some r | None`), and `stop ag` (which deterministically frees the agent's isolated state via `RMM1`/`RMM2`). At compile time, `ridgunc` analyzes each function's call depth and computational cost across `||`, `||map`, `||filter`, and `agent`, executing lightweight functions (`StratInlineSequential`) as zero-overhead *green-thread* steps and automatically dispatching heavy or deeply recursive functions (`StratLazyStealableTask` / `StratKernelWorkerJob`) across real OS kernel threads (`SYS_clone`).
884. **Immutable by Default (`x = ...`) with Explicit `mut` (`mut x = ...`)**:
89 - Local bindings are immutable constants unless declared with `mut x = expr`, preventing accidental rebinding in the same scope.
905. **Deterministic Memory Without a Garbage Collector (`RMM1`, `RMM2` & `with region`)**:
91 - **Policy `RMM1` (Compiled Memory Management)**: Static last-use deallocation (`rg_cmm_drop`), zero-copy return moves (`rg_cmm_move`), and 1-cycle pointer-identity guards (`ptr(a) != ptr(c)`) for branching functions.
92 - **Policy `RMM2` (Assignment & Cascaded Structure Counting)**: Incoming function arguments are untouched (`+0`/`-0`), local/container assignments add `+1`, returned locals are spared in transit, and non-returned locals cascade `-1` through child structures on scope exit.
93 - **Lexical Regions (`with region 64_KB do`) & Timeouts (`with timeout 250_ms`)**: Enforce hard byte quotas and wall-clock execution deadlines on untrusted workloads and free entire arenas in $O(1)$.
946. **Natural-Language Spaces in `fn ::` & `op ::`, Unified Tuple/Sequence Signatures (`FR-COMP-008`), Open `expand type`, Polymorphism (`A`..`Z`), and Symbolic Tensors (`@`)**:
95 - Define multi-word functions and N-ary operators with natural spaces instead of underscores (`fn :: format err AppErr => String`, `op :: clamp Int between Int and Int => Int`).
96 - Argument sequences and tuples are unified (`FR-COMP-008`): write clean space-separated tuple and return type signatures (`=> Int Int [String String] String`, `[String String]`, `(A B)`) without comma clutter.
97 - Share ADT constructor names across types (`Error`, `Var`), use bare type names as variants (`type Something = String | Int | MyStruct Float Int`), extend types via `expand type`, and merge new `fn ::` clauses automatically before `_` catch-alls.
98 - Single uppercase letters (`A`..`Z`) act as zero-noise polymorphic type variables.
99 - Native `Tensor[N, K]` types check symbolic dimensions at compile time and support Einstein summation (`A_ik @ B_kj`) and Hadamard broadcasting (`A_ij * B_j`).
100
101---
102
103## A Taste of Ridgun
104
105```ridgun
106-- 1. Tabular pattern matching, natural spaces & zero-noise polymorphism (A..Z)
107head or [A] A => A
108head or [] d = d
109head or (x:_) _ = x
110
111-- 2. Bare type-name variants, `expand type` & automatic `fn ::` clause merging
112type Something = String | Int | MyStruct Float Int
113
114fn :: describe something Something => String
115 describe something (String s) = "str:{s}"
116 describe something (Int n) = "int:{n}"
117 describe something (MyStruct f count) = "struct:{f}:{count}"
118 describe something _ = "other"
119
120expand type Something = Bool
121
122fn :: describe something Something => String
123 describe something (Bool b) = "bool:{b}"
124
125-- 3. Natural-language mixfix operator with `=` + `|` guards
126op :: clamp Int between Int and Int => Int
127 clamp x between lo and hi =
128 | x < lo = lo
129 | x > hi = hi
130 | otherwise = x
131
132-- 4. Multi-level decision tree with compact inline `|.` closers
133fn :: route request Int Int Bool Bool Int Int => Int
134 route request method role is_signed is_admin payload_kb rate_limited =
135 | method == 0 =
136 | payload_kb == 0 = 204
137 | otherwise = 200 |.
138 | method == 1 =
139 | not is_signed = 401
140 | otherwise =
141 | rate_limited > 0 =
142 | is_admin =
143 | payload_kb > 10000 = 413
144 | otherwise = 202 |.
145 | otherwise = 429 |.
146 | payload_kb > 1000 =
147 | role >= 2 and is_admin = 201
148 | otherwise = 413 |.
149 | otherwise = 201 |.
150 | otherwise = 405
151
152-- 5. Parallel DAG block (`|| expr -> var`) without `where` + `||.` closer
153fn :: quarterly balance Int => Int
154 quarterly balance base =
155 || base * 100 -> gross_sales
156 || base * 40 -> fixed_costs
157 || gross_sales - fixed_costs -> gross_profit ||.
158 gross_profit * 80 / 100
159
160-- 6. Conditional local variable (`x = | ...`) and explicit `mut` accumulator
161fn :: compute total Bool Int => Int
162 compute total is_vip raw =
163 discount =
164 | is_vip and raw > 50 = 20
165 | otherwise = 0
166 mut total = raw - discount
167 total = total + 5
168 total
169
170-- 7. Symbolic-dimension Tensor contraction (Einstein summation `@`)
171fn :: linear layer Tensor[B, In] Tensor[In, Out] Tensor[B, Out] => Tensor[B, Out]
172 linear layer x weights bias = (x_bi @ weights_io) + bias
173
174-- 8. Native Agent Model (`agent`, `send ... to`, `receive from`, `stop`)
175fn :: add visit Int Int => Int
176 add visit acc n = acc + n
177
178fn :: track visits Int => Int
179 track visits delta =
180 counter = agent (add visit) 10
181 send delta to counter
182 total = receive from counter
183 stop counter
184 total
185```
186
187---
188
189## Self-Hosted Architecture (`src/main/ridgun/`)
190
191```text
192Ridgun Source (.rg) + build.rg
193 │
194 ▼
195 ┌────────────────────────┐
196 │ lexer.rg & parser.rg │ Tokenization, Mixfix Resolution, Unified `=` (`|`, `||`, `|.`, `||.`)
197 └───────────┬────────────┘
198 │
199 ▼
200 ┌────────────────────────┐
201 │ typecheck.rg │ Polymorphic `A..Z`, `mut` Immutability Check, Symbolic Tensor Dims
202 └───────────┬────────────┘
203 │
204 ▼
205 ┌────────────────────────┐
206 │ analyzer.rg │ Static `||` DAG Cycle Detection & Work-Stealing Strategy Inference
207 └───────────┬────────────┘
208 │
209 ▼
210 ┌────────────────────────┐
211 │ compiler.rg │ Call-Graph Tree-Shaking, TCO (`loop`), `RMM1` & `RMM2` Lowering
212 └─────┬────────────┬─────┘
213 │ │
214 ▼ ▼
215 ┌───────────┐ Bytecode (`.rgc`)
216 │ x86_64.rg │ (`vm.rg` Stack VM)
217 │ arm64.rg │
218 │ elf64.rg │
219 └─────┬─────┘
220 ▼
221 Native ELF64 (`x86_64` / `ARM64`)
222```
223
224---
225
226## Project Layout
227
228```text
229ridgun/
230├── build.rg # Pure-Ridgun project manifest for `stable`
231├── editors/
232│ └── emacs/ridgun-mode.el # GNU Emacs major mode with tabular alignment
233├── src/
234│ ├── main/ridgun/
235│ │ ├── lib/
236│ │ │ ├── ast.rg # Compiler & VM Algebraic Data Types
237│ │ │ ├── lexer.rg # Lexical scanner & byte-unit literals (_KB, _MB, _GB)
238│ │ │ ├── parser.rg # Recursive-descent parser & mixfix normalizer
239│ │ │ ├── typecheck.rg # Typechecker, A..Z unification & tensor shape checker
240│ │ │ ├── analyzer.rg # Static parallel DAG & strategy analyzer
241│ │ │ ├── compiler.rg # RMM1/RMM2 analyzer, native IR lowerer & .rgc compiler
242│ │ │ ├── elf64.rg # Direct Linux ELF64 executable binary emitter
243│ │ │ ├── x86_64.rg # Pure-Ridgun 2-pass x86_64 machine-code assembler & kernel
244│ │ │ ├── arm64.rg # Pure-Ridgun 2-pass ARM64 (AArch64 / Raspberry Pi) assembler
245│ │ │ ├── formatter.rg # Canonical tabular code formatter (`stable format`)
246│ │ │ └── vm.rg # Meta-circular stack virtual machine
247│ │ ├── ridgunc/main.rg # Self-hosted `ridgunc` CLI entry point
248│ │ └── stable/main.rg # Self-hosted `stable` project manager CLI entry point
249│ └── test/ridgun/ # Self-hosted Ridgun test suites (`test_*.rg`)
250├── docs/
251│ ├── book/ # The Ridgun Book in 7 languages (es, ca, gl, eu, en, pt, sv)
252│ ├── lang/spec.md # Formal language specification
253│ ├── etc/cmm_architecture.md# RMM1 & RMM2 memory management architecture
254│ └── requirements/ # Functional requirements (FR-COMP-*, FR-TOOL-*, FR-VM-*)
255└── target/ # Compiled self-hosted ELF64 binaries (`ridgunc`, `stable`)
256```
257
258---
259
260## Technical Documentation
261
262| Document | Description |
263|---|---|
264| [**Language Specification (`docs/lang/spec.md`)**](docs/lang/spec.md) | Complete syntax, type system, unified `=` blocks, `|` / `|.` decision trees, `||` DAGs, `mut`, ADTs, tensors, and canonical formatting rules |
265| [**RMM Memory Architecture (`docs/etc/cmm_architecture.md`)**](docs/etc/cmm_architecture.md) | Detailed design of `RMM1` (Linear & Conditional CMM) and `RMM2` (Assignment & Cascaded Structure Counting) |
266| [**Low-Level Runtime Design (`docs/etc/low_level_ridgun_runtime_design.md`)**](docs/etc/low_level_ridgun_runtime_design.md) | Architecture and instruction design for the 100% pure-Ridgun symbolic `x86_64` and `ARM64` runtime kernels |
267| [**FR-COMP-009 (`docs/requirements/functional/compiler/`)**](docs/requirements/functional/compiler/FR-COMP-009-native-agent-model-and-message-passing.md) | Requirement specification for Ridgun's Native Agent Model (`agent`, `send ... to`, `receive from`, `try receive from`, `stop`) |
268| [**FR-COMP-008 (`docs/requirements/functional/compiler/`)**](docs/requirements/functional/compiler/FR-COMP-008-space-separated-tuple-and-sequence-type-signatures.md) | Requirement specification for unified space-separated tuple and sequence type signatures |
269| [**FR-COMP-004 (`docs/requirements/functional/compiler/`)**](docs/requirements/functional/compiler/FR-COMP-004-native-elf64-codegen-x86-64-and-arm64.md) | Requirement specification for 100% pure-Ridgun native ELF64 machine-code generation (`x86_64` and `ARM64` Raspberry Pi) without Rust or `rustc` |
270| [**FR-COMP-003 (`docs/requirements/functional/compiler/`)**](docs/requirements/functional/compiler/FR-COMP-003-unified-equals-grammar-guards-parallel-pipes-and-mut.md) | Requirement specification for unified `=` grammar, `|.` / `||.` closers, `mut`, and lowercase logical operators |
271| [**FR-COMP-002 (`docs/requirements/functional/compiler/`)**](docs/requirements/functional/compiler/FR-COMP-002-compiled-memory-management-cmm.md) | Requirement specification for `RMM1` and `RMM2` deterministic memory management |
272| [**Roadmap & Active Tasks (`docs/road-map/INDEX.md`)**](docs/road-map/INDEX.md) | Current session state, completed milestones, and backlog |
273
274---
275
276## License
277
278Apache 2.0 — see [LICENSE](LICENSE).