gitoria

37.0 KB

1. # gitoria.worldapi.org
3. Git hosting for all projects, written in **Hybriel** (hl:web), login via **ident**. Source of truth: `CONCEPT.md`.
4. Built so far: repos with their own address (#6), the repo homepage (#8), code browsing (#9), tickets (#10), pull requests (#11) with a Merge button for the owner (#17), releases (#12),
5. every repo view as its own server-rendered page (#16), push and pull over HTTPS with access tokens and over SSH with keys (#7).
7. ## Hybriel (vendored)
8. * `bin/hybriel` + `plugins/` (core crypto data fetch fs http http1 mpackdb proc time web) = hybriel **master 06617221** (2026-10-03,
9. antcolony mission 074: plugin allocators 3a781359 + 413f60e4 (#126, plugins allocate with malloc via plugin_api.zig), mpackdb
10. frees per-operation buffers 2cb7ae5e, http1 request owns its parse 773de63e, event order f0ac2d2d (plugin ABI field — bin and
11. .so must match); no lambda/parameter semantics change. Built from a read-only `git archive 06617221` into `~/scratch-074/src`
12. (removed), sha256 `21059cc7…d77bdb`; old copy `.scratch/pre-074/` = 190aa11d. Gates 200/0, 46/0, 44/0. Memory proof:
13. `bash .scratch/w074/curlloop.sh <appDir> / 20 <label>` (tracker README), `node .scratch/w074/memtest.mjs <appDir>
14. .scratch/w074/real/storage <label> 0mugibaf6fds` (Chrome, port 8760, Chrome 8761–8764) and `bash .scratch/w074/longcurl.sh
15. <appDir> <label> <N>` (signed-in curl over the 10 memtest pages, RSS every 100) — all need a fresh real-data tar in
16. `.scratch/w074/real/storage` (deleted after the run).
17. Before: master 190aa11d (2026-10-02,
18. antcolony mission 072: #126 GC by bytes + returned closure scopes, #48 lambda params copy (`&p` = reference; audit: nothing in
19. gitoria needs `&`), #127 big SSR pages flat incl. mountKids by reference (04df4428), fc838894 GC correctness (string index /
20. plugin error read freed memory); built from a read-only `git archive 190aa11d` into `~/scratch-072/src` (removed), sha256
21. `860f5e61…0a23626`; old copy `.scratch/pre-072/` = ff51cf46). Tracker README `/` (2.3 MB HTML): 0.56 s per load, RSS flat
22. ~320 MB over 20 loads (old 9 → 16 s, 4.7–6.2 GB). Memory proof: `node .scratch/w072/memtest.mjs <appDir> <storage copy> <label>
23. <userId>` (port 8720, Chrome 8721–8724; env MEM_ONLY/MEM_SKIP/LOADS/CLONES), `bash .scratch/w072/curlloop.sh <appDir> <path> N
24. <label>` (needs a real-data copy in `.scratch/w072/real/storage`, deleted after each run — re-tar it).
25. Before: master ff51cf46 (2026-10-01, antcolony mission 048: #113 mpackdb unique generated ids, #115/#116 lambda members on faces, #118, client SVG namespace, #111 hl:markdown
26. (not vendored — gitoria has its own lib/markdown.hl), session sync 0b17f65b/64527baa — see "Lesson" below; built the same way into
27. `~/scratch-048-gitoria/src`, sha256 `f3b93a53…983588d3`; old copy `.scratch/pre-048/` = 317d4754 + the pre-048 tests/).
28. Before: master 317d4754 (2026-09-26, mission 038; includes 7cb9f8fc = #107: an emit in flight when its socket closes is carried over, a page being left starts nothing
29. — fixes the Firefox pull-back that rolled antcolony mission 037 back; also #103/#104, #105 `headers`, #106 `let` per loop pass, #94 files in
30. emit, #82 reconnect; before: #83 hashed `/__hl/…?v=` URLs + immutable cache, #88–#91 Bytes / chunked bodies / base64 / run stdin,
31. #95–#97, #100, #101, #45 hl:web, #80 `run()`, #81 `*path`, #44 `sessionDomain`), built from a read-only
32. `git archive master` (never inside Anton's repo) into `~/scratch-038/src`: `/media/STORAGE/projects/hybriel/native/zig-toolchain/zig
33. build -Doptimize=ReleaseFast -Dtarget=x86_64-linux-gnu.2.39` in `native/`, binary `native/zig-out/bin/hybriel`, sha256 `fc7481fb…48670a8`.
34. Older copies: `.scratch/pre-038/` (13ef4f9b bin/ + plugins/ + the pre-038 `browser.mjs`), `.scratch/pre-037/`, `.scratch/pre-035/`, `.scratch/pre-033*/`.
35. * **No local patch** (`grep -rn "LOCAL PATCH" plugins` finds nothing): a re-vendor is copy binary + plugins, run ALL THREE gates
36. (browser.mjs, push.mjs, ssh.mjs — deploy.sh only runs browser.mjs).
37. * **069 round 2: master 1a096ad3 (#127) NOT adopted either**: the Markdown component still grows on big READMEs. Repro:
38. `.scratch/w069/repro` (big-md variant, `run.sh <bin> <plugins> N`), see STATUS.
39. * **069 (2026-10-02), master 8efba065 NOT adopted**: the gates were green, but big server-rendered READMEs grow memory and get
40. slower on every load (tracker README 12 s → 47 s). See STATUS. Memory proof: `node .scratch/w069/memtest.mjs <appDir> <storage copy>
41. <label> <userId>`. It uses port 8760 and Chrome 8761–8764, does 200 signed-in loads and 20 clones, and prints RSS as JSON.
42. Lambda audit: `python3 .scratch/w069/lambdas.py` / `lambdas2.py`.
43. * **Lesson (048, hybriel 64527baa)**: a face that takes `session` answers with a `sync` of every component member derived from
44. `session` through server code (`tokens`, `keyItems`, `me`, `codeData` …) — the member is ALREADY fresh when the handler goes
45. on after `emit server`. A handler that adds the returned row must skip a row with the same id (components/tokens.hl, sshkeys.hl,
46. tickets.hl `addTicket` do), or the row is listed twice.
47. The session cookie's `Domain` is the manifest setting `sessionDomain` (project.hl).
49. ## Run (dev, Loreana)
50. ```bash
51. cd /media/STORAGE/projects/gitoria.worldapi.org
52. GITORIA_PORT=8360 GITORIA_PUBLIC_URL=http://localhost:8360 ./bin/hybriel project.hl
53. ```
54. Config (environment, or a `.env` beside `project.hl` — never printed or committed):
56. | Variable | Default | |
57. |---|---|---|
58. | `GITORIA_PUBLIC_URL` | `https://gitoria.worldapi.org` | the main address; a repo lives at `<scheme>://<slug>.<host[:port] of this>` |
59. | `GITORIA_PORT` | 8360 | |
60. | `HL_HOST` | 0.0.0.0 | `127.0.0.1` on Byrodin behind nginx |
61. | `GITORIA_WATCH` | on | `0` = no dev watcher (the container) |
62. | `GITORIA_STORAGE` | `./storage/mpackdb` | table directory (`repos.db`, `users.db`) |
63. | `GITORIA_GIT` | `<launch dir>/storage/git` | the bare git repositories, `<slug>.git` each (absolute path; the `git` binary must be installed) |
64. | `GITORIA_TICKETS_URL` | `https://tickets.worldapi.org` | where a repo's tickets live (see "Tickets") |
65. | `GITORIA_SESSIONS` | `.sessions/` | |
66. | `GITORIA_COOKIE_DOMAIN` | `.<host of the public url>` | the session cookie's Domain (`-` = host-only; a host without a dot or an IP gets host-only) |
67. | `IDENT_URL`, `IDENT_EXCHANGE_URL`, `IDENT_API_KEY`, `IDENT_API_SECRET` | as in tickets | login via ident |
69. ## Files
70. Code order (antcolony `docs/code-order.md`, mission 002): `project.hl` is the MAP (config, routes, who hears what, an index
71. comment); the logic is in `lib/`, one file per topic with its central logic, the noise in a `-helpers` file; API routes,
72. faces and pages are thin wrappers. `let` only where a variable is reassigned (or re-bound in a loop body).
74. | file | what |
75. |---|---|
76. | `project.hl` | the map: PWA settings, routes, audience, session cookie, the server |
77. | `lib/repos.hl` | the repos table and every write to it: create, rows, owner checks (`ownsRepo`, `mayPush` = the push rule of HTTPS and SSH), main-branch setting, tickets connection |
78. | `lib/repos-helpers.hl` | slug rules (`slugError`, `reserved`), the row a page shows, `withChange`, tab titles, the Markdown read view |
79. | `lib/users.hl` | users table, ident exchange (`exchangeCode`, `userOfLoginCode`), display names, session → user, `tagOf` |
80. | `lib/git.hl` | git on the server: `git()` / `gitRaw()`, branches, default branch, `initRepo`, `isEmptyNow`, `tmpDir` |
81. | `lib/git-helpers.hl` | path/name/id checks, tab fields, sizes, the `\|\|\|PR` / `\|\|\|RL` subject parsers |
82. | `lib/homepage.hl` | the repo homepage (README.md / `$docs`) |
83. | `lib/code.hl` | the code browser (`/code`, `/branch`, `/commit`) and "Make main" (`makeMain`) |
84. | `lib/pulls.hl` | pull requests, the Merge (`mergePullNow`), the list after a merge (`pullsView`) |
85. | `lib/releases.hl` | releases |
86. | `lib/tickets.hl` | a repo's tickets (read, open, comment, state), the connect flow (`finishConnect`), `notifyPush` |
87. | `lib/tickets-helpers.hl` | `GITORIA_TICKETS_URL`, answer shapes, `#12` references (`refsOf`, `refParts`, `fixedBy`) |
88. | `lib/transport.hl` / `lib/transport-helpers.hl` | git over HTTPS (`gitTransport`, `callGit`) / Basic password, plain answers, protocol env, pkt-line |
89. | `lib/tokens.hl`, `lib/sshkeys.hl` | access tokens; ssh keys (+ `keyLine` for the sshd container, `sshUrl`) |
90. | `lib/api.hl` / `lib/api-helpers.hl` | the function routes (`/api/repos…`, `/login/callback`, `/settings/connect…`, `/__git/keys`, `/__git/access`) / JSON-Markdown-text answers, `goTo`, `sshGuard`, `safeNext` |
91. | `lib/markdown.hl` | Markdown → blocks (copied from tickets) |
92. | `lib/util.hl` | env, storage dir, addresses (`publicUrl`, `slugOfHost`, `domainFor`), lists, `newestFirst`, text checks, Vienna time |
93. | `components/*.hl` | pages and parts (shell `main.hl`, `list.hl`, `readme.hl`, `code/branch/commit.hl` + `codebrowser.hl`, `pulls`, `releases`, `tickets`, `settings`, `tokens`, `sshkeys`, `repohead`, `markdown`); CSS `components/styles.hl` |
94. | `tools/migrate-short-ids.hl` | one-off (mission 039) |
95. | `tests/` | gates `browser.mjs`, `push.mjs`, `ssh.mjs`; `realdata-baseline.mjs` + `realdata-compare.py` (same output on live data); `letcount.py` |
97. Imports go one way (Hybriel refuses a cycle): util, git-helpers, markdown ← users ← repos-helpers ← git ← repos ← tickets-helpers ←
98. homepage / code / pulls / releases ← tickets ← tokens / sshkeys ← transport ← api-helpers ← api ← project.hl.
100. ## How a repo gets its address
101. * A repo is one row in `storage/mpackdb/repos.db` (`@id` key, unique index on `slug`): `{ slug, description, owner, created }`.
102. * **Slug**: unique in the whole system (one table, unique index); 2–40 characters `a–z 0–9 -`, starts and ends with a
103. letter/digit, no `--`, not one of the reserved names (`lib/repos-helpers.hl` `reserved`: technical host names only — www, api, git, mail, …; app names like ident or tickets are allowed).
104. * **Address** = `<slug>.gitoria.worldapi.org`. nginx sends `gitoria.worldapi.org` and `*.gitoria.worldapi.org` to this one
105. app (wildcard vhost, wildcard DNS and certificate: the architect's).
106. * **Pages (gitoria#16)**: every page component declares `host = null` and hl:web hands it the request's host without port
107. (hybriel#74) — on the server, on the first load and on every hl:web navigation. `lib/util.hl slugOfHost` → '' (main address)
108. or the slug. Routes (`project.hl`): `/` → `components/index.hl` (main address: `list.hl`, the repo list + "Create a repository";
109. repo address: `readme.hl`), `/code` `/code/*` → `code.hl`, `/branch/*` → `branch.hl`, `/commit/*` → `commit.hl` (all three
110. show `codebrowser.hl`), `/pulls` → `pulls.hl`, `/releases` → `releases.hl`, `/tickets` → `tickets.hl`, `/login/failed` →
111. `loginfailed.hl`. Each repo page starts with `repohead.hl` (name, description, nav; unknown address → "No such repository").
112. Links inside a repo are hl:web navigations (no page load). Rendered on the server with their content: the repo list,
113. the repo head, the Readme's address/owner/created, the Tickets list (hl:fetch answers at once).
114. * **Git on the server**: git is read with hl:proc `run()`, which waits for the program (hybriel#80), so the README, the file
115. list / file, branches, commits, pulls and releases are in the first HTML and in every hl:web navigation (`lib/homepage.hl`
116. `homepageNow`, `lib/code.hl browseNow`, `lib/pulls.hl pullsNow`, `lib/releases.hl releasesNow`). No browser script is involved; `/host.js` is gone.
117. * **The path of `/code/*path`, `/branch/*path`, `/commit/*path`**: hl:web binds the rest of the URL to the page member `path`
118. (hybriel#81); the pages hand it to `codebrowser.hl`.
119. * **Login**: ident's login *button* flow (`<ident>/login?key=&return=<main>/login/callback`), the code exchanged server side.
120. * **Identity selector** (gitoria#14, as in tickets): ident's `<ident-selector>` sits beside the button in the header; choosing an identity
121. hands its one-time code (`/login.js` → hidden `#identcode` → face `gitoriaLogin`) to the server for the same exchange — no reload, open
122. tabs of the session follow. ident answers only a registered origin, so the selector shows on the main address only (the shell gives it
123. the class `onrepo` from the request's host, `components/styles.hl` hides it); the button stays the way in there.
124. The app is registered in ident with ONE origin, the main address. The session cookie carries `Domain=.gitoria.worldapi.org`
125. (hl:web `sessionDomain`, hybriel#44), so the login holds on every repo address. A login started at
126. `<slug>.…` returns through the main address and on to that repo (`?next=`, added by `/login.js` at the click: a path, or a full URL of `<valid-slug>.<host>`;
127. `lib/api-helpers.hl` `safeNext`). The first login asks for a display name (shown as the repo's owner). A failed login redirects to
128. `/login/failed` (the reason parked in `session.data.loginError`; since mission 002 also in a browser that already had a session — `&req`, see STATUS "Lessons").
129. * **Short ids (ident#23, antcolony mission 039)**: `users.identity` holds what ident's exchange answers — since ident#23 the identity's
130. public 5-character short id (`a68sz`), before that the per-app id (32 hex); `lib/users.hl isIdentId` accepts both (the old
131. `isHex` check refused short ids). The switch: one-off `tools/migrate-short-ids.hl` (old → short id, idempotent, never
132. `finish`), gate `node tests/short-id-switch.mjs` (ports 8724/8725, no browser), runbook
133. `antcolony-docs/docs/short-id-switch.md` (Byrodin: `/CONTAINERS/projects/antcolony/docs/short-id-switch.md` once synced).
134. * **Create** (web form, face `gitoriaCreate`): any logged-in user with a display name. New repos reach every open list live.
135. * **API** (public reads): `GET /api/repos`, `GET /api/repos/:slug`; `Accept: text/markdown` gives the Markdown read view.
136. Creating is not in the API yet (needs API tokens — as in tickets `users.hl`).
137. * Creating a repo also runs `git init --bare -b main` in `<GITORIA_GIT>/<slug>.git` (`lib/git.hl initRepo`); pushing to it: see "Push and pull".
139. ## Push and pull (gitoria#7)
140. Git over **HTTPS**, answered by this app itself with the `git` binary (`lib/transport.hl`; design: `docs/git-backend.md`). SSH: see below.
141. * **Clone URL**: `https://<slug>.gitoria.worldapi.org/<slug>.git` (on the repo address, so the clone's folder is named like the repo). Paths
142. `/<slug>.git/info/refs?service=…`, `/<slug>.git/git-upload-pack`, `/<slug>.git/git-receive-pack` (function routes, after the page routes in `project.hl`).
143. * **Read** (clone, fetch, pull) needs no login: every repo is public. **Write** (push) needs the **access token of the repo's owner** as the
144. password (any user name); anybody else's token → 403, no/unknown token → 401 with the way to get one. The token check is in `gitTransport`,
145. before git is started.
146. * **Access tokens** (`lib/tokens.hl`, `components/tokens.hl`, section `#tokens` of the main address for a logged-in user): name → token `gtr_` + 40 hex, shown ONCE;
147. only its sha256 is stored (`storage/mpackdb/tokens.db`); list, remove (works at once); at most 20 per user. Faces `gitoriaMakeToken`, `gitoriaRemoveToken`.
148. * **The "Add code to this repository" box** (`components/repohead.hl`, gitoria#20): plain text (no toggle), only on the
149. **Code page** of a repo that has no branch yet (`lib/git.hl isEmptyNow`) — never on Readme / Pull requests / Releases /
150. Tickets / Settings, and never once there is a commit. The clone command, the commands for a new project and for an
151. existing one, and where the token comes from.
152. * **How the body gets to git**: hl:proc `run()` has no stdin, so the request body is written to `<GITORIA_GIT>/.tmp/<random>.in` (a String holds raw bytes;
153. hl:fs writes them exactly) and `sh -c 'git upload-pack|receive-pack --stateless-rpc "$1" < "$2" > "$3"'` (paths as arguments, no user text in the script)
154. writes the answer to `.out`, which is read back as the response; both files are removed. A gzip request is unpacked first. `Git-Protocol: version=…`
155. → `GIT_PROTOCOL` (v0, v1 and v2 tested). No hook: pulls and releases are read from the commits.
156. * **Behind nginx** (the architect's vhost): `client_max_body_size` must allow pushes (say `500m`) and `proxy_request_buffering` stays **on** (default).
157. git sends a big push chunked; hl:http1 reads chunked request bodies since hybriel#89 (antcolony mission 035: the old 411 hint is gone, a direct
158. client without proxy pushes too — gate-checked). `proxy_read_timeout` ≥ 300s. The whole body is held in memory (≤ 500 MB).
159. Kept on purpose (035): the temp files + `sh -c` (hl:proc `run()` could now take `stdin` + `binary`, hybriel#88/#91, but stdin
160. crosses the plugin ABI as hex = twice the memory for a ≤ 500 MB body, and gzip would still need a second program) and `base64 -d`
161. for the Basic header (hl:crypto `fromBase64(...).toString()` aborts the request on bytes that are not UTF-8).
162. * Test: `node tests/push.mjs` (own ident + server + Chrome + the real git client; a small proxy in the gate plays nginx).
163. Other ports: `GITORIA_GATE_PORT=8750 GITORIA_GATE_IDENT_PORT=8751 GITORIA_GATE_PROXY=8752 GITORIA_GATE_CHROME=8753-8757` (048: 46/0).
165. ## Git over SSH (gitoria#7)
166. A small **sshd container** (`docker/sshd`, service `gitoria-sshd` in `docker-compose.yml`) whose only job is `git-upload-pack` / `git-receive-pack`. It keeps no user list:
167. * **Keys** (`lib/sshkeys.hl`, `components/sshkeys.hl`, section `#sshkeys` of the main address for a logged-in user): paste a PUBLIC key + a name; checked with `ssh-keygen -l`
168. (ed25519, ecdsa, sk-…, RSA ≥ 2048; a private key, DSA, junk, a second copy of a key are refused); list with fingerprint; remove; ≤ 20 per user. Table `storage/mpackdb/sshkeys.db`.
169. Faces `gitoria{Add,Remove}Key`. A key works at once and stops at once (sshd asks again on every login).
170. * **Login**: `AuthorizedKeysCommand` (`gitoria-keys`) → `GET /__git/keys?type&key` (`lib/api.hl gitKeys` → `lib/sshkeys.hl keyLine`) → `restrict,command="gitoria-shell <user id>" <key>`. The forced command
171. `gitoria-shell` accepts only `git-upload-pack|git-receive-pack '<slug>.git'` (slug validated, no shell, no forwarding, no tty), asks `GET /__git/access?user&slug&write`
172. and only then runs git on `/repos/<slug>.git`. Same rule as HTTPS: **read = everyone, push = the repo's owner**.
173. * **The two internal routes** exist only when `GITORIA_SSH_SECRET` is set; every call needs `X-Gitoria-Secret` = that secret, and a call that came through the public proxy
174. (`X-Forwarded-For` / `X-Real-IP`) is refused. **SSH is offered on the site only when the secret is set** (the keys section and the ssh commands in the "add code" box are hidden otherwise).
175. * **Deploy (the architect)**: `GITORIA_SSH_SECRET=<long random text>` (and optionally `GITORIA_SSH_PORT`, default 2222) in `.env` beside `docker-compose.yml` (both services read it);
176. open the port in the firewall (port 22 belongs to the host's sshd, hence 2222: address `ssh://[email protected]:2222/<slug>.git`; with `GITORIA_SSH_PORT=22` the box shows `[email protected]:<slug>.git`);
177. `gitoria.worldapi.org` (not only the wildcard) must resolve to Byrodin directly — **Cloudflare's proxy does not carry ssh** (use a grey-cloud/DNS-only record for the ssh host or a
178. separate name; the address in the box uses the site's host name). The Hybriel image needs `openssh-client` (Dockerfile). `./storage/git` is mounted into the sshd container; its `git` account takes
179. the uid of that folder's owner; host keys live in `storage/sshd-hostkeys/` (clients keep trusting the server). `docker compose up -d --build` builds both.
180. * Test: `node tests/ssh.mjs` (048: `… GITORIA_GATE_SSH_PORT=8758` + the push ports → 44/0) (builds and starts the REAL sshd container on port 8708 with host networking; real ssh + git: clone, push 3 MB, RSA + ed25519 keys, pull, unknown key, no shell,
181. path tricks, no forwarding, another user may read not push, removed key stops at once). Needs docker.
183. ## The repo homepage (`/` of a repo address)
184. * The repo's **README.md** (root, any case) is shown as a page; if the repo has a **`$docs`** folder, **all Markdown files inside it**
185. (subfolders too, in path order, at most 30) form the homepage instead and the root README is not shown. Read from the
186. repo's `HEAD` (the branch setting comes with #9). No commit / no README → "This repository has no README.md yet."
187. * Reading = the `git` binary via hl:proc (`lib/homepage.hl`: `ls-tree -r`, `cat-file blob HEAD:<path>`, argv list, paths only from git,
188. 15 s limit, ≤ 5000 lines a file),
189. read while the page is built on the server (`homepageNow`).
190. * Markdown → HTML (`lib/markdown.hl` copied from tickets, plus tables, block quotes, rules; `components/markdown.hl`): built as
191. elements from parsed data, never an HTML string — raw HTML in a README is shown as text, only http(s)/mailto/`/…`/`#…` links
192. are links. Not rendered: images, nested lists (shown as typed).
194. ## Browsing code (`/code`, `/branch/<name>`, `/commit/<id>` of a repo address)
195. * **`/code`** = the repo's main branch at its last commit. Main branch = the owner's setting (repo field `branch`), else `main`,
196. else the first branch. The owner sets it on the code page: "Make main" beside each other branch (only the owner sees it; the
197. face checks the owner again). The homepage (README / `$docs`) reads the same main branch.
198. * **`/branch/<name>`** = that branch at its last commit (a name with slashes works: the longest existing branch name wins).
199. **`/commit/<id>`** = the whole project at that commit (`<id>` = 4–40 hex characters, resolved to the full id).
200. * After each of them a path: `/code/src/a.txt`, `/branch/feature/x/src`, `/commit/<id>/src`. A folder shows its entries
201. (folders first, then files, with size), a file its numbered lines (at most 2000 lines, at most 1 MB). Also on the page: the
202. crumbs, the latest commit, all branches, the latest 20 commits (each links to `/commit/<id>`).
203. * Read with the `git` binary (`lib/code.hl` `browse`: `for-each-ref`, `log`, `cat-file`, `ls-tree`, argv lists, `--literal-pathspecs`).
204. Read on the server while the page is built (`browseNow`). `/code`, `/branch/*`, `/commit/*` are three pages sharing
205. `codebrowser.hl`; a link inside the app is an hl:web navigation. "Make main" (face `gitoriaSetBranch`) answers with the view again.
206. * **Only UTF-8 text is shown**: a binary file or one that is not valid UTF-8 says so instead (its bytes would break the
207. page's socket). Paths with `:`, quotes, backslashes or control characters are not browsable (`lib/git-helpers.hl` `safePath`).
208. * Not built: the Markdown read view / API of code, a diff of a commit, images, syntax highlighting, downloading a tree.
210. ## Tickets (`/tickets` of a repo address)
211. * The tickets are **not stored in gitoria**: they live in tickets.worldapi.org, in a project named `<slug>.<host of GITORIA_PUBLIC_URL>`
212. (e.g. `myrepo.gitoria.worldapi.org`; a repo slug has no dot, so it never meets another repo's project or the dotted app projects).
213. Anyone logged in to tickets sees them like any tickets project. `lib/tickets.hl` talks to tickets' public API.
214. * **List**: `GET <tickets>/api/projects/<project>/tickets` (public), newest update first, at most 200 shown: number, subject (as text), state,
215. last update; each links to the ticket in tickets (read, comment and change the state there — the list view only is in gitoria).
216. No project yet (404) → "No ticket yet". Tickets down → "tickets.worldapi.org did not answer". Read when the page is built on
217. the server (hl:fetch is synchronous): the list is in the first HTML. A ticket opened here shows up live on every open tickets page of that repo.
218. * **Connect (gitoria#18)**: the repo's **settings** (`/settings`, owner only) has "Tickets: connect". It sends the owner to
219. `<tickets>/connect?app=gitoria&label=<slug>&return=<repo address>/settings/connected&state=<nonce>` (tickets asks which project they are
220. admin of); tickets returns `?code&state`; gitoria checks the nonce (parked in the session, bound to the repo), exchanges the code from
221. the server (`POST /api/connect/exchange`) and stores the key per repo in `repos.db` (`tktKey`, never sent to a page). "Forget the
222. connection" clears it locally (tickets can also disconnect). Not connected → `/tickets` says so and points to Settings.
223. `GITORIA_TICKETS_TOKEN` and the auto-created `<slug>.<host>` project are gone (they were the old way).
224. * **Open a ticket**: any logged-in user with a display name: `POST <api>/tickets` with `Authorization: Bearer <key>` and
225. `X-Tickets-Identity: <the user's ident id>` — the person is the author in tickets, under the project's roles (they must have logged
226. in to tickets once).
227. * **`#N` links both ways**: `#12` in a commit subject (code page, latest commits) or a pull request title links ticket 12. A push
228. (and the Merge button) runs `notifyPush`: every commit of the last 100 on any branch whose subject names `#N` posts a comment
229. "Mentioned in commit …" on ticket N (once per commit and ticket, remembered in `ticketlinks.db`), and a **merged** `|||PR` whose
230. title says `fixes|closes|resolves #N` sets ticket N to state `review` with a comment. Done as the repo's owner. Commits that
231. exist when the repo is connected are only marked, not announced.
232. * Not built: showing a ticket's text / comments inside gitoria, editing a ticket from gitoria, a state filter, API of gitoria for tickets.
234. ## Pull requests (`/pulls` of a repo address)
235. * Nothing is stored: the list is read from git each time (`lib/pulls.hl` `pulls`). A **pull request is a commit** whose subject starts
236. `|||PR ` (target = the repo's main branch, i.e. the owner's setting, else `main`) or `|||PR|<branch>] ` (target = `<branch>`).
237. Anything else (marker not at the start, no space after `]`, an invalid branch name) is a normal commit.
238. * Title = the text after the marker (empty → the source branch's name). Source = the branch that holds the commit and is not the target
239. (`for-each-ref --contains`); one request per source branch (its newest marker commit); 50 at most, from the latest 500 commits of all branches.
240. * State: **merged** when the commit is in the target branch (`merge-base --is-ancestor`), else **open**; a target that does not exist is
241. shown as "(no such branch)". Merging is done with git itself (push to the target) — no merge button yet.
242. * Read on the server while the page is built (`pullsNow`).
244. ## Releases (`/releases` of a repo address)
245. * Nothing is stored: read from git each time (`lib/releases.hl` `releases`). A **release is a commit on the repo's main branch** whose subject starts
246. `|||RL ` (patch +1), `|||RL|med ` (minor +1, patch 0), `|||RL|mj ` (major +1, minor and patch 0) or `|||RL|<version> ` (set by hand,
247. `1.1.1a`: three numbers, then letters/digits/`.`/`-`). `|||RL` alone counts as `|||RL `. Anything else is a normal commit.
248. * Versions are counted from 0.0.0 over the main branch's history, oldest to newest (first `|||RL ` = 0.0.1); after a hand-set version the count
249. goes on from its numbers. A hand-set version already released is not a release. Shown newest first (200 at most, latest marked): version, text
250. after the marker (else the short commit id; links to `/commit/<id>`), author, date. Read on the server while the page is built (`releasesNow`).
251. * A release is only that: version + commit. No git tag is written, no archive to download (the concept does not say what else it holds).
253. ## PWA (installable app, antcolony mission 046)
254. The installable app, the same way calendar.worldapi.org and tracker do it: hl:web's own manifest and service worker from
255. settings in `project.hl`, no JavaScript of ours.
257. * `appIcons` (192 + 512 PNG, each `any` and `maskable`), `appTouchIcon` (180 PNG), `appFavicon` (`/icons/favicon.svg`),
258. `appThemeColor` = token `darker` (the header, rgb(15, 20, 25)), `appBackgroundColor` = token `dark` (rgb(25, 30, 35));
259. name = `appTitle` "gitoria". hl:web serves `/__hl/manifest.webmanifest` (start_url/scope `/`, display standalone) and
260. `/__hl/sw.js`, and links manifest, apple-touch-icon and theme-color from every head. `/favicon.ico` is a real icon
261. (16/32/48). Each icon has its own `file` route in `project.hl`. Every repo address is its own origin: it gets the same
262. manifest and its own worker (installed from a repo address, the app opens that repo's Readme).
263. * **Icons** (`icons/`): `icon.svg` is the source (512, hand-written: a git branch — trunk with two commits, a branch
264. curving off to a third; `#569bd4` on rgb(25,30,35); everything inside the maskable safe zone, a circle of radius 204, so
265. one image serves `any` and `maskable`); `favicon.svg` is the same drawing, thicker, cropped tight on a rounded tile.
266. Rendered on Loreana:
267. ```bash
268. rsvg-convert -w 192 -h 192 icons/icon.svg -o icons/icon-192.png
269. rsvg-convert -w 512 -h 512 icons/icon.svg -o icons/icon-512.png
270. rsvg-convert -w 180 -h 180 icons/icon.svg -o icons/apple-touch-icon.png
271. for s in 16 32 48; do rsvg-convert -w $s -h $s icons/favicon.svg -o /tmp/fav-$s.png; done
272. magick /tmp/fav-16.png /tmp/fav-32.png /tmp/fav-48.png icons/favicon.ico
273. ```
274. * **Offline**: `offline = [ Index ]` — the worker precaches the shell (runtime, modules, CSS, manifest, icons) and the
275. document of `/`. Navigations are network-first (an online visit of `/` refreshes the kept copy); without a network `/`
276. comes from the cache AS LAST SEEN (main address: the repo list; repo address: its Readme) and every other page (code,
277. branch, commit, pulls, releases, tickets, settings) gets hl:web's "Unavailable offline" page (503). The shell
278. (`components/main.hl`) shows "You are offline. Repositories and code need the network." while `navigator.onLine` is
279. false: hl:web gives a page no connection state and no mount hook, so an invisible `netProbe` runs an endless 1 s CSS
280. animation (`components/styles.hl` `@keyframes gitoria-net-tick`) whose `animationiteration` handler reads `navigator.onLine`.
281. A server that is down while the device is online shows no note.
283. ## Test
284. `node tests/browser.mjs` (200 checks; commits real files into the gate's bare repos) — own gitoria + own ident + own tickets (copies without `.env`, mail sink; `tests/ticketskit.mjs`) + headless Chromes; `*.gitoria.test`
285. is mapped to 127.0.0.1 inside Chrome (`HL_CHROME_ARGS`, `tests/cdp.mjs`). Ports 8700–8709 (gitoria 8700, ident 8701, tickets 8702, Chromes 8703–8709); another range:
286. `GITORIA_GATE_PORT=8710 GITORIA_GATE_IDENT_PORT=8711 GITORIA_GATE_TICKETS_PORT=8712 GITORIA_GATE_CHROME=8713-8719 node tests/browser.mjs`. Screenshots in `.scratch/gate-*.png`, server log `.scratch/gate-server.log`.
287. Last run (mission 002 code order, ports 8750–8759, tickets from a `git archive HEAD` copy because the tickets working tree was
288. mid-change: `GITORIA_GATE_TICKETS_DIR=$PWD/.scratch/w082/tickets-head GITORIA_GATE_PORT=8750 GITORIA_GATE_IDENT_PORT=8751
289. GITORIA_GATE_TICKETS_PORT=8752 GITORIA_GATE_FIREFOX=8759 GITORIA_GATE_CHROME=8753-8758` → 200/0; push.mjs (`GITORIA_GATE_PROXY=8752
290. GITORIA_GATE_CHROME=8753-8757`) 46/0; ssh.mjs (+ `GITORIA_GATE_SSH_PORT=8758`) 44/0.
291. Before (antcolony mission 074, hybriel 06617221, ports 8760–8769): `GITORIA_GATE_PORT=8760 GITORIA_GATE_IDENT_PORT=8761 GITORIA_GATE_TICKETS_PORT=8762 GITORIA_GATE_FIREFOX=8769 GITORIA_GATE_CHROME=8763-8769` → `200 passed, 0 failed`; push.mjs (`GITORIA_GATE_PROXY=8762`) 46/0, ssh.mjs (`GITORIA_GATE_SSH_PORT=8769 GITORIA_GATE_CHROME=8763-8768`) 44/0. Mission 072 (hybriel 190aa11d): `GITORIA_GATE_PORT=8720 GITORIA_GATE_IDENT_PORT=8721 GITORIA_GATE_TICKETS_PORT=8722 GITORIA_GATE_FIREFOX=8729 GITORIA_GATE_CHROME=8723-8728` → `200 passed, 0 failed`; push.mjs (`GITORIA_GATE_PROXY=8722`) 46/0, ssh.mjs (`GITORIA_GATE_SSH_PORT=8729`) 44/0. Mission 048: ports 8750–8758 → `199 passed, 0 failed`. Before (antcolony mission 046): `GITORIA_GATE_PORT=8720 GITORIA_GATE_IDENT_PORT=8721 GITORIA_GATE_TICKETS_PORT=8722 GITORIA_GATE_CHROME=8723-8727 GITORIA_GATE_FIREFOX=8728 node tests/browser.mjs` → `199 passed, 0 failed` (the Firefox port defaults to 8699 — set it inside your range).
292. * PWA block (antcolony mission 046, at the END of the gate): manifest + icons over HTTP (real PNG sizes, favicon blue, `/favicon.ico` an
293. ICO), then the gate's Chromes are CLOSED and one fresh Chrome gets `--unsafely-treat-insecure-origin-as-secure=<main>,<alpha>`
294. (a service worker needs a secure context; plain-http `*.gitoria.test` is not one — the live https site needs nothing):
295. Chrome's installability + manifest verdict, worker scope `/` on the main AND a repo address; offline: the tab's network
296. is cut (the note appears live), the SERVER is stopped (CDP offline does not reach the worker's own fetches), reload of `/`
297. shows header + note + the list as last seen, alpha's `/` its Readme, `/code` "Unavailable offline"; server restarted,
298. back online. Screenshots `.scratch/gate-pwa-phone-{online,offline}.png` (390 px).
299. * gitoria#16 block: `firstHtml(path, host)` fetches the FIRST HTML with a Host header (node's fetch cannot set one) — `/` (README), /code,
300. /code/src, a file, /branch/main, /branch/feature/x, /commit/<id>, /pulls, /releases, /tickets hold their real content and no "Loading";
301. the main address the repo list; hybriel#43: a code view in a second tab of the session keeps its elements through a login and a
302. logout of the other tab; every view of an unknown address "No such repository"; `/host.js` 404; in Chrome every view loads
303. directly, and Readme → Code → Releases → Tickets → Pulls → Readme plus a folder + Back keep `window.__navMarker` (no page load).
304. * Re-vendor block: `/__hl/app.css` has the token file's `--dark` in `:root` (hybriel#39); a repo description
305. `</script><b id="xss">…` stays inside the seed (`\u003c`) and shows as text (hybriel#34); `Domain=.gitoria.test` on the cookie (`sessionDomain`).
306. * Navigation LOGGED IN (the creator saw full page loads in Firefox): a fresh Chrome logs in with the button on a repo address, then
307. real clicks Readme → Code → Releases → Pulls → Tickets → Code → a folder keep `window.__navMarker`; the same on an empty repo the user
308. owns, and with the WebSocket closed (POST fallback). The same two runs in a **real Firefox** (`tests/firefox.mjs`: headless
309. `/usr/bin/firefox` over WebDriver BiDi, no driver/npm; hosts mapped with the pref `network.dns.localDomains`; BiDi port
310. `GITORIA_GATE_FIREFOX`, default 8699).
311. * By hand: `curl -s -H 'Host: <slug>.gitoria.test:8720' http://127.0.0.1:8720/code` against a running dev server.
312. * Lesson: the views are in the FIRST HTML now, so "content is there" no longer means "page is live" — `await hydrated(page)` before
313. clicking a button with a handler ("Make main" was clicked on the dead SSR page and flaked).
316. ### Same output (a cleanup must not change what the app answers; mission 002)
317. On a COPY of the live data (Byrodin → Loreana, deleted after the run):
318. ```bash
319. # on Byrodin:
320. tar -C /CONTAINERS/projects/gitoria.worldapi.org -cf - storage/mpackdb storage/git | ssh loreana 'mkdir -p /media/STORAGE/projects/gitoria.worldapi.org/.scratch/realdata && tar -C /media/STORAGE/projects/gitoria.worldapi.org/.scratch/realdata -xf -'
321. # on Loreana, in the repo:
322. git archive <old commit> | (mkdir -p .scratch/old && tar -x -C .scratch/old) && cp -a bin .scratch/old/
323. GITORIA_REALDATA=.scratch/realdata node tests/realdata-baseline.mjs .scratch/old 8750 .scratch/out-old # app 8750, sshd 8758, fake tickets 8759
324. GITORIA_REALDATA=.scratch/realdata node tests/realdata-baseline.mjs . 8750 .scratch/out-new
325. python3 tests/realdata-compare.py .scratch/out-old .scratch/out-new -v # exit 0 = the same
326. python3 tests/letcount.py . # root .hl files, project.hl lines, lets
327. ```
328. `realdata-baseline.mjs` signs in as the creator (az5b2 → users @id `0mugibaf6fds`) with a session file and fetches every page of
329. every repo (signed in and out: `/`, code, files, folders, branches, commits, pulls, releases, tickets, settings, unknown
330. addresses), the browser modules, the API (JSON + Markdown), the git refs advertisements, the refusals, the login / connect
331. routes, the internal ssh routes; then WRITES on its own copy: faces (create, token, key, Make main, Merge, …), clone + push over
332. HTTPS (token) and over SSH (the real sshd container), removal of key and token, logout. The connected repo's tickets are a
333. snapshot served locally (nothing reaches live tickets). Two runs of the SAME code compare clean (the masks change nothing alone).
335. ## Deploy
336. `./deploy.sh` (gates → backup → rsync → restart → 200). First deploy = the architect's: folder, `.env`, wildcard vhost
337. (`server_name gitoria.worldapi.org *.gitoria.worldapi.org;`, WebSocket upgrade headers), wildcard DNS + certificate, and the app
338. registered in ident with origin `https://gitoria.worldapi.org`. Container port 45004 (`docker-compose.yml`).
339. deploy.sh's rsync does not delete: after the mission-002 move the old root `.hl` files (api, git, users, …) stay on Byrodin
340. unused (project.hl imports only `lib/`); remove them by hand once if wanted.
342. ## History and worker briefs
343. - `LOG.md` — append-only history, one dated line per step (moved here from the antcolony LOG on 2026-10-01).
344. - `missions/NNN-*.md` — worker briefs for this app; `reports/NNN-*.md` — their reports (same name). Numbered per
345. project since 2026-10-01 (antcolony#40); older text, code comments and commits use the old antcolony numbers →
346. map: `/media/STORAGE/projects/antcolony-docs/docs/mission-map.md` (Byrodin: `/CONTAINERS/projects/antcolony/docs/mission-map.md`).