gitoria

All repositories: gitoria

Readme Code Pull requests Releases Tickets Settings

Commit4a2d7125

4a2d7125initial commitmre2026-09-25 19:10

4a2d7125/README.md

20.7 KB

# gitoria.worldapi.org
Git hosting for all projects, written in **Hybriel** (hl:web), login via **ident**. Source of truth: `CONCEPT.md`.
Built so far: repos with their own address (#6), the repo homepage (#8), code browsing (#9), tickets (#10), pull requests (#11), releases (#12),
every repo view as its own server-rendered page (#16), push and pull over HTTPS with access tokens and over SSH with keys (#7).
## Hybriel (vendored)
* `bin/hybriel` + `plugins/` (core crypto data fetch fs http http1 mpackdb proc time web) = hybriel **master 2fbfe195** (2026-09-25:
#45 hl:web, #80 hl:proc `run()`, #81 `*path` routes, #44 `sessionDomain`, #58), built by mission 033 from a read-only
`git archive` (never inside Anton's repo): `zig build -Doptimize=ReleaseFast` in `native/` (toolchain
`hybriel/native/zig-toolchain/zig`), binary `native/zig-out/bin/hybriel`, sha256 `2a3c2b02…b565`. Older copies:
`.scratch/pre-033/` (e565176b), `.scratch/pre-033b/` (edf27bc1), `.scratch/pre-033d/` (6ae171af).
* **No local patch** (`grep -rn "LOCAL PATCH" plugins` finds nothing): a re-vendor is copy binary + plugins, run the gate.
The session cookie's `Domain` is the manifest setting `sessionDomain` (project.hl).
## Run (dev, Loreana)
```bash
cd /media/STORAGE/projects/gitoria.worldapi.org
GITORIA_PORT=8360 GITORIA_PUBLIC_URL=http://localhost:8360 ./bin/hybriel project.hl
```
Config (environment, or a `.env` beside `project.hl` — never printed or committed):
| Variable | Default | |
|---|---|---|
| `GITORIA_PUBLIC_URL` | `https://gitoria.worldapi.org` | the main address; a repo lives at `<scheme>://<slug>.<host[:port] of this>` |
| `GITORIA_PORT` | 8360 | |
| `HL_HOST` | 0.0.0.0 | `127.0.0.1` on Byrodin behind nginx |
| `GITORIA_WATCH` | on | `0` = no dev watcher (the container) |
| `GITORIA_STORAGE` | `./storage/mpackdb` | table directory (`repos.db`, `users.db`) |
| `GITORIA_GIT` | `<launch dir>/storage/git` | the bare git repositories, `<slug>.git` each (absolute path; the `git` binary must be installed) |
| `GITORIA_TICKETS_URL` | `https://tickets.worldapi.org` | where a repo's tickets live (see "Tickets") |
| `GITORIA_TICKETS_TOKEN` | — | API token (`tkt_…`) of the tickets user "gitoria"; without it the list shows, opening a ticket says it is not set up |
| `GITORIA_SESSIONS` | `.sessions/` | |
| `GITORIA_COOKIE_DOMAIN` | `.<host of the public url>` | the session cookie's Domain (`-` = host-only; a host without a dot or an IP gets host-only) |
| `IDENT_URL`, `IDENT_EXCHANGE_URL`, `IDENT_API_KEY`, `IDENT_API_SECRET` | as in tickets | login via ident |
## How a repo gets its address
* A repo is one row in `storage/mpackdb/repos.db` (`@id` key, unique index on `slug`): `{ slug, description, owner, created }`.
* **Slug**: unique in the whole system (one table, unique index); 2–40 characters `a–z 0–9 -`, starts and ends with a
letter/digit, no `--`, not one of the reserved names (`repos.hl` `reserved`: technical host names only — www, api, git, mail, …; app names like ident or tickets are allowed).
* **Address** = `<slug>.gitoria.worldapi.org`. nginx sends `gitoria.worldapi.org` and `*.gitoria.worldapi.org` to this one
app (wildcard vhost, wildcard DNS and certificate: the architect's).
* **Pages (gitoria#16)**: every page component declares `host = null` and hl:web hands it the request's host without port
(hybriel#74) — on the server, on the first load and on every hl:web navigation. `users.hl slugOfHost` → '' (main address)
or the slug. Routes (`project.hl`): `/` → `components/index.hl` (main address: `list.hl`, the repo list + "Create a repository";
repo address: `readme.hl`), `/code` `/code/*` → `code.hl`, `/branch/*` → `branch.hl`, `/commit/*` → `commit.hl` (all three
show `codebrowser.hl`), `/pulls` → `pulls.hl`, `/releases` → `releases.hl`, `/tickets` → `tickets.hl`, `/login/failed` →
`loginfailed.hl`. Each repo page starts with `repohead.hl` (name, description, nav; unknown address → "No such repository").
Links inside a repo are hl:web navigations (no page load). Rendered on the server with their content: the repo list,
the repo head, the Readme's address/owner/created, the Tickets list (hl:fetch answers at once).
* **Git on the server**: git is read with hl:proc `run()`, which waits for the program (hybriel#80), so the README, the file
list / file, branches, commits, pulls and releases are in the first HTML and in every hl:web navigation (`git.hl`
`homepageNow`, `browseNow`, `pullsNow`, `releasesNow`). No browser script is involved; `/host.js` is gone.
* **The path of `/code/*path`, `/branch/*path`, `/commit/*path`**: hl:web binds the rest of the URL to the page member `path`
(hybriel#81); the pages hand it to `codebrowser.hl`.
* **Login**: ident's login *button* flow (`<ident>/login?key=&return=<main>/login/callback`), the code exchanged server side.
* **Identity selector** (gitoria#14, as in tickets): ident's `<ident-selector>` sits beside the button in the header; choosing an identity
hands its one-time code (`/login.js` → hidden `#identcode` → face `gitoriaLogin`) to the server for the same exchange — no reload, open
tabs of the session follow. ident answers only a registered origin, so the selector shows on the main address only (the shell gives it
the class `onrepo` from the request's host, `styles.hl` hides it); the button stays the way in there.
The app is registered in ident with ONE origin, the main address. The session cookie carries `Domain=.gitoria.worldapi.org`
(hl:web `sessionDomain`, hybriel#44), so the login holds on every repo address. A login started at
`<slug>.…` returns through the main address and on to that repo (`?next=`, added by `/login.js` at the click: a path, or a full URL of `<valid-slug>.<host>`;
`project.hl` `safeNext`). The first login asks for a display name (shown as the repo's owner). A failed login redirects to
`/login/failed` (the reason parked in `session.data.loginError`).
* **Create** (web form, face `gitoriaCreate`): any logged-in user with a display name. New repos reach every open list live.
* **API** (public reads): `GET /api/repos`, `GET /api/repos/:slug`; `Accept: text/markdown` gives the Markdown read view.
Creating is not in the API yet (needs API tokens — as in tickets `users.hl`).
* Creating a repo also runs `git init --bare -b main` in `<GITORIA_GIT>/<slug>.git` (`git.hl`); pushing to it: see "Push and pull".
## Push and pull (gitoria#7)
Git over **HTTPS**, answered by this app itself with the `git` binary (`transport.hl`; design: `docs/git-backend.md`). SSH: see below.
* **Clone URL**: `https://<slug>.gitoria.worldapi.org/<slug>.git` (on the repo address, so the clone's folder is named like the repo). Paths
`/<slug>.git/info/refs?service=…`, `/<slug>.git/git-upload-pack`, `/<slug>.git/git-receive-pack` (function routes, after the page routes in `project.hl`).
* **Read** (clone, fetch, pull) needs no login: every repo is public. **Write** (push) needs the **access token of the repo's owner** as the
password (any user name); anybody else's token → 403, no/unknown token → 401 with the way to get one. The token check is in `gitTransport`,
before git is started.
* **Access tokens** (`tokens.hl`, `components/tokens.hl`, section `#tokens` of the main address for a logged-in user): name → token `gtr_` + 40 hex, shown ONCE;
only its sha256 is stored (`storage/mpackdb/tokens.db`); list, remove (works at once); at most 20 per user. Faces `gitoriaMakeToken`, `gitoriaRemoveToken`.
* **The "Add code to this repository" box** (`components/repohead.hl`, on EVERY repo page, in the first HTML): the clone command, the commands for a
new project and for an existing one, and where the token comes from; open by itself while the repo has no branch (`git.hl isEmptyNow`), else closed.
* **How the body gets to git**: hl:proc `run()` has no stdin, so the request body is written to `<GITORIA_GIT>/.tmp/<random>.in` (a String holds raw bytes;
hl:fs writes them exactly) and `sh -c 'git upload-pack|receive-pack --stateless-rpc "$1" < "$2" > "$3"'` (paths as arguments, no user text in the script)
writes the answer to `.out`, which is read back as the response; both files are removed. A gzip request is unpacked first. `Git-Protocol: version=…`
→ `GIT_PROTOCOL` (v0, v1 and v2 tested). No hook: pulls and releases are read from the commits.
* **Behind nginx** (the architect's vhost): `client_max_body_size` must allow pushes (say `500m`) and `proxy_request_buffering` stays **on** (default).
git sends a big push chunked; hl:http1 does not read a chunked request body, nginx turns it into a request with a length. A direct client without
proxy gets `411` with the hint (`git config http.postBuffer 524288000`). `proxy_read_timeout` ≥ 300s. The whole body is held in memory (≤ 500 MB).
* Test: `node tests/push.mjs` (own ident + server + Chrome + the real git client; a small proxy in the gate plays nginx).
## Git over SSH (gitoria#7)
A small **sshd container** (`docker/sshd`, service `gitoria-sshd` in `docker-compose.yml`) whose only job is `git-upload-pack` / `git-receive-pack`. It keeps no user list:
* **Keys** (`sshkeys.hl`, `components/sshkeys.hl`, section `#sshkeys` of the main address for a logged-in user): paste a PUBLIC key + a name; checked with `ssh-keygen -l`
(ed25519, ecdsa, sk-…, RSA ≥ 2048; a private key, DSA, junk, a second copy of a key are refused); list with fingerprint; remove; ≤ 20 per user. Table `storage/mpackdb/sshkeys.db`.
Faces `gitoria{Add,Remove}Key`. A key works at once and stops at once (sshd asks again on every login).
* **Login**: `AuthorizedKeysCommand` (`gitoria-keys`) → `GET /__git/keys?type&key` (`sshgate.hl`) → `restrict,command="gitoria-shell <user id>" <key>`. The forced command
`gitoria-shell` accepts only `git-upload-pack|git-receive-pack '<slug>.git'` (slug validated, no shell, no forwarding, no tty), asks `GET /__git/access?user&slug&write`
and only then runs git on `/repos/<slug>.git`. Same rule as HTTPS: **read = everyone, push = the repo's owner**.
* **The two internal routes** exist only when `GITORIA_SSH_SECRET` is set; every call needs `X-Gitoria-Secret` = that secret, and a call that came through the public proxy
(`X-Forwarded-For` / `X-Real-IP`) is refused. **SSH is offered on the site only when the secret is set** (the keys section and the ssh commands in the "add code" box are hidden otherwise).
* **Deploy (the architect)**: `GITORIA_SSH_SECRET=<long random text>` (and optionally `GITORIA_SSH_PORT`, default 2222) in `.env` beside `docker-compose.yml` (both services read it);
open the port in the firewall (port 22 belongs to the host's sshd, hence 2222: address `ssh://[email protected]:2222/<slug>.git`; with `GITORIA_SSH_PORT=22` the box shows `[email protected]:<slug>.git`);
`gitoria.worldapi.org` (not only the wildcard) must resolve to Byrodin directly — **Cloudflare's proxy does not carry ssh** (use a grey-cloud/DNS-only record for the ssh host or a
separate name; the address in the box uses the site's host name). The Hybriel image needs `openssh-client` (Dockerfile). `./storage/git` is mounted into the sshd container; its `git` account takes
the uid of that folder's owner; host keys live in `storage/sshd-hostkeys/` (clients keep trusting the server). `docker compose up -d --build` builds both.
* Test: `node tests/ssh.mjs` (builds and starts the REAL sshd container on port 8708 with host networking; real ssh + git: clone, push 3 MB, RSA + ed25519 keys, pull, unknown key, no shell,
path tricks, no forwarding, another user may read not push, removed key stops at once). Needs docker.
## The repo homepage (`/` of a repo address)
* The repo's **README.md** (root, any case) is shown as a page; if the repo has a **`$docs`** folder, **all Markdown files inside it**
(subfolders too, in path order, at most 30) form the homepage instead and the root README is not shown. Read from the
repo's `HEAD` (the branch setting comes with #9). No commit / no README → "This repository has no README.md yet."
* Reading = the `git` binary via hl:proc (`git.hl`: `ls-tree -r`, `cat-file blob HEAD:<path>`, argv list, paths only from git,
15 s limit, ≤ 5000 lines a file),
read while the page is built on the server (`homepageNow`).
* Markdown → HTML (`markdown.hl` copied from tickets, plus tables, block quotes, rules; `components/markdown.hl`): built as
elements from parsed data, never an HTML string — raw HTML in a README is shown as text, only http(s)/mailto/`/…`/`#…` links
are links. Not rendered: images, nested lists (shown as typed).
## Browsing code (`/code`, `/branch/<name>`, `/commit/<id>` of a repo address)
* **`/code`** = the repo's main branch at its last commit. Main branch = the owner's setting (repo field `branch`), else `main`,
else the first branch. The owner sets it on the code page: "Make main" beside each other branch (only the owner sees it; the
face checks the owner again). The homepage (README / `$docs`) reads the same main branch.
* **`/branch/<name>`** = that branch at its last commit (a name with slashes works: the longest existing branch name wins).
**`/commit/<id>`** = the whole project at that commit (`<id>` = 4–40 hex characters, resolved to the full id).
* After each of them a path: `/code/src/a.txt`, `/branch/feature/x/src`, `/commit/<id>/src`. A folder shows its entries
(folders first, then files, with size), a file its numbered lines (at most 2000 lines, at most 1 MB). Also on the page: the
crumbs, the latest commit, all branches, the latest 20 commits (each links to `/commit/<id>`).
* Read with the `git` binary (`git.hl` `browse`: `for-each-ref`, `log`, `cat-file`, `ls-tree`, argv lists, `--literal-pathspecs`).
Read on the server while the page is built (`browseNow`). `/code`, `/branch/*`, `/commit/*` are three pages sharing
`codebrowser.hl`; a link inside the app is an hl:web navigation. "Make main" (face `gitoriaSetBranch`) answers with the view again.
* **Only UTF-8 text is shown**: a binary file or one that is not valid UTF-8 says so instead (its bytes would break the
page's socket). Paths with `:`, quotes, backslashes or control characters are not browsable (`git.hl` `safePath`).
* Not built: the Markdown read view / API of code, a diff of a commit, images, syntax highlighting, downloading a tree.
## Tickets (`/tickets` of a repo address)
* The tickets are **not stored in gitoria**: they live in tickets.worldapi.org, in a project named `<slug>.<host of GITORIA_PUBLIC_URL>`
(e.g. `myrepo.gitoria.worldapi.org`; a repo slug has no dot, so it never meets another repo's project or the dotted app projects).
Anyone logged in to tickets sees them like any tickets project. `tickets.hl` talks to tickets' public API.
* **List**: `GET <tickets>/api/projects/<project>/tickets` (public), newest update first, at most 200 shown: number, subject (as text), state,
last update; each links to the ticket in tickets (read, comment and change the state there — the list view only is in gitoria).
No project yet (404) → "No ticket yet". Tickets down → "tickets.worldapi.org did not answer". Read when the page is built on
the server (hl:fetch is synchronous): the list is in the first HTML. A ticket opened here shows up live on every open tickets page of that repo.
* **Open a ticket**: any logged-in user with a display name (face `gitoriaOpenTicket`): subject (≤ 200, one line) + optional Markdown text
(≤ 20000). Gitoria posts `POST <tickets>/api/projects/<project>/tickets` with **its own token** (`GITORIA_TICKETS_TOKEN`): tickets takes the
author from the token and has no author field, so the ticket is authored by the tickets user "gitoria" and its text ends with
"— opened by <display name> in gitoria". **The first ticket creates the project in tickets** (tickets does that itself).
* **Set up on the live system (architect)**: log in to tickets as a user named `gitoria`, make an API token on `/you`, put
`GITORIA_TICKETS_TOKEN=tkt_…` into gitoria's `.env`, restart. The container reaches tickets over the public HTTPS address.
* Not built: showing a ticket's text / comments inside gitoria, editing a ticket from gitoria, a state filter, the author being the
real person in tickets (needs "act on behalf of" in tickets' API), API of gitoria for tickets.
## Pull requests (`/pulls` of a repo address)
* Nothing is stored: the list is read from git each time (`git.hl` `pulls`). A **pull request is a commit** whose subject starts
`|||PR ` (target = the repo's main branch, i.e. the owner's setting, else `main`) or `|||PR|<branch>] ` (target = `<branch>`).
Anything else (marker not at the start, no space after `]`, an invalid branch name) is a normal commit.
* Title = the text after the marker (empty → the source branch's name). Source = the branch that holds the commit and is not the target
(`for-each-ref --contains`); one request per source branch (its newest marker commit); 50 at most, from the latest 500 commits of all branches.
* State: **merged** when the commit is in the target branch (`merge-base --is-ancestor`), else **open**; a target that does not exist is
shown as "(no such branch)". Merging is done with git itself (push to the target) — no merge button yet.
* Read on the server while the page is built (`pullsNow`).
## Releases (`/releases` of a repo address)
* Nothing is stored: read from git each time (`git.hl` `releases`). A **release is a commit on the repo's main branch** whose subject starts
`|||RL ` (patch +1), `|||RL|med ` (minor +1, patch 0), `|||RL|mj ` (major +1, minor and patch 0) or `|||RL|<version> ` (set by hand,
`1.1.1a`: three numbers, then letters/digits/`.`/`-`). `|||RL` alone counts as `|||RL `. Anything else is a normal commit.
* Versions are counted from 0.0.0 over the main branch's history, oldest to newest (first `|||RL ` = 0.0.1); after a hand-set version the count
goes on from its numbers. A hand-set version already released is not a release. Shown newest first (200 at most, latest marked): version, text
after the marker (else the short commit id; links to `/commit/<id>`), author, date. Read on the server while the page is built (`releasesNow`).
* A release is only that: version + commit. No git tag is written, no archive to download (the concept does not say what else it holds).
## Test
`node tests/browser.mjs` (149 checks; commits real files into the gate's bare repos) — own gitoria + own ident + own tickets (copies without `.env`, mail sink; `tests/ticketskit.mjs`) + headless Chromes; `*.gitoria.test`
is mapped to 127.0.0.1 inside Chrome (`HL_CHROME_ARGS`, `tests/cdp.mjs`). Ports 8700–8709 (gitoria 8700, ident 8701, tickets 8702, Chromes 8703–8709); another range:
`GITORIA_GATE_PORT=8710 GITORIA_GATE_IDENT_PORT=8711 GITORIA_GATE_TICKETS_PORT=8712 GITORIA_GATE_CHROME=8713-8719 node tests/browser.mjs`. Screenshots in `.scratch/gate-*.png`, server log `.scratch/gate-server.log`.
* gitoria#16 block: `firstHtml(path, host)` fetches the FIRST HTML with a Host header (node's fetch cannot set one) — `/` (README), /code,
/code/src, a file, /branch/main, /branch/feature/x, /commit/<id>, /pulls, /releases, /tickets hold their real content and no "Loading";
the main address the repo list; hybriel#43: a code view in a second tab of the session keeps its elements through a login and a
logout of the other tab; every view of an unknown address "No such repository"; `/host.js` 404; in Chrome every view loads
directly, and Readme → Code → Releases → Tickets → Pulls → Readme plus a folder + Back keep `window.__navMarker` (no page load).
* Re-vendor block: `/__hl/app.css` has the token file's `--dark` in `:root` (hybriel#39); a repo description
`</script><b id="xss">…` stays inside the seed (`\u003c`) and shows as text (hybriel#34); `Domain=.gitoria.test` on the cookie (`sessionDomain`).
* Navigation LOGGED IN (the creator saw full page loads in Firefox): a fresh Chrome logs in with the button on a repo address, then
real clicks Readme → Code → Releases → Pulls → Tickets → Code → a folder keep `window.__navMarker`; the same on an empty repo the user
owns, and with the WebSocket closed (POST fallback). The same two runs in a **real Firefox** (`tests/firefox.mjs`: headless
`/usr/bin/firefox` over WebDriver BiDi, no driver/npm; hosts mapped with the pref `network.dns.localDomains`; BiDi port
`GITORIA_GATE_FIREFOX`, default 8699).
* By hand: `curl -s -H 'Host: <slug>.gitoria.test:8720' http://127.0.0.1:8720/code` against a running dev server.
* Lesson: the views are in the FIRST HTML now, so "content is there" no longer means "page is live" — `await hydrated(page)` before
clicking a button with a handler ("Make main" was clicked on the dead SSR page and flaked).
## Deploy
`./deploy.sh` (gates → rsync → restart → 200). First deploy = the architect's: folder, `.env`, wildcard vhost
(`server_name gitoria.worldapi.org *.gitoria.worldapi.org;`, WebSocket upgrade headers), wildcard DNS + certificate, and the app
registered in ident with origin `https://gitoria.worldapi.org`. Container port 45004 (`docker-compose.yml`).

Branches

mainmain branch

Latest commits

4a2d7125initial commitmre2026-09-25 19:10