Architecture

lairkit has two layers:

• A core instance: git, CI, secrets, logs, a static Pages service and backups.
• App instances: one repository per app, deployed onto a core's tailnet.

Both layers use the same operator contract: six intents, six derived states, five exit codes and one evidence tag per change.

Core

The core is one Linux host and ten Kamal accessories. It has no public listener and no application role.

flowchart TB
  user["Tailnet users"]
  ts["Tailscale Serve<br/>TLS on svc: names"]

  subgraph host["Core host — one tailnet node"]
    http["Four HTTP surfaces<br/>Forgejo · Infisical · Dozzle · Pages"]
    db[("Postgres<br/>Forgejo + Infisical")]
    ci["Runner + dind + sidecar"]
  end

  bucket[("S3 backups")]

  user --> ts --> http
  http --> db
  http -->|"Forgejo Actions"| ci
  db --> bucket

Components

Accessory	Purpose	Published on the host
forgejo-postgres	Forgejo database	internal only
forgejo	Git hosting, Actions, package registry	127.0.0.1:3000
dind	Docker-in-Docker runtime for CI jobs	internal
tailscale	Sidecar that gives CI jobs tailnet access	joins the dind network namespace
runner	Forgejo Actions runner, capacity 1	internal
infisical-postgres	Infisical database	internal only
redis	Infisical cache	internal
infisical	Secret store	127.0.0.1:8080
dozzle	Container logs	127.0.0.1:8082
pages	Static site hosting	127.0.0.1:8083

Four names are advertised on the tailnet:

Service	URL
forgejo	https://<slug>-forgejo.<tailnet_dns>
infisical	https://<slug>-infisical.<tailnet_dns>
dozzle	https://<slug>-dozzle.<tailnet_dns>
pages	https://<slug>-pages.<tailnet_dns>

Every port binds 127.0.0.1 on the host. tailscaled terminates TLS and forwards to those loopback ports; tailscale serve advertises each name as svc:<slug>-<name>.

How a command moves

flowchart TB
  L["Laptop<br/>./lk + Kamal"]
  T["Tailscale control plane"]
  H["Core host"]
  C["Containers"]
  E[("evidence tag")]

  L -->|"1. apply policy"| T
  L -->|"2. SSH + Kamal"| H
  H -->|"3. start"| C
  C -->|"4. health"| H
  H -->|"5. exit code"| L
  L -->|"6. write tag"| E

The laptop holds the facts and the dispatcher. The host holds containers, volumes and cron jobs. The host has no checkout and no dispatcher.

Data and backups

Data	Where it lives	How it is backed up
Forgejo repositories, users, Actions	forgejo-postgres + /srv/lk-<slug>/forgejo/data	pgBackRest base + WAL; stop-copy-start of the data volume
Infisical projects and secrets	infisical-postgres + /srv/lk-<slug>/infisical	pgBackRest base + WAL; stop-copy-start of the data volume
Forgejo configuration	config/forgejo/app.ini in git, plus bootstrap values	Rebuilt from git and escrow
Backups	One S3 bucket and one prefix per instance	sets/<UTCstamp>/manifest.json marks a complete set

The default targets are a 24-hour RPO, a 60-minute warm restore and 30-day retention. install ops takes the first backup and the first drill; a core without that action is live but unprotected.

Apps

An app template renders a repository inside the core's repos/ directory. The repository is registered in Forgejo, its secrets are written to Infisical, and every mutation runs as a Forgejo Actions job on the core's runner.

flowchart TB
  repo["Render app repo<br/>inside core/repos/"]
  collect["./lk collect<br/>repo + secrets"]
  push["git push<br/>Forgejo repository"]
  dispatch["workflow_dispatch<br/>core runner"]
  kamal["Kamal over SSH"]
  host["App host<br/>app + database + Dozzle"]

  repo --> collect --> push --> dispatch --> kamal --> host

Apps never write into the core checkout. Each render is its own git repository with its own .git, branch main and remote.

What an app owns

Piece	Example
One or more app roles	ERPNext web, frontend, backend, queue, scheduler
Its database or state	MariaDB, PostgreSQL, volumes
Its own host paths	/var/lib/lairkit-iac-<app>-<slug>
Its own backup path	One S3 prefix per instance
Its exposure	tailnet-only or public
Its own runbooks	BACKUP.md, RESTORE.md, UPGRADE.md, SECRETS.md

Exposure

• tailnet-only: services are advertised as tailnet Services. No public DNS record.
• public: a Bunny pull zone sits in front of the host with a WAF. DNS is delegated to Bunny.
• Jitsi media is the exception: Bunny carries HTTP and websockets, but UDP 10000 must reach the host directly for video.

The core is always private. Only declared app hostnames can be public.

Shared operator contract

Six intents

Intent	Question it answers
install	How does this instance come into existence?
check	Is it healthy, and does it match what is declared?
change	How do I patch, upgrade, rotate or grant access?
recover	How do I take, test and restore a backup?
transfer	How does someone else take ownership?
retire	How do I export and tear it down?

There is no seventh intent. An action that is not in config/intents.yml is refused.

Six derived states

planned, live, degraded, changing, suspended, retired.

State is observed, not stored. check status derives it from containers, tags, backups and reachability.

Note

A fresh render is planned only while the host is reachable. An unreachable host is degraded, because the observation failed rather than proved an empty instance.

Five exit codes

Code	Meaning
0	changed
10	nothing to change
11	drift or a red check; nothing was mutated
2	refused before mutating
1	mutated and failed

Evidence tags

A mutating action with a terminal outcome writes one annotated tag:

evidence/<intent>-<action>-<UTCstamp>

The tag records the reason, result and instance. check actions and exit 10 write nothing.

Names and identity

All names are derived from the instance slug:

Object	Pattern
Container (core)	<component>-<slug>
Container (app)	<role>-<project> or <project>-<role>
Tailnet Service	svc:<slug>-<name>
Host root	/srv/lk-<slug> (core)
App host root	/var/lib/lairkit-iac-<app>-<slug>
Evidence tag	evidence/<intent>-<action>-<stamp>

Apps can share a host as long as they use different slugs. A core is one per host: a second /srv/lk-* fails the host preflight.

Security model

• Root SSH is available over the tailnet only.
• Services bind loopback or a tailnet Service; the core never listens publicly.
• Secrets live in Infisical, with an offline escrow record for the values that cannot be recreated.
• The CI identity is read-only plus one write scope: evidence:tag.
• The app deploy token is scoped to one repository namespace.
• Destructive actions require a typed confirmation.
• Pinned versions are asserted by check; an unpinned image is a red check.