1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
|
# Deploy
## Base Image
The base image is used as a "base" for all images.
It is the config on first boot once Terraform/OpenTofu has finished provisioning.
Build base image:
```sh
nix flake update
nixos-rebuild build-image --image-variant proxmox --flake .#base
```
Copy base image to Proxmox:
```sh
scp result/*.vma.zst lan.hesh:/var/lib/vz/dump/
```
Restore on Proxmox:
```sh
# Destroy the old template 104 is the clone source in TF.
ssh lan.hesh 'qm destroy 104 --purge'
ssh lan.hesh 'qmrestore "$(ls -t /var/lib/vz/dump/vzdump-qemu-nixos-*.vma.zst | head -n1)" 104 --storage local-lvm'
```
Convert to Proxmox Template:
```sh
ssh lan.hesh 'qm template 104'
```
Run TF as usual using clone against the Template.
```sh
cd ../terraform
source con.env
tofu plan -out=tfplan
tofu apply tfplan
```
## Add Host
Scaffold a new host (host keys, `.sops.yaml`, `flake.nix`, `per-host/`, secrets, TF VM if absent, `~/.ssh/config`, Hosts section) and print the exact cmds to create the VM and deploy:
```sh
./add.sh <HOST_NAME> # interactive for cpu/mem/disk/vlan
./add.sh <HOST_NAME> --cpu 1 --mem 1024 --disk 10G --vlan 30
```
Defaults for cpu/mem/disk/vlan are taken from `monitoring-2`. If the VM already exists in TF it is left alone.
The manual equivalent, if needed:
```sh
./generate-keys.sh <HOST_NAME>
cat ./host-keys/<HOST_NAME>/ssh_host_ed25519_key.pub | ssh-to-age
```
Copy the `age` key to the `.sops.yaml` config.
Copy `per-host/<EXISTING_HOST>` to `per-host/<HOST_NAME>`.
Modify the `per-host/<HOST_NAME>/configuration.nix` accordingly.
## Secrets w/ `sops`
Edit the `.sops.yaml` to include the host's age key.
Create new secrets for a host:
```sh
sops secrets/<HOST_NAME>.yaml
```
## Push/Update
```sh
./deploy.sh <HOST_NAME> jefe
# This is not normally required, but if the base image is far enough from the
# current build then NixOS will not allow auto-switch on next boot.
# You can set the boot arg here to force it.
# This is also a good sign that the base image should be re-built and deployed.
./deploy.sh torrents-2 jefe boot
```
## Update all
Update all the hosts.
```sh
nix flake update
for host in $(ls per-host | grep -v '^base$'); do
if ! ssh -o ConnectTimeout=5 "$host" true 2>/dev/null; then
echo "→ [$host] offline, skipping"
continue
fi
./deploy.sh "$host" jefe boot && ssh "$host" sudo reboot
done
```
## References:
https://github.com/nix-community/nixos-generators
https://gist.github.com/joshleecreates/e6892ca21b0e6b7c24d96ca2a24bf23e
## Hosts
This section documents any notes for specific hosts I want to be aware of.
### caldav
For calendar, contacts, and tasks sync.
### torrents-2
For Linux ISOs.
### rproxy-2
TLS termination for upstream services.
### git
For git repos.
Set `~/.ssh/config` to include:
```sshconfig
Host git.kumardamani.net
Port 2222
```
> Non-SSH access (https) is always read-only, and anonymous.
Manage repos with:
```sh
ssh -t git@git.kumardamani.net
== git repo manager ==
l) list repos
c) create repo
p) publish repo
h) hide repo
d) set description
x) delete repo
q) quit
>
```
Without the ssh config entry, use the abs path:
```sh
git clone ssh://git@git.kumardamani.net:2222/srv/git/<name>.git
```
Move an existing local repo onto the server:
```sh
ssh git@git.kumardamani.net create <name>
ssh git@git.kumardamani.net publish <name>
git remote set-url origin git@git.kumardamani.net:<name>
git push --all origin && git push --tags origin
```
Push to both our git and GitLab (for ci/cd repos):
```sh
git config --unset-all remote.origin.pushurl
git remote set-url origin git@git.kumardamani.net:<name>.git
git remote set-url --add --push origin git@git.kumardamani.net:<name>.git
git remote set-url --add --push origin git@gitlab.com:kdam0/<name>.git
git remote -v
```
### photos
For photos.
### s3
For s3 buckets.
Bootstrap one-time with:
```sh
sudo garage status
sudo garage layout assign -z dc1 -c 10G <NODE_ID>
sudo garage layout apply --version 1
sudo garage bucket create <BUCKET_NAME>
# to enable static sites
sudo garage bucket website --allow <BUCKET_NAME>
sudo garage key create site-uploader
```
### monitoring-2
For monitoring.
|
