1 files changed, 148 insertions, 0 deletions
diff --git a/post/nas-upgrade.typ b/post/nas-upgrade.typ
new file mode 100644
index 0000000..9b48839
--- /dev/null
+++ b/post/nas-upgrade.typ
@@ -0,0 +1,148 @@
+#import "/global/common.typ": *
+
+#let doc = [
+Doing it right this time.
+
+Until now I've been using a
+#extlink("https://pine64.org/devices/rockpro64/")[RockPro64] in the (excellent
+for starters) Pine64 #extlink("https://wiki.pine64.org/wiki/NASCase")[NAS case]
+for my NAS. It runs #extlink("https://www.armbian.com/rockpro64/")[Armbian],
+`openzfs`, and I manage my ZFS as needed on the CLI. I share my datasets over
+NFS to my Proxmox guest VMs for access. This has served me reasonably well for
+the past few years. But we have a baby coming soon, and along with that a
+flurry of new photos and videos for my family to store on my NAS. And although
+I _probably_ have enough buffer to survive the initial few months, I certainly
+wish to do better than my current 2xHDD mirrored pool giving me ~4TB of
+storage.
+
+#fig("nas-upgrade/oldnas.png", alt: "nas nas")
+
+= The Plan
+
+Luckily I already possess the hard-drives I need for my desired pool. In total,
+I have 2 HDDs from my current NAS, and 2 spares - all the same 4TB capacity.
+
+I still want to keep my current NAS running as a backup store, but I only need
+1 drive for this purpose (for now). That leaves me with 3 HDDs we can use for
+our new `raidz1` pool which would give me 2 drives for storage and 1 for
+parity.
+
+Now for the platform. I have been a very happy user of
+#extlink("https://www.proxmox.com/en/proxmox-virtual-environment/overview")[Proxmox]
+(on my compute node) and it already comes ready with ZFS support so I'll stick
+with that. I did consider #extlink("https://www.truenas.com")[TrueNAS] (both
+standalone, and virtualized within Proxmox with disk-passthrough), but when it
+comes to storage, I want as few surprises as possible so I stuck with what I
+know. I'm already used to ZFS on the CLI and Proxmox is based on Debian so
+if/when something goes wrong, I don't want to be messing around with an
+unfamiliar UI.
+
+Although I don't want to manage ZFS via a GUI, I'd like a way to manage
+_access_ to my datasets via a GUI.
+#extlink("https://cockpit-project.org")[Cockpit] seems like a good light-weight
+choice. It runs as an LXC container within Proxmox using bind-mounts for the
+dataset paths. I can then select which paths I allow access to over my network
+using NFS, and SMB protocols on a per-user basis.
+
+= Shopping
+
+Needs:
+
+- `>= 3` hot-swappable HDD bays
+- `>= 16` GB RAM (for ZFS)
+- runs Proxmox
+
+Wants:
+
+- IPMI
+- 1U
+- 10Gb networking
+
+On eBay, I ended up checking out with:
+
+#fig("nas-upgrade/nas-chassis.png", alt: "nas chassis")
+#fig("nas-upgrade/ethernet.png", alt: "ethernet adapter")
+
+#table(
+    columns: 2,
+    [Part], [Price (shipped)],
+    [1U 20" Short Depth Supermicro Server X9SCL-F XEON E3-1270 V2 16GB NIC Rails], [\$236],
+    [Mellanox ConnectX-2 PCIe x8 10Gbe SFP+ network card], [\$23],
+)
+
+which gave me everything I needed *and* wanted!
+
+= Build
+
+First I free one of my HDDs from my RockPro64 NAS for use in my new pool. This
+means I am running on a single HDD for the remainder of the migration.
+
+With the free HDD, and my two spares, I get everything assembled, and set up a
+new Proxmox node with a ZFS pool from the UI using the 3 disks.
+
+#fig("nas-upgrade/zfs-pool.png", alt: "proxmox zfs pool")
+
+= Sync
+
+To migrate all of my data from the old pool to the new pool I use
+#extlink("https://github.com/jimsalterjrs/sanoid/tree/master?tab=readme-ov-file#syncoid")[`syncoid`].
+It uses ZFS snapshots to accomplish this. It's awesome.
+
+This takes many hours depending on the size of your pool and your connection,
+but since I saw almost full utilization of my network link I was convinced it's
+the best I can do right now.
+
+I've seen in a few Reddit posts suggesting ways this can be optimized using
+some combination of `zfs send` with `mbuffer` rather than `ssh` which `syncoid`
+uses, but I'm happy with the convenience of `syncoid`.
+
+The best part is that everything can still be running while this is going on!
+This is because only the first sync takes a long time since it needs to copy
+_everything_ over. All subsequent syncs only transfer the delta since the last
+sync which should be pretty quick if your data does not change all that much.
+
+That being said, since I am now running on a single drive, and hitting it hard
+during sync, I do not want to stay in this state for any longer than I need to
+due to potentially leading my single disk to failure.
+
+Once the sync is complete, I do a second sync (for the delta), and promptly
+shut down all of my services to prepare for the cut-over.
+
+= Cockpit
+
+Before I can re-enable my services. I need to expose my datasets on Cockpit.
+
+I create a Debian LXC, and assign the generated MAC a static IP on my network.
+Other than that, I just make sure my container has the NFS feature enabled:
+
+#fig("nas-upgrade/cockpit-lxc.png", alt: "cockpit lxc features")
+
+Setting up user permissions is a pain, but once done I expose the relevant NFS
+as well as Samba paths for my services - replicating my old NAS's shared paths.
+
+= Voila
+
+In my homelab nameserver, I update the A record for my NAS to point to the new
+IP of my Cockpit LXC container, and start my services back up.
+
+I confirm everything is still good by turning off the old NAS, and yep still
+good.
+
+Luckily for me all good on the first try!
+
+= Finishing touches
+
+I set up a cron on my old NAS to sync data nightly from the new pool. This will
+only work while the actual data size on my new pool is less than 4TB, so I'll
+need to get a new drive with higher capacity to keep the backups going in the
+future.
+
+Luckily I should have about a year until I exceed 4TB on the new pool so I'll
+be keeping an eye on deals to snag an 8TB backup drive along with a few spares
+for my new NAS.
+
+= Resources
+
+- #extlink("https://www.apalrd.net/posts/2023/ultimate_nas/")[https://www.apalrd.net/posts/2023/ultimate_nas/]
+- #extlink("https://blog.kye.dev/proxmox-cockpit")[https://blog.kye.dev/proxmox-cockpit]
+]