added content for nas upgrade

6 files changed, 139 insertions, 0 deletions

diff --git a/content/post/nas-upgrade/cockpit-lxc.png b/content/post/nas-upgrade/cockpit-lxc.png
new file mode 100644
index 0000000..a2a9363
--- /dev/null
+++ b/content/post/nas-upgrade/cockpit-lxc.png
diff --git a/content/post/nas-upgrade/ethernet.png b/content/post/nas-upgrade/ethernet.png
new file mode 100644
index 0000000..f39110d
--- /dev/null
+++ b/content/post/nas-upgrade/ethernet.png
diff --git a/content/post/nas-upgrade/index.md b/content/post/nas-upgrade/index.md
new file mode 100644
index 0000000..0ded387
--- /dev/null
+++ b/content/post/nas-upgrade/index.md
@@ -0,0 +1,139 @@
+---
+title: "NAS Upgrade"
+date: 2024-06-01T15:56:10-04:00
+draft: true
+description: "I upgrade my NAS hardware, and the platform it runs on."
+author: "Kumar Damani"
+---
+
+Doing it right this time.
+
+<!--more-->
+
+Upto now I've been using a [RockPro64](https://pine64.org/devices/rockpro64/) in the (excellent for starters) Pine64 [NAS case](https://wiki.pine64.org/wiki/NASCase) for my NAS.
+It runs [Armbian](https://www.armbian.com/rockpro64/), and `openzfs`, and I managed my ZFS as needed on the CLI.
+I share my datasets over NFS to my Proxmox guest VMs for access.
+This has served me reasonably well for the past few years.
+But we have a baby comming soon, and along with that a flurry of new photos and videos
+for my family to store on my NAS.
+And although I *probably* have enough buffer to survive the inital few months,
+I certainly wish to do better than my current 2xHDD mirrorred pool giving me ~4TB of storage.
+
+![nas nas](oldnas.png)
+
+# The Plan
+Luckily I already possess the hard-drives I need for my desired pool. 
+In total I have 2 HDDs from my current NAS, and 2 spares -
+all the same 4TB capacity. 
+
+I still want to keep my current NAS running as a backup store, 
+but I only need 1 drive for this purpose (for now).
+That leaves me with 3 HDDs we can use for our new `raidz1` pool which would give 
+me 2 drives for storage and 1 for parity.
+
+Now for the platform. I have been a very happy user of [Proxmox](https://www.proxmox.com/en/proxmox-virtual-environment/overview) (on my compute node) 
+and it already comes ready with ZFS support so I'll stick with that. 
+I did consider [TrueNAS](https://www.truenas.com) (both standalone, and virtualized within Proxmox with 
+disk-passthrough), but I when it comes to storage, I want as few surprises as possible 
+so I stuck with what I know. I'm already used to ZFS on the CLI and Proxmox is basically 
+Debian so if/when something goes wrong, I don't want to be messing around with an unfamiliar UI.
+
+Although I don't want to manage ZFS via a GUI, 
+I'd like a way to manage *access* to my datasets via a GUI.
+[Cockpit](https://cockpit-project.org) seems like a good light-weight choice. 
+It runs as an LXC container within Proxmox using bind-mounts for the dataset paths. 
+I can then select which paths I allow access to over my network 
+using NFS, and SMB protocols on a per-user basis.
+
+# Shopping
+Needs: 
+- `>= 3` hot-swappable HDD bays
+- `>= 16` GB RAM (for ZFS)
+- runs Proxmox
+
+Wants:
+- IPMI
+- 1U
+- 10Gb networking
+
+On Ebay, I ended up checking-out with:
+
+![nas chassis](nas-chassis.png)
+![ethernet adapter](ethernet.png)
+
+| Part | Price (shipped) |
+| -------------- | --------------- |
+| 1U 20" Short Depth Supermicro Server X9SCL-F XEON E3-1270 V2 16GB NIC Rails | $236 |
+| Mellanox ConnectX-2 PCIe x8 10Gbe SFP+ network card | $23 |
+
+which gave me everything I needed **and** wanted!
+
+# Build
+First I free one of my HDDs from my RockPro64 NAS for use in my new pool.
+This means I am running on a single HDD for the remainder of the migration.
+
+With the free HDD, and my two spares,
+I get everything assembled, and setup a new Proxmox node with a ZFS pool 
+from the UI using the 3 disks.
+
+![proxmox zfs pool](zfs-pool.png)
+
+# Sync
+To migrate all of my data from the old pool to the new pool I use
+[`syncoid`](https://github.com/jimsalterjrs/sanoid/tree/master?tab=readme-ov-file#syncoid). 
+It uses ZFS snapshots to accomplish this. Its awesome.
+
+This takes many hours depending on the size of your pool and your connection, 
+but I since I saw almost full utilization of my network link
+I was convinced its the best I can do right now.
+
+I've seen in a few Reddit posts suggesting ways this can be optimized using
+some combination of `zfs send` with `mbuffer` rather than `ssh` which
+`syncoid` uses, but I'm happy with the convenience of `syncoid`.
+
+The best part is that everything can still be running while this is going on!
+This is because only the first sync takes a long time since it needs to copy
+*everything* over. All subsequent syncs only transfer the delta 
+since the last sync which should be pretty quick if your data does not change
+all that much.
+
+That being said, since I am now running on a single drive, and hitting it hard during
+sync, I do not want to stay in this state for any longer than I need to due to
+potentially leading my single disk to failure.
+
+Once the sync is complete, I do a second sync (for the delta), and promptly 
+shutdown all of my services to prepare for the cut-over.
+
+# Cockpit
+Before I can re-enable my services. I need to expose my datasets on Cockpit.
+
+I create a Debian LXC, and assign the generated MAC a static IP on my network.
+Other than that, I just make sure my container has the NFS feature enabled:
+
+![cockpit lxc features](cockpit-lxc.png)
+
+Setting up user permissions is a pain, but once done
+I expose the relevant NFS as well as Samba paths for my services -
+basically replicating my old NAS's shared paths.
+
+# Voila
+In my homelab nameserver, I update the A record 
+for my NAS to point to the new IP of my Cockpit LXC container,
+and start my services back up.
+
+I confirm everything is still good by turning off the old NAS, and yep still good.
+
+Luckily for me all good on the first try!
+
+# Finishing touches
+I setup a cron on my old NAS to sync data nightly from the new pool.
+This will only work while my the actual data size on my new pool is less than 4TB, 
+so I'll need to get a new drive with higher capacity to keep the backups going
+in the future.
+
+Luckily I should have about a year until I exceed 4TB on the new pool so I'll be keeping 
+an eye on deals to snag an 8TB backup drive along with a few spares for my new NAS.
+
+# Resources
+- https://www.apalrd.net/posts/2023/ultimate_nas/
+- https://blog.kye.dev/proxmox-cockpit
diff --git a/content/post/nas-upgrade/nas-chassis.png b/content/post/nas-upgrade/nas-chassis.png
new file mode 100644
index 0000000..5d70b60
--- /dev/null
+++ b/content/post/nas-upgrade/nas-chassis.png
diff --git a/content/post/nas-upgrade/oldnas.png b/content/post/nas-upgrade/oldnas.png
new file mode 100644
index 0000000..15ad609
--- /dev/null
+++ b/content/post/nas-upgrade/oldnas.png
diff --git a/content/post/nas-upgrade/zfs-pool.png b/content/post/nas-upgrade/zfs-pool.png
new file mode 100644
index 0000000..ecf9605
--- /dev/null
+++ b/content/post/nas-upgrade/zfs-pool.png