ZFS

ZFS Whole Disk vs. Partition…

Posted on by dk

So, with the latest replacement of disks in my RAIDZ2, I used zpool replace <pool> <old ID> /dev/sdx. Previously, while replacing with like-sized drives, it was not an issue (unless your replacement drives had “less space”).

But using the new 16TBs, I realised that ZFS decided to create one single honking 16TB partition (and a “partition #9” 8MB “buffer”), instead of matching the required 6TB and leaving empty space for future use, even when the pool had “autoexpand=off“.

So I should have replaced using a manually created partition instead of assigning the whole disk…

Sigh… Let’s see what we can do…

Continue reading →

Replacing Multiple Spinning Disks Simultaneously or Serially…

Posted on by dk

So, with a 6-drive RAIDZ2, I faced a drive failure over a year ago with a “hung” Windows host (hosting the Ubuntu Server LTS Hyper-V VM with pass-through, direct access to the 6 physical HDDs used for the RAIDZ2 array) – the Windows UI was still responsive but any drive reads (e.g. Windows Explorer navigation, starting an app) “hung” the offending app attempting the drive reads (even if the dying drive was not the drive being read from)… With 2x 6TB “spares” on hand purchased over time (2017, 2018) for just such an event, a VM-and-host shutdown, HDD swap and a quick zfs replace <pool> <old GUID> <new /dev/sdx> and a “quick” resilver brought everything back to normal.

Then, three months back, I then started facing 2 failed drives – I had the one remaining 6TB “spare” replacement drive for the first, but after a 2nd failure in the span of these three months (without purchasing another set of standby replacements), it was time to start considering replacing all the drives (slowly).

Not too shabby, with ~7+ years’ lifespan of near 24/7 powered-on, low-drive write loads, with some pretty bad temperatures (near constant 50°+C to 60°C, no matter how I tried to force air flow when these were still in the DS380):

  • 2x Seagate ST6000DX001:
    • from March 2015
      • 1x failed in August 2016; RMA/replacement still running
  • 2x Seagate ST6000DM001:
    • from November 2015
      • 1x failed in November 2022
      • 1x failed in November 2023
  • 4x Toshiba X300 HDWE160:
    • 2x from July 2016
    • 1x from November 2017 (spare)
      • 1x (surprisingly the replacement drive from November 2017 that was “just” plugged in in November 2022) (just) failed in February 2024
    • 1x from November 2018 (spare)

I therefore purchased 2x Seagate Exos X18 16TB HDDs, with another still on the way… Wanting to minimise the number of resilver attempts (straining the surviving 6TBs), I attempted to pull a working drive from the degraded 5-drive RAIDZ2 array and plugged both new 16TBs in, fingers crossed that none of the remaining 4 drives give up the ghost while resilvering (confident I had important data backed up elsewhere).

I gave the replacement commands one after another:

2024/03/03 Update: Don’t assign the whole disk, manually create a partition instead and assign that as replacement instead!

And that seems to work… So, 11+ hours later, nearing the end of the resilver process, I was eagerly checking the status…

Wha..?!? Resilvering only completed on one drive (and was only now starting on the other)!

Continue reading →

Missing The (Mount) Point…

Posted on by dk

So my Silverstone DS-380 casing’s power LED seems to have bought it… In an attempt to try fix it (or at least test it), I had to get to the motherboard and that meant I had to remove all the drives, drive cage, etc… Since piecing everything back together again was a pain, I left the 3.5″ spinning media drives out to boot the system several times during testing.

After giving up on the power LED, I re-plugged in everything + the drives… Only to find that, of some 11 different ZFS sub pools, 10 were missing…

My heart stopped and the universe whirled around me…

zpool status showed the drives were all present and accounted for…

Thankfully, zfs list showed all my ZFS sub pools/”partitions” were still there… So, what gives?

Continue reading →

Sharing The Love…

Posted on by dk

So, the ZFS datasets have been created, and we now have to start creating the shares…

<rant>No thanks to Micro$oft, NFS support on is now only available on Enterprise versions of Windows 10, and therefore I will cannot use (the more efficient) NFS in my largely Windows environment network.</rant>

Goals

My goal was to:

  1. set up several different shares, mapping directly to the ZFS datasets
    1. this meant setting up SAMBA in a “WORKGROUP” environment
  2. allow different users to map this share on Windows, and be able to view only, or modify any objects within (add/delete/edit) according to their permissions per share

What follows is the steps required to:

  • create the SAMBA share(s) and securing it (or at least setting the correct permissions)
  • set-up the SAMBA user(s)

Note: There is a difference between Linux uid/user and gid/group and SAMBA SID/user and GID/group; within this context, I attempt to refer to the latter as “SMB User” and “SMB group” for disambiguation.

Continue reading →

RAIDZ2… Finally

Posted on by dk

Finally, we start installing the ZFS packages and start the RAIDZ2 set-up (no screenshot):

apt-get zfsutils-linux

For several reasons, I chose to use:

  • vdevs created “by-id”:
    • as per advice from multiple places online, using the “easier” /dev/sdx identifier may break things should the order be changed (e.g. you swapped cable ends or drive cages, etc.)
  • lz4 compression
    • as per statistics from some places, the “cost” of LZ4 is relatively low/negligible

Continue reading →

Reading Up On ZFS…

Posted on by dk

Attempting to implement ZFS and the decisions that go into it is enough to make any newbie give up (and/or get terribly confused)…

Unfortunately, Google does not help, what with old information (e.g. installation instructions: old vs. new; possibly outdated information) and different “package variations” of ZFS (e.g. Ubuntu-native ZFS packages vs. ZFS-native), and the endless arguments on the “correct number of disks” for RAIDZ/2 (mirrored vdevs only? RAIDZ? RAIDZ2? mirrored RAIDZ/2? block size?)…

I read through all those articles linked above, including some other helpful(?) ones:

Some other “associated” links/articles/comments/posts that I came across:

My decision?

Continue reading →