Replacing Multiple Spinning Disks Simultaneously or Serially…

So, with a 6-drive RAIDZ2, I faced a drive failure over a year ago with a “hung” Windows host (hosting the Ubuntu Server LTS Hyper-V VM with pass-through, direct access to the 6 physical HDDs used for the RAIDZ2 array) – the Windows UI was still responsive but any drive reads (e.g. Windows Explorer navigation, starting an app) “hung” the offending app attempting the drive reads (even if the dying drive was not the drive being read from)… With 2x 6TB “spares” on hand purchased over time (2017, 2018) for just such an event, a VM-and-host shutdown, HDD swap and a quick zfs replace <pool> <old GUID> <new /dev/sdx> and a “quick” resilver brought everything back to normal.

Then, three months back, I then started facing 2 failed drives – I had the one remaining 6TB “spare” replacement drive for the first, but after a 2nd failure in the span of these three months (without purchasing another set of standby replacements), it was time to start considering replacing all the drives (slowly).

Not too shabby, with ~7+ years’ lifespan of near 24/7 powered-on, low-drive write loads, with some pretty bad temperatures (near constant 50°+C to 60°C, no matter how I tried to force air flow when these were still in the DS380):

  • 2x Seagate ST6000DX001:
    • from March 2015
      • 1x failed in August 2016; RMA/replacement still running
  • 2x Seagate ST6000DM001:
    • from November 2015
      • 1x failed in November 2022
      • 1x failed in November 2023
  • 4x Toshiba X300 HDWE160:
    • 2x from July 2016
    • 1x from November 2017 (spare)
      • 1x (surprisingly the replacement drive from November 2017 that was “just” plugged in in November 2022) (just) failed in February 2024
    • 1x from November 2018 (spare)

I therefore purchased 2x Seagate Exos X18 16TB HDDs, with another still on the way… Wanting to minimise the number of resilver attempts (straining the surviving 6TBs), I attempted to pull a working drive from the degraded 5-drive RAIDZ2 array and plugged both new 16TBs in, fingers crossed that none of the remaining 4 drives give up the ghost while resilvering (confident I had important data backed up elsewhere).

I gave the replacement commands one after another:

2024/03/03 Update: Don’t assign the whole disk, manually create a partition instead and assign that as replacement instead!

And that seems to work… So, 11+ hours later, nearing the end of the resilver process, I was eagerly checking the status…

Wha..?!? Resilvering only completed on one drive (and was only now starting on the other)!

Continue reading

Breaking My Bluetooth Disconnecting-Reconnecting Cycle…

For some odd reason, some Windows 10 update or software of some sort seems to have suddenly borked my Bluetooth connection/s, whereby any Bluetooth headset and/or speaker would disconnect and reconnect repeatedly from my Bluetooth 5.0 radio (that came with the after-market Intel AX200 Wi-Fi 6/802.11ax NGFF M.2 card I installed on my PC)… I do not have other Bluetooth device types to test, so cannot say if other devices would have also experienced the same connect/disconnect cycle.

Nevertheless, I quickly found a solution: disabling power saving on the “USB hub” devices the “Intel Wireless Bluetooth” device was hanging off… (Assuming you have administrator rights/permissions on Windows…)

Continue reading

Logitech G933 Microphone Borked by Own Logitech Software…

OK, so I love headsets that have their own microphone toggle switch that is not tied to some software switch requiring a mouse click or even a keyboard shortcut or press. The switch can be a physical in-line flick or push button switch for wired headsets, or, in my current case, a flick up/down microphone on my wireless Logitech G933 headset I purchased quite a while back…

Recently, with the pandemic raging all around and Work-From-Home (WFH) being the norm, a good headset with microphone control was a must.

Unfortunately, for some odd reason, my Logitech G933 headset microphone started to “crap out” – once muted (i.e. flipped up) the microphone would never re-activate, even with repeated power cycling of the headset. Only a reboot would work…

Frustrated, I was finally ready to throw the headset away when I tried it with another laptop… It worked without issues… I suddenly remembered about the Logitech G Hub software that was running (and not running very well at that – it hangs on the loading screen, which I previously tried troubleshooting but never got anywhere).

I killed the G Hub process and suddenly, my headset was working just fine again!

My Background…

Skeletons In My Closet

(or in my case, Diners In The Background, aka the Case of the Faceless Man)…

With the recent global pandemic, working from home means parking myself at my (messy) workbench, work laptop tucked under my triple monitor set-up… that, due to my small apartment, is actually in my dining area (with the dining table at my back). This means I (usually) do not enable my video/webcam feed during virtual meetings/teleconferences, mainly to

  • retain some modicum of privacy for the rest of my household
  • avoid the extremely distracting situation of having people walking past behind me (and given that my webcam is perched on top of my monitor that has been raised to eye height and therefore is pointing downwards, means headless torsos traversing stage left to right or vice versa)
  • avoid having some person/s sitting down at the dining table behind me, eating or whatever

Unfortunately, forced to show my ugly mug via video/webcam feed during work calls, due to guilt for not doing so while everyone else was, or because of conducting customer training, I had to scramble to find a solution to “hide” my background.

You Feel Sense Me?

I immediately thought of depth-sensing cameras, hopeful that the days of the Xbox Kinect have been left behind in the name of maturity… A full day of researching depth-sensing capable cameras came up with disappointing results, however.

The Intel D435 is the only widely available retail webcam considered “current”, with everything else, like the Intel SR305 and similar third-party Intel SR300-based products (like the Creative BlasterX Senz3D and the Razer Stargazer) currently relegated as “discontinued”/”unsupported” (due to the latest Intel SDK dropping support for SR300) or “limited support” (i.e. using only the normal, 2D RGB sensor, like any other normal HD/FHD webcam)

Countless other pages I forgot to bookmark/note down simply distilled into the single fact that dabbling with depth sensing cameras was a hit-or-miss affair, with actual software support not universal, despite Intel’s SDK having being already been provided… Maybe if someone can write a “tween” application (here or here) that would create a fake green screen that (most) other “normal” software with native support for green screens (like OBS and Zoom) could use, this may be the ideal (compared to what follows below)…

If you have to ask why: some of the solutions proffered still pick up persons moving behind me, and often incorrectly, resulting in some disembodied torso crossing behind me; the use of depth sensing would result in picking up only me, assuming some ability to control/select the distance/depth…

Since I did not want to spend money on something that may work (but most likely not), that brought me back to the “standard” webcam + “human shape/form sensing” and “‘smart‘ background removal”…

I tested several options I could find, namely:

One key point for me, which may not be applicable to you, is that I needed the software to work with BlueJeans, the “video conferencing application of choice” for work.

2021/07/01 Update: With my work moving over from BlueJeans to Zoom (citing the latter is more commonly found everywhere, probably due to their offering of a “free” tier), this is no longer a requirement. Zoom itself had introduced non-green screen background replacement shortly after this article was written also, and to date, it’s outline detection appears to equal or surpass Snap Chat’s.

2020/07/13 Update: BlueJeans surprised me today with an updated version, with a new “background” feature ala Zoom’s virtual background feature (i.e. not requiring chroma-keying/green screen)…

The good: less CPU intensive, less lag than routing through Snap Camera (as expected).

The bad: Shape/person object detection is still wanting, in comparison with Snap Camera (better) or even PerfectCam’s (best) shape detection – persons walking in my background still “appear” in BlueJeans as compared to the other two, despite the rather large distance separation.

Jump past the break to see what I eventually use now…

Continue reading

Exporting Clips Off A Dahua NVR Part Deux…

So, another day, another kid, another requirement to pull video off the DVR…

I totally forgot about my previous post and attempted to use the in-built NVR software (which requires ye ‘old Internet Explorer because it’s an ActiveX plugin) and, to my amazement, discovered that the video extraction actually works (although you still need to manually convert .dav video to something else that’s, you know, a known standard that is universally playable)… I am not sure how I missed this the last time, but oh well…

Anyway, as per usual, the quick instructions are here for posterity and future reference…

Continue reading

Playing and Converting Dahua NVR’s .dav H264 Videos…

Well, as if fighting the NVR in attempting to export video clips ain’t enough. There’s a need to convert stuff just to play ’em back…

I tried a solution using VideoLAN’s VLC Media Player, but that was a hit-or-miss affair (i.e. sometimes it worked, other times, not).

Enter FFMPEG

Continue reading

Exporting Clips Off A Dahua NVR…

I recently had an unfortunate episode requiring the exporting of a clip off a Dahua NVR, but I needed someone else to do it… (and no, I was not in jail, trying to instruct someone else to pull exonerating evidence off some CCTV, if that was what you were thinking)…

My original method of using the in-built web UI off the NVR, manual streaming conversion via VLC Player (after fiddling with it for a temporal configuration change to “enable” playback of .dav files) was just not going to “cut it”…

So, once I had the time, I waded into the battle of the forever-changing, forever-beta nature of Dahua firmware, software and hodge-podge end-user solutions…

SmartPSS

There happened to be a “new” version of the SmartPSS software (v2.002.0000008.0.T.190801); which was supposedly released a month ago (2019/08/01, as of writing this). This software was “supposed” to have had some (old) “enforced 1 hour video export” bug fixed. However, I was pulling my hair out as to why I could not seem to limit the video clip I chose to export. I finally found my clue and therefore am posting this here for posterity (actually, to enable other people to follow the same instructions to do this).

Continue reading

Forcing dhcpd On pfSense To Forget…

So I was playing around with 2 new TP-Link HS110 units I bought from Amazon, and I was attempting to swap the two with two other existing units (which had older firmware), having the two new units take the identity (name/ID and pfSense dhcpd statically defined IP addresses) of the “older” units… Review of that device aside, I was pulling my hair out because the old units kept getting their old IPs, as did the new units!

Funnily enough, the units appear to “take turns” “phoning home” – such that in the Kasa control application, there were duplicates of the old unit name/IDs shown, and it “flip-flopped” on showing the availability (and also the recorded energy consumption).

I figured it must be the DHCP service/daemon, since I kept seeing the DHCPDISCOVER and DHCPOFFER in the dhcpd logs.

Time to roll up my sleeves and fix this…

Continue reading

Unifi Controller vs. MongoDB Debacle

Ubiquiti’s Motto: If It Ain’t Broke, Don’t Won’t Fix It

After my upgrade of my Ubuntu LTS 16.04 to 18.04, I discovered some things had broken, including the Unifi Controller used for my UAP-HD. Apparently, the entire /usr/lib/unifi directory disappeared (alongside with MongoDB)!

Rooting around the Internet turned this thread up… And accordingly, there is a work-around, with some “clean-up” work.

The “official” fix is relatively useless, but that is another shouting match argument with another idiot person for a different time…

Anyway, on to the fix!

NOTE

The “fix” offerred below is not really one – it does not restore your data, although you could conceivably do so if you got creative in restoring some data files before re-installing MongoDB version 3.4 as per below…
Continue reading