I noticed that none of my Seagate Hard Drives were spinning down while being idle. This is after I upgraded to Ubuntu 20.04 as well as installing completely new drives. For some reason or other hdparm no longer seem to be working on my system.
My Linux server runs 24/7 and most of the time the hard drives are idle. So for me it is important to save as much power as needed while at the same time to not wear the drives out by constantly loading and unloading the heads.
NOTE: Some of my scripts can be found here.
What have I tried?
This being Linux, I just know things are never going to be easy and most of the time you end up deep diving and learning a ton of stuff along the way (whether it is useful or not).
- I can put all of the disks to sleep using
sudo hdparm -y /dev/___with no issues. - They don’t spin down as I would expect. Just when I think I finally nailed it, then a reboot proofs my wrong.
- Tried logging what is causing drives to spin up using a script that turns on
/proc/sys/vm/block_dumpand that monitors the power state of each drive. - Installed
inotify-toolsand wrote a script to log which directories are being accessed and at what time. This did reveal thatlost+foundis accessed at 12am and 8am everyday like clockwork. - Tried someone else’s “bug fix” for
/usr/lib/hdparm/hdparm-functions. - Took a deep dive into
hdparmand instrumented the scripts with logging to a file to see what is going on. Note that at this point I have tried many many things. - Discovered that the new Seagate IronWolf (and Pro) drives use EPC (Extended Power Conditions) to manage power states.
- Discovered that my older Seagate drives (4TB and 3TB) don’t actually support APM (advanced power management).
- Tried manually adding
udevcommands to runhdparmand also triedopenSeaChest. - And many more things that will take a whole book to document.
What worked?
- Using
openSeaChestto set the different timers for the newer Seagate IronWolf and IronWolf Pro drives. More details in the next section. - Installing Prometheus and monitoring drive power states.
- Installing and using
hd-idleto manage the older Seagate drives. - Setting the
noatimeflag when mounting theext4filesystem.noatimespecifies that the access time for files and directories should not be written. This solved thelost+founddirectories being accessed everyday at 12am and 8am and that results in the drives being spin up.
Seagate’s openSeaChest tools
Turns out that Seagate has free and open source tools for Linux to manage their drives.
https://github.com/Seagate/openSeaChest
The next steps will show you how I installed it on Ubuntu 20.04.
- openSeaChest uses meson and ninja to build.
- Install the 3rd party dependencies.
$ sudo apt-get install python3 python3-pip python3-setuptools python3-wheel ninja-build$ pip3 install --user meson- Compile openSeaChest.
$ git clone --recurse-submodules -j8 https://github.com/Seagate/openSeaChest.git$ cd openSeaChest/$ /home/andre/.local/bin/meson --buildtype=release builddir$ ninja -C builddir# All the binaries are located inside the builddir directory- Get info about a drive.
$ sudo ./openSeaChest_Info -d /dev/___ -i- Manage APM (which none of my drives support).
# Most of my drives don't have APM (so this is useless)$ sudo ./openSeaChest_PowerControl -d /dev/___ --showAPMLevel$ sudo ./openSeaChest_PowerControl -d /dev/___ --setAPMLevel 127- Check the current power state. This is handy and I ended up writing a script to capture this and export to prometheus.
$ sudo ./openSeaChest_PowerControl -d /dev/___ --checkPowerMode- Getting the EPC settings. My new drives manage power using EPC.
$ sudo ./openSeaChest_PowerControl -d /dev/___ --showEPCSettings===EPC Settings===* = timer is enabledC column = ChangeableS column = SavableAll times are in 100 millisecondsName Current Timer Default Timer Saved Timer Recovery Time C SIdle A *1 *1 *1 1 Y YIdle B *1200 *1200 *1200 4 Y YIdle C 0 6000 6000 37 Y YStandby Z 0 9000 9000 105 Y YLooking at this I can see that none of the new drives have a timer set to go into Idle C or Standby Z mode.
- Changing the Idle C and Standby Z timers. I set mine to 10 minutes for Idle C and 30 minutes to go into Standby Z. The values are in milliseconds so 5 minutes would be: ((5 * 60) * 1000) = 300000.
$ sudo ./openSeaChest_PowerControl -d /dev/... --idle_c 600000 --standby_z 1800000$ sudo ./openSeaChest_PowerControl -d /dev/... --showEPCSettings...===EPC Settings===* = timer is enabledC column = ChangeableS column = SavableAll times are in 100 millisecondsName Current Timer Default Timer Saved Timer Recovery Time C SIdle A *1 *1 *1 1 Y YIdle B *1200 *1200 *1200 4 Y YIdle C *6000 6000 *6000 37 Y YStandby Z *18000 9000 *18000 105 Y Y- This ^^ solved it for my new drives and the best is the settings survive a reboot.
noatime
Disable writing file and directory access times to disk. As mentioned earlier that directories like lost+found gets accessed periodically and then the drives get woken up just to write the access times.
$ sudo vi /etc/fstab# Add noatime option to each drive.../dev/mapper/megalodon /media/megalodon ext4 defaults,noatime 0 2smartd scans the drives every 30 minutes
https://unix.stackexchange.com/questions/366438/hard-disk-not-going-to-standby-automatically
I changed the scan time to be once every hour (eventually ended up once every 6 hours). Note: something else still seem to access the drives every 30 minutes-ish.
$ sudo vi /etc/default/smartmontools# Allow drives a chance to spindown and only scan every hoursmartd_opts="--interval=3600"$ sudo systemctl restart smartd smartmontoolhd-idle
Ok thus far I have 2x IronWolf Pro drives and 2x IronWolf drives RAID1 that spins down after applying openSeaChest. However I still have 1x 4TB and 2x 3TB drives that do not spin down for no love or money.
- Time to install hd-idle.
$ sudo apt install hd-idle- First do a debug run (e.g. not as a daemon).
# 5 minute = 300 seconds# -d = debug# -i 0 = ignore idle time for all drives# -a followed by -i = sets the idle time for a specific device$ sudo hd-idle -d -i 0 -a /dev/disk/by-id/ata-ST4000VN000-1H4168_W301BX76 -i 300- Ok this did work for me on those pesky drives and they did go to sleep after 5 minutes of idleness.
- Time to configure hd-idle to run as a daemon. Through trial and error I discovered the sweet spot to be 25 minutes for my system. Something is still keeping the drives awake around 28 – 30 minutes just before hd-idle had a chance to spin down the drives.
$ sudo vi /etc/default/hd-idle# Set HD_IDLE_OPTS# Spindown the difficult Seagate drives down after 25 minutesHD_IDLE_OPTS="-i 0 -a /dev/disk/by-id/ata-ST4000VN000-1H4168_W301BX76 -i 1500 -a /dev/disk/by-id/ata-ST3000VN000-1H4167_Z3100CPP -i 1500 -a /dev/disk/by-id/ata-ST3000VN000-1H4167_Z300PDD0 -i 1500"# Note: forgetting to daemon-reload means the service does not start in my case$ sudo systemctl daemon-reload$ sudo systemctl stop hd-idle$ sudo systemctl start hd-idle# Double check the status!$ sudo systemctl status hd-idle# You can also check the log file$ sudo journalctl -u hd-idle- After many trial and error attempts this now seems to be working!
