Proxmox UPS Graceful Shutdown Integration with NUT

This setup integrates Proxmox with the Network UPS Tools (NUT) system to perform a graceful shutdown of all VMs and containers in case of a power outage, and optionally cancel shutdown if power is restored.

This guide focuces only on the client side (Proxmox), if you need a guide for the server part check out this excellent guide by Techno Tim!

To make the best of this guide, it’s better to have a reparate machine running nut-server like a Raspberry Pi.

📁 Files Included

You can find all files on my github repo here

• upsmon.conf - NUT monitoring client configuration.

RUN_AS_USER root
MONITOR apc-modem@ip.address.of.nut.server 1 admin secret slave
MINSUPPLIES 1
SHUTDOWNCMD /usr/local/sbin/pve-shutdown.sh
NOTIFYCMD /usr/sbin/upssched
POLLFREQ 2
POLLFREQALERT 1
HOSTSYNC 15
DEADTIME 15
POWERDOWNFLAG /etc/killpower
NOTIFYMSG ONLINE    "UPS %s on line power"
NOTIFYMSG ONBATT    "UPS %s on battery"
NOTIFYMSG LOWBATT   "UPS %s battery is low"
NOTIFYMSG FSD       "UPS %s: forced shutdown in progress"
NOTIFYMSG COMMOK    "Communications with UPS %s established"
NOTIFYMSG COMMBAD   "Communications with UPS %s lost"
NOTIFYMSG SHUTDOWN  "Auto logout and shutdown proceeding"
NOTIFYMSG REPLBATT  "UPS %s battery needs to be replaced"
NOTIFYMSG NOCOMM    "UPS %s is unavailable"
NOTIFYMSG NOPARENT  "upsmon parent process died - shutdown impossible"
NOTIFYFLAG ONLINE   SYSLOG+WALL+EXEC
NOTIFYFLAG ONBATT   SYSLOG+WALL+EXEC
NOTIFYFLAG LOWBATT  SYSLOG+WALL
NOTIFYFLAG FSD      SYSLOG+WALL+EXEC
NOTIFYFLAG COMMOK   SYSLOG+WALL+EXEC
NOTIFYFLAG COMMBAD  SYSLOG+WALL+EXEC
NOTIFYFLAG SHUTDOWN SYSLOG+WALL+EXEC
NOTIFYFLAG REPLBATT SYSLOG+WALL
NOTIFYFLAG NOCOMM   SYSLOG+WALL+EXEC
NOTIFYFLAG NOPARENT SYSLOG+WALL
RBWARNTIME 43200
NOCOMMWARNTIME 600
FINALDELAY 5

• upssched.conf - Scheduler rules for conditional shutdowns.

CMDSCRIPT /etc/nut/upssched-cmd.sh
PIPEFN /var/run/nut/upssched.pipe
LOCKFN /var/run/nut/upssched.lock
AT COMMBAD * EXECUTE oncommbad
AT COMMOK * EXECUTE oncommok
AT ONLINE * EXECUTE online
AT ONBATT * START-TIMER onbatt 5
AT ONBATT * START-TIMER shutdown30 60
AT LOWBATT * EXECUTE lowbatt
AT FSD * EXECUTE forcedshutdown

• upssched-cmd.sh - Script triggered by upssched events.

#!/bin/bash
LOGFILE="/var/log/upssched.log"
UPS="[email protected]"

echo "[$(date)] upssched-cmd triggered: $1" >> "$LOGFILE"

case $1 in
  shutdown30)
    charge=$(upsc $UPS battery.charge 2>/dev/null)
    echo "[$(date)] Battery charge: $charge%" >> "$LOGFILE"

    if [ -n "$charge" ] && [ "$charge" -le 30 ]; then
        echo "[$(date)] Battery at or below 30%. Initiating shutdown." >> "$LOGFILE"
        /usr/local/sbin/pve-shutdown.sh
    else
        echo "[$(date)] Battery still above 30%. Shutdown canceled." >> "$LOGFILE"
    fi
    ;;
  lowbatt)
    echo "[$(date)] LOWBATT received. Forcing shutdown." >> "$LOGFILE"
    /usr/local/sbin/pve-shutdown.sh
    ;;
  forcedshutdown)
    echo "[$(date)] FSD received. Forcing shutdown." >> "$LOGFILE"
    /usr/local/sbin/pve-shutdown.sh
    ;;
  *)
    echo "[$(date)] Unknown event: $1" >> "$LOGFILE"
    ;;
esac

• pve-shutdown.sh - Graceful shutdown logic for VMs, CTs, and power state checks.

#!/bin/bash
# Place the script in /usr/local/sbin/
LOGFILE="/var/log/pve-shutdown.log"
UPS_NAME="[email protected]"
GRACE_PERIOD=180
CHECK_INTERVAL=10

echo "[$(date)] Starting Proxmox shutdown procedure via NUT" >> "$LOGFILE"

echo "[$(date)] Shutting down all VMs..." >> "$LOGFILE"
for vmid in $(qm list | awk 'NR>1 {print $1}'); do
    echo "[$(date)] Shutting down VM ID $vmid" >> "$LOGFILE"
    qm shutdown $vmid &
done

echo "[$(date)] Shutting down all LXC containers..." >> "$LOGFILE"
for ctid in $(pct list | awk 'NR>1 {print $1}'); do
    echo "[$(date)] Shutting down CT ID $ctid" >> "$LOGFILE"
    pct shutdown $ctid &
done

echo "[$(date)] Waiting for all guests to shut down..." >> "$LOGFILE"
timeout=300
while [ $timeout -gt 0 ]; do
    running_vms=$(qm list | awk 'NR>1 && $3 == "running" {print $1}')
    running_cts=$(pct list | awk 'NR>1 && $2 == "running" {print $1}')

    if [ -z "$running_vms" ] && [ -z "$running_cts" ]; then
        echo "[$(date)] All guests shut down successfully." >> "$LOGFILE"
        break
    fi

    echo "[$(date)] Still shutting down... ($timeout seconds left)" >> "$LOGFILE"
    sleep 10
    timeout=$((timeout - 10))
done

echo "[$(date)] Entering $GRACE_PERIOD second grace period before shutdown." >> "$LOGFILE"
remaining=$GRACE_PERIOD
while [ $remaining -gt 0 ]; do
    status=$(upsc "$UPS_NAME" ups.status 2>/dev/null)

    if echo "$status" | grep -q "OL"; then
        echo "[$(date)] Power returned. Canceling shutdown." >> "$LOGFILE"
        for vmid in $(qm list | awk 'NR>1 {print $1}'); do
            echo "[$(date)] Restarting VM ID $vmid" >> "$LOGFILE"
            qm start $vmid
        done

        for ctid in $(pct list | awk 'NR>1 {print $1}'); do
            echo "[$(date)] Restarting CT ID $ctid" >> "$LOGFILE"
            pct start $ctid
        done

        echo "[$(date)] Shutdown canceled. System remains up." >> "$LOGFILE"
        exit 0
    fi

    echo "[$(date)] Power still out. ($remaining seconds left)" >> "$LOGFILE"
    sleep $CHECK_INTERVAL
    remaining=$((remaining - CHECK_INTERVAL))
done

echo "[$(date)] Grace period over. Power not restored. Proceeding with shutdown." >> "$LOGFILE"
shutdown -h now

---

🧰 Setup Instructions

1. Copy the Config and Scripts

cp upsmon.conf /etc/nut/upsmon.conf
cp upssched.conf /etc/nut/upssched.conf
cp upssched-cmd.sh /etc/nut/upssched-cmd.sh
cp pve-shutdown.sh /usr/local/sbin/pve-shutdown.sh
chmod +x /etc/nut/upssched-cmd.sh /usr/local/sbin/pve-shutdown.sh

2. Make Sure upsmon Runs as Root

Ensure upsmon.conf contains:

RUN_AS_USER root

3. Update UPS Server Address

Replace ip.address.of.nut.server in all files with the actual IP or hostname of your NUT server.

---

⚙️ Configuration Overview

upsmon.conf Highlights

• Monitors remote UPS via MONITOR line.
• Triggers /usr/sbin/upssched for advanced scheduling.
• Executes shutdown via pve-shutdown.sh when needed.
• Notifies system users via WALL, logs events, and runs scripts.

upssched.conf Behavior

Schedules actions on power events:

• Starts a 60-second timer after ONBATT to check battery level.
• Executes shutdown if battery is at or below 30%.
• Executes immediate shutdown on LOWBATT or FSD.

upssched-cmd.sh Logic

• Logs all events.
• Executes pve-shutdown.sh if battery.charge <= 30.
• Cancels shutdown if power returns.

pve-shutdown.sh

1. Shuts down all VMs and containers.
2. Waits up to 5 minutes for graceful shutdown.
3. Waits a configurable grace period (default: 180 seconds).
4. Cancels shutdown and restarts services if power returns.
5. Executes shutdown -h now if not.

---

🔄 Testing the Setup

1. Simulate a power failure by disconnecting the UPS from wall power.
2. Observe /var/log/upssched.log and /var/log/pve-shutdown.log.
3. Confirm proper shutdown behavior and power restoration handling.

---

✅ Final Notes

• Ensure upsc is installed for UPS status checks.
• Adjust timing and thresholds as needed in the script.
• Test in a safe environment before production rollout.