• 2 min read
  • I recently ran into an issue where a bug in one of my Docker containers would intermittently chew through CPU until restarted. I wanted Monit to automatically restart the service when it was eating CPU (which ordinarily is trivial to do), but due to the mapped volume, I only wanted it to stop & start the service if it was already running. Otherwise, Monit would proceed to start the container on boot prior to the mounted drive being present, resulting in a bunch of headaches.

    “if already running” turned out to be a little more complicated than I expected. Monit doesn’t have a good answer for this built-in, so the key is to override the start action by executing a no-op when the service isn’t running:

    only check process home-assistant MATCHING ^python.+homeassistant
       start program = "/usr/bin/docker start home-assistant"
       stop  program = "/usr/bin/docker stop home-assistant"
       if cpu usage > 13% for 3 cycles then restart
       if does not exist then exec /bin/true

    Monit considers 100% CPU usage to be full utilization on all cores, which is why you see 13% (you can also verify current service CPU usage checking the output of monit status). In my case, 13% is about 65% CPU on a single core which (over 3 minutes) I deemed enough to recognize when the bug had occurred.

    Here you see I’m also using the MATCHING syntax because the entrypoint for Docker containers may change in the future (I don’t maintain this one myself).

    The only downside to this method is that Monit will log about the service not running repeatedly until it is started. In my case, because I start docker services on boot it wasn’t an issue.