File Monitor Development

Bcfg2 depends heavily on file activity monitoring (FAM) to reload data from disk when it changes. A number of FAM backends are supported (documented thoroughly below), but you may wish to develop additional backends. For instance, the current best FAM backend on Linux is INotify, but if you are running a non-Linux system that lacks INotify support you may wish to write a backend for your OS (e.g., a kqueue backend for BSD-based Bcfg2 servers). This page documents the FAM API and the existing FAM backends.

Event Codes

Five event codes are generally understood:

Event Description
exists Produced when a monitor is added to a file or directory that exists, and produced for all files or directories inside a directory that is monitored (non-recursively).
endExist Produced immediately after exists. No plugins should process this event meaningfully, so FAM backends do not need to produce it.
created Produced when a file is created inside a monitored directory.
changed Produced when a monitored file, or a file inside a monitored directory, is changed.
deleted Produced when a monitored file, or a file inside a monitored directory, is deleted.

Basics

Bcfg2.Server.FileMonitor provides the support for monitoring files. The FAM acts as a dispatcher for events: An event is detected on a file (e.g., the file content is changed), and then that event is dispatched to the HandleEvent method of an object that knows how to handle the event. Consequently, Bcfg2.Server.FileMonitor.FileMonitor.AddMonitor() takes two arguments: the path to monitor, and the object that handles events detected on that event.

HandleEvent is called with a single argument, the Bcfg2.Server.FileMonitor.Event object to be handled.

Assumptions

The FAM API Bcfg2 uses is based on the API of SGI’s File Alteration Monitor (also called “FAM”). Consequently, a few assumptions apply:

  • When a file or directory is monitored for changes, we call that a “monitor”; other backends my use the term “watch,” but for consistency we will use “monitor.”
  • Monitors can be set on files or directories.
  • A monitor set on a directory monitors all files within that directory, non-recursively. If the object that requested the monitor wishes to monitor recursively, it must implement that itself.
  • Setting a monitor immediately produces “exists” and “endExist” events for the monitored file or directory and all files or directories contained within it (non-recursively).
  • An event on a file or directory that is monitored directly yields the full path to the file or directory.
  • An event on a file or directory that is only contained within a monitored directory yields the relative path to the file or directory within the monitored parent. It is the responsibility of the handler to reconstruct full paths as necessary.
  • Each monitor that is set must have a unique ID that identifies it, in order to make it possible to reconstruct full paths as necessary. This ID will be stored in Bcfg2.Server.FileMonitor.FileMonitor.handles. It may be any hashable value; some FAM backends use monotonically increasing integers, while others use the path to the monitor.

Base Classes

class Bcfg2.Server.FileMonitor.Event(request_id, filename, code)[source]

Bases: object

Base class for all FAM events.

Parameters:
  • request_id (Varies) – The handler ID of the monitor that produced this event
  • filename (string) – The file or directory on which the event was detected. An event on a file or directory that is monitored directly yields the full path to the file or directory; an event on a file or directory that is only contained within a monitored directory yields the relative path to the file or directory within the monitored parent.
  • code (string) – The event code produced. I.e., the type of event.
action = None

The event code produced. I.e., the type of event.

code2str()[source]

Return the event code for this event. This is just an alias for action.

filename = None

The file or directory on which the event was detected. An event on a file or directory that is monitored directly yields the full path to the file or directory; an event on a file or directory that is only contained within a monitored directory yields the relative path to the file or directory within the monitored parent.

requestID = None

The handler ID of the monitor that produced this event

class Bcfg2.Server.FileMonitor.FileMonitor(ignore=None, debug=False)[source]

Bases: Bcfg2.Server.Plugin.base.Debuggable

The base class that all FAM implementions must inherit.

The simplest instance of a FileMonitor subclass needs only to add monitor objects to handles and received events to events; the basic interface will handle the rest.

Parameters:
  • ignore (list of strings (filename globs)) – A list of filename globs describing events that should be ignored (i.e., not processed by any object)
  • debug (bool) – Produce debugging information about the events received and handled.
__priority__ = -1

The relative priority of this FAM backend. Better backends should have higher priorities.

AddMonitor(path, obj, handleID=None)[source]

Monitor the specified path, alerting obj to events. This method must be overridden by a subclass of Bcfg2.Server.FileMonitor.FileMonitor.

Parameters:
  • path (string) – The path to monitor
  • obj (Varies) – The object whose HandleEvent method will be called when an event is produced.
  • handleID (Varies) – The handle ID to use for the monitor. This is useful when requests to add a monitor must be enqueued and the actual monitors added after start() is called.
Returns:

Varies - The handler ID for the newly created monitor
events = None

Queue of events to handle

fileno()[source]

Get the file descriptor of the file monitor thread.

Returns:int - The FD number
get_event()[source]

Get the oldest pending event in events.

Returns:Bcfg2.Server.FileMonitor.Event
handle_event_set(lock=None)[source]

Handle all pending events.

Parameters:lock (threading.Lock) – A thread lock to use while handling events. If None, then no thread locking will be performed. This can possibly lead to race conditions in event handling, although it’s unlikely to cause any real problems.
Returns:None
handle_events_in_interval(interval)[source]

Handle events for the specified period of time (in seconds). This call will block for interval seconds and handle all events received during that period by calling handle_event_set().

Parameters:interval (int) – The interval, in seconds, during which events should be handled. Any events that are already pending when handle_events_in_interval() is called will also be handled.
Returns:None
handle_one_event(event)[source]

Handle the given event by dispatching it to the object that handles it. This is only called by handle_event_set(), so if a backend overrides that method it does not necessarily need to implement this function.

Parameters:event (Bcfg2.Server.FileMonitor.Event) – The event to handle.
Returns:None
handles = None

A dict that records which objects handle which events. Keys are monitor handle IDs and values are objects whose HandleEvent method will be called to handle an event

ignore = None

List of filename globs to ignore events for. For events that include the full path, both the full path and the bare filename will be checked against ignore.

list_event_handlers()[source]

XML-RPC that returns Bcfg2.Server.FileMonitor.FileMonitor.handles for debugging purposes.

pending()[source]

Returns True if there are pending events (i.e., events in events that have not been processed), False otherwise.

should_ignore(event)[source]

Returns True if an event should be ignored, False otherwise. For events that include the full path, both the full path and the bare filename will be checked against ignore. If the event is ignored, a debug message will be logged with debug_log().

Parameters:event (Bcfg2.Server.FileMonitor.Event) – Check if this event matches ignore
Returns:bool - Whether not to ignore the event
shutdown()[source]

Handle any tasks required to shut down the monitor.

start()[source]

Start threads or anything else that needs to be done after the server forks and daemonizes. Note that monitors may (and almost certainly will) be added before start() is called, so if a backend depends on being started to add monitors, those requests will need to be enqueued and added after start(). See Bcfg2.Server.FileMonitor.Inotify.Inotify for an example of this.

started = None

Whether or not the FAM has been started. See start().

Bcfg2.Server.FileMonitor.available = {'default': <class 'Bcfg2.Server.FileMonitor.Pseudo.Pseudo'>, 'pseudo': <class 'Bcfg2.Server.FileMonitor.Pseudo.Pseudo'>}

A dict of all available FAM backends. Keys are the human-readable names of the backends, which are used in bcfg2.conf to select a backend; values are the backend classes. In addition, the default key will be set to the best FAM backend as determined by Bcfg2.Server.FileMonitor.FileMonitor.__priority__

Existing FAM Backends

Pseudo

Pseudo provides static monitor support for file alteration events. That is, it only produces “exists” and “endExist” events and does not monitor for ongoing changes.

class Bcfg2.Server.FileMonitor.Pseudo.Pseudo(ignore=None, debug=False)[source]

Bases: Bcfg2.Server.FileMonitor.FileMonitor

File monitor that only produces events on server startup and doesn’t actually monitor for ongoing changes at all.

Parameters:
  • ignore (list of strings (filename globs)) – A list of filename globs describing events that should be ignored (i.e., not processed by any object)
  • debug (bool) – Produce debugging information about the events received and handled.
__priority__ = 1

The Pseudo monitor should only be used if no other FAM backends are available.

Fam

Gamin

Inotify