NAME

The watchdog thread

DESCRIPTION

The watchdog is forked away on startup and takes care of paused workflows. The system has a default configuration but you can override it via the system configuration.

The namespace is system.watchdog. The properties are:

max_fork_redo: Retry this often to fork away the initial watchdog process before failing finally. default: 5
max_exception_threshhold: There are situations (database locks, no free resources) where a watchdog can not fork away a new worker. After max_exception_threshhold errors occured, we kill the watchdog. This is a fatal error that must be handled! default: 10
interval_sleep_exception: The number of seconds to sleep after the watchdog ran into an exception. default: 60
interval_sleep_overload: The number of seconds to sleep after the watchdog ran into an exception. default: 15
max_tries_hanging_workflows: Try to restarted stale workflows this often before failing them. default: 3
max_instance_count: Allow multiple watchdogs in parallel. This controls the number of control process, setting this to more than one is usually not necessary (and also not wise).
default: 1
max_worker_count: Maximum number of workers that the watchdog can run in parallel. No new workflows are woke up if this limit is reached and the watchdog will sleep for interval_sleep_overload seconds.
default: 50
interval_wait_initial: Seconds to wait after server start before the watchdog starts scanning. default: 10;
interval_loop_idle: Seconds between two scan runs if no result was found on last run. default: 5
interval_loop_run: Seconds between two scan runs if a result was found on last run. default: 1

STATIC METHODS

_sig_hup

Signal handler for SIGHUP registered with the forked worker process.

Triggered by the master process when a reload happens.

_sig_term

Signal handler for SIGTERM registered with the forked worker process.

Trigger by the master process to terminate the worker.

start_or_reload

Static method to instantiate and start the watchdog or make it reload it's config.

terminate

Static method that looks for watchdog instances and sends them a SIGHUP signal.

This will NOT kill the watchdog but tell it to gracefully stop.

METHODS

run

Forks away a worker child, returns the pid of the worker

__main_loop

Watchdog main loop (child process).

Runs until the package scope variable $TERMINATE is set to 1.

__purge_expired_sessions

Purge expired sessions from backend if enough time elapsed.

__scan_for_paused_workflows

Do a select on the database to check for waiting or stale workflows, if found, the workflow is marked and reinstantiated, the id of the workflow is returned. Returns undef, if nothing is found.

__flag_for_wakeup( wf_id )

Flag the workflow with the given ID as "being woken up" via database.

To prevent a workflow from being reloaded by two watchdog instances, this method first writes a random marker to create "row lock" and tries to reload the row using this marker. If either one fails, returnes undef.

__wake_up_workflow

Restore the session environment and execute the action, runs in eval block and returns the error message in case of error.

__auto_archive_workflows

Archive workflows whose "archive_at" date was exceeded.

__flag_for_archiving( wf_id )

Flag the workflow with the given ID as "being archived" via database to prevent a workflow from being archived by two watchdog instances.

Flagging is done by updating DB field "workflow_archive_at" with an intermediate value of 0. It is updated to "null" once archiving is finished, so a permanent value of 0 indicates a severe error.

Returns 1 upon success or "undef" if workflow is/was archived by another process.