hastd
—
Highly Available Storage daemon
hastd |
[-dFh ] [-c
config] [-P
pidfile] |
The hastd
daemon is responsible for managing highly
available GEOM providers.
hastd
allows the transparent storage of
data on two physically separated machines connected over a TCP/IP network.
Only one machine (cluster node) can actively use storage provided by
hastd
. This machine is called primary. The
hastd
daemon operates on block level, which makes it
transparent to file systems and applications.
There is one main hastd
daemon which
starts new worker process as soon as a role for the given resource is
changed to primary or as soon as a role for the given resource is changed to
secondary and remote (primary) node will successfully connect to it. Every
worker process gets a new process title (see
setproctitle(3)),
which describes its role and resource it controls. The exact format is:
hastd: <resource name> (<role>)
If (and only if) hastd
operates in primary
role for the given resource, a corresponding
/dev/hast/<name> disk-like device (GEOM
provider) is created. File systems and applications can use this provider to
send I/O requests to. Every write, delete and flush operation
(BIO_WRITE
, BIO_DELETE
,
BIO_FLUSH
) is sent to the local component and
replicated on the remote (secondary) node if it is available. Read
operations (BIO_READ
) are handled locally unless an
I/O error occurs or the local version of the data is not up-to-date yet
(synchronization is in progress).
The hastd
daemon uses the GEOM Gate class
to receive I/O requests from the in-kernel GEOM infrastructure. The
geom_gate.ko
module is loaded automatically if the
kernel was not compiled with the following option:
options GEOM_GATE
The connection between two hastd
daemons
is always initiated from the one running as primary to the one running as
secondary. When the primary hastd
is unable to
connect or the connection fails, it will try to re-establish the connection
every few seconds. Once the connection is established, the primary
hastd
will synchronize every extent that was
modified during connection outage to the secondary
hastd
.
It is possible that in the case of a connection outage between the
nodes the hastd
primary role for the given resource
will be configured on both nodes. This in turn leads to incompatible data
modifications. Such a condition is called a split-brain and cannot be
automatically resolved by the hastd
daemon as this
will lead most likely to data corruption or loss of important changes. Even
though it cannot be fixed by hastd
itself, it will
be detected and a further connection between independently modified nodes
will not be possible. Once this situation is manually resolved by an
administrator, the resource on one of the nodes can be initialized (erasing
local data), which makes a connection to the remote node possible again.
Connection of the freshly initialized component will trigger full resource
synchronization.
A hastd
daemon never picks its role
automatically. The role has to be configured with the
hastctl(8)
control utility by additional software like ucarp
or
heartbeat
that can reliably manage role separation
and switch secondary node to primary role in case of the primary's
failure.
The hastd
daemon can be started with the
following command line arguments:
-c
config
- Specify alternative location of the configuration file. The default
location is /etc/hast.conf.
-d
- Print or log debugging information. This option can be specified multiple
times to raise the verbosity level.
-F
- Start the
hastd
daemon in the foreground. By
default hastd
starts in the background.
-h
- Print the
hastd
usage message.
-P
pidfile
- Specify alternative location of a file where main process PID will be
stored. The default location is
/var/run/hastd.pid.
- /etc/hast.conf
- The configuration file for
hastd
and
hastctl(8).
- /var/run/hastctl
- Control socket used by the
hastctl(8)
control utility to communicate with
hastd
.
- /var/run/hastd.pid
- The default location of the
hastd
PID file.
Exit status is 0 on success, or one of the values described in
sysexits(3)
on failure.
Launch hastd
on both nodes. Set role for resource
shared
to primary on nodeA
and
to secondary on nodeB
. Create file system on
/dev/hast/shared provider and mount it.
nodeB# hastd
nodeB# hastctl role secondary shared
nodeA# hastd
nodeA# hastctl role primary shared
nodeA# newfs -U /dev/hast/shared
nodeA# mount -o noatime /dev/hast/shared /shared
The hastd
utility appeared in FreeBSD
8.1.
The hastd
was developed by Pawel Jakub
Dawidek
<pjd@FreeBSD.org> under
sponsorship of the FreeBSD Foundation.