|
|
| |
CVSUPD(8) |
FreeBSD System Manager's Manual |
CVSUPD(8) |
cvsupd —
network distribution server for CVS repositories
cvsupd |
[-efv ] [-A
addr] [-b
base] [-c
collPath] [-C
maxClients] [-l
log] [-p
port] [-P
range] [-s
scanDir] [-Z
comprLevel] |
cvsupd is the server program for the
CVSup network distribution package. For information
about the companion client program, see
cvsup(1).
In normal usage, cvsupd should be started
with the ‘-C
maxClients ’ option. Unless
-f is specified, it runs as a background daemon,
fielding connection requests from remote clients. For each connection,
cvsupd forks a child process to serve the files
requested by the client.
The following options are supported:
-A
addr
- Specifies a local address (dotted quad or hostname) on which the server
will accept connections. This may be useful on hosts which have multiple
IP addresses.
-b
base
- Uses base as the base directory for the
configuration files. The default is
/usr/local/etc/cvsup.
-c
collPath
- Searches the specified directories for information about the collections
being offered. collPath contains of one or more
directory names, separated by colons. Non-absolute pathnames are
interpreted relative to the base directory. The default search path is
‘
sup ’.
-C
maxClients
- Limits the number of simultaneous clients to
maxClients. Clients beyond the specified maximum are
politely refused service.
If this option is not specified,
cvsupd serves one client in the foreground and
then exits.
-e
- Suppresses the redirection of the standard output and standard error when
running as a daemon and logging to a syslog facility. Otherwise, these
descriptors are redirected to /dev/null. This
option is useful for capturing any panic messages which might be emitted
in the rare event of a crash of the server. Such messages are very helpful
for debugging, but there is no reliable way to send them to syslog. A
recommended way to use the option is like this:
cvsupd -e ...
>>/var/tmp/cvsupd.out 2>&1
assuming that the command line syntax of
sh(1) is
used.
-f
- When
-C maxClients is
specified, causes cvsupd to stay in the foreground
instead of becoming a background daemon.
-l
log
- Directs log messages to log. If
log is of the form
@ facility (e.g.,
‘@local0 ’) then logging is done via
syslog to the indicated facility. Otherwise, log is
treated as the name of a log file. If the file already exists, new
messages are appended to its end.
For each client served, at least two messages are logged. The
first message identifies the client by username and hostname. The last
message reports the success or failure of the update and gives the total
network I/O in Kbytes (1K = 1024). Additional messages may be emitted to
report errors or other noteworthy conditions.
-p
port
- Listens for connections on an alternate TCP port. The default port is
5999. When not in passive mode, the server also uses the next lower port
to establish a second connection (called the data connection) back to the
client.
-P
range
- Sets the range of server TCP ports to be used for the data connection,
when in passive mode. Range may be a single integer
or a range given as ‘
lo-hi ’.
-s
scanDir
- Enables mirror mode, and specifies the directory under which the scan
files can be found. If it is not an absolute pathname,
scanDir is interpreted relative to the base
directory. See RUNNING A
CVSup MIRROR SITE, below.
-v
- Prints the version number and exits, without serving any clients.
-Z
comprLevel
- Sets the compression level to comprLevel. The
compression level must be between 0 and 9. A level of 0 means no
compression, while 9 provides maximum compression. The default level is 1.
Higher levels provide relatively little improvement, at a rather high cost
in CPU cycles.
The file collections which cvsupd makes available to
clients are described by various configuration files. The configuration files
reside under the directory
base/collDir, where
base is the directory specified by the
-b base command line option, or
/usr/local/etc/cvsup by default.
collDir is any of the directories specified with the
-c option, or
‘sup ’ by default.
Each collection's configuration files are kept in a separate
subdirectory of base/collDir,
named after the collection itself. For example, the configuration for the
‘src-base ’ collection would be found
in
base/collDir/src-base.
Within the collection's subdirectory, there should be a
releases file and a list file. The
releases file contains one line per release. The
first word of each line is the name of the release, e.g.,
‘cvs ’. That must be followed by the
following phrases, in any order:
list= file
- Specifies the name of the list file, relative to the collection's
subdirectory. The list file is described below.
prefix= directory
- Specifies where the files making up the collection are to be found. If
directory is not an absolute pathname, it is
interpreted relative to the base directory base. If
there is no
prefix clause, the prefix defaults to
base.
keywordprefix= directory
- Specifies a “pseudo-prefix” which is used only for expanding
the absolute pathnames for the
‘
$Header$ ’ and
‘$Source$ ’ RCS keywords. The
directory should normally be the absolute pathname
of the CVS repository on the machine containing the master copy of the
repository. The use of keywordprefix ensures that
CVSup will expand the RCS keywords identically on
all machines, even though the repositories may reside in different
directories.
super= collection
- Specifies the immediate super-collection of the current collection. In
large distributions, the collections are often organized in a hierarchical
manner. At the top of the hierarchy is a collection containing all of the
distributed files. At the next level are several sub-collections, each
containing a subset of the files. Each sub-collection may in turn have
further sub-collections, and so forth. The
super
keyword specifies the collection's parent in such a hierarchical
arrangement.
This keyword is optional. If it is omitted,
cvsupd assumes nothing about the relationship
between the current collection and the other available collections.
Information from the super keywords is
used for finding the appropriate scan files when the server is running
as a mirror site. See
RUNNING A CVSup MIRROR
SITE.
nocheckrcs
- Disables the comparison of MD5 checksums for updated RCS files. Checksum
mismatches for RCS files may not be meaningful, since a given logical RCS
file can have many different textual renditions.
norcs
- Disables special processing for RCS files. They will be treated the same
as other files.
norsync
- Disables the use of the rsync algorithm.
Note: this keyword is deprecated in the
releases file. Use
norsync
or rnorsync in the list file instead. (See
below.)
Unrecognized phrases are accepted but ignored, for backward
compatibility with the sup package. Note that even
though cvsupd often serves only a single release,
the releases file is still required.
The list file specifies the details of how to upgrade the client's
version of the collection. Each line contains a single command. Blank lines
are ignored, as are lines beginning with
‘# ’. Any filenames specified are taken
as relative to the prefix directory given in the
releases file.
Many of the list file commands accept file name patterns as
arguments. These are similar to the patterns accepted by
sh(1), and
may include the wild card constructs
‘* ’,
‘ ’?, and
‘[...] ’. With the exception of
omitany patterns, any slash characters in file names
must be matched explicitly by slash characters in the pattern. Leading
periods in file names are not treated specially. For example, the pattern
‘* ’ matches the filename
‘.profile ’.
upgrade
pattern ...
- All files and directories matching any of the given patterns will be
included in the list of files to be upgraded. If a directory name is
matched, it recursively includes all files and subdirectories within
it.
always
pattern ...
- This command is the same as
upgrade , except that
it overrides any omitany commands.
omitany
pattern ...
- Files or directories matching any of the given patterns will be omitted
from the upgrade. If a directory name is matched, then it and all files
and subdirectories beneath it are omitted.
The patterns for omitany are
interpreted differently than other patterns. For normal patterns, a
slash character in a pathname must be matched explicitly by a slash
character in the pattern. For omitany patterns,
slashes are treated the same as other characters. Thus the pattern
‘*.c ’ will match any pathname
ending in ‘.c ’, including, for
example, ‘foo/bar/lam.c ’.
symlink
pattern ...
- Symbolic links matching any of the given patterns will be upgraded as
symbolic links, rather than as the files they refer to. Otherwise,
symbolic links are followed and their target files are sent to the
client.
rsymlink
pattern ...
- This command is similar to
symlink , but if a
directory is matched, all symbolic links beneath it in the tree are
treated as matched.
norsync
pattern ...
- The rsync algorithm will not be used for updating files matching any of
the given patterns. This is useful for log files, where
CVSup 's “append” optimization is
more effective than the rsync algorithm.
rnorsync
pattern ...
- This command is similar to
norsync , but if a
directory is matched, all files beneath it in the tree are treated as
matched.
execute
command (pattern ...)
- The given command will be executed by the client
whenever a file matching one of the patterns is
successfully updated. The command comprises all
words up to the first (‘
’ character.
Any occurrences of the string ‘%s ’
are replaced by the pathname of the updated file on the client host.
Occurrences of ‘%% ’ are replaced by
‘% ’. The command is executed by
passing it to /bin/sh.
There may be multiple patterns, separated by white space. They
are interpreted relative to the prefix
directory. Each pattern should be written to match the appropriate files
as they exist on the server. For example, the
‘,v ’ suffix of an RCS file name
must be matched, even though that suffix will not be present on the
client if checkout mode is in effect. Slashes in file names must be
matched by explicit slashes in the pattern. CVS
‘Attic ’ directories are implicitly
included in the matching process, and should not be specified directly
in the patterns. A matching file will be found whether it is in the
Attic or not.
If an execute statement matches a
directory, its command is executed if the directory is created for the
first time, or if its attributes are changed. The command is executed
when ascending out of the directory, i.e., after its files and
subdirectories have been processed.
If multiple execute statements match a
file, all of the associated commands are executed in sequence.
For security reasons, the client may choose to ignore all
execute statements.
Unrecognized commands are accepted but ignored, for backward
compatibility with sup .
A mirror is a server which obtains and updates its files from a master site by
means of CVSup , and redistributes them via
CVSup to other sites. Mirror sites are commonly used
in large projects to spread the load among a number of servers. The files
being distributed originate at a master site. Each mirror site updates its own
copies of the files periodically from there. Clients in turn obtain their
updates from any of the mirror sites.
cvsupd has a special mode of operation for
mirror sites that dramatically increases its efficiency. This mode is
enabled by the -s scanDir
option on the command line. Without the -s option,
cvsupd makes a full file tree traversal over the
files in each requested collection, performing a
stat(2)
system call on every file. It does this for each client that connects to it,
on the assumption that any of the files could change at any time. Such a
traversal imposes a heavy seek load on the disks containing the files, and
limits the number of clients that can be served simultaneously.
On a mirror site, the files in the collections are known to change
only when new versions of them come in via CVSup .
The -s option allows cvsupd
to take advantage of this property to completely avoid the file tree
traversals. This reduces the disk load on the server by orders of magnitude.
In place of the file tree traversal, cvsupd gets the
necessary information about the files in the collections by reading
scan files. The scan files are created by the
cvsup client when it updates the files on the mirror
site from the originals at the master site. In
CVSUP(1),
these files are referred to as list files. Both names
refer to the same files. Each time cvsupd serves a
client, it finds the scan files created by the most recent update from the
master site. Thus the server always has fully up-to-date information about
the files in the collections, without the need to perform a file tree
traversal.
The -s option is followed by a directory
name which specifies where the scan files can be found. It is normally a
subdirectory of the base directory, and it must match the location where the
cvsup client maintains its list files. By default,
cvsup locates these files under the
sup subdirectory of the base directory. To match
this, cvsupd should be run with
‘-s sup ’. If
cvsup 's list file location is changed from the
default using the -c option, then
cvsupd 's scan directory must be changed the same
way. There is no default for the -s option. If it is
not given explicitly on the command line, no scan files are used.
There does not need to be a scan file for every collection.
cvsupd first looks for the scan file for the
collection requested by the client. If that scan file does not exist,
cvsupd tries the scan files for each successive
super-collection, and uses the first one it finds. (See the description of
the super keyword in
PREPARING A
FILE COLLECTION REPOSITORY for details.) If no suitable scan file is
located, cvsupd falls back on performing a file tree
traversal.
Access to the server is unrestricted by default, but there is a reasonably
flexible mechanism for limiting access based on the IP addresses of connecting
clients. It is enabled by placing a set of rules into the access file
base/cvsupd.access. The access
file is a text file with one rule per line. Comments begin with
‘# ’ and extend to the end of the line.
White space is ignored except where it is needed to separate adjacent tokens.
Blank lines are ignored.
Each rule consists of the following components:
- A flag indicating whether the rule is a permit rule, an
authenticate rule, or a deny rule. The
flag is expressed as a single character:
‘
+ ’ means permit,
‘* ’ means
authenticate, and
‘- ’ means
deny.
- An IP address to compare with the client's IP address to determine whether
the rule applies to the client. This may be expressed either as a numeric
IP address or as a host name. Numeric addresses consist of 1 to 4 octet
values, separated by dots. If fewer than 4 octets are specified, the
trailing octets are assumed to contain 0.
Host names are converted to numeric addresses when they are
read. If a host has multiple addresses, a separate rule is added for
each address. This may or may not have the desired effect.
Host names should be used with caution. A name that is slow to
resolve can bog down the server significantly.
- A matching mask to be ANDed with the IP addresses of the
rule and the client before comparing the addresses. This mask is specified
as ‘
/ ’ followed by an integer giving
the number of high-order 1s in the mask. For example,
‘/24 ’ specifies a mask of
0xffffff00. The matching mask is optional; if omitted,
it defaults to ‘/32 ’.
- A counting mask that determines how the clients that
match the rule are counted. (See below.) It is specified the same way as
the matching mask. The counting mask
is optional; if omitted, it defaults to the same value as the
matching mask.
- A limit specifying the maximum number of matching
clients allowed at the same time. This is specified as a decimal integer,
preceded by white space to separate it from the preceding component. The
limit is optional. If omitted, it defaults to 0 for a
deny rule, or to infinity for a permit
rule.
When a client connects to the server, its IP address is checked
against successive rules in sequence. Each rule is processed as follows:
- The IP address of the rule is compared with the IP address of the client,
after ANDing each address with the matching mask. If the
addresses do not match, the rule is ignored.
- The IP addresses of all other currently connected clients are compared
with the IP address of the connecting client, after ANDing each address
with the counting mask. If the number of matching
clients (not counting the connecting client) is less than the
limit, then the rule succeeds.
Otherwise, the rule fails.
- If the rule is a permit rule and it succeeded, the
client is allowed access, and the rest of the rules are ignored.
- If the rule is an authenticate rule and it succeeded,
the server attempts to verify the client's identity using a
challenge-response protocol (see
AUTHENTICATION, below). Access is
granted or denied based on the outcome of authentication. The rest of the
rules are ignored.
- If the rule is a deny rule and it failed, the client is
denied access, and the rest of the rules are ignored.
- Otherwise, processing continues with the next rule.
There is an implicit authenticate rule at the
end of the list which matches any IP address. Thus, if the processing
reaches the end of the list of rules without having allowed or denied
access, access is controlled by the authentication mechanism.
Here are some examples illustrating how the rules are commonly
used.
-spam.cyberpromo.com
Deny all access from a specific host.
+mirror.FreeBSD.org
Permit unlimited access from a specific host.
-user.FreeBSD.org 1
Limit a specific host to at most 1 connection at a time.
-198.211.214/24
Deny all access from hosts in a specific class C address block.
-198.211.214/24 3
Allow at most 3 connections total from the hosts in a specific class C address
block.
-198.211.214/24/32 3
Allow at most 3 connections from each of the hosts in a specific class C address
block.
Note the difference between the previous two examples. The first
example imposes a per-network limit, while the second example imposes a
per-host limit. The difference is in the counting mask.
The 24 bit mask in the first example produces a single counter that is
shared by all of the hosts in the specified address block. The 32 bit mask
in this example produces a separate counter for each host.
-0.0.0/0/24 1
Allow no more than 1 connection at a time from each block of 256 addresses.
*0.0.0.0/0
For all clients, use authentication to decide whether access is allowed.
When updating the access file, it is not necessary to halt the
server. But the file should be copied for editing, and then the new version
should be moved atomically into place. There is no need to signal the server
after updating the file. The server will notice that the file has been
touched, and will rescan it automatically. In addition, the server rescans
the file every 3 hours to keep up with DNS changes that might affect the
resolved addresses of host names.
Syntax errors in individual rules are logged, and the offending
rules are ignored. Host name lookup failures are also logged.
CVSup implements an authentication mechanism which can
be used to control access to the server. It uses a challenge-response protocol
which is immune to packet sniffing and replay attacks. No passwords are sent
over the network in either direction. Both the client and the server can
independently verify the identities of each other.
Authentication of the client is invoked by a successful
authenticate rule in the
base/cvsupd.access file, or by
“falling off the end” of the file. If there is no
cvsupd.access file, clients are not
authenticated.
The file
base/cvsupd.passwd holds the
information used for performing authentication. This file contains a record
for each client who is allowed access to the server. Each record occupies
one line in the file. Lines beginning with
‘# ’ are ignored, as are lines
containing only white space. White space is significant everywhere else in
the file. Fields are separated by ‘: ’
characters.
The first record of the file is special. It specifies the identity
of the server itself. This server record has the form:
serverName:privateKey
ServerName is the canonical name of the
server, e.g., ‘CVSup.FreeBSD.ORG ’.
This name is sent to the client, which uses it to choose an appropriate
client name and shared secret for authentication. The name is
case-insensitive.
PrivateKey is any string of characters
except ‘: ’. When the server generates
random challenges to send to the client, it uses
privateKey to make the challenges harder to guess.
Challenges are random and quite unpredictable in any case, so the
privateKey isn't really very important. It can be left
empty if desired, but the ‘: ’ that
precedes it must be present.
All of the remaining records in the file correspond to individual
clients. Each client record has the following form:
clientName
: sharedSecret
:
class
: comment
All fields must be present even if some of them are empty.
ClientName is the name of the client to which the
record applies. By convention, e-mail addresses are used for all client
names, e.g., ‘BillyJoe@FreeBSD.ORG ’.
Client names are case-insensitive.
SharedSecret is a secret string of
characters known only to the client and the server. It is generated from a
password chosen by the client, using the cvpasswd
utility. The client proves its identity to the server (and vice versa) by
demonstrating that it knows the sharedSecret. A
client's access may be disabled by changing its
sharedSecret field to
‘* ’.
The shared secret is never sent across the network, nor can it be
used to find out the client's password. However, given the shared secret, a
modified version of cvsup could impersonate the
client. Thus, care must be taken to ensure that the
cvsupd.passwd file is readable only by the
server.
Class is reserved for future use. It should
be empty.
Comment contains any additional information
about the client that might be useful to the server administrator. For
example, it may contain the client's full name and other contact
information.
When updating the cvsupd.passwd file, it
is not necessary to halt the server. But the file should be copied for
editing, and then the new version should be moved atomically into place.
There is no need to signal the server after updating the file.
Syntax errors in individual records of the
cvsupd.passwd file are logged, and the offending
records are ignored.
Here is a summary of the interactions between the access control and
authentication mechanisms. The key principle is that access control takes
place first. The outcome of access control determines whether authentication
is performed too.
- If there is no cvsupd.access file, then all
clients are granted access. No authentication is done, even if
cvsupd.passwd exists.
- If the cvsupd.access file exists but is empty, all
clients are subjected to authentication. If
cvsupd.passwd does not exist, nobody can access
the server.
- If cvsupd.access exists and has some rules in it,
but there is no cvsupd.passwd file, then
successful authenticate rules cause access to be denied.
Access is still granted to those who match successful
permit rules. Falling off the end of the
cvsupd.access file results in denial of
access.
- If both the cvsupd.access and
cvsupd.passwd files exist, then:
- Successful permit rules cause access to be granted
without authentication.
- Successful authenticate rules cause authentication
to be performed. Access is granted or denied based on the outcome of
that. Falling off the end of the cvsupd.access
file is included in this case.
- Failing deny rules cause access to be denied.
In checkout mode, CVSup expands RCS keywords as
described in
co(1). It
expands all of the standard keywords, and also the non-standard
‘$CVSHeader$ ’ keyword. This expands the
same as ‘$Header$ ’, except that the RCS
file's pathname is expressed relative to the prefix
directory, rather than as an absolute pathname. The
prefix is assumed to be the root of the CVS
repository.
It is also possible to define aliases for the standard RCS
keywords, and to selectively enable or disable the recognition of individual
keywords. These properties are controlled on a repository-wide basis by
directives in a file named
prefix/CVSROOT/options. Each
directive occupies one line of the file. Comments begin with
‘# ’ and extend to the end of line.
Blank lines are ignored. The syntax is ridiculous, for historical
reasons.
To define a keyword alias, use a line of the form:
tag=alias[= keyword]
For example:
tag=FreeBSD=CVSHeader
defines a new RCS keyword ‘$FreeBSD$ ’,
which expands the same as ‘$CVSHeader$ ’.
If the second ‘= ’ and the
keyword are omitted, the keyword defaults to
‘Id ’.
To disable all but certain selected keywords, use a line of the
form:
tagexpand=ikeyword
[,...]
For example:
tagexpand=iFreeBSD,Id
disables the expansion of all keywords except
‘$FreeBSD$ ’ and
‘$Id$ ’. The leading
‘i ’ stands for “include”.
To enable all but certain selected keywords, use a line of the
form:
tagexpand=ekeyword
[,...]
For example:
tagexpand=eFreeBSD,Id
enables the expansion of all keywords except
‘$FreeBSD$ ’ and
‘$Id$ ’. The leading
‘e ’ stands for “exclude”.
If there exists a file
base/cvsupd.HALT that is newer
than the time when the server was started, then the server will reject all new
incoming connection requests. Clients which had already started will run to
completion, but no new ones will be accepted. This mechanism is awkward and
weak, and will probably be changed in a future release.
cvsupd does not create or write any files, except for
its log file if one is specified on the command line. There is thus little
risk that cvsupd can be subverted into damaging the
system on which it is running. A more likely risk is that
cvsupd might be spoofed into sending out files that
are not intended to be publicly distributed. cvsupd is
very careful to prevent this from happening. Nevertheless, for maximum
protection it is a good idea to run cvsupd as a
completely unprivileged user analogous to
‘nobody ’, serving only files that are
readable by everyone.
CVSup has no provision for encrypting the
data sent across the network. If secrecy is desired then the connection can
be tunneled through ssh .
- /usr/local/etc/cvsup
- Default base directory.
- sup
- Default collDir subdirectory.
- base/collDir/collection/releases
- Releases file.
- base/collDir/collection/list
- List file.
- base/cvsupd.HALT
- Shutdown file.
- base/cvsupd.access
- Access control file.
- base/cvsupd.passwd
- Authentication password file.
- prefix/CVSROOT/options
- RCS keyword configuration file.
John Polstra ⟨jdp@polstra.com⟩.
CVSup is a registered trademark of John D. Polstra.
An RCS file is not recognized as such unless its name ends with
‘,v ’.
Any directory named ‘Attic ’
is assumed to be a CVS Attic, and is treated specially.
Visit the GSP FreeBSD Man Page Interface. Output converted with ManDoc. |