NAME

SYNOPSIS

DESCRIPTION

The qjail utility is used to manage the qjail environment and all the jails inside of qjail's scope. Qjail's administration ease does not evaporate as jails deployed grow beyond 5 jails. For the deployment of a large number of jails, qjail provides two facilities designed to make their management easy. The First facility is the group prefix selection ability, which is advantageous in managing both small and large jail deployments. The group prefix equal sign "=" wildcard when used on the jailname allows for management of jails based on common jailname group prefixes. The second facility is qjail's ability to create multiple unique jail environments, thus providing another method to group common jails together for easier management. A large deployment of hundreds of jails is possible if your host system resources are adequate and a jail naming convention is used to segregate jails into manageable groups.

This utility deploys two different jail types. The first type is based on a Directory tree. This type has unlimited disk space growth potential, it shares the host's disk space. The jail will never run out of space until the host does. The second type is based on a sparse image file. A sparse file is one that occupies only the sum size of its contents, not its allocation size. IE; a sparse file allocated size of 5M, but only having 7 files, each 1k in size, only occupies 7k of physical disk space. As content is added, additional physical disk space is occupied up to the 5M allocation ceiling. The sparse file is mounted as a memory disk using the mdconfig command and populated with the directory tree content of a jail. This configuration is called a sparse image jail. Its major benefits is it provides a way to put a hard limit on the maximum amount of disk space a jail can consume. This provides an addition level of protection to the host from intentional or unintentional run-a-way processes inside of a jail consuming disk space until the host system dies.

Adding qjail_enable="YES" to the "host's" /etc/rc.conf file, will cause all jails to be started when the system is booted.

Following the command "qjail" is the function subcommand. Each function subcommand has its own list of unique options. Qjail is executed from /usr/local/bin/ and is a command interpreter Bourne type (shell) script that has to be run from user root.

qjail install

In most cases the production versions of the operating system is used. These are identified by versions labeled as "X.X-RELEASE" and have both iso image files and original distribution files available for download from the FreeBSD FTP servers.

The pre-release versions "X.X-BETAx and X.X-RCx" iso image files and original distribution files are only made available by Release Engineering during the process of creating the next production version and are removed after the new production version is published. If your host is running one of these pre-release versions, qjail will use it's distribution file. It's the user's responsibity to update the host to the newly published production version.

The Development "X.X-STABLE, and X.X-CURRENT" branchs have snapshot versions of the iso image files and original distribution files that are recreated weekly. These Development branchs are the bleeding edge and may cause problems for qjail. If your host is running one of these and you want to run qjail. Manually fetch distribution base.txz file from the FTP snapshot directory that matches what the host is running. Use option -f to create a clean qjail system. If you compiled your OS from source, most likly it's revision number is out of sync with the snapshot revision number. To update the qjail system to same revision number as what's running on the host. Issue 'qjail update -b' command to clone the host's running libs to the sharedfs directory tree.

During the "qjail install" process the following directory structure is allocated with the default path of /usr/jails. This default may be modified by following comments in the qjail script around line 43.

sharedfs contains all of the operating system's executable libraries as read-only files and is mounted as an "nullfs" that is shared between all the individual jails. It's populated with a pristine version of the operating systems binaries. This design effectively secures all the executable files from being updated or deleted and also secures the directories containing the executable files from having new files inserted by any process running inside of the jail. The "/usr/src" and "/usr/ports" directories are also included. The hosts "/usr/ports" filesystem can temporarily be made available to the jails by using the "mv" command like this: mv /usr/ports /usr/jails/sharedfs/usr and returned doing mv /usr/jails/sharedfs/usr/ports /usr

Or use "update" subcommand to populate src and/or ports permanently.

template contains the operating system configuration files. It is copied to form the base filesystem of each jail.

archive is where the archive files are stored that are created by the "qjail archive" command.

flavors contains the "default" and "ssh-default" system flavors and any user created custom flavors.

Six internal administration directories are created and get populated with information unique to each jail. /usr/local/etc/qjail.config /usr/local/etc/qjail.fstab /usr/local/etc/qjail.local /usr/local/etc/qjail.global /usr/local/etc/qjail.packages /usr/local/etc/qjail.vnetctl

This command can be run any time to rebuild the sharedfs and the template from scratch while not disturbing the existing jails, or your customized flavors. The "default and ssh-default" flavors are renamed with "users.saved." prefix before being replaced with fresh versions.

If rebuilding using a newer major RELEASE, IE: 10.3 to 11.0, then remember, all existing jails that have ports or packages in them will need them updated to versions compatible with the new major RELEASE version. If going from a subversion to a newer subversion within the same major RELEASE, IE: 10.0 to 10.1, then there is no need to update your installed ports/packages.

The options are as follows:

-z

Code this option to create multiple unique qjail environments. The coded zone value is appended to /usr/jail as /usr/jail.zone and to /usr/local/etc/fstab.qjail.zone and /usr/local/etc/qjail.local.zone which uniquely segregates the qjail environments. All ". - /" in the zone name are converted to "_" underscores to standardize zone names. All the other qjail subcommands "MUST" code the same zone value to process against the zone created here. If absent /usr/jails and /usr/local/etc/qjail.fstab and /usr/local/etc/qjail.local/ are used.

-h

Code the URL of the remote host to fetch the original distribution files from. If this option is absent the default host ftp2.freebsd.org is used. You may change the default using the -h ftp7.freebsd.org option. Read this for complete list of FTP servers to choose from. www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/mirrors-ftp.html

-f

Code the complete path to the location where any of three RELEASE sources are to target as the source to populate qjail's directory structure from. That could be the mounted disc1 cdrom, or the downloaded disc1.iso image file, or the downloaded original distribution files.

-l

This enables logging of all qjail commands and error messages to /var/log/qjail.log file. Each log entry is prefixed with a date/time stamp and the user account name of the user entering the commands. An entry is also made in /etc/newsyslog.conf to auto rotate the qjail.log file.

qjail install examples

2. qjail install -h ftp6.freebsd.org -l
Same behavior as above, except the FreeBSD ftp server specified
in the -h option is used, and the qjail system wide logging
is enabled.

3. mount_cd9660 /dev/cd0 /mnt
qjail install -z env1 -f /mnt/usr/freebsd-dist
Use this option to target a mounted disc1 RELEASE cdrom
as the source of the original distribution files used to
populate the qjail directory structure. Plus a uniquely named
qjail zone is created named "env1".

After the install completes, execute the following commands
to release the disc1 RELEASE cdrom.
cd /usr
umount /mnt

qjail create

During the creation process four administration files are created for each jail. They are /usr/local/etc/qjail.fstab/jailname file, /usr/local/etc/qjail.local/jailname file, /usr/local/etc/qjail.global/jailname file, and the /usr/local/etc/qjail.config/jailname file.

The options are as follows:

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

-n

This is the "network interface name" servicing the jails IP addresses. When qjail starts a non-vnet jail it will automatically create an alias for the jails IP address on that "network interface name". When qjail stops the jail, it will automatically remove the alias. If "-n value" is absent, the "route command" is used to identify the default network interface device name which is the device connected to the public internet and automatically populate the "-n value".

For multiple static public routable IP addresses, the correct "network interface name" to code is the name of the NIC facing the public internet where these IP addresses enter your host. For jails on the hosts private LAN, the correct "network interface name" to code is the name of the NIC facing the hosts private LAN where those IP addresses exit and enter your host. For jails assigned IP addresses reserved for private LAN use to be able to access the public internet, you must configure your firewall to perform NAT on them. See -4 option for more details.

Very important CAUTIONARY note: Be aware of the LAN IP address range your DHCP server is dynamically assigning. Do not assign those IP addresses to jails or your LAN users will instantly lose their network access when the jail is started and its alias gets created.

-a

You can use an archive file as the template to create your new jail from. If just the archived jailname is coded, then the most current archive file matching that jailname will be used as the source. The full archive file name can also be coded. Its prefixed with the jailname and has the date & time the archive was created appended as a suffix. Coding the full archive file name is how you select an archive file other than the most current one. This option is normally used to clone multiple jails with the same status as the archived jail has. If the -a flag is absent, the template is used. Note: The -a and -f options cannot be used together. By design jails created from a archive file cannot be flavored. Use "ls /usr/jails/archive/" to list all archive file names.

An archive of a image jail can be used to create a new directory tree jail or a new image jail with a larger sized sparse file image jail. An archive of a directory tree jail can be used to create a new directory tree jail or a new image jail. The -n interface nic name from the archive file is dropped.

-A

Using the -A archive zone option you can select an archive file from a different zone as the template to create your new jail from. The -A option is only valid when used together with the -a option. Using a combination of the -z, -a and -A options allows the selection of archive files from other zones.

The default jail environment IE; no -z option coded on the qjail install command is a special case. Manipulation of the values in -z, -a, and -A covers everything but selecting a jail archive from the default jail environment. A special reserved value named nozone may be coded in the -A value to point to the default jail environment archives. Normally the -z value is the zone the create command applies to, the -A value is the zone where the selected archive file is located, and the -a value is the archive jailname or full archive file name in the zone pointed by the -A value.

-f

Using the flavor option you can apply an qjail flavor to your new jailname. If the -f flavor option is coded, the selected flavor directory tree is merged into the new jail's directory tree. If no flavor option is coded, the "default" flavor is merged into the new jail's directory tree. Qjail has no function to delete unwanted flavor directories. It's the users responsibility to delete unwanted flavor's using the host's rm -rf /user/jails/flavor/name command. Note: The -f and -a options cannot be used together. By design jails created from a archive file cannot be flavored.

As part of the "install" subcommand, a flavor base directory was created as /usr/jails/flavors and populated with two flavors, one named default and the other named ssh-default. Both of these flavors contains 3 files customized for running in a jail (make.conf, periodic.conf, rc.conf). In addition these customized host files /etc/resolv.conf and /etc/localtime are copied to default and ssh-default to facilitate jail usage. On inspection you will see that these files are in their normal directory tree locations. When customizing your own flavors you have to manually create your own flavor directory tree populating it with your customized files in their correct paths for merging into the new jail.

The ssh-default flavor contains everything the default flavor has, but in addition it has been customized to enable ssh support, and has a predefined standard user account named qjail with a password of "qjail". Every jail you use this ssh-default flavor on will have this predefined standard user account qjail. On first login the user will be prompted to enter a new password to address basic security concerns. The qjail user belongs to the "wheel" group so it has "su" access to "root".

When creating your own flavor always copy the "default" flavor or the "ssh-default" flavor as your starting base.

-c

This option will enable ssh and create a user account having the login ID and password of the jailname. To address basic security concerns, on first login the user will be prompted to enter a new password. The jailname user account belongs to the "wheel" group so it has "su" access to "root".

When the jails created with the -c option are started for the first time, the changes to configure ssh and create the user account for that jail are applied. Doing a qjail restart jailname or a qjail stop jailname followed by qjail start jailname is required to enable the changes which will be in effect from that point on.

-i

When coded means create a sparse file image type jail. When absent an directory tree type jail is created. When the -i option is coded, it must be followed by a size value which is the allocation ceiling size of the sparse file. Only suffixes m|M for megabytes or g|G for gigabytes are valid entries. The sparse image file has a .img suffix and resides in the jailname directory as a single file. When the image jail is stopped the jailname.img file will be visible. Issuing ls -lh jailname.img will show you the allocated size, issuing du -h jailname.img will show you the amount of space used. If a image jail should consume all of its disk space allocation, you can increase it by following this procedure, archive it, delete it, and create it using the -a option, using the image archive as input with a larger -i value. A -i value of 10m is the bare minimum size for a image jail.

-d

Enter a numeric number representing the number of times you want this jailname duplicated. A suffix number starting at one and incremented by one for each duplication is appended to each newly created jailname. Any number greater than 100 is invalid. A single IPv4 address is required. For each repetition of the duplication cycle the last octal of the IPv4 address increments by 1.

-4

This is the network address used to communicate with the jail. Both non-vnet jails and vnet jails use it for the same purpose. This is either a public IPv4 address or a private IPv4 address. More than a single IPv4 address can be assigned to a jail. Multiple IPv4 addresses have to be a list of IP addresses separated by a comma "," without spaces before or after. Example 10.0.0.2,10.0.0.3,10.0.0.4 A second format is also available "xl0|10.0.02,lo1|127.0.2.1" To the left of the "|" symbol is the network device name that the ip address is assigned to. Note the surrounding " " they are required.

According to RFC 1918, you can use the following IP address ranges for private IPv4 networks which will never be connected to the Internet. This is normally intended for Local Area Networks.
#
# 10.0.0.0 - 10.255.255.255
# 172.16.0.0 - 172.31.255.255
# 192.168.0.0 - 192.168.255.255
#

Static IP address (permanent, never changes) public Internet routable IP addresses are assigned to you by your ISP. If you purchased a continuous block of static public internet routable IP addresses, then each jail could be assigned one of those individual IP addresses from that block.

Normally cable providers and DSL providers assign dynamic IP addresses. The assigned IP address may change when the lease time expires or you reboot your system.

-6

This is a IPv6 address that is to be assigned to the jail. More than a single IPv6 address can be assigned to a jail. Multiple IPv6 addresses have to be a list of IP addresses separated by a comma "," without spaces before or after. Both IPv4 and IPv6 addresses may be assigned to a jail by coding both the -4 and -6 options. An single IPv4 address is required if the -d option is coded to enable jail duplication.

jailname

Only a single jailname is valid. The jailname can only contain alphanumeric, dash, and underscore characters, all numeric jailnames are invalid. To better manage large jail deployments a jail naming convention that groups jails by common function or user groups is advised. The maximum jailname size is 50 characters. Jailnames have to be unique across all the zones. Just remember that you will be typing in this jailname or some prefix of it on all the subcommands you use, so try to keep the jailname short but meaningful.

Jails are started, stopped, and restarted in ascending alphabetical order, "a to z" based on the spelling of the jailname. If you want selected jails to start before other jails prefix those jailnames with numbers.

qjail create examples

2. qjail create -n rl0 -c -f myflavor -4 10.0.10.20 bld21a-floorA-cell01
This creates a single new directory tree type jail as
/usr/jails/bld21a-floorA-cell01 from the template
and copies the myflavor directory tree onto the
bld21a-floorA-cell01 directory tree.
The auto alias function is enabled and ssh access is enabled.

3. qjail create -a cell-a -4 10.0.10.20,10.0.10.30 prison-B
This creates a single new directory tree type jail as
/usr/jails/prison-B using the archive file named cell-a as
the template directory tree for the new jailname.
The auto alias default function is enabled.

4. qjail create -a cell-a -d 15 -4 10.0.10.20 room
This creates a new directory tree type jail using the archive
file named cell-a as the template for the new jailname, and
then duplicates it 15 times.
Creating jailnames room-1 through room-15.
At the same time the last octet of the IP address
10.0.10.20 is incremented by one.
room-1 10.0.10.20 room-2 10.0.10.21 room-15 10.0.10.34
The auto alias default function is enabled

5. qjail create -n rl0 -d 15 -4 10.0.10.20 room
This creates a new directory tree type jail using the
template directory tree for the new jailname, and then
duplicates it 15 times creating jailnames
room-1 through room-15. The auto alias function is enabled
At the same time the last octet of the IP address
10.0.10.20 is incremented by one.
room-1 10.0.10.20 room-2 10.0.10.21 room-15 10.0.10.34

6. qjail create -n rl0 -d 15 -c -4 10.0.10.20 room
This does the same as the previous one except these jails
also has ssh access enabled.

7. qjail create -i 100m -4 10.0.10.20 class
This creates a single new sparse image type jail using the
template directory tree to populate the image with a
maximum allocation size of 100 megabytes.
The auto alias default function is enabled

8. qjail create -d 15 -c -i 100m -4 10.0.10.20 class
This does the same as the previous one except this jail
also has ssh access enabled, and duplicates it self
15 times creating jailnames class-1 through class-15.
At the same time the last octet of the IP address
10.0.10.20 is incremented by one giving.
class-1 10.0.10.21 class-2 10.0.10.22 class-15 10.0.10.34

9. qjail create -c -a cell-a -i 1g -4 10.0.10.20 room
This creates a new single sparse image type jail with a
maximum allocation size of 1 gigabyte, using the archive
file named cell-a as the template directory tree for
populating the image jail.
The auto alias default function is enabled and ssh access
is enabled.

10. qjail create -z env1 -a cell-a -i 1G -4 10.0.10.20 room
This does the same as the previous one except this jail is
being created in the "env1" zone.

11. qjail create -z env1 -a cell-a -A env2 -4 10.0.10.20 room
This creates a new jail named "room" in zone "env1" using a
archive file named "cell-a" which is located in zone "env2".

12. qjail create -z env1 -a cell-0 -A nozone -4 10.0.10.20 room
This creates a new jail named "room" in zone "env1" using a
archive file named "cell-0" which is located in the qjail
default environment. The "nozone" keyword is used.

qjail list

The list displays 5 columns of information. Starting from the left, Column (1) Contains status flags consisting of 6 positions.   Position one can be the letter (D) for Directory tree based jail, or the letter (I) for a image file based jail. Position two can be the letter (R) meaning the jail is currently running, or the letter (S) meaning the jail is stopped. Position three through six can contain optional letters depending on if the option is emabled. The letter (N) means the jail is in norun status or the letter (M) which means the jail is in manual start only status. The letter (V) means this is a vnet jail, and it's followed by a space and number 0 through 3 indicating which firewall is enabled in the vnet jail. 0-none, 1=ipfw, 2=pf, 3=ipfilter. See the qjail config subcommand
-r -R -m -M -v -V for details Column (2)   Contains the jail's jid number if the jail is started, and N/A for not available if the jail is stoped.   Column (3) is the network interface device name, this is the value entered on the "create -n" option, or if a vnet jail the "config -w option. If the "create -n value" was not used, the "route command" is used to identify the default network interface device name which is the device connected to the public internet and then automatically populates the "-n value". Column (4)   is the jails IPv4 and IPv6 address or addresses, entered with the "create" command.   Column (5) is the jails jailname.

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone. When this option is coded an addition heading "Jails in zone xxxx" displays right above the normal heading. "xxxx" is the zone name.

jailname

If absent all the jails are listed. Multiple jailnames separated by a space are allowed on the command. The group prefix option is enabled. xxxx= will cause only those jailnames matching the xxxx characters to be selected for processing. The equal sign "=" is the wildcard symbol that signifies all the characters to its left are to be used to match on jailname to create a list of jailnames to be processed.

qjail [start | stop | restart] jailname.....

Jails are started, stopped, and restarted in ascending alphabetical order, "a to z" based on the spelling of the jailname. If you want selected jails to start before other jails prefix those jailnames with numbers. All norun status jails are ignored.

The function subcommands are as follows:

start Start all jails at once if jailname is absent.

stop Stop all jails at once if jailname is absent.

restart Restart all jails at once if jailname is absent.

The options are as follows:

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

jailname

If absent all the jails are used. Multiple jailnames separated by a space are allowed on the command. The group prefix option is enabled for these subcommands. xxxx= will cause only those jailnames matching the "xxxx" to be selected for processing. The equal sign "=" is the wildcard symbol that signifies all the characters to its left are to be used to match on jailname to create a list of jailnames to be processed. Use the qjail "list" subcommand to list all the jails under qjail's scope.

qjail console

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

-c

This option is for submitting console commands to a jail. If the command has arguments they need to be enclosed in quotes. This option can be used with the -U option to target other user ids. Some examples; -c 'ls -l' or -c "ps ax" or -c ls

-u

If this is absent, the /usr/bin/login -f root command is executed logging you in as root. Coding the -u option with the user id of some user account already created in the jail will cause the console to be attached to that user. The "Welcome" message is displayed.

jailname

Jailname is a mandatory parameter. Only a single jailname is valid. Use the subcommand list to display list of all jailnames.

qjail archive

There is no qjail function to delete archive files. It's the users responsibility to delete unwanted archives using the host's rm command. It's also the user responsibility to keep a log of archive file names with a description of why the archive was created, so the correct archive can be restored if desired.

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

-s

This option makes a backup of the qjail internal definition and control files to the archive directory as "system.bkup" plus the qjail executable scripts to the archive directory as "pgm.bkup". VIMAGE (virtualized network stack) is a highly experimental feature that may cause the host system to freeze up and wipe out the contents of any open files. Experience has shown that at times this has happened to the qjail internal definition and control files, and with less frequency to the qjail executable scripts. If your going to be using vnet jails, it's highly recommended you have a current backup.

-A

When used with no other parameters all jails are archived. Any other parameter coded with -A is an syntax error.

jailname

Multiple jailnames separated by a space are allowed on this command. The group prefix option is enabled. xxxx= will cause only those jailnames matching the xxxx character to be selected for processing. The equal sign "=" is the wildcard symbol that signifies all the characters to its left are to be used to match on jailname to create a list of jailnames to be processed. Jailname is a mandatory parameter. Jails in "norun" status or "man" manual start status are also candidates for archiving.

If jailname is sharedfs or template it will be archived. A sharedfs containing only the minimum system install, takes less than one minute elapse time to complete. A sharedfs containing portsnap downloaded ports tree may take up to 7 minutes elapse time to complete. Template and all other jails takes less than 15 seconds elapse time to complete. Use the subcommand list to display list of all jailnames.

Use qjail restore to restore an archive.

qjail delete

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

-A

This option will delete all the jails under qjail's control. You are advised to archive all your jails before doing this. This will also reset the qjail.vnetctl counter to zero.

jailname

Multiple jailnames separated by a space are allowed on this command. The group prefix option is enabled. xxxx= will cause only those jailnames matching the xxxx character to be selected for processing. The equal sign "=" is the wildcard symbol that signifies all the characters to its left are to be used to match on jailname to create a list of jailnames to be processed. Jailname is a mandatory parameter. Jails in "norun" status are NOT excluded from being deleted.

qjail restore

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

-s

This option restores the backup of the qjail internal definition and control files from the archive "system.bkup" directory, plus the qjail executable scripts from the archive "pgm.bkup" directory. VIMAGE (virtualized network stack) is a highly experimental feature that may cause the host system to freeze up and wipe out the contents of any open files. Experience has shown that at times this has happened to the qjail internal definition and control files, and with less frequency to the qjail executable scripts. If your going to be using vnet jails, it's highly recommended you have a current backup.

jailname

The most current archive file matching the jailname will be restored. To restore an older file you have to specify the full archive file name with the date and time of the archive appended to it. Multiple jailnames separated by a space are allowed on the command. The group prefix option is disabled for this subcommand. Jailname is a mandatory parameter. To view all the full archive file names, use this command ls /usr/jails/archive/.

If jailname is sharedfs or template and it's the only jailname on the command, it will be restored. A sharedfs containing only the minimum system install, takes less than one minute elapse time to complete. A sharedfs with source and full ports tree may take up to 7 minutes elapse time to complete. The existing sharedfs or template will be renamed to previous.sharedfs and previous.template before restoring begins.

qjail config

The options are as follows:

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

-A

This option is valid with all options except -i, -n and -d. When coded, a jailname is invalid. This -A option means to set the selected option on "ALL" the jailnames including those in "norun" and "man" status.

-b

Lower case b populates the devfs_ruleset per-jail option with the rule number representing the custom ruleset you added to the host's /etc/devfs.rules file. If this -b rule# is omitted the default ruleset number 4 is used and in most cases is sufficient.

To create your custom devfs_ruleset always copy rule number 4 from the hosts /etc/defaults/devfs.rules and create a host's /etc/devfs.rules file renaming number 4 to a unused number of your choice. Never grant access to raw disk devices inside of a jail, this may permit processes to exit the jail container and modify files outside of the jail. To enable your new rule issue "service devfs restart" from the hosts console. Type "man devfs" or "man devfs.rules" for information on how to create devfs rulesets to limit access and expose only appropriate device nodes to a jail.

Example; If a jail were to run OpenVPN in a jail it requires access to the "tun" device which rule number 4 does not include. This is when a customized ruleset would be called for.

There is a customized ruleset number 50 that adds the "bfp" device which allows dhclient(8) and tcpdump(1) and maybe some others to work inside both non-vnet jails and vnet jails. It's not added by default but the first time you use "-b 50" command it will be created automatically.

-B

Upper case B disables the devfs_ruleset per-jail option and reverts the jail to the default #4 devfs.rules for jails.

-c

The new network interface device name you want to replace the selected jailname "NIC" network interface device name with. Coded -c NIC jailname.

-d

Display's the jails jail(8) format definition from "/usr/local/etc/qjail.config/jailname" which shows the values set for the jail. Coded "-d jailname"

-f

Set the FIB (routing table) to use when running commands from inside the jail. This is a very advanced function used under very special conditions. Coded "-f numeric value" to identify which routing table the jail is to use.

First you need to increase the number of host routing tables by compiling the kernel with "option ROUTETABLES=3" or use the net.fibs=3 option in /boot/loader.config. The result would be 0 = default host routing table, 1 = first additional routing table, 2 = second additional routing table. Then issue host console command;   setfib 1 route add default "That jails default route ip address" Where 1 represents the first additional routing table. This route table will remain in effect until the next boot. Adding that command to /etc/rc.conf would make it happen on every boot of the host system.

Then use -f option with value of 1 to assign that routing table to the selected jail. See setfib(8) for more details.

-F

Upper case F disables the setfib per-jail option.

-h

This option will enable ssh and create a user account having the login ID and password of the jailname. To address basic security concerns, on first login the user will be prompted to enter a new password. The jailname user account belongs to the "wheel" group so it has "su" access to "root".

When the jails modified with the -h option are started for the first time, the changes to configure ssh and create the user account for that jail are applied. Doing a qjail restart jailname or a qjail stop jailname followed by qjail start jailname is required to enable the changes which will be in effect from that point on.

-4

The new IPv4 addresses you want to replace the selected jailname IPv4 address with. More than a single IPv4 address can be assigned to a jail. Multiple IPv4 addresses have to be a list of IPv4 addresses separated by a comma "," without spaces before or after. Example 10.0.0.2,10.0.0.3,10.0.0.4 Coded "-4 new-IPv4 jailname"

-6

The new IPv6 addresses you want to replace the selected jailname IPv6 address with. More than a single IPv6 address can be assigned to a jail. Multiple IPv6 addresses have to be a list of IPv6 addresses separated by a comma "," without spaces before or after. Coded "-6 new-IPv6 jailname"

-k

This negates the security of the jail concept and should never be used on a jail accessible from the public Internet. Lower case "k" enables the allow.raw_sockets per-jail option. Normally the ping command will get "Operation not permitted" error when issued from inside of a jail. This is a security design default of the jail environment. This security feature does not allow users or jail applications to create raw sockets. With raw sockets enabled a jail user could use perl or python or some other port utilities to create raw sockets and launch attacks on the host or the public network. If the jail has public internet access, an public attacker may compromise the jail and launch attacks on the host or the public network. Consideration of the security risk verses the convenience of using the ping command from inside of the jail is in order. However this restriction may be nullified by coding this option. Recommend using dig or whois commands to verify public access.

There are some valid situations for enabling allow.raw_sockets for an individual jail, such as running mysql or php or PostgreSQL which require local Unix sockets to work.

-K

Upper case K disables the allow.raw_sockets per-jail option.

-l

Lower case l enables the allow.mount.nullfs per-jail option. This is restricted to nullfs mounts on directories inside of the jail. It has nothing to do with mounting nullfs from the host to the jail filesystem which is always available to do. Any mount_nullfs commands issued from within the jail are only in effect for the duration the jail is running. When the jail is stopped the established nullfs mount is neutralized. The exec.prestart or exec.poststart parameters may be used to automate the issuing of the desired mount_nullfs commands. Documented in jail(8).

-L

Upper case L disables the allow.mount.nullfs per-jail option.

-m

Means put this jail in "manual start" status. If qjail_enable="YES" is present in the "host's" /etc/rc.conf file, then all jails in "man" status will be bypassed when the host is booted or powered up.

-M

Upper case M disables the "man" per-jail option.

-n

The new jailname you want to replace the selected jailname with. This changes the jailname and the jails directory name that the jail is known by. Coded "-n new-jail-name current-jail-name". The new jail name can only have alphanumeric, dash, and underscore characters and all numeric jail names are invalid.

-p

Edits a text file containing the names of packages you want installed in the jail. Enter one package name per line.

-P

Upper case P runs a "pkg install" command reading the content of the text file created by the lower case -p option. The host must have public Internet access to download the package. The target jail must be running.

-q

Lower case q enables the allow.quotas per-jail option. Quota has to be compiled into the host's system kernel first for this option to function. "option QUOTA" is the statement you have to add to your kernel definition source. Documented in jail(8).

-Q

Upper case Q disables the allow.quotas per-jail option.

-r

Means put this jail in "norun" status. If qjail_enable="YES" is present in the "host's" /etc/rc.conf file, then all jails in "norun" status will be bypassed when the system is booted and also when the start command is used.

-R

Upper case R disables the "norun" per-jail option.

-s

Enables the securelevel per-jail option. Documented in jail(8). There are five different security levels. Any super-user process can raise the level, but no process can lower it. The security levels are:

-1 Permanently insecure mode - always run the system in
insecure mode. This is the default initial value.

0 Insecure mode - immutable and append-only flags may be
turned off. All devices may be read or written subject
to their permissions.

1 Secure mode - the system immutable and system append-only
flags may not be turned off; disks for mounted file
systems, /dev/mem and /dev/kmem may not be opened for
writing.

2 Highly secure mode - same as secure mode, plus disks may
not be opened for writing (except by mount(2)) whether
mounted or not. This level precludes tampering with file
systems by un-mounting them, but also inhibits running
newfs(8) while the system is in multiuser. In addition,
kernel time changes are restricted to less than or equal
to one second. Attempts to change the time by more than
this will log the message "Time adjustment clamped to
+1 second".

3 Network secure mode - same as highly secure mode, plus IP
packet filter rules (see ipfw(8), ipfirewall(4) and
pfctl(8) cannot be changed and dummynet(4) or pf(4)
configuration cannot be adjusted.

This does not really apply to the qjail jail system because all of the system executables are in an read only nullfs mounted filesystem which makes it impossible to change file content or add files to those directories. This is by far a stronger form of jail security than the securelevel parameter can provide on an per-jail basis.

-S

Upper case S disables the securelevel per-jail option.

-t

Lower case t enables the allow.mount.tmpfs per-jail option. This is restricted to tmpfs mounts on directories inside of the jail.

-T

Upper case T disables the allow.mount.tmpfs per-jail option.

-v

Lower case v enables the vnet [vimage] per-jail option. WARNING: The vnet/vimage configuration used here is designed for host systems running version 11.0-RELEASE or newer. VIMAGE (virtualized network stack) is a highly experimental feature. Information given here maybe different in newer RELEASES. Vimage has to be compiled into the host's kernel before the vnet function will work. "option VIMAGE" is the only statement you have to add to your kernel definition source.

The "bridge/epair" method is used to establish network access for the vnet jail. The /usr/local/etc/qjail.vnetctl file contains the counter whos number is used to assign a unique permanent number to the vnet jail's virtual ethernet epair interfaces. The epair pair are named epair<n>[ab] where <n> is the unique number generated using the value in the qjail.vnetctl file. This means the names of the first pair of epair interfaces would be named epair1a and epair1b. The bridge qjail uses is named bridge10 to greatly reduce the likelyhood of the host administrator creating a bridge with the same name.

The -v option requires a value of none, ipfw, pf, or ipf, to select which firewall to run in the vnet jail. The host and the vnet jail must be running the same firewall type, meaning if the vnet jail is using ipfw, then the host must also be running ipfw. Note: ipf is short for ipfilter. The "config -w and -v" flags can be coded together.

"IPFW" is currently the only one that can run on the host and inside the vnet jail at the same time and function correctly. IPFW dummynet and "in kernel NAT" has not been tested in a vnet jail. The vnet jail's IPFW logging records are intermingled with the host's IPFW logging records on the host. "IPF" can run on the host and inside the vnet jail at the same time, except the ipf rules in the vnet jail are not enforced.   "PF" can run on the host and inside the vnet jail at the same time, except the pf rules in the vnet jail are not enforced. "In conclusion; At the current time [RELEASE-11.0], PF and IPF firewalls in a vnet jail do not work and are useless for controlling access to a vnet jail on a per vnet jail basis."

Because of the increased likely hood of host system freezes or page faults, vnet jails are restricted to directory type filesystem jails only. No image jail type are allowed because of the manually effort required to recover them. Experience has shown that at times a vnet system freeze may cause the contents of any open files to be wiped out leaving only the file name. This effects the qjail internal definition and control files. It's highly recommended you use the "archive -s" option to create a current backup of these files. Using the "restore -s" option will restore those files after a vnet system freeze greatly simplifying the recovery process.

-V

Upper case V disables the vnet [vimage] per-jail option.

-w

Enables the vnet.interface per-jail option. Populate with the network interface device name of the NIC facing the public internet or facing the LAN server you want vnet [vimage] to exchange traffic with.

-W

Upper case W disables the vnet.interface per-jail option.

-x

Lower case x enables the allow.mount.zfs per-jail option. This option has mandatory host requirements before it's useful. The host must have all or some part of it's hard drive space defined to zfs and actively using it. See zfs(8) for information on how to configure the zfs filesystem to operate from inside a jail. The exec.prestart or exec.poststart parameters may be used to automate the issuing of the desired zfs commands. Documented in jail(8).

-X

Upper case X disables the allow.mount.zfs per-jail option.

-y

Lower case y enables the allow.sysvipc per-jail option. Grant processes within the jail access to System V IPC (semaphores). Enabling sysvipc makes the jail considerably less secure in respect to shared memory. This feature should not be used unless absolutely necessary when no other option is available. Documented in ipcs(1).

Example: Zabbix (a system monitoring tool) which is the major competitor of naigos would need allow.sysvipc enabled so it will start.

-Y

Upper case Y disables the allow.sysvipc per-jail option.

jailname

For all options except -d, -i, and -n, multiple jailnames separated by a space are allowed on the command. The group prefix option is enabled. xxxx= will cause only those jailnames matching the xxxx characters to be selected for processing. The equal sign "=" is the wildcard symbol that signifies all the characters to its left are to be used to match on jailname to create a list of jailnames to be processed. Jailname is a mandatory parameter. Use subcommand "list" to show a list of all jailnames.

qjail update

-z

Code the same zone value used with the "install" subcommand to have this subcommand process against that zone.

-b

The basic requirement of FreeBSD jails is the jail environment and the host run the same version of the systems binaries. Since the FreeBSD-update utility only inspects the host system to determine the systems RELEASE level it's not applicable in a jailed environment. Performing a make buildworld/installworld on sharedfs's source is such a waste of effort and resources after having done this already for the host system. This option makes the buildworld/installworld obsolete for the qjail environment.

This option deletes all the system binaries from the sharedfs and them copies the host's system binaries to sharedfs. It's intended to be used after running the FreeBSD-update utility on the host to apply security updates or to upgrade the GENERIC host from one RELEASE to another newer RELEASE, or after performing a make buildworld/installworld on the host updating its system binaries. Basically update the host and copy your work to the sharedfs getting both environments synchronized.

Note: When going from one subversion to a newer subversion within the same major RELEASE, IE: 8.0 to 8.1 there is no need to update your installed ports/packages. When going to a newer major RELEASE IE; 8.1 to 9.0 then your installed ports/packages need updateing.

-p

This option Invokes the portsnap utility to fetch and extract a FreeBSD ports tree from "portsnap.FreeBSD.org" (475MB). By design the "sharedfs" filesystem includes the "/usr/ports" directory which is not automatically populated by "qjail install".

An alternative to executing portsnap to populate "sharedfs/usr/ports" would be to temporarily make the hosts "/usr/ports" directory tree available to the jails by using the "mv" command like this: mv /usr/ports /usr/jails/sharedfs/usr and returned doing mv /usr/jails/sharedfs/usr/ports /usr

Portsnap will initially download a compressed file containing the complete ports tree. Elapse download time greater than 15 minutes is normal. On its initial execution, an extract is performed creating the /usr/ports sub-directories and populating them. Subsequent executions, the /usr/ports directory exists, so an update is done populating the /usr/ports directory tree with only things that have been changed or added. This is portsnap's default behavior. This behavior can be somewhat modified by changing the content of the /usr/local/etc/qjail.portsnap.conf file. Add REFUSE statements to select the ports categories you don't want populated to your /usr/ports directory tree. Ideal candidates to REFUSE are the non-English languages, astro, biology, cad, finance, games, math, mbone, and science. From there you can select additional categories to REFUSE based on your normal jail port usage. For more details see Appendix A.6-Using Portsnap and Chapter 24.3 Portsnap in the FreeBSD Handbook or "man portsnap".

-P

This is an upper case P. This option copies the hosts /usr/ports directory tree to the /usr/jails/sharedfs/usr/ports directory tree after first deleting the existing one.

-S

This is an upper case S. This option copies the hosts /usr/src directory tree to the /usr/jails/sharedfs/usr/src directory tree after first deleting the existing one.

-l

This enables or disables [on | off] logging of all qjail commands and error messages to /var/log/qjail.log file. Each log entry is prefixed with a date/time stamp and the user account name of the user entering the commands. An entry is also made in /etc/newsyslog.conf to auto rotate the qjail.log file.

qjail logmsg

qjail help

GENERAL QJAIL USAGE TIPS

* For a jail to receive unsolicited inbound traffic from the
public internet, the host's firewall must have a rule to allow
the desired port number to pass through the firewall and a NAT
forward rule targeting the IP address and port number of the
desired jail. For additional details read qjail-howto man page.

* You can remotely access a jail through ssh if that option was
selected during the jail create process and the host's firewall
and NAT forwarding rules are in place.

* The orderly stopping of jails that have databases or other
applications that may have delayed buffered writes to files is
accomplished by the use of the "qjail stop" command or issuing
the "shutdown now" command on the host. Using the host's halt
and reboot commands or pressing the computers reset or power on
buttons results in the running jails being instantly terminated
which some applications can not tolerate. Always use the host's
shutdown command or "qjail stop" command.

* By design the "sharedfs" filesystem includes the "/usr/ports" and
"/usr/src" directories which are not automatically populated by
"qjail install". You can temporarily make the hosts "/usr/ports"
or "/usr/src" directory trees available to the jails by using the
"mv" command like this:
mv /usr/ports /usr/jails/sharedfs/usr and returned doing
mv /usr/jails/sharedfs/usr/ports /usr

# There are some common utilities that require the "bfp" device to
function in a jail such as dhclient(8) and tcpdump(1). There is
a customized ruleset number 50 that adds the "bfp" device. The
first time you use the "qjail config -b 50" command it will be
created automatically.

* In environments where you want all the jails to use the same set
of ports but don't want to have to compile these ports in every jail,
you can do the following. Populate sharedfs/usr/ports/packages/
directory with the packages you want. All jails have access to this
shared directory. Then create a SEED jail to be used as the source
to clone all of the other jails from. First create your basic SEED
jail using the template. You may wish to customize a flavor
to contain any desired /etc config files unique to that seed.
Additionally you can start the SEED jails console and perform any
other customization such as "pkg install" for the pre-staged
packages or "make install" on ports you want. When your satisfied
with the SEED jail's configuration, archive it. Then use the SEED's
archive file jailname in the -a option of the create subcommand so
it's used as the source template to create the other jails from.
Optionally you could use the -d and or -I options with the -a
option for mass duplication of jails based on that SEED
configuration.

* In the situation where you want "all" the jails that you EVER create
to have the same selection of ports, create a "SEED" jail as
described above. When your satisfied with your "SEED" jail, delete
the /usr/jails/template directory and rename your "SEED" jail to
/usr/jails/template directory.
mv /usr/jails/SEED /usr/jails/template
rm /usr/local/etc/qjail.local/SEED
rm /usr/local/etc/qjail.global/SEED
rm /usr/local/etc/qjail.fstab/SEED
From that point on, all new jails created using the template
will contain your standard ports.

* The /etc/rc.conf in the default flavor has this statement;
cron_flags="$cron_flags -J 60" this enables time jitter
for all /etc/crontab jobs run by the superuser, which on a
pristine jail environment is everything in the crontab file.
Time jitter works this way: Prior to executing commands in the
/etc/crontab file, cron will sleep a random number of seconds
in the range from 1 to 60 seconds. This option greatly helps
to reduce host system load spikes during moments when a
lot of cron jobs are likely to start at once, IE, at the
beginning of the first minute of each hour. Without this
statement in every deployed jail to randomly spread the
starting of cron tasks over the first minute, most likely
the host system would come to a darn near halt. The default
flavor has another customized configuration file just for
jails. The /etc/periodic.conf overrides the normal emailing
of reports and instead creates daily, weekly, and monthly
logs within each jails /var/log directory. These logs get
rotated and deleted as specified in the jails
/etc/newsyslog.conf.

* Its a mandatory requirement of the FreeBSD "jail" system that the
host and the jails are both running the same version of the operating
system binaries. First you have to get your host system running at
the newer RELEASE version. You can do the fresh install from scratch
method, or update your host's current RELEASE version by using the
Freebsd-update utility or svn update your system source and make
buildworld/installworld. After the host is running the new RELEASE
version and before starting any qjail's. You can run the "install"
subcommand again and re-install with the newer RELEASE version
matching what is on the host, without disturbing the existing
installed jails, or run the "update" subcommand with the -b option
to copy the hosts operating system binaries to the sharedfs.
If going to a newer major RELEASE, IE: 6.4 to 7.1; 7.2 to 8.0;
then remember, all existing jails that have ports or packages in
them will need them updated to versions compatible with the new
major RELEASE version. On the other hand, if going from a
subversion to a newer subversion within the same major RELEASE,
IE: 7.1 to 7.2; 8.0 to 8.1, then there is no need to update your
installed ports/packages.

* Each jail has a console log located in the host's /var/log/
directory named jail_*_console.log. Where "*" = jailname.
These logs don't grow much but if the jails are going to be
used long term, their names should be added to the hosts
/etc/newsyslog.conf so they get auto rotated and deleted.
You don't want some jail user to cause console messages and
flood the jails log until all the host's disk space is
consumed bring the host to a abrupt stop.

* If you have qjail start a image jail, then the contents of its
sparse image file are accessible by the host system. From the host
you can "cd" into the image jails jailname directory and access
the directory tree there just like any other directory tree.

* The ping command will get "Operation not permitted." error when
issued from inside of a jail. This is not a qjail restriction, but
a design default of the FreeBSD jail command. This default does not
allow users or jail applications to create raw sockets. This is a
security feature. With raw sockets a jail user could use perl or
python or some other port utilities to create raw sockets and launch
attacks on the host or the public network. The config -k option
maybe used to enable allow.raw_sockets function on a per-jail basis.

* Once your jail has public network access, (test with whois)
then all your normal application install functions are available,
(ports tree update, svn update, ports and package installs) right
from the jails console using the "qjail console" command.

* Jails do not have a network stack of their own, so they can't have
a firewall. The host's firewall and network is in control.

* If you want absolute control over starting your Jails. (IE. no boot
time auto-start of the jails), then don't put the qjail_enable="YES"
statement in the hosts rc.conf file.

* If for whatever reason you want to completely delete the qjail
jail environment so you can start over with the install
subcommand from scratch, execute these commands;
rm -rf /usr/jails
rm -rf /usr/local/etc/qjail.local
rm -rf /usr/local/etc/qjail.global
rm -rf /usr/local/etc/qjail.config
rm -rf /usr/local/etc/qjail.packages
rm -rf /usr/local/etc/qjail.fstab
rm /var/log/jail_*
rm /var/log/jails.lo*

FILES

SEE ALSO

AUTHOR

qjail versions 1.0 through 2.2 written by
The Qjail project team, Angeles City, Philippines
Joe Barbish <qjail1@a1poweruser.com> for the Qjail project team.

qjail version 3.0 and newer maintained by
Joe Barbish <qjail1@a1poweruser.com>
http://qjail.sourceforge.net/