NAME

portsreinstall - ports upgrading utility for massive forced reinstallation

SYNOPSIS

portsreinstall [ OPTIONS ] [ -- ] [ command ]

DESCRIPTION

This utility is an alternative to portupgrade(1) and portmaster(8), and designed to be suitable for reinstallation/upgrade of all packages after major version upgrade of the system or very long absence of ports upgrade, or for entire correction of confusion among installed packages.

ROBUSTNESS AGAINST TERMINATION AND RESTART

Execution of do, redo and packupgrade commands can be terminated at any points and restarted from there with the same options saved in the temporary database (independent among utilities). The saved options can be changed by the ways depending on the assigned groups of the options. Some options can be reset only by redo command. The options are reset to the default by clean and reset all commands.

FLAVOR SUPPORT

As of the release time of this software, the flavor support of the Ports Collection has many incomplete features. Therefore, users should note that problems may arrise due to the specification changes related to flavors. Currenly, coexistence with portupgrade(1) is suspended due to the lack of flavor support. The same port configuration options of the same ports are shared among the all flavors because they are not distinguishable in the current implementation of the Ports Collection.

GLOB PATTERNS

A glob is an extended regular expression or a shell-type glob pattern matching either of a unique name, package names, or unflavored port origins possibly followed by flaver. The format of a glob may be either:

unique_name_glob
package_name_glob
port_origin_glob
port_origin_glob@flaver_glob
:unique_name_regexp
:package_name_regexp
:port_origin_regexp
:port_origin_regexp@flaver_regexp

where each regular expression (*_regexp) and glob (_glob) pattern must not include any at sign (@). Port origin patterns without at sign match all available flavors if deined. As long as the specified port origin pattern represents ports wihout available flavor, the glob syntax is compatible with portupgrade(1) and its accompaniments such as portsdb(1), pkg_glob(1) and ports_glob(1). If a glob starts with a colon ":", the following string is parsed as an extended regular expression, and otherwise a shell-type glob. A shell-type glob is evaluated by matching an asterisk "*" to an arbitrary string of an arbitrary length, a question mark "?" to an arbitrary single character, characters "..." enclosed in a bracket "[...]" to a single character as one of the characters and characters "..." in a bracket "[!...]" to a single character matching none of the characters. Here, the evaluation of the bracket is actually done by passing it to an extended regular expression by simply converting the prefix "[!" to "[^". A glob is recognized as a unique name (package name without the version part) if no slash "/" or any special character for an extended regular expression or a shell-type glob is included in the pattern. For example, all of "zip" "zip-3.?", "zip-*", "zip-[0-9]*" and ":^zip-[[:digit:]]+.*" can match "zip-3.0"; "archivers/unzip*" can match both of "archivers/unzip" and "archivers/unzip-iconv". When the glob should match the all flavors of a port, a regular expression can be used such as ":devel/py-setuptools(|@.*)".

ARGUMENTS

One of the following commands can be given for optional operations or confirmation.

do
do all: (Default) Full execution.
prepare
do prepare: Just build (or continue to build or rebuild) the temporary database and stop before the actual operations to the ports/packages.
redo
redo all: Execute again for failed ports and their dependents.
redo prepare: Just rebuild the temporary database and stop before the actual operations for the redo process. In case of restarting a stopped redo prepare process, execute by prepare or do prepare command instead of this command.
clean
clean [normal]: Clean up the temporary database.
clean force: Attempt to clean up the temporary database without checking the lock and privilege.
reset
reset all: Reset the temporary database by preserving the initial snapshot of installed packages. This command can be used to restart reinstallation when the ports tree is updated after the previous run or to rescue the temporary database from destruction.
reset keepopts: Reset the temporary database by preserving the initial snapshot of installed packages, option settings and manually added taboo/need/noneed ports. This command can be used to restart reinstallation when the ports tree is updated after the previous run or to rescue the temporary database from destruction.
reset keepstatus: Reset the temporary database by preserving the progress status to reuse for unchanged successfully (re)installed ports. The initial snapshot of installed packages, option settings and manually added taboo/need/noneed ports are preserved in the same way as reset keepopts command. This command can be used to more smartly restart reinstallation when the ports tree is updated after the previous run.
ok add glob1 [glob2 ...]: Register manually resolved ports. The port globs are separated from each other by comma or space.
ok del glob1 [glob2 ...]: deregister manually resolved ports. The port globs are separated from each other by comma or space.
taboo add glob1 [glob2 ...]: Register taboo ports that must not be built or newly (re)installed. The port globs are separated from each other by comma or space. Registration to be "taboo" practically invalidates "necessary" registered by noneed add command. Ports specified in the HOLD section in the configuration file are protected from deinstallation. This functionality is intended to avoid manually deinstalled conflict ports or ports whose build or installation operations can damage the system, e.g., by flooding the file systems or kernel panic. This command is for temporal definitions and permanent definitions should be given as TABOO_* values in ${LOCALBASE}/etc/portsreinstall.conf instead.
taboo del glob1 [glob2 ...]: Deregister taboo ports. The port globs are separated from each other by comma or space.
freeze add glob1 [glob2 ...]: Register ports to freeze that must not be built but the package installtion by any available version may be safe. If the specified ports failed to (re)install by packages, build of their dependents are also suppressed. The port globs are separated from each other by comma or space. This functionality is intended to build of ports which can damage the system, e.g., by flooding the file systems or kernel panic, while their installed packages will not be harmful. This command is for temporal definitions and permanent definitions should be given as FREEZE_* values in ${LOCALBASE}/etc/portsreinstall.conf instead.
freeze del glob1 [glob2 ...]: Deregister ports to freeze. The port globs are separated from each other by comma or space.
need add glob1 [glob2 ...]: Register necessary ports that should be newly installed if not yet and kept installed even if being obsolete. The port globs are separated from each other by comma or space. Registration to be "necessary" takes priority over "unnecessary" registered by noneed add command. In other words, the registered ports are excluded from deinstallation candidates if they are obsolete, leaf or not-initially-installed ports, and otherwise have no effect on the actual operations.
need del glob1 [glob2 ...]: Deregister necessary ports. The port globs are separated from each other by comma or space. The deregistered ports are added to leaf port candidates if they are not initially installed. In this case, information on the deregistered ports remain until removal by forget command.
noneed add glob1 [glob2 ...]: Register unnecessary ports that should be deinstalled unless required by other non-leaf ports. The port globs are separated from each other by comma or space. In other words, the registered ports are added to leaf ports if they have no non-leaf dependent, and otherwise have no effect on the actual operations. Requirements of the registered ports are also recognized as leaves if all of their dependents are leaves. Ports specified in the HOLD section in the configuration file are protected. For example, just after installing new ports due to need add command or -i option as requirements of a specified target, registering only the target will let the all required ports be leaves as well as the target. Actually deinstalled ports are selected via dialog together with the original leaf ports.
noneed del glob1 [glob2 ...]: Deregister unnecessary ports. The port globs are separated from each other by comma or space.
reselect leaves: Reselect leaf ports to delete. This command is effective only after the temporary database is once built.
reselect obsolete: Reselect obsolete ports to delete. This command is effective only after the temporary database is once built.
save [dir]: Save the current temporary database as a .tar.gz archive. The file name is created automatically by containing the current date time. By default save directory is the current directory.
load path: Load a temporary database archive. Note that the currently temporary database is removed.
rmconf glob [glob2 ...]: Reset port options for the specified ports to the default and reset affected parts in the temporary database.
reconf glob [glob2 ...]: Reconfigure port options for the specified ports. The specified ports are re-inspected even when the options are unchanged or undefined. All affected ports are inspected again in the following runs by do or redo command.
forget glob [glob2 ...]: Try to let the temporary database forget about the specified ports as much as possible. Concretely, the data on each of the specified ports and their requirements/dependents is unless initially installed or required by other preserved ports. This command is mainly for removing information on ports which had been added due to -i option or need add command but have become unneeded again. In advance to executing this command, deinstallation for the specified ports should be completed by noneed add command followed by redo command. It is noted that ports registered by ok, taboo, freeze and noneed commands are NOT deleted. Internal tables used for show command keep their information regardless of this command.
pkgsanity [glob ...]: Examine the sanity of installed packages, i.e., whether the installed files exist and match the checksum. If arguments globs are given, only the matching packages are examined. For each insane package, the user is queried whether to reinstall it forcedly in the following do/redo runs. The default choice is "yes" for packages which have any ELF binaries or include files changed or any type of files overwritten by other packages, and "no" for the others. Insane packages already assigned to be reinstalled are skipped. With -a enabled, all insane packages are assigned to be the above mentioned default and the output is given in a form that the first to fourth columns denote the package name, flavored port origin, whether assigned to be reinstalled (yes or no), and file path with a notice, respectively. It is noted that this sanity check can detect some configuration or cache files which are changed during normal use and need not to be recovered.
escape glob [glob2 ...]: Back up and delete packages specified by globs for a temporary escape mainly for resolving undeclared conflicts. The escaped packages are registered as taboo as same as taboo add glob [glob2 ...].
restore glob [glob2 ...]: Restore packages specified by globs escaped by escape command. The escaped packages are deregistered from taboo as same as taboo del glob [glob2 ...].
packupgrade create: Create packages of the all reinstalled or newly installed ports (at a "builder" environment) with a set of scripts for upgrading another system with the same initial configuration of installed packages ("target" environment). One of typical examples of application can be such that the builder is a "forked" jail/chroot system constructed onto a host system by using nullfs and unionfs and the target is the host system (Refer to "Package build in a chroot system" Section). Another typical example can be such that the builder is one node of a cluster system and the targets are the other nodes. If the execution is terminated before completion, it can be restarted from the terminated point by re-running this command. The created packages are saved at ${PACKAGES}/${PKGREPOSITORYSUBDIR}, which is usually /usr/ports/packages/All. In order to crop the created set of scripts, packupgrade crop command must be executed.
packupgrade crop [path]: Crop the created set of scripts as a tar.gz archive, where its path name is specified by path argument (if path is a directory, the file name is set to "portsreinstall-upgrade.tar.gz"; the default of path is the current directory). This command must be executed after completion of packupgrade create command. The obtained archive is to be conveyed to the target together with the created packages. Then "portsreinstall-upgrade.sh" obtained by extracting the archive is to be run at the target.
packupgrade clean: Reset the execution progress of packupgrade create command and discard the internally saved created set of scripts. This does not affect any already cropped sets of scripts or created packages. This command is provided for cases that ports/packages are reinstalled or reconfigured during/after the execution of packupgrade create command. After execution of this command, packupgrade create command will overwrite the packages by creating them again.
make glob [target ...] [arguments ...]: Execute make(1) command for ports matching the glob pattern. The taget and arguments are passed as they are with the arguments and environment variables customized for each port. For flavored ports, the appropriate argument (FLAVOR variable) is automatically set. The main purpose of this command is diagnosis and experimental build of unsuccessful ports. The result of this command will not be recognized by the (re)installation processes of do and redo commands. The execution is carried out for each matching port in the alphanumeric order. If any port failed to execute make(1) command, its exit status is returned by terminating immediately (so the following ports in the queue are skipped).
glob glob [glob2 ...]: Evaluate port globs and show matched flavored origins both for installed and uninstalled packages. The exit status is always 0.
pkg glob [glob2 ...]: Evaluate port globs and show matched installed package names. The returned exit status is 1 if no installed package matches any of the globs.
options: Show saved option settings and expected effects of option-resetting options -L, -M and -N. With -a option, the first, second and third columns denote the option-resetting options, reset options and remaining options, respectively.
show [subject] [@[run|build|all][,[direct|full]]] [arguments]: Show the list of ports to be reinstalled. The applied scope of dependencies can be controlled by "show option" proceeded by "@"; run, build and all employ the run-time, build-time or both-time ones, respectively; direct and full include only the direct or fully recursive ones, respectively. Here, the full build-time requirements are defined as the direct build-time requirements and their full run-time requirements. The full build-time dependents are also defined in accordance. The default scope is determined by the saved option settings for -B, -b and -o. Some subject's require arguments. The following subject's are available.

todo: Ports to be reinstalled in the current do/redo process (default). With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
done: Ports which have been already reinstalled to be up-to-date with their all requirements. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
resolved: Manually reinstalled ports registered by ok command. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
failure: Failed ports. With -a option, the first, second and third columns denote the flavored origin, initial/new/current package name, failed make target and manually resolved status (yes or no), respectively.
redo: Ports to be reinstalled after success in any of their failed requirements. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
inst_by_pkg: Ports installed by the default packages because their configurations are default. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
inst_built_default: Ports locally built and installed with the default configurations including their all requirements. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
inst_built_custom: Ports locally built and installed with the non-default configurations or having any of their requirements non-default. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
taboo: Taboo ports registered by taboo command. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
freeze: Taboo ports registered by freeze command. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
need: Necessary ports registered by need command. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
noneed: Unnecessary ports registered by noneed command. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
restored: Leaf, obsolete or unneeded ports which had been once deleted but are to be or have been restored. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
deleted: Leaf, obsolete or unneeded ports are to be or have been deleted. With -a option, the first and second columns denote the flavored origin and initial/new/current package name, respectively.
fossil: Installed ports which have not been either upgraded or reinstalled since the initial state. With -a option, the first and second columns denote the flavored origin and package name, respectively.
conflict: Conflicting ports which are temporarily deleted. With -a option, the first, second and third columns denote the flavored origin, initial/new/current package name, and opponent ports concatenated by comma, respectively.
moved: Moved or replaced ports. With -a option, the first, second, third and fourth columns denote the flavored initial origin, initial package name, flavored alternative origin and alternative package name, respectively.
build_conflict_pkgs glob [glob2...]: Installed packages which conflict with ports matching globs in the build. The package names are listed in a single column.
inst_conflict_pkgs glob [glob2...]: Installed packages which conflict with ports matching globs in the installation. The package names are listed in a single column.
leaves [selected | unselected]: All detected leaf ports. Keywords selected and unselected are for filtering only selected (to be deleted) and unselected (to be preserved) ones, respectively. With -a option, the first, second and third columns denote the flavored origin, initial/new/current package name, and opponent ports concatenated by comma respectively.
obsolete [selected | unselected]: All detected obsolete ports. Keywords selected and unselected are for filtering only selected (to be deleted) and unselected (to be preserved) ones, respectively. With -a option, the first, second and third columns denote the flavored origin, initial/new/current package name, and opponent ports concatenated by comma respectively.
requirements glob1 [glob2 ...]: Ports required by matching ports/packages. With -a option, the first, second, third and fourth columns denote the flavored origin of the queried port, initial/new/current package name of the queried port, flavored origin of a requirement of the queried port and initial/new/current package name of the requirement, respectively.
dependents glob1 [glob2 ...]: Ports depending on matching ports/packages. With -a option, the first, second, third and fourth columns denote the flavored origin of the queried port, initial/new/current package name of the queried port, flavored origin of a dependent of the queried port and initial/new/current package name of the dependent, respectively.
initrequirements glob1 [glob2 ...]: Ports initially required by matching initially installed ports/packages. With -a option, the first, second, third and fourth columns denote the flavored origin of the queried port, package name of the queried port, flavored origin of a requirement of the queried port and package name of the requirement, respectively.
initdependents glob1 [glob2 ...]: Ports initially depending on matching initially installed ports/packages. With -a option, the first, second, third and fourth columns denote the flavored origin of the queried port, package name of the queried port, flavored origin of a dependent of the queried port and package name of the dependent, respectively.
conflict_files glob1 [glob2 ...]: Possible additional conflict files of matching ports/packages. The first and second columns denote the possible conflict package and file path, respectively. For officially registered conflict, nothing is outputted. In other words, if something is outputted, the port Makefile can have some defect or the custom modification of the system can be confusing the port or the conflicting packages.
status glob1 [glob2 ...]: Current success/failure status in (re)installation of matching ports/packages. Returned values are null, "todo", "done", "resolved", "failure", "redo", "inst_by_pkg", "inst_built_default", "inst_built_custom", "taboo", "freeze", "need", "noneed", "restored", "deleted", "fossil", or "conflict", where null means that they are untouched in the current option configuration or temporary reset due to configuration changes.
errormessage glob1 [glob2 ...]: Error message in (re)installation of matching ports/packages. Returned values are null for successful or skipped ports/packages.

OPTIONS

If duplicated or conflicting ones are set, the last ones are effective. The end of options can be explicitly specified by --. Short options can be given in compact forms, for example, -i -q -P to be -iqP.

*NOTE*: The configuration of options annotated as "saved and transferred to restarted/following runs" are saved in the temporary database by the first run after cleaning the database. In the following runs for any commands, the corresponding option settings are loaded from the saved configuration unless explicitly reset by -L, -M or -N option. The saved options can be checked by portsreinstall options command.

Group 1: Just show messages and exit without operation

-H

--long-help: Show a long help whose content is the same as the manual page.

-h

--help
--short-help: Show a short help.

-V

--show-version: Show the current version.

Group 2: Effective anytime

-a

--batch-mode: Suppress messages so as to be friendly for batch operations. The output formats for options and show commands and -V option are arranged to be more batch-friendly. It is noted that log output in build/installation processes are not suppressed.

-i

--allow-new-targets: Allow -O, -T or -t options to specify not-yet-installed ports. If any of not-yet-installed ports matching the target globs are ambiguous, a dialog box is open for each of them to select the actual targets.

-M

--reset-minor-options: Reset option settings for minor controls. Option settings for group 4 are once reset and replaced with the newly specified ones. Check the saved options by portsreinstall options command before specifying this option.

-S

--no-opening-message: Suppress the credit, opening and terminating messages. Option -a takes higher priority over this option.

Group 3: Effective only with redo command

-L

--reload-conf: Reload configuration files. This option is effective only with redo command. Option settings for group 6 are once reset and replaced with the newly specified ones. Check the saved options by portsreinstall options command before specifying this option.

-N

--reset-targets: Re-scan installed packages and reset option settings for target specification. This option is effective only with redo command. Option settings for group 5 are once reset and replaced with the newly specified ones. Check the saved options by portsreinstall options command before specifying this option.

Group 4: Saved and transferred to restarted runs, renewable by -M option

Option settings in this group are saved at the first do or prepare run, and transferred to the following runs. Reset of the saved values for this group is available by appending -M with newly specified options.

-A

--non-interactive-ports-only: Operations of (re)installation are made only on ports which do not require manual interaction. This option conflicts with -I.

-B

--exclude-runtime-dependencies: Exclude run-time dependencies in evaluation of dependencies. This option affects behaviors of -t, -T and -q options. Use of this option should be just for temporary diagnosis or salvage.

-b

--include-buildtime-dependencies: Include build-time dependencies in evaluation of dependencies. This option affects behaviors of -t, -T and -q options. Use of this option will result in installation of the all build-time requirements regardless of the actual necessity for upgrade of installed packages.

-C

--apply-default-config: The temporary database is built by skipping executing make config-conditional so that the port options are unchanged. The default values are applied to unconfigured port options. The port options are unchanged and unsaved (they are saved in case of old ports trees in which dialog(1) is used instead of dialog4ports(1)).

-c

--suppress-cleaning-obsolete-database: Suppress cleaning the temporary database even if its obsolete. This option suppresses the default behavior that the temporary database is automatically cleaned up if it is older than the ports tree or portsreinstall itself is to be upgraded. Use of this option may cause unexpected results and basically unrecommended.

-D

--suppress-entire-inspection-distinfo: Suppress entire inspection of distinfo files in the ports tree as a preparation for deleting obsolete distfiles. By default, viz., without this option, all distfiles are preserved unless being obsolete in the current version of the ports tree. For this purpose, entire inspection of distinfo files in the ports tree is carried out in order to get the complete list of distfiles. This inspection can take an extremely long time if the ports tree is located in file systems with low access speeds. With this option specified, distfiles for ports are deleted unless they are initially installed, added due to -i option, or required by any of the installed or added ports.

-d

--keep-distfiles: Do not clean up obsolete or unused distfiles.

-F

--fetch-only: Execute fetch and checksum verification only at the (re)installation process for each port. Packages are also fetched if applicable. Regardless of this mode, missing essential tools for package management such as pkg(8) and dialog4ports(1) will be installed.

-f

--disallow-force-continuation-for-looped-dependency: Disallow forcible continuation in case that looped dependencies are found. Without this option, workarounds are carried out at the phase of "Completion of *-time requirement lists" of the preparation stage. In this case, when a looped dependency is detected, the first found dependency which is not run-and-build-time dependency (that is a typical pattern of dynamically linked libraries) is marked ignored to break the loop; if no such dependency is found, one connecting the end back to the start of a loop cycle is ignored for the time being.

-G

--use-prebuilt-package: Use prebuilt packages for ports with the default configurations, i.e., in which and in the all required ports of which no port option is changed from the default and no knob (make environment variable or make argument) or replacement is defined in${LOCALBASE}/etc/portsreinstall.conf . As long as no knob is defined in /etc/make.conf and no environment variable has effect on package builds, this option will accelerate the (re)installtion processes without harm. It is noted that some ports still may be problematic with this assumption by automatic dependencies on other system configurations. This problem can be resolved by configuring REBUILD_* section in ${LOCALBASE}/etc/portsreinstall.conf.

Historical background: The actual merit of this option depended on the service levels and qualities of the remote package sites; since October 31, 2013, weekly-updated pkgng packages were provided for major architectures (at November 15, 2013, i386 and amd64 only) until their end-of-life while legacy packages were provided only at each release timing; the situations were different for STABLE and CURRENT versions and more various architectures were supported for legacy packages. The official support of legacy packages were abandoned at September 1, 2014, but third-party services might be used by configuring PACKAGECHECKSUMROOTS and PACKAGEROOTS if available somewhere.

-g

--suppress-pkgtools-upadte: Keep indispensable packages for the standard function of the ports/packages system untouched. Concretely, this option suppresses upgrade, de/re-installation of the currently installed ports-mgmt/pkg(-devel) and ports-mgmt/dialog4ports.

-I

--interactive-ports-only: Operations of (re)installation are made only on ports which require manual interaction. This option conflicts with -A.

-j

--delete-then-reinstall: Delete unneeded leaf and obsolete packages before (re)installation of needed ports. The effects of this option will be manifested as possible shortening of the total time for (re)installation (by avoiding unrecognized conflict with moved ports) in exchange of possible longer abscent times of moved packages. This option is suitable for builder chroot environments.

-k

--suppress-self-upadte: Keep portsreinstall itself untouched. This option suppresses upgrade, deinstallation and reinstallation of the currently installed portsreinstall.

-l

-use-legacy-package-for-missing-pkgng: If prebuilt modern-style packages are missing, use of corresponding legacy ones is attempted instead by converting them to the modern-style using pkg2ng. This option is effective only when -G option is enabled and the modern-style is employed for the current packages system, but will not be harmful even in the other cases.

-n

--dry-run: No operation is carried out (just for seeing what will be done). This option is effective for do and redo commands so that no deinstallation and (re)installation process is actually carried out. By this option, ports/packages to be deinstalled or (re)installed can be confirmed without making changes to the current situation of packages.

-q

--skip-unchanged-ports: Only new ports and their dependents are reinstalled. This option is convenient when the all of the major version of the system and configurations of ports (options and knobs) are unchanged. The behavior can be modified by -b option.

-s

--avoid-vulnerability-check: Build of vulnerable ports are avoided by triggering errors. Note that already installed vulnerable packages are untouched. If you desire to uninstall them, do it manually.

-X

--deselect-all: Automatically deselect all candidates for deinstallation of leaf or obsolete ports. This option conflicts with -Y option.

-x

--no-exec-inst-script: Execution of scripts in pre-installation, post-installation and deinstallation defined for each package is skipped. This option is suitable for package build in portsreinstall-chroot(8) or other change-rooted/jailed environment.

-Y

--select-all: Automatically select all candidates for deinstallation of leaf or obsolete ports. This option conflicts with -X option.

Group 5: Saved and transferred to restarted runs, renewable by -N option

Option settings in this group are saved at the first do or prepare run, and transferred to the following runs. Reset of the saved values for this group is available in the initial run of redo command by appending -N with newly specified options.

-O glob1[:glob2[:...]]

--target-only-itself=glob1[:glob2[:...]]: Restrict (re/de)installation within a scope consisting of the specified target ports and their missing build-time requirements. Available ports matching the specified target globs are automatically registered as necessary ports equivalently to portsreinstall need add command. Records of successful (re)installation for the all ports in the scope are deleted so that they are to be reinstalled forcedly. So this option is usable for fixing packages whose data in the package database or content files are broken. Without -i option, target ports must be already installed or inspected. Without -o option, the temporary database is maintained to have complete data on dependencies of all installed and necessary ports. Combination with options -T and -t is available. The behavior can be modified by -B, -b, -i and -o options. Use of this option should be just for temporary diagnosis or salvage.

-o

--only-target-scope: Ignore ports which are outside of target scopes of -O, -T or -t options. Inspection of dependencies is made within the least-required scope for (re)installing the targets. If given with -O, targets themselves and their missing direct build-time requirements are in the scope. If given with -T, targets themselves, their direct requirements and their missing direct build-time requirements are in the scope. If given with -t, targets themselves, their already-inspected dependents and their missing direct build-time requirements are in the scope. Ports outside of the scopes are kept untouched even if they are updated. It is noted that this option disables detection and deinstallation of new leaf ports. This option will be useful for quick upgrades or new installation (only) before complete construction of the temporary database for the all installed ports. Use of this option should be just for temporary diagnosis or salvage.

-T glob1[:glob2[:...]]

--target-and-requirements=glob1[:glob2[:...]]: Restrict (re/de)installation within a scope consisting of the specified target ports, their requirements and their missing build-time requirements. Available ports matching the specified target globs are automatically registered as necessary ports equivalently to portsreinstall need add command. Records of successful (re)installation for the all ports in the scope are deleted so that they are to be reinstalled forcedly. So this option is usable for fixing packages whose data in the package database or content files are broken. Without -i option, target ports must be already installed or inspected. Without -o option, the temporary database is maintained to have complete data on dependencies of all installed and necessary ports. Combination with options -T and -t is available. The behavior can be modified by -B, -b, -i and -o options. Use of this option should be just for temporary diagnosis or salvage.

-t glob1[:glob2[:...]]

--target-and-dependents=glob1[:glob2[:...]]: Restrict (re/de)installation within a scope consisting of the specified target ports, their dependents and their missing build-time requirements. Available ports matching the specified target globs are automatically registered as necessary ports equivalently to portsreinstall need add command. Records of successful (re)installation for the all ports in the scope are deleted so that they are to be reinstalled forcedly. So this option is usable for fixing packages whose data in the package database or content files are broken. Without -i option, target ports must be already installed or inspected. Without -o option, the temporary database is maintained to have complete data on dependencies of all installed and necessary ports. Combination with options -T and -t is available. The behavior can be modified by -B, -b, -i and -o options. Use of this option should be just for temporary diagnosis or salvage.

Group 6: Saved and transferred to restarted runs, renewable by -L option

-P

--load-pkgtoolsconf-as-override: Import settings from pkgtools.conf(5) as the secondary. This option is effective only when portupgrade(1) is installed. For duplicated configurations, values in portsreinstall.conf are applied first and then those in pkgtools.conf(5) are. This option overrides preceding -p option.

-p

--load-pkgtoolsconf-as-default: Import settings from pkgtools.conf(5) as the primary (default). This option is effective only when portupgrade(1) is installed. For duplicated configurations, values in pkgtools.conf(5) are applied first and then those in portsreinstall.conf are. This option overrides preceding -P option.

-Q

--ignore-pkgtoolsconf: Ignore pkgtools.conf(5) even if it exists.

DETAILS

Overview

This utility is a ports/packages management tool which upgrades packages to be as much as consistent regarding their dependencies by allowing repetitional retrials called "redo runs". The implementation of this utility is designed for smart entire reinstallation of installed packages which takes a very long time. A temporary database is used for managing the starting point, intermediate status and goal of the upgrade processes. The starting point is determined by the initially installed packages. The intermediate status contains various information on successes and failures of ports to judge the necessity of upgrade considering dependencies during redo runs. The goal is determined by the ports tree, replacement to compatible ports and configurations for each port, i.e., port options configured by make config and knobs (make environment variables and make arguments). In order to make this utility work smartly, knob should be defined in ${LOCALBASE}/etc/portsreinstall.conf (or pkgtools.conf(5) if portupgrade(1) is installed) but not in /etc/make.conf The temporary database is available until the ports tree is updated. Each port is built for its (re)installation in principle, however, -G option enables a function that use of prebuilt packages in remote servers are attempted for each port matching a condition that all configurations of the port and its requirements are unchanged from the default.

The algorithms of this utility are originally optimized for massive reinstallation to be invoked after major upgrade of the system where reinstallation of all third-party applications is encouraged before cleaning up obsolete system libraries. Nevertheless, the all functionalities of this utility is applicable to any situations where complete reinstallation is preferred for the whole or typical packages, e.g., when you have been lazy in upgrade of ports for a too long time.

In a standard case, the entire reinstallation will proceed in the following step:

1.: Update the ports tree and (except in case of the legacy package system) the pkg(8) repository catalog;
2.: Clean up a temporary database which stores the all information used for the whole task of reinstallation (first do or prepare run);
3.: Record the snapshot of the all installed packages as the initial point of reinstallation (first do or prepare run);
4.: Build up the database to store data on configurations and dependencies (which may depend on the configurations) of the all (re/de)installing packages to be the latest version according to the current ports tree (do or prepare run);
5.: Select obsolete packages or new leaf ports/packages to deinstall or exclude if any (do or prepare run);
6.: Execute the actual (re/de)installation (do run);
7.: If some ports failed, attempt manual troubleshooting by make modification to the temporary database or manually re/de-install packages in concern (ok, taboo, freeze, noneed, reconf, etc.).
8.: Execute the retrial of the (re/de)installation (redo run);
9.: Repeat from 7 to 8 until the all available ports succeed.

A typical instance of actually executed commands for upgrading packages will be in the following flow:

1.: portsnap fetch update
2.: pkg update

(No need for the legacy packages system)

3.: script

(Enter script(1) environment for logging)

4.: portsreinstall

(Possibly end up with some failed ports)

5.: ... Countermeasure to failures ...
6.: portsreinstall redo

(Possibly end up with some failed ports)

7.: ... Repetition from 6 to 7 ...
8.: portsreinstall redo

(All available ports succeed)

Here, in case of entire reinstallation after a major version upgrade of the operating system, -q option should be removed. It is usually encouraged to execute the command on script(1) in order to record the make outputs for catching reasons of failures if any. In many cases, the user may undergo failures in build or installation of some ports during do/redo runs. Refer to subsections entitled "Workaround for failed ports: ..." for the techniques and procedures to resolve the problems.

This utility is implemented to be flexible and robust about interrupt/restart operations so as to allow the users to run only when the machine is free and terminate when it becomes busy on demand. Concretely, the users can stop the process by CTRL+C (or even by unexpected termination) and restart from the stopped point at any stage throughout the whole task, i.e., from the beginning of preparation of the temporary database to the end of (re/de)installation. This functionality allows the users, for example, to start this utility before lunch, terminate after lunch, restart before dinner, terminate after dinner, restart before going to bed, terminate after breakfast, restart before lunch, ..., and finally complete.

Compatibility with portupgrade(1) is well considered if it is installed; settings in pkgtools.conf(5) are reflected and the portupgrade database is updated at the end of each do/redo run. The main difference of this utility with portupgrade(1) or portmaster(8) is that this utility is optimized for on-the-fly entire upgrade of the packages environment to be as complete as possible while the latter are for partial upgrade to get the latest versions available as quickly as possible by applying the least "patches" to the environment. While this utility also has options for partial upgrade (-t, -T and -O), they are intended as auxiliary modification of (re)installing packages. This utility resolves conflicts between old and new packages automatically by referring to CONFLICTS, CONFLICTS_BUILD and CONFLICTS_INSTALL defined for each port or by redo processes for ports missing appropriate definition of them. Many of the UPDATING advisories on trouble shooting in packages upgrade will be unnecessary to follow if the users use this utility instead of portupgrade(1) or portmaster(8).

If this utility has been installed by ports/packages and the corresponding port is renewed, upgrade of this utility is carried out first and then the following processes are continued by the new version after cleaning up the temporary database.

New leaf ports (primary leaves) and their exclusive requirements (secondary leaves) are automatically detected. The user may specify unneeded ports explicitly by portsreinstall noneed command so that the specified ports are added to the new leaves if they have no non-leaf dependents. If any leaf ports are detected, the user is prompted to select ones to delete by dialog. Similarly, obsolete ports to delete are also selectable by dialog. Here, both for leaf and obsolete ports, the selection can be modified afterward and unselected ones can be restored by following redo runs.

The scheme of this utility is divided into stages of temporary database construction and actual (re/de)installation. Execution by portsreinstall prepare procedes to the end the first stage, and that without any argument procedes to the end of the second stage. Each of these two major stages is divided into minor stages. When a previously terminated process is restarted, completed minor stages are skipped. Changes of the configurations made or notified by corresponding options or commands of this utility are reflected to the database by automatically re-executing the concerned stages in the following do/redo runs.

When option(s) -O, -t or -T is/are specified, only the targets and their requirements or dependents within the specified scope are inspected and (re/de)installed. Inspected data on the ports are preserved in the temporary database and reused in the following runs even if the targets are reset or changed with aid of -N option. The preserved data are basically harmless even if the corresponding ports are out of scope of new targets unless conflicts arise.

New installation of ports: case 1

If the user has not executed this utility for entire reinstallation/upgrade never or after the final clean up of the temporary database,

portsreinstall -i -Glq -oO globs

will be a quick way to install new packages matching glob patters globs (here the options are separated into each group just for easy understanding). More automated operation is possible by appending -C and -Y options so as to skip all dialog queries by letting all configurations default.

New installation of ports: case 2

If the user already has a complete temporary database by executing entire reinstallation/upgrade,

portsreinstall need add globs

followed by

portsreinstall -MGlq -N redo

will be a quick and smart way to install new packages matching glob patters globs. More automated operation is possible by appending -C and -Y options so as to skip all dialog queries by letting all configurations default.

Deinstallation and restoration of leaf and obsolete ports

If any new leaf ports and obsolete (lost) ports are detected, dialogs are displayed for selecting which ports/packages to delete. Here it is noted that detection and operations on new leaf ports are invalidated when the temporary database is incomplete due to -o option. Deinstallation and restoration of the packages are carried out after reinstallation of the all ports are attempted. The selected packages are backed up before deinstallation.

The selection can be modified by executing

portsreinstall reselect leaves

for new leaf ports, and

portsreinstall reselect obsolete

for obsolete ports.

Then following execution of

portsreinstall

deinstalls newly selected packages and restores unselected ones.

Package build in a chroot system

For smooth and safe on-the-fly port upgrading, the pakage build can be carried out in a chroot(8) system using portsreinstall-chroot(8). Refer to its manual page for the detail.

Port upgrading of a cluster system

In a cluster system where multiple nodes have the same software configuration, packages built in one node can be used for upgrading ports in the others. The conventional flow will be as follows:

1. Share file systems for /usr/ports/packages and /home among the nodes by NFS. Here we assume that the mount points are the same among the all nodes and user "admin" is used for the system administration.

2. In the builder node, update the ports tree and the package repository catalog.

root@[builder.cluster.intranet] portsnap fetch update

root@[builder.cluster.intranet] pkg update

3. Upgrade the all pakages completely by portsreinstall (the command line options are just an example).

root@[builder.cluster.intranet] portsreinstall -qC

4. Create the packages and dispatching script set. This process can be skipped when portsreinstall-chroot(8) was used in the previous stage.

root@[builder.cluster.intranet] portsreinstall packupgrade create

5. Crop and extract the archive of the created dispatching script set.

root@[builder.cluster.intranet] portsreinstall packupgrade crop /home/admin/portsreinstall-upgrade.tar.gz

root@[builder.cluster.intranet] rm -rf /home/admin/portsreinstall-upgrade

root@[builder.cluster.intranet] mkdir -p /home/admin/portsreinstall-upgrade

root@[builder.cluster.intranet] tar xzf /home/admin/portsreinstall-upgrade.tar.gz -C /home/admin/portsreinstall-upgrade

6. Execute the dispatched script at each target node.

ssh -l admin target01.cluster.intranet sudo /home/admin/portsreinstall-upgrade/portsreinstall-upgrade.sh

Workaround for failed ports: conflicts: case 1

This utility automatically resolves conflicts between ports/packages by temporary deinstallation during concerned build or installation processes according to CONFLICTS, CONFLICTS_BUILD and CONFLICTS_INSTALL defined by each port. This means that ports lacking proper CONFLICTS, CONFLICTS_BUILD and CONFLICTS_INSTALL definitions may undergo failures.

Some problems may be due to coexistence of the same or related software of different versions of the requirements or the unsuccessful ports themselves. For analysis of the upgraded and initial requirements,

portsreinstall show requirements @all,direct glob

and

portsreinstall show initrequirements @all,direct glob

can be used, respectively, where glob denotes a glob specifying the unsuccessful ports. For analysis of ports for the same software of different versions,

portsreinstall glob glob_req

can be used, where glob_req denotes a glob specifying the suspicious requirements.

If a conflict is found, first escape (back up and delete) the conflict by

portsreinstall escape package_conflict

where package_conflict is the conflicting package.

Then execute

portsreinstall redo

for completing (re)installation of the dependents of the resolved port.

If the (re)installation is successful, execute

portsreinstall restore package_conflict

to restore the escaped package (reinstallation may fail if a newer version is installed but there is no problem). If any conflicts still remain unresolved, refer to case 2.

Workaround for failed ports: conflicts: case 2

If some upgraded ports fundamentally conflict with each other, the output log of this utility for do/redo processes will report a message entitled "The following ports are temporarily deleted due to conflicts". In this case, although not always encouraged, the user may consider replacement of the conflicting ones with one of them by assuming and expecting compatibility between them.

This workaround can be done by the following procedure. First edit the configuration file ${LOCALBASE}/etc/portsreinstall.conf so as to define the corresponding REPLACE_* values. If portupgrade(1) is installed, the corresponding configuration should be made in ALT_PKGDEP section of pkgtools.conf(5).

Then execute

portsreinstall -L redo

(Recover options reset by -L if any by checking with the aid of portsreinstall options.)

for (re)installation of the concerned port by reflecting the configuration changes.

Workaround for failed ports: conflicts: case 3

If some conflicts are found to be unnecessary, they can be removed in the following way. The unnecessary ports are registered by executing

portsreinstall noneed add globs_unnecessary

where globs_unnecessary denotes globs specifying the unnecessary ports.

Next execute

portsreinstall show dependents @run,full globs_unnecessary

for checking whether any dependents seem unnecessary for the user. Next execute

portsreinstall noneed add globs_unnecessary_dependents

to register them, where globs_unnecessary_dependents denotes globs specifying the unnecessary dependents.

Then execute

portsreinstall

(If it seems effective, reconfigure options for -B, -b and -o by using -M and -N.)

for automatic evaluation and deinstallation of new leaf ports redefined by the new unnecessary ports. If the registered ports are required by any non-leaf or non-unnecessary ports, this process will end up with no practical progress.

Workaround for failed ports: reconfiguration of port options

If the problems may be resolved by reconfiguration of the port options, execute portsreinstall reconf glob which invokes the dialog for reselecting options. Here glob denotes a glob specifying the concerned port.

If any change was made here, execute

portsreinstall

for retrial of (re)installation.

Workaround for failed ports: reconfiguration of knobs

If the problems may be resolved by reconfiguration of the knobs, first edit the configuration file ${LOCALBASE}/etc/portsreinstall.conf so as to redefine the corresponding CONFLICT_*, MARG_*, MENV_*, BEFOREBUILD_*, BEFOREDEINSTALL_* or AFTERINSTALL_* values. If portupgrade(1) is installed, the corresponding configuration should be made in MAKE_ARGS, MAKE_ENV, BEFOREBUILD, BEFOREDEINSTALL or AFTERINSTALL section of pkgtools.conf(5).

Then execute

portsreinstall -L redo

(Recover options reset by -L if any by checking with the aid of portsreinstall options.)

for (re)installation of the concerned port by reflecting the configuration changes.

Workaround for failed ports: update of ports tree

If the ports are fundamentally broken, updating the ports tree may fix the problems. In this case, execute

portsnap fetch update

and, followed by

pkg update

and then

portsreinstall reset keepstatus

to clean everything but the option settings, information of the initially installed packages and the progress of unchanged successful ports. Then execute

portsreinstall

and see whether the problems are resolved.

ENVIRONMENT VARIABLES

The following environment variables can be used to change the behavior of portsreinstall. Some of them are the same as defined in ports(7). Usually they should be kept to be the system default or empty. The configuration file (${LOCALBASE}/etc/portsreinstall.conf) takes priority over environment variables.

LOCALBASE: Where to install files of native applications. The default is /usr/local.
LINUXBASE: Where to install files of Linux applications. The default is /compat/linux.
PREFIX: Where to install things in general. The default value is defined for each port; it is usually ${LOCALBASE} for most native applications and ${LINUXBASE} for Linux applications. However, some ports customize it to their own defaults, e.g., ${LOCALBASE}/kde4 for KDE4-related ones. Therefore, it is safer to keep this variable undefined by the user so as to define it automatically although traditionally this variable has been used for controlling installation paths by users.
PORT_DBDIR: Where to store port option values. The default is /var/db/ports.
PORTSDIR: Location of the ports tree. The default is /usr/ports.
DISTDIR: Where to store distfiles. The default is ${PORTSDIR}/distfiles.
PACKAGES: Where to store package archives. The default is ${PORTSDIR}/packages.
PKGREPOSITORYSUBDIR: Subdirectory under ${PACKAGES} to store substances of package archives. The default is "All".
PKGREPOSITORY: Where to store substances of package archives. The default is ${PACKAGES}/${PKGREPOSITORYSUBDIR}.
PKG_PATH: Overriding Synonym to PKGREPOSITORY for compatibility with pkgtools.conf(5).
PACKAGECHECKSUMROOTS: Roots of available sites for legacy package check sum to be randomly selected. Each site is separated by "|" (vertical bar). The default consists of ftp://ftp.FreeBSD.org/ and from ftp://ftp1.FreeBSD.org/ to ftp://ftp14.FreeBSD.org/.
PACKAGECHECKSUMDIR: Subdirectory pattern of legacy package check sum sites. The first and second %s are substituted with the platform and version of OS, respectively. The default is pub/FreeBSD/ports/%s/packages-%s/All/.
PACKAGEROOTS: Roots of available legacy package sites to be randomly selected. Each site is separated by "|" (vertical bar). The default is ${PACKAGECHECKSUMROOTS}, but it is relatively secure to be different from it.
PACKAGEDIR: Subdirectory pattern of legacy package sites. The first and second %s are substituted with the platform and version of OS, respectively. The default is ${PACKAGECHECKSUMDIR}.

FILES/DIRECTORIES

The following files and directories are referred to.

${LOCALBASE}/etc/portsreinstall.conf: Configuration file.
/var/tmp/portsreinstall.db: Temporal database directory. The whole contents can be saved by portsreinstall save command.
${LOCALBASE}/etc/pkgtools.conf: Configuration file of portupgrade(1).

HISTORY

This utility first appeared as Version 0.9.0 released on November 21, 2010. pkg(8) is supported since Version 2.1.0 released on December 10 2012. Flavor is supported since Version 4.0.0 released on June 29, 2018.

APPENDIX: Conditions determining the package system (general specification of FreeBSD Ports/Packages)

The current package system is automatically detected according to the OS version whose serial number is obtained by

sysctl -n kern.osreldate

and the configuration of /etc/make.conf. Pkgng is supported for systems with serial numbers of 800505 (just before 8.1-RELEASE) or later. For systems of 1000017 (10-CURRENT) or later, Pkgng is the default.

In order to choose pkgng in systems from 800505 (just before 8.1-RELEASE) to just before 1000017 (10-CURRENT), put

WITH_PKGNG=yes

WITH_PKG=yes

in /etc/make.conf.

In order to choose the legacy one in systems of 1000017 (10-CURRENT) or later, put

WITHOUT_PKGNG=yes

WITHOUT_PKG=yes

in /etc/make.conf.

COPYRIGHT

This software is distributed under the 2-Clause BSD License.

Email: sakaue.mamoru@samurai.mwghennn.net

Homepage: http://www.mwghennndo.com/software/portsreinstall/