Introduction

At this level, the libs can be used for any number of tasks. In the examples directory in the source distribution there is, for example, a script that lets one monitor the status of a domain via the Nagios monitoring system.

The system as distributed, however, contains more than the lib. There are also a couple of programs and a web interface. We will not talk about the web interface here, except to mention that it like the rest of the programs communicate with the other parts via the database tables to be mentioned later.

The programs

The program that actually runs tests is dnscheck-dispatch. It polls the table "queue" table in the database every 0.25 seconds, and forks children to run tests on what it finds there. It monitors the state of the children, and cleans up after they exit. If a child exits uncleanly, the test it was running is re-scheduled. If the same domain causes unclean exits five times or more, that domain will be blacklisted and will not be tested again until the dispatcher is restarted.

If the dispatcher receives a TERM signal, it will stop starting new tests, wait for the ones currently running to exit and then exit in good order. This may at worst take several minutes.

If the dispatcher receives a HUP signal, it will do an orderly exit as above, and then re-execute itself. If it was not originally started with a full pathname, the re-execution may fail.

The dispatcher is written to do a small number of tasks (get queue entry, start child, clean up after it) in as robust fashion as possible. As of this writing, the program appears to be very reliable as long as the database server is not too heavily loaded. Exactly what "too heavily loaded" means in practice will vary, of course, but as a rule of thumb, if a simple UPDATE takes more than a second to finish the system is almost certainly there.

It is vital that the queue table use the InnoDB storage engine. If it is MyISAM, there will be data corruption under heavy load. Other storage engines with sane transaction handling should also work, but have not been tested.

At the moment, only MySQL is supported. Patches to add support for more databases are welcome.

The dispatcher's child processes run the default tests on the given domain, write the results to the "tests" and "results" tables in the database and then exits.

The zone change detector

One of the programs that insert entries into the "queue" table is dnscheck-zonediff. This program does a zone transfer of a specified domain and looks at all the "NS" and "DS" records. For ever subdomain, it'll concatenate the textual representations of the records and calculate an MD5 sum of the resulting string. If that sum is not identical with what it was the previous time the program was run, the subdomain in question will be scheduled for testing (that is, it'll be written to the "queue" table, where the dispatcher will pick it up).

The change detector stores data between runs in a simple text file, with a zone name and its corresponding MD5 sum on each line.

At the moment, the time it takes to do the AXFR is limited by the speed at which the Perl module Net::DNS can create new records. It also uses a fair bit of RAM (100-150MB for around 750 thousand domains), so if you're planning to run it against ".com" you'll want a pretty hefty machine.

The results mailer

The dnscheck-12hourmailer script reads results of tests from the database tables "tests" and "results", aggregates them per registrar and sends an email to each registrar with a summary of serious problems found among their domains.

This is pretty specific to ".SE", particularly the bit that maps domain names to registrars and fetches contact information for the registrars. If you want to do something similar to this, you'll probably need to modify the code of the script. At the moment, the emails are also generated entirely programmatically, so changing their formatting also involves changing code.

The preflight script

dnscheck-preflight is used by the web GUI. It takes a name as a command line argument and prints the string TRUE if it's a domain and FALSE otherwise.

The hostsyntax script

dnscheck-hostsyntax is also used by the web GUI. It takes a name as a command line argument, and prints the string TRUE if it is a syntactically valid hostname and FALSE otherwise.

The CLI interface

dnscheck is a command line tool to check domains. Run it with the switch "--help" to get a short description of how to use it.

The database

There are a number of tables. Let's start looking at them from the top of, so to speak, the food chain.

The source table

Anythingt that can write to the "queue" table can be a source of domains to be tested. In order to make it possible to keep track of which source requested which test, there is the "source" table. It has only three fields, and is supposed to be mostly read-only. The fields are "id", "name" and "contact". The id is an autoincrementing integer as usual. The name is a human-readable name for the source. The contact is some sort of human-useable contact information for the source in question (email address, phone number, ham radio call sign,...).

One example of the use of source information is the mailer described above, which only mails results about tests that were scheduled for testing by the zone change detector, not once that came from (for example) the web GUI.

The queue table

This table has six fields apart from the autoincrementing "id". There is "domain", which is the name of a domain to be tested. There is "priority", which is an integer. dnscheck-dispatch will always pick the queue entry with the highest priority first (and if there are several with the same priority, the one with the lowest id). There is "inprogress", which is NULL to begin with and gets a timestamp written to it by the dispatcher when it's been scheduled for testing. There is a "tester_pid" field, where the particular child process that's running the test for the domain writes it's system PID (if we ever need to have more than one machince running tests, thre maybe a host field added here). "source_id" pints into the source table. "source_data" is any data specific to this test that is relevant for the specific source. The two source fields are simply copied over to the "tests" table when a test is started.

The tests table

This table has one row for every test run, an contains information on the domain tested, the start and finish times of the test, a summary of the results and the source information that was copied from the queue table. The fields are: "id" as usual, "domain" for the domain name, "begin" for the start time (with whole-second resolution), "end" for the finish time, "count_critical", "count_error", "count_warning", "count_notice" and "count_info" for the number of problems at the respecive severity levels, "source_id" to point into the source table and finally "source_data" to hold whatever data that source wants.

The results table

This table has a number of rows for every test. They reflect the list of results returned after a test is run, and several of the fields are rather specific to the web GUI application. The fields that can be useful to others are: "test_id", which points into the "tests" table. "timestamp" with is the time at which the log entry was entered (with microsecond resolution). "level" which is one of the all-caps strings CRITICAL, ERROR, WARNING, NOTICE or INFO. "message" which is the message tag that describes the reason the log entry was made. "arg0" to "arg9", which are arguments to the message.

The message tags are specified in YAML files.

The messages table

The messages table contains a copy of a localy YAML file. It is used by the web GUI.

The delegation_history table

This table contains information about which nameservers has historically been authoritative for domains. Some tests use it, if it's available.

The nameservers and nsp tables

The function of these tables have been obsoleted by the registrar system.

The domains table

A list of domains. More of a convenience thing than an actual part of the DNSCheck system.

Needed permissions

The various programs need different sets of permissions in the database in order to work correctly. It breaks down as follows:

• Everything that schedules tests needs INSERT, UPDATE and SELECT on "sources" and INSERT on "queue".
• Everything that reads test results needs SELECT on "sources", "tests" and "results".
• Anything that runs tests needs SELECT on "delegation_history" (or no database access at all).
• dnscheck-dispatcher needs a lot: INSERT, SELECT, UPDATE and DELETE on "queue", INSERT, SELECT and UPDATE on "tests", INSERT on "results" plus everything listed above.

In practice, the easiest reasonably secure setup is to give the web GUI one user with very limited permissions (INSERT, UPDATE, SELECT on "sources" and INSERT on "queue"), and one user with full rights on all tables for the rest of the system. Indeed, since all of the programs use the database connection info given in the site_config.yaml file, you will need to point out a different such file to any programs you want to use different connection information.

Files

config.yaml and site_config.yaml

These contain configuration information in YAML format. The "config.yaml" file is installed with DNSCheck, and should not be modified (if you do modify it, your changes will be overwritten when you upgrade DNSCheck). "site_config.yaml" overrides what is in "config.yaml", and is where local changes (such as database access information and TSIG keys) should be entered. It will not be touched by the installation process.

One key that is usually not in either of these files but that can be added if needed is the "disable" key. It has two levels under it, being the name a test module (converted to lowercase) and the name of a method in that module. If the value for the method is true, that method will not be run. Note that no kind of sanity checking is done, and that some tests rely on each other, so disabling one may make others give false results. Be cautious and know what you're doing. As an example, the following would disable all DNSSEC tests:

  disable:
      dnssec:
         test: 1

All test methods mentioned in the documentation for the respective modules are possible to disable this way.

policy.yaml and site_policy.yaml

"policy.yaml" specifies the severity of various problems. As distributed, it represents .SE's view on best practices for DNS. If you want to change anything, do so in "site_policy.yaml", or your changes will be overwritten on the next install.

locale/en.yaml and locale/sv.yaml

These files contain information on how to transform the raw test log entries into human-readable messages. If you want to do your own translation, you need to know that the "format" key is a sprintf() format string, and "args" the number of arguments after the format. If the latter number does not match the Perl code, a warning will be printed at runtime.