NAME

SYNOPSIS

DESCRIPTION

dreaderd maintains overview information locally but expects to retrieve article bodies from remote spool sources via the 'article <message-id>' NNTP command. It implements the entire NNTP command set including xover and xhdr extensions. It is also able to maintain a local article cache though this feature is not always appropriate. You can create a multi-level cache by having the leaf dreaderd use another dreaderd for its spool access. Several topologies are typical.

First, you run the Diablo feeder side (the diablo server) on the same machine as you run dreaderd. This allows you to send multiple feeds to the diablo server and then have the server send a single header-only feed to dreaderd running on the same machine. In this configuration you normally run dreaderd on ports 119 and 434 and run diablo on port 435 and you normally turn OFF article caching in dreaderd, instead using diablo's spool on the same box for your first level cache. The diablo feeder side can be configured with a small spool or a huge spool depending on what kind of caching you want to do before dreaderd backs off to a remote spool. If you configure a huge diablo feeder spool, it can act as the sole spool source to dreaderd.

Second, you run dreaderd standalone on a machine. You must still supply a feed to dreaderd from which it can extract the headers and maintain its overview database (a 9G disk is recommended for /news/spool/group for overview storage). Note that you should supply only a single feed to dreaderd because dreaderd itself does not maintain a history file. dreaderd can take a header-only feed or a full-article feed depending on what you use to feed it. You then configure dreaderd to obtain it's spool from one or more off-machine servers.

Third, you can run dreaderd as a mid-level cache for a leaf dreaderd. In this configuration dreaderd is used solely for 'article <message-id>' retrievals and not for news reading. You do NOT have to supply a feed to dreaderd in this case so you do not need a 9G disk for overview information. Your active file can be minimal since it is not really needed. You need a disk sufficiently sized to handle the article cache you wish to maintain on the machine. In this configuration you use the intermediate dreaderd to cache articles from offsite spools and have your leaf dreaderd talk to the midlevel dreaderd (the leaf dreaderd still needs a feed in order to maintain overview information).

dreaderd maintains an active file, dactive.kp, which is a KP database. That is, it is human-readable and machine-writable (human editable only if no processes have the file locked). The

dsyncgroups program may be used to initially create dactive.kp from an INN box. There are several ways to maintain dactive.kp ... you can use dsyncgroups to maintain the article range from a remote server even if the remote server's article numbers are not synchronized with yours. The expireover program will also adjust the beginning article number in an attempt to maintain sufficient free space in the overview partition. The expireover program then deletes overview data files containing articles that are outside the article range. Both methods may be used together. dreaderd maintains overview data files in blocks of 256 articles and can only delete whole blocks, so no less then a 4G disk should be used for the overview partition.

dreaderd requires dactive.kp, dreader.access, dserver.hosts, and diablo.config to be configured properly. It also requires the /news/log directory to exist. Realtime human-readable status information is stored in /news/log/dreaderd.status as well as on the process command line (i.e. via 'ps' on many systems). See the sample files for more information. The most critical configuration for dreaderd is dserver.hosts and the command line arguments given to dreaderd when it is run. See the sample rc.news file for more information.

The dreaderd cache is optional but usually used, even if only a small partition is available, to reduce the bandwidth required to pull articles from the spool machine. However, if you are running the diablo feeder on the same machine you usually use that as dreaderd's first level cache and thus do not use dreaderd's own internal cache. dreaderd's cache configuration is stored in diablo.config. The default is for the cache to be turned on.

-d[#] turns on debugging.

-f[0] Turns on or off the fast-copy feature. The default is ON. -f0 will turn it off. The fast-copy feature allows Diablo to start sending data received from remote spools to requesting clients as it is received from the remote spool. However, this means that if the remote spool dies in the middle of the transfer, the client will receive a truncated article. If the fast-copy option is off, dreaderd buffers the entire article before sending it to the client and is able to transparently restart the request if the remote spool dies in the middle of the transfer.

-p newspathname/0 specify host name for Path: header, '-p 0' for none. See also the ``readerpath'' option in diablo.config(8).

-s argv-buffer-space-for-ps-status specify an argv buffer area that status can be stored into to show up in a ps.

-T txbufsize specify the size of the transmit buffer for NNTP connections. The default is 4K.

-R rxbufsize specify the size of the receive buffer for NNTP connections. The default is 2K.

-F maxfeedforks specify the maximum number of non-NNTP forks from feeds. 4 is a good number. Any connections which resolves to 'f' in dreader.access where the 'r' and 'p' flags are NOT also set will be routed to one of the feed-only forks. This is desireable because an incoming feed is a high-load item and can slow down reader threads running on the same process. This option will override its diablo.config counterpart.

-D numdnsprocs dreaderd forks off a number of processes to handle DNS authentication, allowing several incoming connections to be resolved in parallel. Typically you want one dns thread for every 90 or so users with a minimum of four. So, for example, if you configure dreaderd to handle up to 750 users, you would want 8 or 9 DNS forks. This option will override its diablo.config counterpart.

-M numreaderprocs specify the number of reader processes, default is 10. Each reader process can handle several connections in parallel but opens a connection to each spool and post server, so be sure that the servers can handle the number of reader processes you fork (for example, if the diablo server is doing spool/post duty, make sure the maxconnect field in diablo.config and the maxconnect directive in dnewsfeeds is set high enough). This option will override its diablo.config counterpart.

-N maxconnectperreaderproc specify the number of connections (threads) per reader process. Default is 40 (10x40 = 400 maximum reader connections by default). You need to be careful of hitting file descriptor limits and I would not set it higher then 40. It is usually a better idea to set this limit lower, to around 25, and to increase the number of forks (-M option) in order to increase I/O parallelism. This option will override its diablo.config counterpart.

-c delayedclosesecs enable and specify the number of seconds to delay the closing of a failed connection attempt. This can help against clients that continually attempt to connect multiple times per second, even when rejected with a 500 error code. The number of seconds is not an absolute value and can vary up to an extra 10 seconds, depending on how often new connections are made to the server. The minimum value is 10 seconds if the server receives no new connections during that time.

-h reportedhostname specify the reported host name, otherwise the hostname() system call is used. This option will override its diablo.config counterpart.

-P [bindhost:]port specify the host (if not INADDR_ANY) and port to bind to. Default is INADDR_ANY port 119. If you specify this option once you override the default. If you specify it a second time (third, fourth, etc...) you add additional bindings. Normally one uses '-P 119 -P 434' to support traditional inews/trn. See also the ``readerbindport'' and ``readerbindhost'' options in diablo.config.

-B bindhost:[port] This is a synonym for -P so that the same option is used by other diablo utils, although the bindhost is the default option if no options in diablo.config.

-x xrefhost specify the hostname as it appears in the Xref: line of the host we accept Xref: headers from. If not specified, or the host does not match, the Xref: header will be ignored and article numbers will be assigned from the active file. See also the ``readerslavexrefhost'' option in diablo.config.

dreaderd will assign article numbers based on the supplied Xref: header or, if the header does not exist, will assign article numbers in sequence.

CRON JOBS

SEE ALSO