.TH SLURP 1 "7 Feburary 1995" "V1.10" .SH NAME .I slurp \- retrieve netnews articles from a remote NNTP server .SH SYNOPSIS .I slurp [ .B \-g .I newsgroups/distributions ] [ .B \-t .I time ] [ .B \-a .I username/password ] [ .B \-o .I output ] [ .B \-p .I portno ] [ .B \-cdhilmnrwx ] .I hostname[ /sublist ][ :timefile ] .SH DESCRIPTION .PP .I Slurp is an advanced passive NNTP client for UNIX. It will connect to a remote NNTP server and retrieve articles in a specified set of Usenet newsgroups that have arrived after a particular time (typically the last time it was invoked) for processing by your local news system. .SH OPTIONS .TP .BI \-g " newsgroups/distributions" Retrieve articles from .IR newsgroups , and optionally with .IR distributions \. This option will override all entries for .I hostname in the .B slurp.sys file, including flags, output and authinfo settings. .TP .BI \-t " time" Retrieve articles that have arrived at the server after .IR time , rather than using the time taken from .BR slurp. \. This is in the standard NNTP time format of 'YYMMDD HHMMSS' GMT. For example, midnight GMT on the 1st of December 1992 will be '921201 000000'. Note that you will need to use quotes around the time and remember that it is is in GMT, not the local or server time. .TP .BI \-a " username/password" Send .I username and .I password to the server using the NNTP simple authorisation protocol. .TP .BI \-o " output" Set the directory in which news batches will be written or, if .I output begins with a '|', the name of a program to which articles will be piped. .TP .BI \-p " portno" Specify the port number to use for the NNTP server, rather than using the one normally defined for "nntp" in .BR /etc/services \. .TP .B \-c Omit the article collection stage. This can be used to write the Message-IDs of waiting articles to .B slurp. so they can then be further processed a later collection stage. .TP .B \-d Enable debugging. This diverts reporting to stderr instead of syslog and turns on extra debugging output. .TP .B \-h Don't check if article already exists on the local news system. This can be used when creating news batches for a remote system or when debugging .I Slurp. .TP .B \-i Don't load any uncollected Message-IDs that are waiting in .BR slurp. \. .TP .B \-l Local time is used to set the start time in .B slurp. for the next news retrieval, rather than setting the start time through a call to the tcp time service at the remote server. .TP .B \-m Don't write the start time for the next news retrieval to .BR slurp. \. .TP .B \-n Omit the NEWNEWS stage, which collects the Message-IDs of articles waiting at the server. This can be used when it is required to only retrieve articles whose Message-IDs are stored in .BR slurp. \. .TP .B \-r A 'MODE READER' command is sent to the remote server. This is used when connecting with an INN site that needs to be switched from innd to nnrpd so the NEWNEWS and ARTICLE commands can be issued. .TP .B \-w Don't write the Message-IDs of any uncollected articles to .BR slurp. \. .TP .B \-x Display progress indication. During the NEWNEWS stage, the number of Message-IDs collected so far is displayed on stdout. During the article collection stage, the number of articles collected so far out of the total waiting is displayed. .TP .I hostname The hostname of the remote NNTP server to connect to. This must be specified. .TP .I [/sublist] If this is specified, then the entry in .B slurp.sys which contains .I /sublist after the hostname will be used. This lets you take different groups from the same server at different times. .TP .I [:timefile] Use this to specify an alternate extension to .BR slurp. \. Normally .I Slurp will use the hostname, but this can cause problems on file systems with short filenames. .SH "CONFIGURATION FILES" .PP There are two configuration files used by .IR slurp \. .SS slurp.sys .PP Entries in .B slurp.sys take the form .IP hostname/sublist:groups/distributions:flags:username/password:output .PP This format should be familar to people who have used the C News .B sys file. Entries for a particular host can be continued on more than one line by using a '\\' at the end of the line. e.g. .IP hostname:group1,group2,\\ .br group3,group4,group5 .PP .I Slurp is even more picky about the presence of whitespace than C News. It can only appear in comments. Comments begin with a '#' and continue to the end of the line. .PP Using distributions is .B not recommended - they're only really included for completeness. Under many NNTP implementations they will result in a large extra load on the server and increase the total amount of time for your connection. .PP There are 4 possible flags: .BR i , .BR l , .BR r " and" .BR w , which have the same meaning as the command line options. If present, .I username and .I password will be sent to the server using the NNTP simple authorisation protocol when the connection is first made. If present, .I output sets the directory in which news batches are to be written or, if it begins with a '|', the name of a program to which articles will be piped. .SS slurp.[.] .PP The file .B slurp. contains the time when .I Slurp last connected to the NNTP server at .IR hostname . If a sublist has been specified with the .B \-s option then this will be appended with a period to the name. .I Slurp can then use this time to pick up all the articles that have arrived at the server since the last session. It may be followed on subsequent lines by a list of Message-IDs of articles that are to be retrieved from the server in the next session. .PP Each time .I Slurp is run and .B slurp. updated, the current .B slurp. will be backed up in the file .BR slurp..o \. .SH OPERATION .PP When run, .I Slurp will first retrieve the appropriate newsgroup list, distribution list and start time for the specified server, either from the configuration files or overriding those settings with the command line options. .PP If the .B \-m option is not set, then the current time will be obtained to use as the start time for the next session. If the .B \-l option is specified, this will be taken from the local machine, otherwise it will be retrieved from the remote server through a call to the tcp time service there. If .B \-i is not specified, then the Message-IDs of any articles which were not retrieved in the last session will be loaded from .BR slurp. \. .PP .I Slurp will now connect to the NNTP server at the remote host. If a username and password for use with the NNTP simple authorisation protocol have been supplied then they will be sent to the server. If the .B \-r option is specified, then a 'MODE READER' command will be sent, to ensure at INN sites that .I Slurp is talking to nnrpd. .PP If the .B -n option has not been specified, a NEWNEWS request will now be issued, asking for all the articles that have arrived in the specified list of newsgroups since the specified time. The server will respond with a list of Message-IDs. If the .B -h option has been specified or a Message-ID is not already present in the local history file, then it will be stored in memory. If the list of newsgroups is too large to fit on one line (NNTP has a maximum line length of 512 characters) then a series of NEWNEWS requests will be carried out, adding further Message-IDs to the memory list if they are not already present. If there are too many Message-IDs to fit in memory then the overflow will be later written to .BR slurp. \. .PP Once the NEWNEWS has been completed, and if the .B -c option has not been specified, .I Slurp moves into an article retrieval stage. It will go through the list of Message-IDs in memory and request them in turn from the server, adding each article to the batch of articles being either stored in the incoming news directory or piped to another program, usually .I rnews. When a batch is found to be larger than the maximum size, it will be submitted to the news system. .PP Once all the articles have been retrieved, the final batch of articles will be submitted. If the .B \-m option has not been set, then the previously obtained time to use for the next NEWNEWS will be written to .BR slurp. \. If the .B -w option has not been set then any uncollected Message-IDs are also written to this file for possible later retrieval. .PP Statistics on the connection will be logged to syslog (or stderr if syslog is not available). The new article count is the total number of articles that have been submitted to the new system. The duplicate count is how many Message-IDs were found to already exist on the local system. If two NEWNEWS requests are necessary and a message ID was returned by both requests, then it will be included twice in the duplicate count. The missing count is those articles which were in the server's history file but didn't exist as actual article files, usually because they have been cancelled. If configured, the speed of transfer of the article retrieval stage will also be logged. .SH "RETURN CODES" .PP Slurp returns a series of return codes which may be useful to controlling programs:- .RS 0 - Successful completion .br 1 - General system error .br 2 - Incorrect arguments supplied or incorrect configuration files. .br 3 - Error occurred during attempt to connect to remote host .br 4 - NNTP Protocol error .br 5 - Insufficient disk space available for news batch. .RE .SH FILES /usr/lib/news/slurp.sys .br /usr/lib/news/slurp. .SH AUTHOR Stephen Hebditch .SH "SEE ALSO" news(5) .br rnews(8) .br RFC977 \- Network News Transfer Protocol (NNTP), .br RFC1036 \- Usenet Article Format standard.