|
NAMEbbftp - transfer files using compression and parallel streamsSYNOPSISbbftp -vbbftp [Options] [-u RemoteUsername] -i ControlFile [RemoteHost] bbftp [Options] [-u RemoteUsername] -e ControlCommands [RemoteHost] DESCRIPTIONUse the bbftp command to transfer files between the local host and a remote host. In order to get better performance on a loaded Wide Area Network than transferring with the standard ftp command, use the bbftp command. bbftp has been designed to take advantage of the RFC 1323 and uses multiple streams is order to speed up transfer. It also implements an automatic retry in case of failure of commands contained in the ControlFile or in the ControlCommands.bbftp allows two methods of connection to the remote host, a direct connection on the control port of the remote bbftpd daemon or the ability to remotely start a bbftpd daemon through an ssh tunnel. For the first method, the bbftpd daemon has to be started (by inetd or as a standalone daemon) on the remote host and all user information (username, password) are transmitted encrypted to the daemon; for the second the bbftpd binary has to be accessible somewhere on the remote host and all control data are transmitted through the ssh tunnel. A third additionnal authenticate mode allows to use certificates to log on. This mode is based on the Grid Security Infrastructure and requires Globus software to be installed. The client side needs a certificate to identify itself and the daemon needs a host certificate. The behaviour of bbftp is controled by the ControlFile which contains the commands to be executed (option -i) or by commands separated by semicolons (option -e). The format of these commands are given in CONTROL COMMANDS. bbftp may be used in one of the following ways: % bbftp to print a short help % bbftp -v to write the version of the software and default values to standard output. % bbftp [Options] -i ControlFile [-u RemoteUsername] [RemoteHost] % bbftp [Options] -e ControlCommands [-u RemoteUsername] [RemoteHost] to request the execution of commands contained in the control file ControlFile or the ControlCommands using RemoteUsername on RemoteHost. Depending on the Options a password for the RemoteUsername on RemoteHost may be requested interactively (see OPTIONS section, for a full description of the Options). The RemoteUsername value is ignored in the certificate authentication mode. The remote user is automatically detected by the daemon using the Globus gridmap file. OPTIONSOptions can be separated into two classes, those describing the way bbftp connects to the bbftpd daemon and those modifying the behaviour of the control commands.CONNECTION OPTIONS
PROTOCOL OPTIONS
BEHAVIOUR OPTIONS
CONTROL COMMANDSThe control commands are either contained by an ASCII file ( -i option) or written on the bbftp line ( -e option). They can be divided into two classes, the "File related commands" and the "Behaviour commands".All "Behaviour commands" may be put in a .bbftprc file, but all "File related commands" are forbiden in that file. IMPORTANT NOTE Under the RFIO mode (see setoption remoterfio and
setoption localrfio ) all file related commands have to be given in
absolute mode.
FILE RELATED COMMANDS
BEHAVIOUR COMMANDS setoption " Option"
To negate an option just add "no" before the option (ie setoption nocreatedir). The options are the following : createdir All file-related commands will create missing directories if needed (default createdir). gzip All file transfers will be compressed using the gzip algorythm (default nogzip). keepaccess The access time and modify time will be kept on each file transferred (default keepaccess). keepmode The file mode will be kept on each file transferred (default keepmode). localrfio All local files will be created with RFIO functions (default nolocalrfio). remoterfio All remote files will be created with RFIO functions (default noremoterfio). qbss All the packets will be marked for QBSS (default noqbss). tmpfile All files will be created under a temporary name (FileName.bbftp.tmp.HostName.Pid) and renamed to the correct file name if transfer is successful (default tmpfile) setbuffersize " Buffersize" Set the size in Kbytes of the buffer used for reading or writing the files. This command set the local and remote buffer size. (Each stream will use the same buffer size) setlocalcos " LocalCos" Set the local COS to the value specified by LocalCos. This COS will be used for further rfio funtions. It is used if the setoption localrfio has been set and if the file is a HPSS file. A value of 0 allows to select the COS according to the file size. A negative value allows to not set the COS. The default value is 0. setlocalumask " LocalUmask" Set the local umask to the value specified by LocalUmask. This umask will be used for further i/o funtions. The LocalUmask has to be given in OCTAL setnbstream " NumberOfParallelStreams" Set the number of parallel streams to NumberOfParallelStreams. This number will be used for further transfer commands. setremotecos " RemoteCos" Set the remote COS to the value specified by RemoteCos. This COS will be used for further rfio funtions. It is used if the setoption remoterfio has been set and if the file is a HPSS file. A value of 0 allows to select the COS according to the file size. A negative value allows to not set the COS. The default value is 0. setremoteumask " RemoteUmask" Set the remote umask to the value specified by RemoteUmask. This remote umask will be used for further i/o funtions. The RemoteUmask has to be given in OCTAL setrecvwinsize " WindowSize" Set size in Kbytes of the receive TCP window of each stream of the bbftpd daemon. This also set the send window size of the client to the same value. setsendwinsize " WindowSize" Set size in Kbytes of the send TCP window of each stream of the bbftpd daemon. This also set the receive window size of the client to the same value. setackto " Acknowledge time-out" Set time-out (in seconds) to wait for an acknowledge. Default value is 100 setrecvcontrolto " Input control time-out" Set time-out (in seconds) to wait while reading on the control socket. Default value is 180 setsendcontrolto " Output control time-out" Set time-out (in seconds) to wait while writing on the control socket. Default value is 180 setdatato " Data time-out" Set time-out (in seconds) to wait while reading on the data socket. Default value is 300 NOTES If the option tmpfile is used then if the new file ( RemoteFile for a put or LocalFile for a get) did not exist before, bbftp ensures that the file transfer was correct if the file exists. In case of an already existing file, if the size, the last access and modification time are correct (if option keepaccess has been set) bbftp ensures that the file transfer was correct. TURL NOTATIONThe TURL notation can be used with commands relatives to files and directory.If you use this notation, you should not specify the server name in command. BBFTP support 3 types of TURL:
Remote TURL bbftp://bbftp_servername/path_to_file_or_directory The path to file can be relative or absolute (start with '/') For example:
Local TURL file://path_to_file The path to file can be relative or absolute (start with '/') For example:
Local rfio TURL rfio://path_to_file The path to file can be relative or absolute (start with '/') For example:
When the 'rfio://' notation is used, the option 'localrfio' is no
longer necessary.
Remote TURL with rfio It is possible to use rfio on the remote side using mixed TURL like: bbftp://bbftp_servername/rfio://path_to_rfio_file When the 'bbftp://bbftp_servername/rfio://' notation is used, the option 'remoterfio' is no longer necessary. RETURN VALUESThe following exit values are returned:
It may happend that a non-zero value is returned even if all files were correctly transfered, if during one transfer a retry was needed. This will be corrected in future releases. MESSAGES AND ERRORSAll informative messages are written to the standard ouput (or to the OutputFile ). All error messages are written to the standard error (or to the ErrorFile ).WARNINGThe bbftp client version 2.0.0 is unable to talk with a daemon in release 1.x.x.The rfioxxx or xxxrfio commands are no longer supported, use instead the options localrfio or remoterfio in conjunction with put and get commands to obtain the same result. RESULT FILEIf the -i option was used a result file will be created in the same directory as the ControlFile. Its name is ControlFile with the extension ".res". It contains the same lines as the ControlFile plus the keyword "OK", in case of success, or "FAILED", in case of failure.If the -e option was used and the -V option was not used, the software will print the command executed plus the keyword "OK", in case of success, or "FAILED", in case of failure to standard output. CONNECTION EXAMPLESbbftp -i ctrlfile -u jon -p 5 -c cchost.in2p3.frmeans that bbftp is going to connect to remote host cchost.in2p3.fr using username jon. If the connection is successful then the commands in ctrlfile will be executed. All transfer commands will use five streams and gzip compression. bbftp -i ctrlfile -u phg -s cchost.in2p3.fr means that bbftp is going to start a remote bbftp via sshd on host cchost.in2p3.fr using username phg. ssh will first try an RSAAuthentication if it is allowed by cchost.in2p3.fr; otherwise ssh will ask for a password for user phg on cchost.in2p3.fr. Then the sshd on cchost.in2p3.fr will log user phg and try to start the command "bbftpd -s" bbftp -i ctrlfile -u jon -E '/tmp/bbftpd -s' cchost.in2p3.fr Same behaviour as preceding, except that the remote command will be "/tmp/bbftpd -s" bbftp -i ctrlfile -u gilles -S cchost.in2p3.fr means that bbftp is going to start using ssh a remote bbftpd on host cchost.in2p3.fr using username gilles. ssh will try an RSAAuthentication if it is allowed by cchost.in2p3.fr, otherwise the connection will be broken. bbftp -e 'setrecvwinsize 1024 ; put file1 file2' -u jon cchost.in2p3.fr means that bbftp is going to connect to remote host cchost.in2p3.fr using jon username. If the connection is successful then the commands setrecvwinsize 1024 and put file1 file2 will be executed. All tranfer commands will use one stream. bbftp -e 'put file1 file2' cchost.in2p3.fr means (in the certification authentication mode) that bbftp is
going to connect to remote host cchost.in2p3.fr using a certificate. The
remote user will be detected by the daemon which will check for the
certificate provided and will accept or not the connection.
Using SSH to start a BBFTPD daemon linked with dynamic libraries If you have linked the daemon with dynamic libraries with -L/path/to/lib option, you need to specify this location in $LD_LIBRARY_PATH. To be taken into account by SSHD, this environment variable must be modified in the $HOME/.ssh/environment file. See your SSH or SSHD manual for more details. EXAMPLESUser jon want to transfer files from host localhost to remotehost on the account bbrdist. The bbrdist account has /home/babar/bbrdist as default directory on remotehost but has no subdirectories. We are going to study a control file in order to understand bbftp behaviour (we do not care here about the connection method; see CONNECTION EXAMPLES for that).User jon on the local host is on the /home/babar/jon directory and has the following control file (all lines have a number which must not exists but which are there just for clarity) : 1 setnbstream 20 2 setremoteumask 022 3 setoption nocreatedir 4 put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1 5 setoption createdir 6 put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1 7 setnbstream 5 8 setrecvwinsize 1024 9 setoption gzip 10 put /home/babar/jon/f2 /home/babar/bbrdist/newfiles/f2 Command 1 just sets the number of parallel streams to 20 for further get or put commands. Command 2 sets the remote umask for further put commands. Command 3 indicates that no directory has to be created on further put or get commands. Command 4 tries to send the local file /home/babar/jon/f1 to /home/babar/bbrdist/newfiles/f1. This command will fail because bbrdist has no subdirectory and directory creation is inhibed. Command 5 resets the createdir option. Command 6 will be successful (if the connection does not break), because the creation of the directory /home/babar/bbrdist/newfiles has been authorized by the createdir option. Command 7 reduces the number of streams to 5 Command 8 sets the receive TCP window size to 1024 Kbytes on remotehost and the send TCP window size to 1024 Kbytes on localhost. Command 9 sets the gzip option for further get or put commands. Command 10 will transfer file /home/babar/jon/f2 to /home/babar/bbrdist/newfiles/f2 with 5 streams in compressed mode. AUTHORSbbftp was developed by Gilles Farrache. It is now maintained by Lionel Schwarz and Pierre-Emmanuel Brinette at IN2P3 Computing Center , Villeurbanne (FRANCE).CONTRIBUTORSTim Adye (Idea and implementation of ssh mode)Gilles Gallot (Mutli-IP addresses support, secondary groups support, port on various systems and bug fixes) Pierre-Emmanuel Brinette (Bug fix) Andrew Goodney (Port to Darwin) Paola Grosso (Idea and implementation of the -q client option) Petr Holub (Port to Windows cygwin) Dan Schrager (Idea and implementation of the -D client option) Rod Walker & Kostas Georgiou (Idea and implementation of the -g client option) Shuwei Ye (Bug fix) BUGSSend bugs / comments to bbftp@in2p3.frSEE ALSObbftpd(1).POD ERRORSHey! The above document had some coding errors, which are explained below:
Visit the GSP FreeBSD Man Page Interface. |