Reliable File Transfer
From TeraGrid Wiki
Contents |
Description
The Reliable File Transfer (RTF) Web Service[1] is a WSRF [2] based service used by Teragrid for transfer of of files to and from compute resources. The RFT is provided by the Globus Toolkit, which is included in the CTSS. (See CTSS_-_Coordinated_TeraGrid_Software_and_Services) The RTF provides a programmatic mechanism for the transfer of files using GridFTP servers via the a 3rd party transfer. The RFT adds some features like reliability and recovery over simply using the globus-url-copy command-line utility to transfer files to or from a GridFTP server.
Services
RFT Factory Service
The RFT implements a factory service model, meaning that there is a service that is used to create instances of another service. In this case, the RFT Factory Service (RFT-FS) is used to create an instance of the RFT service (RFT-S) by sending the RFT-FS a standard SOAP message over HTTP protocol a list of URL pairs of files to be deleted or transferred, and a time to live for the resource, or service instance.
Operation
- createReliableFileTransfer: Creates a Reliable File Transfer resource.
- Input Parameters: Initial Termination time, Transfer Request or Delete Request.
- Output parameters: Termination time, Current time, Endpoint reference of the Resource created. This should be stored by the user, as it is needed to query the status of the resource and to perform any further operations on the resource.
- Fault: createReliableFileTransferFault.
RFT Service
Used to manage the Resource created using the RFT Factory Service.
Operations
- start:Starts executing the transfers/deletes.
- Input Parameters: None
- Output Parameters: None
- Fault: RepeatedlyStartedFault
- getStatus:To get the status of a particular file.
- Input Parameters: A source URL of the file that is part of the request.
- Output Parameters: Transfer Status Type
- Fault: RFTDatabaseFault
- getStatusSet: To get the status of a set of files in a request.
- Input Parameters: int from (the relative position of the transfer in the request) and int offset (the number of files queried).
- Output Parameters: An array of TransferStatusType.
- Fault: RFTDatabaseFault
- cancel: To cancel a transfer that is part of a resource.
- Input Parameters: int from (the relative position of the transfer in the request) and int to.
- Output Parameters: None
- Fault: RFTDatabaseFault
WSDL
See the Reliable File Transfer Factory Service WSDL here [3].
See the Reliable File Transfer Service WSDL here [4].
See the associated port_types, bindings, and data type wdls and schemas here [5].
Examples
Command Line Example
The RFT service can be invoked via the command line using the rft command found in the $GLOBUS_LOCATION/bin directory.
The rft command usage is as follows:
rft [options] -f <path to transfers file> where options can be -h host, defaults to localhost. -r port, defaults to 8080 -l lifetime in minutes, defaults to 60mins -m security mechanism. 'msg' for secure message or 'conv' for secure conversation and 'trans' for transport. Defaults to secure transport. -p protection type, 'sig' signature and 'enc' encryption, defaults to signature -z authorization, defaults to Host Authorization. 'self' for self authorization -file file to write EPR of created Reliable File Transfer Resource
For instance:
lslogin1% rft -h lslogin1.tacc.utexas.edu -p 8443 -f transfer.xfr
where transfer.xfr is:
#true=binary false=ascii true #Block size in bytes 16000 #TCP Buffer size in bytes 16000 #Notpt (No thirdPartyTransfer) false #Number of parallel streams 1 #Data Channel Authentication (DCAU) true # Concurrency of the request 1 #Grid Subject name of the source gridftp server /C=US/O=UTAustin/OU=TACC/CN=lslogin1.tacc.utexas.edu #Grid Subject name of the destination gridftp server /C=US/O=UTAustin/OU=TACC/CN=lslogin1.tacc.utexas.edu #Transfer all or none of the transfers false #Maximum number of retries 10 #Source/Dest URL Pairs gsiftp://lslogin1.tacc.utexas.edu:2811/home/teragrid/<<your username>>/rftTest.tmp gsiftp://lslogin1.tacc.utexas.edu:2811/home/teragrid/<<your username>>/rftTest_Done.tmp
Please insert a path that makes sense into the last two lines of the xfr file, and make sure that you can read from and write to the source and destination.
Programmatic Examples
The RFT transfer client code included with GT4 can be viewed here [6]. This is the code called by the rft command line client mentioned above. You can find client code to delete a file here [7].
Compiling
To compile and run these clients you'll need the supporting classes. One of the most straightforward ways of acquiring them is by referencing the jarfiles from a GT4 installation's lib directory. You will need to add these jarfiles from $GLOBUS_LOCATION/lib to your $CLASSPATH environment variable.
Executing
If you want notifications from the RFTS, then the code needs a working GT4 installation's path in the $CLASSPATH, as well as the jarfiles from the $GLOBUS_LOCATION/lib.
This example demonstrates using the RFT Service at the SDSC DTF-IA64 Linux Cluster to perform a gsiftp transfer of a file from the SDSC Linux Cluster to the TACC XEON IA-32 Linux Cluster.
java -DGLOBUS_LOCATION=$GLOBUS_LOCATION org.globus.transfer.reliable.client.ReliableFileTransferClient -h tg-login.sdsc.teragrid.org -p 8443 -f ./transfer.xfr
With the contents of transfer.xfr as follows:
#true=binary false=ascii true #Block size in bytes 16000 #TCP Buffer size in bytes 16000 #Notpt (No thirdPartyTransfer) false #Number of parallel streams 1 #Data Channel Authentication (DCAU) true # Concurrency of the request 1 #Grid Subject name of the source gridftp server null #Grid Subject name of the destination gridftp server null #Transfer all or none of the transfers false #Maximum number of retries 10 #Source/Dest URL Pairs gsiftp://lslogin1.tacc.utexas.edu:2811/home/teragrid/<<your username>>/rft/gt4-admin.pdf gsiftp://tg-login.sdsc.teragrid.org:2811/users/<<your username>>/rft/gt4-admin.pdf
Please make sure the paths in the Source/Dest URLs make sense.
Links & References
RFT Public Interfaces [8]
--Steve Mock, mock@sdsc.edu, 22:09, 25 January 2007 (GMT)
