Access Keys:
Skip to content (Access Key - 0)

caGrid Transfer

caGrid Transfer 1.2 Developers Guide

Contents


Before You Begin


Proficiency in developing, deploying, and invoking grid services with Introduce is critical to successful use of the caGrid Transfer Tools; they are simple extensions to the normal service generation process.
Information and tutorials are available on the Introduce web site. We recommend you take the Introduce 1.2 tutorial and familiarize yourself with the Transfer Service Architecture. An example service, which is detailed in the guide below, can be found here.

Utilizing the Transfer Service


Introduce Configuration and Usage

Add the Transfer Extension

In Introduce, add the caGrid_Transfer extension to your service to utilize Grid Transfer. Prior to service generation, this is found in the Advanced tab of the creation panel. After the service has been generated, select caGrid_Transfer in the Extensions tab as shown in the image below:

The TransferServiceHelper API allows either the staging of data (in the service) for later retrieval by a client, or creation of a resource that allows a user to upload data for the service to retrieve. The first three methods can be used to stage data and return a TransferServiceContextReference, which contains an EndPointReference that can be used to retrieve the data on the client. The last method creates a resource where the client can upload data, and the service can locally retrieve.

Create an Operation that Uses Transfer

Using Introduce, you create a new Operation on your service in the usual manner (either on the main service or on a service context). The screenshot below shows the user creating a new operation on the main service context named "getSomeData". This operation returns a TransferServiceContextReference.

Please refer to the next section for example code showing the service method implementation.

Using Transfer to Enable Clients to Download Data


Code Needed in the Client

On the client side, if a method from your service utilized the TransferServiceHelper to locally stage data, and returned the type TransferServiceContextReference, the TransferServiceContextClient and TransferClientHelper can be used to retrieve the data on the client. Below is a client example of how this can be executed in an insecure environment:

                    HelloWorldClient client = new HelloWorldClient(args[1]);
                    //create transfer is a method that staged some data and returned the Reference
                    TransferServiceContextReference ref = client.getSomeData();
                    //create a client that enables me to talk to my transfer resource
                    TransferServiceContextClient tclient = new TransferServiceContextClient(ref.getEndpointReference());
                    //use the TransferClientHelper to get an InputStream to the data
                    InputStream stream = TransferClientHelper.getData(tclient.getDataTransferDescriptor());

Again, in a secure environment:

                    HelloWorldClient client = new HelloWorldClient(args[1],GlobusCredential.getDefaultCredential());
                    //create transfer is a method that staged some data and returned the Reference
                    TransferServiceContextReference ref = client.getSomeData();
                    //create a client that enables me to talk to my transfer resource
                    TransferServiceContextClient tclient = new TransferServiceContextClient(ref.getEndpointReference(),GlobusCredential.getDefaultCredential());
                    //use the TransferClientHelper to get an InputStream to the data
                    InputStream stream = TransferClientHelper.getData(tclient.getDataTransferDescriptor(),GlobusCredential.getDefaultCredential());

You can always use the secure method of retrieving data, but if the connection URL is http then the credentials will not be used, and regular sockets will be used to retrieve the data.

Code Needed in the Service

Code inside the service is straightforward: when the user calls a method you created (i.e. getSomeData()) that you provided them, your service must find their data, create the TransferContextResourceReference, and return the reference. Once the user has received the data, they can call destroy() on the resource. An example:

    public org.cagrid.transfer.context.stubs.types.TransferServiceContextReference getSomeData() throws RemoteException {
        // create some data to sent
        byte[] data = new String("This is my bulk data").getBytes();
        // create a descriptor for that data
        DataDescriptor dd = new DataDescriptor(null, "My Data");
        // create the transfer resource that will handle delivering the data and
        // return the reference to the user
        return TransferServiceHelper.createTransferContext(data, dd);
    }

Using Transfer to Enable Clients to Upload Data


Code Needed in the Client

The caGrid Transfer service can also be used to upload data from client to service. The service, as in the download case, must provide a method which returns an TransferServiceContextReference. Thay can do this by using the TransferServiceHelper API, except, in this case they will call the createTransferContext(DataDescriptor dd) operation. This operation does not take any data in, as the intent is that the data is staged in by something else, however, it does generate the resource and reserve a url where that data can be written and then eventually stored on the local machine. Below is an example of how the client is to use the TransferServiceContextClient and TransferClientHelper in order to upload data to grid service so that it can be used:

                    HelloWorldClient client = new HelloWorldClient(args[1]);
                    // upload example
                    TransferServiceContextReference ref1 = client.putSomeData();
                    // use the EPR from the reference to create a client to talk
                    // to my resource
                    TransferServiceContextClient tclient1 = new TransferServiceContextClient(ref1.getEndpointReference());
                    // use the helper to get the output stream that I can upload
                    // my data with
                    DataDescriptor dd = new DataDescriptor(null, "My Data");
                    String myDataToSend = new String("New upload test");
                    InputStream is = new ByteArrayInputStream(myDataToSend.getBytes());
                    TransferClientHelper.putData(is, myDataToSend.length(), tclient1.getDataTransferDescriptor());
                    // tell the resource that the data has been uploaded.
                    tclient1.setStatus(Status.Staged);

Again, in a secure environment:

                    HelloWorldClient client = new HelloWorldClient(args[1],GlobusCredential.getDefaultCredential());
                    // upload example
                    TransferServiceContextReference ref1 = client.putSomeData();
                    // use the EPR from the reference to create a client to talk
                    // to my resource
                    TransferServiceContextClient tclient1 = new TransferServiceContextClient(ref1.getEndpointReference(),GlobusCredential.getDefaultCredential());
                    // use the helper to get the output stream that I can upload
                    // my data with
                    DataDescriptor dd = new DataDescriptor(null, "My Data");
                    String myDataToSend = new String("New upload test");
                    InputStream is = new ByteArrayInputStream(myDataToSend.getBytes());
                    TransferClientHelper.putData(is, myDataToSend.length(), tclient1.getDataTransferDescriptor(),GlobusCredential.getDefaultCredential());
                    // tell the resource that the data has been uploaded.
                    tclient1.setStatus(Status.Staged);

Again, as described before, you can always use the secure method of retrieving data but if the connection URL is http, the credentials will not be use and regular sockets will be used to upload the data.

Code Needed in the Service

When a user calls the createTransfer method that you provided them: the TransferContextResourceReference is created, a DataStagedCallback is registed, and the reference returned. Once the the data is uploaded, and the setStatus operation is called on the resource (to let the resource know that the data is staged), the DataStagedCallback that was registered with the resource is executed. This allows your implementation to process the uploaded data. An example of this code is as follows:

    public org.cagrid.transfer.context.stubs.types.TransferServiceContextReference putSomeData() throws RemoteException {
        // create a data descriptor for the upload for thed ata to be uploaded
        DataDescriptor dd = new DataDescriptor(null, "Users Data");
        // create a callback that will handle the data once it is uploaded
        DataStagedCallback callback = new DataStagedCallback() {

            public void dataStaged(TransferServiceContextResource resource) {
                File dataFileUserSentMe = new File(resource.getDataStorageDescriptor().getLocation());
                // do something with this data
            }

        };

        // create the transfer resource that will handle receiving the data and
        // return the reference to the user
        return TransferServiceHelper.createTransferContext(dd, callback);
    }

About the DataDescriptor Constructor (Optional Metadata on the Transfer Context)


As of caGrid 1.2, the DataDescriptor constructor can take in an Object as the first parameter. Technically, anything that the service (and the transfer context specifically) knows how to serialize (as specified in both .wsdd files) can be handed to this constructor to pass some metadata about the transfer to the client. In practice, this Object is a placeholder for a future iteration of the transfer service that will add in standardized support for describing the Transfer streams. At this time, it is not recommended to put anything other than null in the constructor, because a new client of your service would not have the beans and type descriptor files to be able to retrieve your metadata without customizing their client.
However, if you would like to customize the metadata on a Transfer context using a basic Axis type (such as a String or another basic type that is known to all service and doesn't require custom serialization), pass in the data to the server operation that creates the Transfer context. Then you can use this String in your DataDescriptor constructor to set a String descriptor (metadata) on the DataDescriptor object:

public org.cagrid.transfer.context.stubs.types.TransferServiceContextReference getContext(String metadata) throws RemoteException {

   DataDescriptor dd = new DataDescriptor(metadata, "Users Data");
 ...
}

The client can retrieve this metadata when it receives the transfer context:

  String metadata = "JPEG";
  TransferServiceContextReference ref = client.getContext(metadata);
  TransferServiceContextClient tclient = new TransferServiceContextClient(ref.getEndpointReference());

  DataTransferDescriptor d = tclient.getDataTransferDescriptor();
  String metadataValue = (String)d.getDataDescriptor().getMetadata();
  String dataDescriptorName = d.getDataDescriptor().getName();

Note that trying to set DataDescriptor fields client-side will NOT update the metadata value and data descriptor name for others. (i.e., the "set" methods on the DataDescriptor object, when used client-side, have no effect).

Deploying a Service that Uses caGrid Transfer


The caGrid Transfer service MUST be deployed to the same container that the invoking grid service in deployed. Like all caGrid services, only Tomcat and JBoss are currently supported. Prior to deployment of caGrid Transfer, Globus must be deployed to the container, and (if desired) security configuration must be completed. This can be accomplished by the caGrid Installer, or manually by following the instructions provided by Globus.
After you have acquired caGrid and setup your container, you can deploy the caGrod Transfer Service. This is accomplished by entering the caGrid Transfer project directory: From the top of the caGrid release:

cd projects/transfer

Then deploy to either Tomcat or JBoss (tomcat example below):

ant deployTomcat

Next, deploy your service by entering the services directory:

cd <my services directory>

Then deploy to either Tomcat or JBoss (tomcat example below):

ant deployTomcat

Last edited by
Knowledge Center (1855 days ago) , ...
Adaptavist Theme Builder Powered by Atlassian Confluence