Access Keys:
Skip to content (Access Key - 0)

Data Services

Client API

caGrid Data Services provide a standard query method which all data services have in common. This query method is exposed to the grid via a uniform WSDL using request and response messages of a consistent namespace and type. This allows for a uniform client to invoke any arbitrary data service. Such a client is provided with the data services infrastructure, along with several helper classes. This is also true for data services which support WS-Enumeration.

Creating a Query

Data services in caGrid use CQL or CQL 2 to compose queries. A query can be produced programmatically, building up parts of the query using the supplied object model.

    CQLQuery query = new CQLQuery();
    Object target = new Object();
    target.setName(Gene.class.getName());
    Attribute symbolAttribute = new Attribute("symbol", Predicate.LIKE, "IL%");
    target.setAttribute(symbolAttribute);
    query.setTarget(target);

Alternatively, a CQL query can be loaded from a string of XML text, or an XML file and deserialized into the object model.

    // from a string
    CQLQuery query2 = (CQLQuery) Utils.deserializeObject(
         new StringReader("<CQLQuery ... />"), CQLQuery.class);
    // from a file
    CQLQuery query3 = (CQLQuery) Utils.deserializeObject(
         new FileReader(cqlFile), CQLQuery.class);

Standard Data Services

Standard data services are those which support the basic query and executeQuery methods. A standard data service client may be used to query any caGrid data service via these methods, rather than requiring a separate client for each data service on the grid. This is true of all caGrid data services, and other specialized query methods may be included in addition to this one.

The executeQuery method takes a CQL 2 query parameter and returns CQL 2 query results. Since CQL 2 is a new feature of caGrid 1.4, attempting to invoke data services created with caGrid 1.3 and earlier using this method will fail with an error similar to 'operation not defined'

Initializing the Client

The generic data service client can be initialized simply by passing the URL of the remote data service service to the constructor:

    String url = "http://localhost:8080/wsrf/services/cagrid/ExampleDataService";
    DataServiceClient standardClient = new DataServiceClient(url);

The generic data service client has only the query and executeQuery methods in its public API. To make use of any additional methods you may have added to your service, the specific client generated by Introduce for your service must be used.

client-config.wsdd
All CaGrid client programs need to have the file client-config.wsdd on their classpath. There are two recommended ways to arrange this:
  • Include $GLOBUS_LOCATION in the classpath.
  • Copy client-config.wsdd from $GLOBUS_LOCATION to a directory that is already on the classpath.

Submitting a Query

A query can be submitted to the standard data service client, which will return the query results when processing is complete. The query method can throw two typed exceptions:

  • Query Processing Exception
    • Thrown when an error occurs in actually processing the query or formulating a result set. This might include database errors and misconfiguration of the query processor.
  • Malformed Query Exception
    • This is generally thrown from the query validation process, and indicates a query which violates the CQL schema, or attempts to reach parts of a domain model which are either excluded from queries or do not exist.
    • Query processor implementations may also throw this exception in response to queries which attempt to use features of CQL which are not supported by that particular implementation (eg. Attribute Results)
    try {
         CQLQueryResults results = standardClient.query(query);
    } catch (QueryProcessingExceptionType ex) {
         // handle processing exception
    } catch (MalformedQueryExceptionType ex) {
         // handle malformed query
    } catch (RemoteException ex) {
         // handle remote exception
    }

Handling the Results

The results of a CQL query are returned in an object model which can hold one of either object results, attribute results, or a count result. These results can be iterated as single items using a specialized implementation of the standard Java Iterator interface.

    Iterator iter = new CQLQueryResultsIterator(results,
         InvocationExample.class.getResourceAsStream("client-config.wsdd"));
    while (iter.hasNext()) {
         Gene g = (Gene) iter.next();
         System.out.println("Found a gene:");
         System.out.println(g.getFullName() + " -- " + g.getSymbol());
    }

Alternatively, the data service infrastructure provides a simplified handle class which hides the complexity of setting up an iterator to handle the query results.

    try {
         String url = "http://localhost:8080/wsrf/services/cagrid/ExampleDataService";
         DataServiceClient standardClient = new DataServiceClient(url);
         DataServiceHandle handle = new DataServiceHandle(standardClient);
         Iterator i = handle.query(query);
         // iterate
    } catch (QueryProcessingExceptionType ex) {
         // handle query processing exception
    } catch (MalformedQueryExceptionType ex) {
         // handle malformed query
    } catch (MalformedURIException ex) {
         // malformed URI
    } catch (RemoteException ex) {
         // handle remote exception
    }

Handling of CQL 2 query results is described in the CQL 2 documentation.

Enumeration Data Services

Data Services optionally support use of WS-Enumeration for retrieving query results. Since this is also a standard operation with a consistent WSDL, a single Enumeration Data Service Client can be used to query any data service which supports Enumeration.

Initializing the Client

Just as the standard data service client, the enumeration data services client can be initialized with the URL of your grid service:

    String url = "http://localhost:8080/wsrf/services/cagrid/ExampleDataService";
    EnumerationDataServiceClient client = new EnumerationDataServiceClient(url);

Submitting a Query

The generic enumeration client supports a single method:

    EnumerationResponseContainer responseContainer = client.enumerationQuery(query);

When this method is invoked, the service executes the CQL query as it would for a standard data service, but rather than returning the result directly, an enumeration resource is created to store the results, and an EnumerationResponseContainer instance is returned.

The EnumerationResponseContainer is a caGrid data type which encapsulates both the WS-Enumeration spec EnumerationContext and the endpoint reference (EPR) of the enumeration service context and the resource key of created enumeration resource. More information on caGrid's support for WS-Enumeration is provided here.

Handling the Results

The data service infrastructure provides tooling to assist in creating a ClientEnumIterator instance which can communicate with the remote enumeration service context.

    EnumerationResponseContainer responseContainer = client.enumerationQuery(query);
    InputStream wsddStream =
         ClassUtils.getResourceAsStream(getClass(), "client-config.wsdd");
    ClientEnumIterator iterator = EnumerationResponseHelper.createClientIterator(
        responseContainer, wsddStream);
    // constraints to retrieve 10 results, no char limit, timeout of 5 sec
    IterationConstraints cons = new IterationConstraints(10, -1, DurationUtils.toDuration(5000));
    iterator.setIterationConstraints(cons);
    while (iterator.hasNext()) {
         SOAPElement elem = (SOAPElement) iterator.next();
         // process element
    }

The contents of each SOAP Element are an XML representation CQLObjectResult instances, and can be deserialized as such.

Alternatively, the whole querying and result handling process can be condensed to a few lines using the supplied EnumDataServiceHandle class:

    EnumDataServiceHandle handle = new EnumDataServiceHandle(client);
    Iterator i = handle.query(query);
    // iterate

Transfer Data Services

Data Services optionally support the use of caGrid Transfer for retrieving query results. Since this is also a standardized operation with consistent WSDL, a single Transfer Data Service Client can be used to query any data service which supports Transfer.

The support of caGrid Transfer is a new feature in caGrid 1.4, and attempting to query a data service generated with caGrid 1.3 or earlier using this client and the associated operations will result in an error similar to 'operation not supported'

Submitting a Query

The generic transfer data service client supports two public operations; one for CQL named transferQuery, and one for CQL 2 named executeTransferQuery.

    String url = "http://localhost:8080/wsrf/services/ExampleTransferDataService";
    TransferDataServiceClient client = new TransferDataServiceClient(url);
    TransferServiceContextReference transferRef = client.transferQuery(cqlQuery);

Once the TransferServiceContextReference has been retrieved from the data service, the client can obtain the query results from it by following the [caGrid Transfer developers guide]

Handling the Results

The transfer context will return an input stream. This input stream will retrieve the query results as XML text. This text can be deserialized as CQL or CQL 2 query results and handled with the standard results handling utility APIs.

Other Data Service APIs

The caGrid data services infrastructure supplies many client and service side APIs.

Utility Client APIs

The caGrid Data Services infrastructure provides a number of utility classes to make invocation of remote data services a simpler process for the application developer. The package gov.nih.nci.cagrid.data.utilities contains utilities can invoke standard enumeration as well as tools for handling domain models and working with wsdd and castor mapping files.

Iterating Query Results

When a query is performed using the standard caGrid Data Service client's query method, a CQLQueryResults object is returned. This object is a container for both the results themselves and some information pertaining to their type. This container can contain object results, attribute name/value pairs, or a count of the total number of items in the result set. The difficulty of manipulating a container which may contain such a wide variety of result types stored in it may be handled by an iterator class provided with the data service infrastructure.

The interface DataServiceIterator specifies a single query() method, which takes a CQL Query and returns an instance of java.util.Iterator. The iterator can be used to walk through the results of a query issued to a data service. Three concrete implementations of the DataServiceIterator interface are provided, each for communicating with a different type of data service. The DataServiceHandle class can be used to invoke a standard caGrid Data Service while the EnumerationDataServiceHandle is designed for WS-Enumeration.

Additionally, this package contains an Iterator utility for handling CQLQueryResults instances directly. The class CQLQueryResultsIterator implements the java.util.Iterator interface, and has three constructors. The choice of constructor affects the behavior of calling the next() method.

  • CQLQueryResultsIterator(CQLQueryResults)
    • Creates an Iterator over the results which will return materialized objects deserialized using the default type mappings.
  • CQLQueryResultsIterator(CQLQueryResults, boolean)
    • Creates an Iterator over the results, and the value of the Boolean parameter indicates if XML strings should be returned from the next() method.
    • If the Boolean value is true, XML text of each item is returned, otherwise the results will be deserialized using the default type mappings.
  • CQLQueryResultsIterator(CQLQueryResults, InputStream)
    • Creates an Iterator over the results, and expects the InputStream will point to a client or server side wsdd.
      • The contents of this wsdd file will be used to configure deserialization of the objects contained in the results.

The class gov.nih.nci.cagrid.data.utilities.CQLQueryResultsIterator implements the java.util.Iterator interface, and so can be used in a while() loop like any other iterator over a Java collection. Depending on what the query to the data service asked for, calls to the next() method of this iterator will return different types of objects.

  • If the query was for object results, then:
    • The iterator returns objects of the type specified as the target for the query.
    • Objects which require custom serialization and/or deserialization require that the iterator be configured with an InputStream to the client-config.wsdd file containing the type mappings for the objects.
    • Alternatively, the iterator can be configured to return only the XML representation of those objects.
  • If the query was for attribute results, including distinct attributes, then:
    • Each successive call to next() returns an array of TargetAttribute types.
    • These types contain the name of an attribute and its value.
    • The value in the TargetAttribute instance will be null if the value was null on the object satisfying the query.Each array of TargetAttributes corresponds to one object instance which satisfied the CQL query criteria.
  • If the query was for a count of object instances, then:
    • The iterator returns a single java.lang.Long value.

An example usage of this iterator is below:

import gov.nih.nci.cagrid.cqlquery.CQLQuery;
import gov.nih.nci.cagrid.cqlresultset.CQLQueryResults;
import gov.nih.nci.cagrid.data.utilities.CQLQueryResultsIterator;

import java.util.Iterator;

public class SampleDataServiceInvocation {

    public static void main(String[] args) {
        try {
	    DataServiceClient client = new DataServiceClient(args[0]);
	    CQLQuery query = new CQLQuery();
	    // build up the query
	    CQLQueryResults results = client.query(query);
	    Iterator iter = new CQLQueryResultsIterator(results,
	        SampleDataServiceInvocation.class
                    .getResourceAsStream(
                        "client-config.wsdd"));
            while (iter.hasNext()) {
		java.lang.Object result = iter.next();
		// do something with the result object
	    }
	} catch (Exception ex) {
	    ex.printStackTrace();
	    System.exit(1);
	}
    }
}

Determining Service Capabilities

Since caGrid 1.4 data services introduce several new features, a utility API has been added which allows clients to determine what features a given caGrid data service supports. The gov.nih.nci.cagrid.data.utilities.DataServiceFeatureDiscoveryUtil class provides several static methods which return boolean flags indicating if the service supports a given feature.

  • serviceSupportsCql2(EndpointReferenceType)
    • Returns true if the Endpoint Reference points to a caGrid data service which supports CQL 2.
  • serviceHasCql2TransferOperation(EndpointReferenceType)
    • Returns true if the Endpoint Reference points to a data service which supports CQL 2 and has the transfer operation available
  • serviceHasCql1TransferOperation(EndpointReferenceType)
    • Returns true if the Endpoint Reference points to a data service which supports CQL 1 and has the transfer operation available. Note that if the service supports CQL 2, support for CQL 1 is implicit.
  • serviceHasCql2EnumerationOperation(EndpointReferenceType)
    • Returns true if the Endpoint Reference points to a data service which supports CQL 2 and WS-Enumeration.
  • serviceHasCql1EnumerationOperation(EndpointReferenceType)
    Returns true if the Endpoint Reference points to a data service which supports CQL 1 and WS-Enumeration. Note that if the service supports CQL 2, support for CQL 1 is implicit.

This utility class also contains an additional method which can be used to determine more generally if a given service supports a specific WSDL operation:

  • getOperation(EndpointReferenceType epr,
    QName inputMessage, QName outputMessage, String operationName) -> Operation
    • Returns the Operation from the service identified by the endpoint reference with input and output messages of the given QNames and of the name specified.
Last edited by
Mark Grand (296 days ago) , ...
Adaptavist Theme Builder Powered by Atlassian Confluence