Data Services 1.2 Introduce Extensions

	Contents Data Services 1.2 Introduce Extensions Functionality User Interface Creation Interface Styles and Options Service Modification Domain Model Query Processor Details Auditing Enumeration

caGrid Data Services can be built with a set of extensions to the Introduce Toolkit. This provides grid service developers with a simple and well defined starting point to create caBIG gold compliant Data Services. When the data service extension is selected in creating new services, the user is presented with the both standard service modification interface, and a new section containing options specific to configuring data services. This extension may be manually selected by using the 'Advanced' tab of the caGrid Service Creation interface in Introduce, or by simply selecting the 'Data Service' radio button on the standard view.

Functionality

The Introduce Toolkit allows for extensions to be included in the service development process which add functionality to almost any step of the service build process. The Data Service extension makes use of three of these extension points. The first extension point used is immediately following service creation. This post-creation operation makes the following changes to the generated service:

• The data service WSDL file is copied into the service.
  • Optionally, if WS-Enumeration and / or BDT support is enabled, the WSDLs for these will be included as well.
  • Additionally, selecting WS-Enumeration and / or BDT will cause those extensions to be added to the service and run before the data service extension.
• The data service schemas for CQL and Domain Models are copied into the generated service.
• The data service libraries are copied in to the service.
• The base data service query method is added, and its implementation is defined in the copied libraries and provided in the copied WSDL.
  • If WS-Enumeration and / or BDT support is enabled, specialized query methods for each of these are added, as well as their implementation.
• Data Service specific service properties are added.

The next step in the build process added by the Data Service extension is invoked when modifications to the service are saved. This operation happens before the standard operations provided by Introduce are executed which generate code for user defined methods, edit WSDL files, and copy schemas. This pre code generation operation makes the following changes to the service:

• Service properties are modified.
  • The property defining the query processor class implementation is populated with the user's defined value.
  • Properties required by the query processor implementation for initialization are synchronized with the service model.
• Adds domain model metadata.
  • The caDSR grid service may be contacted to generate one a domain model. This process can be time consuming depending on the network connection to the caDSR and the specific package and project combination selected.
  • If the service developer has defined an XML file containing the domain model definition, it will be copied into the service.

Following the execution of the pre code generation extension, the standard Introduce build operations are invoked. When this is complete, the final extension operation is invoked on the new service. This operation modifies the generated Eclipse files to add the new jar library files to the project's classpath.

User Interface

Creation Interface

Creation

The process of creating a caGrid data service begins with the Introduce creation interface. A developer can use this interface to control some of the low level implementation details of a grid service, including the service's name, package, and namespace. To create a new data service, the service developer only needs to change the radio button selection in the lower portion of this dialog from 'Analytical Service' to 'Data Service' and click the 'Create' button. For more control over which extensions are added to the service, the developer may use the 'Advanced' tab, and add and remove any extensions installed into Introduce. Adding the 'Data Service' extension on this tab has the same effect as selecting the radio button on the 'Standard' tab.

For a detailed discussion of service creation using a data service style, see the Tutorial on creating a data service backed by the caCORE SDK version 4.0

Style Selection

Styles and Options

When the data service extension has been selected from the caBIG creation dialog in Introduce, and the user clicks 'Create', the service developer is presented with a dialog allowing selection of a data service style, which can be used to repeatedly create highly customized data services. This dialog also allows the developer to enable the WS-Enumeration or Bulk Data Transfer features of the data service. When the initial creation process is complete, the user is presented with the service modification interface.

Service Modification

Introduce 1.2 using the Data Service extension

The Data Service extension adds a tab component to the Introduce Toolkit's service modification view. This tab contains several sub tabs, each containing major points of configuration for the data service:

• Domain Model
  • Provides facilities for selecting the domain of data types available to be queried by the data service
• Query Processor
  • Allows the user to select an implementation of CQL to use in the data service and provides for configuration of its parameters
• Details
  • Serialization of data types, mapping classes to XML Schema element names, and enabling / disabling query validation
• Auditing
  • Selection and configuration of auditors for various processes of the data service infrastructure
• Enumeration (Optional)
  • If the WS-Enumeration or BDT feature is enabled for data services, this tab will be available to choose a server side implementation of the WS-Enumeration spec. Generally this will not need to be changed from the default implementation

Domain Model

On the Domain Model tab, the user may select the types exposed by this data service. Data types are derived from information stored in the caDSR.

Domain Model selection

The top field labeled 'caDSR' indicates the URL of the caDSR grid service to access. The button 'Refresh from caDSR Service' queries the grid service to retrieve a list of Projects and UML Packages available in the caDSR. Projects and packages can be selected with the two dropdown menus provided. The 'Add Full Project' button will insert all packages and classes from the project into the domain model. 'Add Package' will insert a single selected package, and 'Remove Package' will take a package out of the domain model. When a package is added, its contents will appear in the tree below the buttons. This tree contains check boxes next to each package and its individual classes. Placing a check next to a class will add it to the domain model as an object which can be queried, and by default, used as a type which can be targeted with a query. Placing a check by a package will automatically add all the classes in that package in the same way.

Each package selected to participate in the domain model must be mapped to an XML schema for serialization across the grid. If a package is selected for which no schema can be found, the data service extension will prompt the service developer to locate it.

Graphical Representation of a Domain Model

The domain model may be viewed graphically by clicking the 'Visualize Domain Model' button. Note that this may cause the domain model information to be retrieved from the caDSR service before it can be displayed, which can be a time consuming process. A dialog with a progress bar keeps the service developer appraised of the progress of this process.

The 'Advanced Options' button shows a dialog which enables the service developer to control advanced options for domain model selection.

Advanced Options

The 'No Domain Model' check box disables use of a domain model in the data service. Selection of this option is intended for use when testing other aspects of a service. The option to use a domain model from the file system is also intended for testing purposes. Since building a domain mode from the caDSR can be a time consuming process, the domain model may be generated from the caDSR elsewhere and specified here. Selecting either of these options will disable selection of the domain model from the caDSR on the Domain Model configuration tab.

Query Processor

Query Processor Configuration

The Query Processor tab shows a drop down of all currently available CQL Query Processor implementations. If a query processor other than the one displayed is required, the jar file containing it may be added to the service by clicking the 'Add Jar' button. When a new jar file is added to the service, both the list of jars and the query processor drop down will update to reflect the changes. If a query processor class depends on other classes, the jars containing those dependencies must be added this way as well, and before the jar containing the query processor itself.

The class selection is not limited to the displayed classes specifically, as the drop down is manually editable. The added jar files will be copied into the service's library directory, and deployed with the service. Selecting a class from the drop down will make it the query processor implementation which is invoked by the data service to handle queries at runtime. Its configuration properties (if any) will be shown in the table at the bottom of the screen. This table shows the name of every parameter, its default value, and an editable field where a custom value may be entered. These parameters are stored as service properties in the generated service and are made available at runtime through JNDI. Query processors may optionally supply their own configuration user interface, which can be displayed by clicking the 'Launch Query Processor Configurator' button.

Custom CQL Query processors may be implemented to support querying over a specialized data source by extending a provided base class.

Main Article: How-to implement a CQL Query Processor

Details

The Details page allows configuration of lower level functionality in the data service. This tab contains a table allowing configuration of the XML schema element each data type maps to, the way each data type in the domain model will be serialized and deserialized, and a flag to indicate if each type may be targeted and returned by a CQL query.

Data Service Details

Types can be selected either individually or in groups and their serialization is configured by a right-click popup menu. This menu allows selection of either the default serialization, SDK serialization for caCORE SDK generated objects, or a custom serialization. Selecting SDK Serialization assigns the SDK serializer and deserializer factories to the selected types. Custom serialization presents the user with a dialog box in which to enter the serializer and deserializer classes which will be assigned to the schema type. Changes made to type serialization on this tab will be reflected on Introduce's built-in 'Types' tab as well. This information is recorded in the generated client-config.wsdd file, as well as the server-config.wsdd. These configuration files may be used later by the data service client tools to properly deserialize results of queries. This tab also allows the service developer to select what type of query validation they desire to be performed. Selecting to validate CQL syntax will cause the Data Service to ensure that the submitted CQL query conforms to the CQL schema. Selecting the validate domain model option will cause the data service to parse the CQL query against the domain model, ensuring that all aspects of the query remain within the confines of the exposed domain model. Using these options in conjunction ensures that every query that reaches the CQL query processor implementation is both syntactically correct and conforms to the domain model, which can substantially simplify checking for potential errors in the query processor.

Auditing

The auditing page allows auditing settings to be configured for the data service. Multiple auditors may be added to the service, each of which may handle auditing events in its own way. Multiple instances of the same auditor type may be added as well, allowing configuration options to dictate their behavior and handling of auditing events.

Auditing Configuration

Four points of the data service query process may be audited:

• Query Begins
  • This event is fired when a query is first submitted to the data service before any validation or processing has been done.
• Validation Failure
  • This event is fired when a query fails the validation process. If validation is not enabled, this event will never be fired.
• Query Processing Failure
  • If a query should fail to process correctly, this event is fired. The reason (exception) causing the failure is included in this event.
• Query Results
  • When a query completes successfully, this event is fired. The results of the query are available at this point as well.

Enumeration

Enumeration Implementation Selection

The enumeration page appears only when a data service's enumeration or BDT support features are enabled. This tab allows the service developer to select a server side implementation of the EnumIterator interface to support the WS-Enumeration specification. In caGrid 1.2, there are five implementations which may be selected. Two are supplied with the Globus WS-Enumeration support, and three are part of caGrid itself. The default selection is the caGrid implementation which uses Java 5's concurrent package to fully support the specification, including timeouts, and maximum characters. The other implementations support fewer features, but may be desirable for duplicating the functionality of other WS-Enumeration enabled services.