Part Four: Discovery of Services on the Grid

Part Three - Authentication

Client Application Guide

Part Five - Data Service Querying

	Table of Contents 1 Introduction 2 Desired Client Functionality 2.1 Search for Services by Name 2.2 Search for the BootCamp Service in the Portal using NCI Public ID 3 Creating Client Functionality 3.1 Step 1: List and Search for Registered Services 3.2 Step 2: Querying metadata from Service Endpoints. 3.3 Step 3: Test the Client 4 Conclusion 5 References

Introduction

In order to successfully utilize the Grid it will be necessary to locate available Grid Services from which you can obtain data and analyze data. This is called discovery of Grid Services.

Discovery involves the Grid Index Service and caGrid Portal, which serve as a registry of each available grid service. Both services contain metadata that identify the service hosting research facility, point of contact, and the metadata that identifies the data made available by the grid service. Developers creating grid clients will create solutions that utilize the Index Service to programmatically discover grid services.

The caGrid distribution contains tools to perform the steps that we are implementing. You may find it useful to look at the tools that perform the functionality that we will be implementing in our client application.

Desired Client Functionality

Search for Services by Name

1. Open a web browser and visit http://portal.training.cagrid.org
2. Click the Services link in the horizontal menu bar.
3. Click the Search tab in the Discovery portlet.
4. Enter the search term BootCamp, select Name as the Search field and click the Search button.
5. You will see the BootCampDataSvc in the search results.
6. Click on the BootCampDataSvc name.
7. In the results you will see that the Portal provides the ability to browse the service metadata and query the service.

Search for the BootCamp Service in the Portal using NCI Public ID

1. Open a web browser and visit http://portal.training.cagrid.org
2. Click the Services"link in the horizontal menu bar.
3. Click the Search tab in the Discovery portlet.
4. Enter the search term 2322246, select caDSR Public ID (CDE) as the Search field and click the Search button.
   1. This is the public id for the entrezGeneId data element that exists in our BootCamp domain model.
5. You will see the BootCampDataSvc in the search results.

Creating Client Functionality

Now, we will add the ability to discover grid services using the Discovery Client and the Index Service, a registry of all services that are available on the Grid. The caGrid Portal performs these searches using the same grid APIs.

The Discovery Client getAllServices method takes a Boolean input parameter. This parameter provides the ability to return only services that meet service metadata requirements. We will use this parameter to filter out non-compliant services. For this reason, you will not see errors when retrieving metadata from the getAllServices method, but you will see exceptions from the getAllDataServices and getAllAnalyticalServices methods.

Our search functionality uses the discoverServicesBySearchString method. This is a rather broad way to match services. For example, using the search string "emory" to discover services from Emory University will also return services that contain the word "memory". It is better to call one of the other available methods to search against the exact field in which you are interested.

Step 1: List and Search for Registered Services

In this section we will implement the ability to connect to the Index service and obtain a complete list of registered services and search for specific services.

Note: Not all services returned by the Index Service are active. Some may be behind firewalls, have restricted access or are incorrectly configured.

Update Ivy Dependencies

1. Open ivy.xml and update the project dependencies by adding the following entries within the '<dependencies>' section.

   <!-- Discovery Dependencies -->
   <dependency rev="${repository.version}" org="caGrid" name="discovery" conf="default->default"/>
   <dependency rev="${repository.version}" org="caGrid" name="metadatautils" conf="default->default"/>
   

2. Save the ivy.xml file.
3. From the command line run ant all to retrieve the new jars.

Import Jars into the Project

1. Right-click on the caGridClient project in the Eclipse Package Explorer.
2. Select the Properties entry at the bottom of the list.
3. Select Java Build Path, then Libraries.
4. Select the lib entry and click the Edit button.
5. Click User Libraries
6. Click Add Jars
7. Navigate to the newly added jars in the caGridClient/lib directory and add them to the lib user library.
8. In the Preferences dialog, click OK.
9. In the Add Library dialog, click Finish.

Import Required Globus Jars

1. To add the Globus Jars to the project properties, right-click on the "caGridClient" project in the Eclipse Package Explorer.
2. Select the Properties entry at the bottom of the list.
3. Select Java Build Path, then Libraries.
4. Select the Add External Jars... entry.
5. Navigate to the Globus ws-core-4.0.3/lib directory that was installed by the caGrid installer.
6. Select the Globus Jars files, listed above, then click Open.
7. Click OK to close the Properties dialog.

Create the DiscoveryActions Class

1. Right-click org.cagrid.client and select New->Class
2. Set the class name as DiscoveryActions and click Finish.
3. Add imports

   import gov.nih.nci.cagrid.discovery.client.DiscoveryClient;
   import gov.nih.nci.cagrid.metadata.exceptions.QueryInvalidException;
   import gov.nih.nci.cagrid.metadata.exceptions.RemoteResourcePropertyRetrievalException;
   import gov.nih.nci.cagrid.metadata.exceptions.ResourcePropertyRetrievalException;
   
   import org.apache.axis.message.addressing.EndpointReferenceType;
   import org.apache.axis.types.URI.MalformedURIException;
   

4. Add DiscoveryClient member variable.

   private DiscoveryClient discClient = null;
   

5. Add the constructor.

   public DiscoveryActions(String serviceUrl)
   throws MalformedURIException{
   	this.discClient = new DiscoveryClient(serviceUrl);
   }
   

6. Add public methods.

   	public EndpointReferenceType[] getServices () throws
   RemoteResourcePropertyRetrievalException,
   QueryInvalidException, ResourcePropertyRetrievalException {
   	    	EndpointReferenceType[] allServices = null;
   			allServices = discClient.getAllServices(true);
   			return allServices;
           }
   
   	public EndpointReferenceType[]  getDataServices () throws
   RemoteResourcePropertyRetrievalException,
   QueryInvalidException,
   ResourcePropertyRetrievalException {
   
   		EndpointReferenceType[] dataServices =
   discClient.getAllDataServices();
   		return dataServices;
   	}
   
   	public EndpointReferenceType[]  getAnalyticalServices ()
   throws RemoteResourcePropertyRetrievalException,
   QueryInvalidException,
   ResourcePropertyRetrievalException {
   		EndpointReferenceType[] analyticalServices =
   discClient.getAllAnalyticalServices();
   		return analyticalServices;
   	}
   
           public EndpointReferenceType[] searchServices () throws
               RemoteResourcePropertyRetrievalException,
               QueryInvalidException, ResourcePropertyRetrievalException, MalformedURIException {
   
               String searchTerm = UserInteraction.getStringFromUser("Enter Search Term");
   
   	    System.out.println ("Searching for '" + searchTerm + "' services");
       	    EndpointReferenceType[] searchedServices = null;
   
          	    searchedServices = discClient.discoverServicesBySearchString(searchTerm);
   
       	    return searchedServices;
           }
   
   	public EndpointReferenceType[] searchServices (String searchTerm) 
   		throws RemoteResourcePropertyRetrievalException, QueryInvalidException, 
   		ResourcePropertyRetrievalException, MalformedURIException {	
   		
   		System.out.println ("Searching for '" + searchTerm + "' services");
       	EndpointReferenceType[] searchedServices = discClient.discoverServicesBySearchString(searchTerm);
       	
       	return searchedServices;
   	}
   

7. Save the file.

Update the GridClient Class

1. Open GridClient.java for editing.
2. Add imports.

   import org.apache.axis.message.addressing.EndpointReferenceType;
   

3. Add a new constants.

   private static final String DEFAULT_INDEX_SERVICE_URL_PROP = "default.index.service.url";
   
   private static final int SERVICE_TYPE_ALL = 1;
   private static final int SERVICE_TYPE_DATA = 2;
   private static final int SERVICE_TYPE_ANALYTICAL = 3;
   

4. Add methods.

   // search for services
   private EndpointReferenceType[] discoverServices () {
   		System.out.println("Discovering Grid Services");
   
   		String indexServiceUrl = props.getProperty(DEFAULT_INDEX_SERVICE_URL_PROP);
   		try {
   			DiscoveryActions discActions = new DiscoveryActions(indexServiceUrl);
   
   			// search for services
   			EndpointReferenceType[] serviceEndpoints = discActions.searchServices();
   			if (null != serviceEndpoints && serviceEndpoints.length > 0) {
   
   				System.out.println ("Available services:");
   				for (int i=0; i< serviceEndpoints.length; i++){
   					System.out.println("    " + i + ": " + serviceEndpoints[i].getAddress().toString());
   				}
   
   				return serviceEndpoints;
   			}
   			else {
   				System.out.println("No matching Services found");
   			}
   
   
   		} catch (Exception e) {
   			// TODO: Display appropriate client error
   			System.out.println(e.getClass().getName() + ": " + e.toString());
   		}
   
   		return null;
   	}
   
   	// Obtain a list of available services by service type
   	private EndpointReferenceType[] listServices () {
   		System.out.println("Discovering Grid Services");
   
   		// get service type for query
   		System.out.println("--------------------------");
   		System.out.println("1 : All Services");
   		System.out.println("2 : Data Services");
   		System.out.println("3 : Analytical Services");
   		int serviceType = UserInteraction.getIntegerFromUser("Select Service Type", 1, 3);
   
   		// get list of services
   		String indexServiceUrl = props.getProperty(DEFAULT_INDEX_SERVICE_URL_PROP);
   		try {
   			DiscoveryActions discActions = new DiscoveryActions(indexServiceUrl);
   
   			// search for services
   			EndpointReferenceType[] serviceEndpoints = null;
   			if (serviceType == SERVICE_TYPE_DATA) {
   				serviceEndpoints = discActions.getDataServices();
   			} else if (serviceType == SERVICE_TYPE_ANALYTICAL) {
   				serviceEndpoints = discActions.getAnalyticalServices();
   			} else {
   				serviceEndpoints = discActions.getServices();
   			}
   
   			if (null != serviceEndpoints && serviceEndpoints.length > 0) {
   
   				for (int i=0; i < serviceEndpoints.length; i++) {
   					System.out.println("    " + i + ": " + serviceEndpoints[i].getAddress().toString());
   				}
   			}
   			else {
   				System.out.println("No matching Services found");
   			}
   
   			return serviceEndpoints;
   		} catch (Exception e) {
   			// TODO: Display appropriate client error
   			System.out.println(e.getClass().getName() + ": " + e.toString());
   		}
   
   		return null;
   	}
   
   

5. Add call to invoke the discoverServices method in main where we are evaluating the user input of '4'.

   	client.discoverServices();
   

6. Add a call to invoke the listServices method where we are evaluating the userinput of '5'.

          client.listServices();
   

7. Save the file.
8. Build the project from the command line using the 'ant all' command.
9. Fix all compilation errors.

Step 2: Querying metadata from Service Endpoints.

Once we have obtained the endpoint of a service it will be useful to query the service metadata to learn about the service. We will perform these actions using the caGrid MetadataUtils.

Update the GridClient Class

1. Open GridClient.java for editing.
2. Add import statements.

   import gov.nih.nci.cagrid.metadata.MetadataUtils;
   import gov.nih.nci.cagrid.metadata.ServiceMetadata;
   

3. Go to the listServices method. Add code to obtain metadata within the for loop after where we are printing the service name.

   try {
       ServiceMetadata metadata=  MetadataUtils.getServiceMetadata(serviceEndpoints[i]);
       String description = metadata.getServiceDescription().getService().getDescription();
       System.out.println("       Description: " + description);
       String hostingCenter = metadata.getHostingResearchCenter().getResearchCenter().getDisplayName();
       System.out.println("       Hosting Center: " + hostingCenter);
      }
      catch (Exception e) {
   	// TODO: Display appropriate client error
   	System.out.println(e.getClass().getName() + ": " + e.toString());
      }
   

4. Save the file.

Edit the Client Properties File

1. Open the conf/client.properties for editing.
2. Add the property that defines the Index Service URL.

   default.index.service.url=http://index.training.cagrid.org:8080/wsrf/services/DefaultIndexService
   

3. Save the file.

Build the Application

1. Build the project from the command line using the ant all command.
2. Fix all compilation errors.

Step 3: Test the Client

The Ant build file includes a target for running the client. It will create a classpath from the jars included by Ivy.

1. Open a command prompt.
2. Change directory to your caGridClient location
3. Type ant run. This will build and execute the project
4. When prompted, type 4 to search for services.
5. When prompted for a search term enter protein
6. You should see results similar to the following:
   

         [java] Provide Search Term:  protein
   
         [java] Searching for 'protein' services
   
         [java] Available services:
   
         [java]     0: http://kc-vm2.bmi.ohio-state.edu:18080/wsrf/services/cagrid/CaArraySvc
   
         [java]     1: https://tutorials.training.cagrid.org:8443/wsrf/services/cagrid/BootCampDataSvc
   
         [java]     2: http://caarraygrid.c2b2.columbia.edu:80/wsrf/services/cagrid/CaArraySvc ...
   

7. When prompted, type 5 to list services.
8. When prompted select the type of service that you wish to find.
9. You should see all services listed including the URL, description and HostingResearch Center:
   

        [java]     Select Service Type [1..3]: 1
   
        [java]     0: http://kc-vm2.bmi.ohio-state.edu:18080/wsrf/services/cagrid/CaArraySvc
   
        [java]        Description: This is the caArray grid service.
   
        [java]        Hosting Center: Ohio State University
   
        [java]     1: https://mastergts.training.cagrid.org:8443/wsrf/services/cagrid/GTS
   
        [java]        Description: The Grid Trust Service (GTS) is a grid-wide mechanism for maintaining and provisioning a federated trust fabric consisting of trusted certificate authorities, such that grid services may make authentication decisions against the most up to date information.
   
        [java]        Hosting Center: Ohio State University
   
   ...
   

10. Now, experiment with other searches.

Conclusion

1. Here we listed all services registered in the Training Grid Index Service.
2. We also searched the Index Service for services using search strings.
3. It is far more specific to search is using public IDs that have been used to semantically annotate the domain models.

References

For additional information on discovery of grid services, please read the following documents.

• caGrid Portal
• Discovery Client
• Discovery using the Grid Portal
• Training Grid Portal: http://portal.training.cagrid.org
  
  

  Part Three - Authentication

  Client Application Guide

  Part Five - Data Service Querying