Sight for Java programmers

Sight generates modules that can be very easily integrated into the custom java applications. In the older versions you only could integrate a single agents. However since 3.0.0 the whole workflow (possibly consisting of 10-20 or even more agents) can be converted into an agent and later used in the independent java application. Integrated modules still use Sight caching and security systems.

This chapter explains how to integrate Sight agents into any Java program. The author now supposes that you have at least some practical experience in java programming and can use some Java developer environment. If you do not, please start from http://java.sun.com/docs/books/tutorial/java/index.html

Agents, generated by Sight are complete, working java classes that can be used in your own bioinformatical project. You only need to add the Sight_200.jar to your source path and know which method to call in order to get a response from request.

The Response and Request classes are inner classes for each agent, and you can see their declarations in the generated source code. The request vary significantly depending from the agent task. However it always contains a single set of public fields. Before submitting request, you must assign at least some of these fields. The most of the fields also have default values, initialised in constructor of response. To get an instance of Request with default values (that you may later modify before submission), call the static agent method getDefaultRequest().

There are several ways for submitting the modified request to agent.

Create a new instance of agent (it has a parameterless constructor). If needed, set the cache expiration time by calling setCacheExpirationTime(int days), default being 42 days. Then call the method Object go(Object request, int key). The first parameter for this method must have the agent request type. The second parameter is a key, used to store the result value in the cache (after it will be computed). The method will return the agent response for this request. If you cannot provide the key for the agent, compute it using the request method getKey(). Hence the simplest way of submission would look like:

result = agent.go(request, request.getKey());

This is a simple method, but it blocks the current thread until the remote server completes the task. If you need to submit several requests at time, use another way:

Call submit() method of the created request. It returns an object, having the Agents.Request type. The submit() method returns immediately, and the agent task starts in a separate thread. When finally you cannot continue without knowing the result, call getResult() method that stalls the current thread before the result is ready and then returns it (cast the returned object to the inner Response class of the agent). Note: multiple paralled submissions to the same server will by blocked by security system.
If you do not want any caching, you can call alternatively call result = agent.go(request). This is typically much slower and inefficient, but if the agent should return a different results to the same request (for example, in a time-dependent manner), this may be the only alternative. This method blocks until agent completes the request.

The returned response type is similar for the most of the agents. It always contains an array (typically having name a) of records. Similary as Request and Response, the Record is also an inner agent class. Each Record contains public fields, describing various information, related to the single item of the response. Responses usually contain multiple items (multiple predicted genes for the given DNA sequence, multiple predicted transmembrane segments for the given protein sequence and so on). The most of the fields have CharSequence type. To convert into numeric values (where appropriate), use int Integer.parseInt(String x);

Each generated agent contains the public static main(...) method. The purpose of this method is to provide short and clear illustration, how it would be possible to use this agent from any java program. Hence if you need more explanations about the agent structure and usage, try to look at the generated code.

Here is the simple complete example, illustrating how to read the sequence from NCBI (underlined lines means followable hyperlinks):

import impl.sequence_readers.fastaReaderNCBI;
import impl.protein_analysers.TmPred;

public class Simple {

  public static void main(String[] args) {
    String idToRead = "NP_000059"; // NCBI database identifier
    try {
     // create instance of agent:
     fastaReaderNCBI agent_0 = new fastaReaderNCBI();
     // get default request
     fastaReaderNCBI.Request request = agent_0.getDefaultRequest();
     // modify it:
     request.Id = idToRead;
     // submit it:
     System.out.println(agent_0.getDescription());
     fastaReaderNCBI.Result result
      = (fastaReaderNCBI.Result) agent_0.go(request, request.getKey());

     System.out.println("The sequence:");
     System.out.println(result.a[0].Sequence);
     // (only one member is expected in the array of records)

     // now the same with TmPred:
     TmPred agent_t = new TmPred();
     TmPred.Request request_t = agent_t.getDefaultRequest();
     request_t.Sequence = result.a[0].Sequence.toString(); // was CharSequence
     System.out.println(agent_t.getDescription());

     System.out.println(agent_t.getDescription());
     TmPred.Result helixes = (TmPred.Result) agent_t.go(request_t, request_t.getKey());

     System.out.println(helixes.a.length+" transmembrane helixes found.");
     for (int i = 0; i < helixes.a.length; i++) {
       System.out.println("Helix from "+helixes.a[i].Region.from+ " to "+helixes.a[i].Region.to);
     }

    } catch (Exception exc)
    { exc.printStackTrace(); };
  }

}

This file can be compiled using your preferred Java developer environment, from just javac to Forte and JBuilder. Do not forget to add Sight_200.jar to all required library pathes.

Manual programming allows to create very complicated agent teams that can only be described in the form of the workflow network, not just in the form of tree like in the Sight automatic team generator. It also allows to store results immediately to some database via database bridge or in some other ways. The agents still use cache to avoid unnecessary repetetive submissions, and the Sight execution window is still opened to provide more control on the process being executed. If you want to display your own messages inSight GUI, use the static method Sight.Agents.util.Pind.w(String label, String message),

Sight agents and all Sight system obviously contain much more methods and a number of different classes. However at present, it would be an advantage not to use the internal Sight classes very intensively as this may cause incompatibility with the future versions.

Using sample generator

Sight contains code generator that builds a sample, demonstrating, how to use the selected robot. It can be used to create your template, where correct robot, robot request, robot response and robot record class names are defined. The generator is accessible from the "Java programming" sub menu in the mane ("Create team") window.

Writing memory leak - resistant applications

During the days and weeks of execution, your application may run out of any clever memory limits. Even the newest java runtime environments still produce some memory leaks. The simples way to solve this program is to plan auto-restart after the OutOfMemoryError is catched. To write such applications, you can use the Sight class Sight.util.ListProcessor. Here is the typical skeleton:

package Samples;
import java.io.*;
import Sight.util.ListProcessor;
/**
 * This class provides example how to create the simple
 * genome walker that walks through the list of accession
 * numbers.
 */

public class ListWalker {

  public ListWalker() {
  }

  static public void main(String[] args) {
    // process fragment of the chromosome 8, located in H8.txt
    // (program start folder)
    new ListWalker().processList("H8.txt");
  }

  /**
   * Analyses the given list.
   */
  public void processList(String fileToLoad)
   {
    try {
    ourListProcessor lp = new ourListProcessor();
    lp.walker = this;
    lp.setList(new File(fileToLoad));
    lp.run();
    } catch (Exception exc)
       { if (exc!=null) System.out.println(exc.getMessage());
         exc.printStackTrace();
       };
   };
  }

 /* We need a new child of ListProcessor. */
 class ourListProcessor extends Sight.util.ListProcessor
  {

    ListWalker walker;

    /** Operations, required only when starting the task list.
     *  For example, write header of the log files. */
    public void do_on_start()
     {
       System.out.println("Starting");
     };

    /** Operations, required only when finishig the task list.
     *  For example, write footers of the log files. */
    public void do_on_finish()
     {
       System.out.println("Finishing.");
     };

    /** Analyse the item. */
    public void process(String accession) throws Exception
     {
      // Do something great with your accession number here:
       System.out.println(accession);
       // make a pause or you will see nothing.
       try { Thread.sleep(1000); } catch (InterruptedException ex) {}

     };
  };

Similar files can be written for other operating systems. For deeper understanding, you can look at the ListProcessor source code, included in the distribution.

Writing your own robot

Automatic generation of web robots is easy, but you may need to create a module that implements your own algorithm or communicates with the server with CORBA, RMI and so on. Sight can generate a template for the robot that accepts the request data structure you expect and returns the array of result data structures you plan. You need to complete a single method (go) that actually implements your part. This option is accessible via menu item Special robots|Java module. In the first step, you need to specify your input and output data structures by adding the fields to these records. After generating, you need to complete the go method. It takes your request data structure (it is passed as Object and must be casted). The method must return the array of your result data structures. If you create just one result from your request, return a single-size array. If no result is available (for example, no hits in the database), return the zero size array. After a potentially transient error throw the IOException, this will make Sight to retry your request after several minutes. Finally, after heavy error, throw FatalAgentException. The .java file, containing the code of such robot, can be added to any Sight application in the same way as any other generated web robot. The example of a simple robot with the built-in trivial algorithm could be impl.manipulators.rangeSelector

Writing your own web robot generator

All Sight generators are placed in folder bin/Generators. They are presented in the form of .jar files with specific manifest. For the built-in generators these archives only contain this manifest, but they also can contain your specific classes. The manifest contains the following entries:

Sight-210-generator_frame: The main code generator class. Must implement Sight.Generators.AbstractRobotGenerator interface. This class gets the examples of the server response. It gets the rest of the user data from its formProvider that can provide abstractForm. Note that all WebMacro classes are available.

Sight-210-generator_name: Name for the generator.
Sight-210-generator_def: Description of the generator.

Sight-210-generator_priority: Priority by that the generators are sorted in the selection dialog (optional).

Sight scans the mentioned folder and loads all found generators. In the generator selection dialog, you can also pick the generator file directly.

For more information, please study the provided example, located in the Example folder (both source and executables are provided).

For more understanding, see the source code of all built-in web robots.

Sight application as the Web server

Since Sight 2.1.2, the Sight application can start as the web server. By default, it starts on the port 4525 on the local host. To activate this mode, you need to select the corresponding initiator. Next, you must add and configure at least one DAS server endpoint. This agent is responsible for sending the results of analysis back to the client.

Sight server request is similar to DAS server request to get the sequence feature: all you need is to read from the url like http://<you-ip-address>/das/DSN/features?segment=REF:start,stop. Read the DAS specification for more details. The server mode is still experimental, and the other types of the DAS request are not yet supported. The server mode is now mainly used to make Sight callable from the Perl, Python and other applications, running at the same or nearby-standing computer (this is why it runs on the non-standard port).