Friday, August 31, 2007

Building an async JAX-WS from the bottom up.

Now and then in JDeveloper land we have an organization wide bug hunt to improve the overall quality of our product. For the most recent one I decided to try to write a async web service from the bottom up.

For the purposes of this experiment I am going to create a simple HelloWorld example so first off lets created the primary interface:

package model;

import ...;

@WebService
public class Hello
{

  public void sayHello(String name)
  {
  ...
  }
}

And of course we need to define the interface for the callback service:

package model;

@WebService
public interface HelloCallback
{
  public void response(String message);
}

Simple enough, right? Now I we are going to create a static wsdl under public_html/wsdl. Use the normal wizard to create a blank document then cut and paste the results of "Show WSDL for annotations" from the Hello class. You also need to do something similar for the HelloCallback service. Unfortunately you can't show the WSDL for a SEI, although I have raised that bug, so the quickest way to generate the WSDL content is to temporarily convert HelloCallback to a class.

To complete the WSDL add a partnerLinkType, this is used by tools such as BPEL to link the two halves of the equation:

    <partnerLinkType name="HelloService"
                     xmlns="http://schemas.xmlsoap.org/ws/2003/05/partner-link/">
        <role name="In">
            <portType name="tns:Hello"/>
        </role>
        <role name="Out">
            <portType name="tns:HelloCallBack"/>
        </role>
    </partnerLinkType>

Then it is worth updating the Hello class to reference this WSDL in the annotations:

package model;

@WebService(wsdlLocation = "/wsdl/Hello.wsdl")
public class Hello 
{
  ...
}

Okay so the specifications are in place lets look at a trivial implementation of the sayHello method. Note that in this case the response can arrive before the original call has completed; but this will do for this example.

SOAPMessageContext smc = (SOAPMessageContext)_context.getMessageContext();
        
// Get hold of the AddressingProperties and look up the reply to address
// and later the return message id
//
        
AddressingProperties ap = AddressingUtils.getAddressingProperties(smc.getMessage());
EndpointReference endpointReference = ap.getReplyTo();
AttributedURI address = endpointReference.getAddress();
URI replyTo = address.getURI();
        
// 

try
{

  // Use the servlet context to get a reference to the local wsdl
  //  

  ServletContext sc = (ServletContext)_context.getMessageContext().get(
    "javax.xml.ws.servlet.context");
  URL found = sc.getResource("/wsdl/Hello.wsdl");
 
  // Create a new service instance on the callback

  Service s = Service.create(found, new QName(
                             "http://model/",
                             "HelloCallBackService"));
            
  // Create a port

  HelloCallBack acb = s.getPort(
           new QName(
               "http://model/",
               "HelloCallBackPort"),
               HelloCallBack.class); 
            
  BindingProvider bp = (BindingProvider)acb;
  Map requestProperties = bp.getRequestContext();

  // Change the endpoint address to the match the replyTo address
            
  requestProperties.put(
    BindingProvider.ENDPOINT_ADDRESS_PROPERTY,
    replyTo.toString());

  // Make sure that this request has the correct addressing headers
            
  AddressingUtils.setupClientSideWSAddressing(requestProperties);
  String t = requestProperties.keySet().toString();
  AddressingProperties rap = 
    AddressingUtils.getOutboundAddressingProperties(requestProperties);
  rap.setRelatesTo(
    ap.getMessageID());
       
  // Invoke the response address
  //
     
  acb.response("Hello " + name);
}
catch (Exception ex) {
  ex.printStackTrace();            
}
        

Part of the pleasure of using JAX-WS over JAX-RPC is the simplicity of invoking services dynamically in this way. As you can see above this is actually really quite straight forward.

So how would you go about testing this, well the HTTP Analyzer in JDeveloper doesn't support WS-Addressing headers properly although we are working on this. You can of course create your own headers in the HTTP Content view, it might look like this:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Header xmlns:ns1="http://www.w3.org/2005/08/addressing">
      <ns1:MessageID>ws:uniqueAddress</ns1:MessageID>
      <ns1:ReplyTo>
         <ns1:Address>http://returnAddress.com/example</ns1:Address>
      </ns1:ReplyTo>
   </soap:Header>
   <soap:Body xmlns:ns2="http://model/">
      <ns2:sayHello>
         <arg0>John</arg0>
      </ns2:sayHello>
   </soap:Body>
</soap:Envelope>

If you start up the HTTP Analyzer before the embedded server you can send his message to the endpoint and you should see the failed attempt to send the message to the returnAddress url in the analyzer log. It should look something like:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="http://www.w3.org/2005/08/addressing">
   <env:Header>
      <wsa:To>http://returnAddress.com/example</wsa:To>
      <ns1:MessageID>ws:anotherUniqueAddress</ns1:MessageID>
      <wsa:RelatesTo xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" soap:mustUnderstand="0">ws:uniqueAddress</wsa:RelatesTo>
   </env:Header>
   <env:Body>
      <ns0:response xmlns:ns0="http://model/">
         <arg0 xmlns="">Hello John</arg0>
      </ns0:response>
   </env:Body>
</env:Envelope>

This is worth remembering this trick if you are trying to test a async web service and don't want to go to the bother of setting up a valid return address to connect to.

Next time I get a few minutes I will take a look at the client side of this equation. Back to fixing the bugs I have just found.

Monday, August 13, 2007

A viewlet showing Abbot being used to test relatively complex UI

As I have mentioned previously we have been playing with Abbot and Costello internally in order to automate testing of some of our UI. When you start working Abbot/Costello there is a steep learning curve with people not sure that they will ever get there tests run more than once. (Some of the reasons I have talked about before.) Once you get the hang of things though setting up tests becomes much easier with recording tests becoming a normal part of the working day.

I figured it would be usefully to publish a viewlet showing abbot testing some relatively meaty UI. So here is a recording of current smoke tests for JDeveloper web services tooling which were created in Costello and run using Abbot.

There are a few glitches in that some context menu are not recorded and the playback speed is slower than the tests actually run on my machine. In general though this gives a fair representation of a test run on my machine. Also if you look carefully you will notice some bugs that the tests are not picking up; but I will leave that as an exercise for the reader to figure out what is wrong.

Friday, August 10, 2007

More than meets the eye, or JAX-WS and the not so obvious interfaces.

I really enjoy using JAX-WS I have only come to web services relatively recently but I have found the programming model to be relatively easy to use. Certainly removing the need to create n xml mapping files has made the developer's life much easier.

One problem I have with the design of JAX-WS it the way that you end up having to cast common objects to other helper interfaces. Now most of the implementation is done using proxies so in most debuggers you are going to have a hard time understanding the objects you are passed. Lets just knock up a quick method to list all the interfaces for a bit of debugging:

public class Util
{
    public static void dumpInterfaces(Class t)
    {
       System.out.println("=== Interfaces for " + t.getSimpleName());
       List interfaces = new ArrayList();
       interfaces.add(t);
       for (int i=0; i < interfaces.size(); i++) {
           Class it = interfaces.get(i);
           for (Class sp : it.getInterfaces()) {
               if (!interfaces.contains(sp))
               {
                 interfaces.add(sp);
               }
           }
       }
       
       // List them out
       //
       
       for (Class sp : interfaces)
       {
         System.out.println(sp.getName());
       }
    }

    public static  void dumpMap(String name, Map map) 
    {
        System.out.println("=== " + name);
        for(Map.Entry e : map.entrySet()) {
            System.out.println(e.getKey() + " : " + e.getValue());
        }
    }
}

On the server side lets take a look at what interfaces that might not otherwise be obvious, take this simple web service example:

@WebService
public class Hello {
    
    @Resource
    private WebServiceContext context;
    
    
    public String sayHello(String name) {
        
        Util.dumpInterfaces(context.getClass());
        Util.dumpInterfaces(context.getMessageContext().getClass());
        
        Util.dumpMap("Dumping message context properties",context.getMessageContext());        
        
        return "Hello " + name;
    }
}

This gives the following output for the interfaces. The web service context is less interesting that the message context. This can be cast to SOAPMessageContext which can give you access to the original SOAP message along with the headers. This is an alternative to using Handlers in order to process headers.

=== Interfaces for WebServiceContextImpl
oracle.j2ee.ws.server.jaxws.WebServiceContextImpl
javax.xml.ws.WebServiceContext
=== Interfaces for SOAPMessageContextImpl
oracle.j2ee.ws.common.jaxws.SOAPMessageContextImpl
javax.xml.ws.handler.soap.SOAPMessageContext
javax.xml.ws.handler.MessageContext
java.util.Map

The since the message context is also a map it is relatively easy to list all of the properties defined in there. Here is what I see when I run the service defined above. Part of the problem with JAX-WS is that these properties are extensible so it is hard to get a definitive list of all of them. Something for a future blog I think.

=== Dumping message context properties
javax.xml.ws.binding.attachments.inbound : {}
javax.xml.ws.http.request.pathinfo : null
incomming.addressing.properties : amespace=http://www.w3.org/2005/08/addressing
javax.xml.ws.servlet.context : Application1-Project1-webapp web-app
javax.xml.ws.http.request.querystring : null
javax.xml.ws.http.response.code : null
javax.xml.ws.http.response.headers : null
javax.xml.ws.servlet.request : com.evermind.server.http.EvermindHttpServletRequest@109b23b
oracle.j2ee.ws.context.Message : oracle.j2ee.ws.saaj.soap.soap11.Message11@1945696
javax.xml.ws.binding.attachments.outbound : {}
javax.xml.ws.servlet.response : com.evermind.server.http.EvermindHttpServletResponse@1c0bec5
javax.xml.ws.http.request.headers : {SOAPACTION=[""], CONNECTION=[Keep-Alive, TE], CONTENT-LENGTH=[202], USER-AGENT=[Oracle HTTPClient Version 10h], TE=[trailers, deflate, gzip, compress], ACCEPT-ENCODING=[gzip, x-gzip, compress, x-compress], CONTENT-TYPE=[text/xml; charset=ASCII], HOST=[localhost:8988], ECID-CONTEXT=[HIINUXQV`[YegG@DKKGSLNWWZZ[Zbef@BKCAGSLNS[TQ]}
javax.xml.ws.http.request.method : POST
javax.xml.ws.handler.message.outbound : false

We can do something similar with the on the client side, so if we were to create a simple client:

public class Client {
    public Client() {
    }
    
    public static void main(String argv[]) {
        HelloService hs = new HelloService();
        Hello h = hs.getHelloPort();
        Util.dumpInterfaces(h.getClass());
        
        BindingProvider bp = (BindingProvider)h;
        Util.dumpInterfaces(bp.getBinding().getClass());
        Util.dumpMap("Request context",bp.getRequestContext());
        Util.dumpMap("Response context",bp.getResponseContext());
    }
}

We can see that the port is implemented using a proxy object that implement both the SEI and the BindingProvider interface. The BindingProvider interface is perhaps not that much of a secret, although I would have thought that a more usable design for the client generation would have seen the SEI extend this interface to make the programming model cleaner.

=== Interfaces for $Proxy12
$Proxy12
project1.proxy.Hello
javax.xml.ws.BindingProvider
=== Interfaces for SOAPBindingImpl
oracle.j2ee.ws.client.jaxws.SOAPBindingImpl
javax.xml.ws.soap.SOAPBinding
javax.xml.ws.Binding
=== Request context
javax.xml.ws.binding.attachments.outbound : {}
=== Response context

It is worth noting that again there is a SOAP variant that you need to cast to in order to get all of the functionality available to you. Of course you have to be careful that you understand the context of your service. For example if you were to using the HTTP binding than the Binding object would of course be HTTP not SOAPBinding.

Finally where I have dumped out a the contents of a map, these represent the current state when running on an interim build of OC4J 11. As noted before there are many more properties that are available that can be added depending on what platform and extensions you are using.

Update: A nice summary document for the standard MessageContext and BindingContext parameters can be found here.

Friday, August 3, 2007

Q: How do I monitor outgoing HTTP Traffic in OC4J, or any other java app for that matter

Make sure that -Dhttp.proxyHost or -Dhttp.proxyPort is set

When you use the HTTP Analyzer within JDeveloper you will find that all java processes started from then on will send there traffic via the HTTP Analyzer. This includes the embedded oc4j instance. This can be useful if your service is using external services such as a credit card validation system.

Is the relatively straight forward to do this trick for any java process that might have been started outside of JDeveloper. You simply need to supply something like -Dhttp.proxyHost=localhost -Dhttp.proxyPort=8099 to the java executable. You can of course alter the host name if the process is running on a different machine.

If you are running an instance of oc4j you tend to do this with with oc4j or oc4j.bat. These both read the environment variable OC4J_JVM_ARGS which can contain the parameters mentioned above.

This may not capture all traffic between SOA services, such as BPEL, as in some cases inter-process communicate will skip the "HTTP Protocol" stack and exchange infosets more directly. This is something we will of course look at in future releases.

Friday, July 27, 2007

"Wait, Don't Delay" or how to test complex async Java GUIs

I have been working with abbot and costello for some time now and people tend to have one of two problems getting there scripts to run reliably: the first is not being able to find a component again; and the second is the real time nature of most complex swing UIs.

The first issue can usually be solved by ensuring the that program is accessible and that use of the "name" property where required. Things do get more complicated with nested component; but they can generally be solved with a little bit of common sense.

The second issue is the real topic of this post and it sometimes takes quite a while to change around the mindset required to create tests that will run reliably on a range of platforms and systems.

Take for example an wizard that needs to query a database between pages. This means that the required components might not be enabled instantly after the user action. Now a naive user might try to include a "delay" step because they know how long the action might take; but they will find that eventually the test will fail on one platform or another. They might also try to use the waitForIdle action which is a better guess; but unfortunately not reliable and it might never return in the case of a dialog with an ongoing animation.

In all cases where delays and waits have been used in our tests we have seen then resulting in unstable test suites. The only thing I can think that delay should be used for in day to day tests is using it to ensure a piece of functionality doesn't take too long to run. Even then it will be hard to pick a value that will work on all platforms being tested upon.

Lets look at a few ideas on how you can detect whether the task has completed.

The first and most important thing you can look at is obvious UI changes that signal that the asynchronous task as finished. This might be as simple as waiting for a component to be showing using "Insert"->"Wait For"->"Component Showing". Or in the case where the UI component is always displayed you can use an assert step to ensure that a given component is enabled. In Costello this is done by selecting the "References" tab, then go to the "Properties" tab and find the "enabled" property, hold down Control and use the button that appears to insert a "Wait For" step.

The one exception to this are progress dialogs which are generally not a reliably way of tracking a task. If a progress dialog is implemented properly they might never show if the task is really quite quick.

You can also monitor lists and trees for the addition of new nodes. In costello you only need to add a selectRow step for a tree and code being the scenes will wait until that node path becomes available. (I did some of the work for this). You do find that when you have some experience with costello that you instinctively select elements in list and tree because they provide you will a cheap way to test the current state of the model.

If you are working in a product such as an IDE then you will find that many asynchronous tasks write to a log window of some kind. The example of this in JDeveloper is when we try to start and embedded oc4j instance and we have to monitor the log window to know when it has finished starting. To deal with this we use the "Call" step to invoke a static java method that does the work for us. We might also have achieved the same results with an "Expression" step; but they can be harder debug.

If the worst comes to the worst then you do end up having to provide some kind of behind the scenes API. In the work I have been doing on JDeveloper it proved to be easier to invoke the build system programatically using an Expression step rather than the UI directly. You can also use "Call" or "Expression" steps to query the state of UI components that you don't have tester for, although in the long term you are much better or writing the testers in the first place. An example I currently have on my queue waiting is a custom tester for the "gutter" in the code editor in JDeveloper. Without that it is very hard to work out when "Audit" has finished applying its rules in order to invoke a "Quick Fix". Currently a test is this area is using a very long delay which I am less than pleased with; but I hope to remove this soon.

Finally we have implemented a kind of "internal" log window using Logger for those cases where we can't think of something more creative. This provide a way for actions to say "I am finished" and for abbot to be able to pick this up. In general this is only for use as a sticking plaster until we can put the right tester in place.

Thursday, July 26, 2007

Configuring the HTTP Analyzer to record HTTPS traffic

One of the cool new feature we added to the HTTP Analyzer in 11 is the ability to monitor https traffic. This gives you the ability to see all of the crossing traffic in plain text. The configuration for this though is a mite complicated and didn't make it into the documentation for the Technology Preview so I thought that I should write it up here. To make things simple I am just going to look at try to sniff my password from amazon.co.uk; but the same principles apply when working with other HTTP traffic such as SOAP messages.

The first thing you need to do is to use the keytool command to create a server side key. For various security and legal reasons we can't ship a default keystore with the product; but it is fairly easily to create one yourself:

keytool -genkey -alias oracle-server -dname "CN=Gerard Davison, OU=HTTPAnalyzer O=Oracle L=Reading, S=Berkshire, C=UK" -keyalg RSA -keypass welcome -storepass welcome -keystore server.keystore

You then need to configure the HTTP Analyzer, bring up the preferences dialog and set this keystore for both the client and the server. You can of course use two separate keystores if you wish. This is more likely to be useful when working with web services rather than standard web traffic.

In this example I am going to look at using the HTTP Analyzer to look at traffic going to amazon.co.uk. To do this I need to configure my web browser to use the analyzer as its proxy. This by default will be localhost with port 8099.

For the moment, although this is something I am working on, you also need to manually import any certificates for website you are going to visit. Some ideas on how your might go about this are in my previous blog entry on this topic. The snippet of java code appears to be the most reliable option. Update: Or you can use the root certificate keystore that comes with java with a workaround. He have made this much easier in more recent internal builds of JDeveloper.

Now start the HTTP Analyzer and try to log into www.amazon.co.uk using this proxy configuration. You will see a bunch of paranoid messages complaining about how the certificate is not right. This is normal and your should just accept the certificate for the session. (But it is a reminder of why you should never ignore these dialogs in the real world)

If you continue to log in the you should see that there is a bunch of requests, one of these will contain your username and password in plain text. (I am not going to show you a screen grab of this). You will also notice a whole bunch of error that relate to fetching images. This is because you need to import a certificate for each server you are talking to. In the case of amazon.co.uk images are served from "https://images-eu.ssl-images-amazon.com".

Being able to watch traffic in this way is very valuable tool when debugging interactions. AJAX services are really quite interesting to watch in this way. You can also record, replay and mock requests as you would un-encrypted traffic.

Tuesday, July 24, 2007

Have-A-Job Junior Test Engineer. Reading, UK

We have an opening for a junior test engineer working on Oracle JDeveloper in the Reading, UK who has an interest in Java and a desire to learn new things. The successful candidate will gain experience in testing a range of JDeveloper components both manually and using automated testing tools. They will also have the benefit of being co-located with the development team and be able to work closely with them on a day to day basis.

We have a lovely office overlooking the Thames which is currently in the process of flooding. Thankfully the lovely riverside and nature reserve provide a great flood plain so the office stays dry. We have a great benefits package including the ability to buy up to 35 days leave a year, not including bank holidays.

If you are interested, drop me a comment and I will put you in touch with the right people.