Making your JSON-P (JSR-353) code slightly prettier

The JSON-P API as described in JSR-353 is a limited API for working with JSON; but at the basic level it will do the job. It is possible to add a few utility methods that can make your code, in my eyes, prettier.

The first think that annoyed me was the use of Json.createObjectBuilder() and Json.createArrayBuilder() when trying to construct a JSONObject.

So lets create a nice helper class with some very short method names for both:

public static JsonObjectBuilder ob() {
  return Json.createObjectBuilder();
}

public static JsonArrayBuilder ab() {
  return Json.createArrayBuilder();
}

This makes the creation of object just that little bit less wordy:

JsonObject object = ob()
  .add("hello", "world")
  .add("fark", ".com")
  .add("fish", ob()
    .add("child", "child"))
  .add("array", ab()
    .add("one")
    .add("two")
    .add(ob()
      .add("boolean",true)))
  .build();

The second problem is accessing element in a JsonObject can be wordy; but it is relatively easy to knock up a method that would allow some simply XPath like accessor:

public static <T extends JsonValue> T get(JsonStructure structure, 
  String path, Class<T> type) {

  String segments[] = path.split("/");
  JsonValue currentValue = structure;
  for (String segment : segments) {

    if (segment.length() == 0) {
      continue;
    }

    if (currentValue instanceof JsonObject) {
      JsonObject currentObject = (JsonObject) currentValue;
      currentValue = currentObject.get(segment);
    } else if (currentValue instanceof JsonArray) {
      if (segment.startsWith("[") && segment.endsWith("]")) {
        int index = Integer.parseInt(segment.substring(1, segment.length() - 1));
        currentValue = ((JsonArray) currentValue).get(index);
      } else {
        throw new IllegalArgumentException("Array type requires key of the form [n]");
      }
    } else {
      throw new IllegalStateException("Value types are not decomposible" 
        + currentValue.getValueType());
    }

  }

  return type.cast(currentValue);

}


// Example to get hold of a string value

System.out.println(get(object, "/fish/child", JsonString.class));

Of course even this is a little bit untidy as you have to care about the internal JsonValue types which is a pain; and deal with possibly null pointers without the aid of Optional. It doesn't take much to wrap these up though.

public static String getString(JsonStructure structure, String path) {
  JsonString value = get(structure, path, JsonString.class);
  return value != null ? value.getString() : null;
}

public static boolean is(JsonStructure structure, String path) {
  JsonValue value = get(structure, path, JsonValue.class);
  return value != null ? value == JsonValue.TRUE : false;
}

public static boolean isNull(JsonStructure structure, String path) {
  JsonValue value = get(structure, path, JsonValue.class);
  return value != null ? value == JsonValue.NULL : false;
}

public static BigInteger getBigDecimal(JsonStructure structure, String path) {
  JsonNumber value = get(structure, path, JsonNumber.class);
  return value != null ? value.bigIntegerValue() : null;
}

public static int getInt(JsonStructure structure, String path) {
  JsonNumber value = get(structure, path, JsonNumber.class);
  return value != null ? value.intValue() : null;
}

public static JsonArray getArray(JsonStructure structure, String path) {
  return get(structure, path, JsonArray.class);
}

public static JsonObject getObject(JsonStructure structure, String path) {
  return get(structure, path, JsonObject.class);
}

This means you can write more direct code such as:

if (is(object, "/array/[2]/boolean")) {
  System.out.println(getString(object, "/fish/child"));
}

Sometimes it only take a few statically imported methods to make a API more useful / easier to read.

Selecting level of detail returned by varying the content type, part II

In my previous entry, we looked at using the feature of MOXy to control the level of data output for a particular entity. This post looks at an abstraction provided by Jersey 2.x that allows you to define a custom set of annotations to have the same effect.

As before we have an almost trivial resource that returns an object that Jersey will covert to JSON for us, note that for the moment there is nothing in this code to do the filtering - I am not going to pass in annotations to the Response object as in the Jersey examples:

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

@Path("hello")
public class SelectableHello {


  @GET
  @Produces({ "application/json; level=detailed", "application/json; level=summary", "application/json; level=normal" })
  public Message hello() {

    return new Message();

  }
}

In my design I am going to define four annotations: NoView, SummaryView, NormalView and DetailedView. All root objects have to implement the NoView annotation to prevent un-annotated fields from being exposed - you might not feel this is necessary in your design. All of these classes look the same so I am going to only show one. Note that the factory method creating a AnnotationLiteral has to be used in preference to a factory that would create a dynamic proxy to have the same effect. There is code in 2.5 that will ignore any annotation implemented by a java.lang.reflect.Proxy object, this includes any annotations you may have retrieved from a class. I am working on submitting a fix for this.

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import javax.enterprise.util.AnnotationLiteral;

import org.glassfish.jersey.message.filtering.EntityFiltering;

@Target({ ElementType.TYPE, 
  ElementType.METHOD, ElementType.FIELD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@EntityFiltering
public @interface NoView {

  /**
   * Factory class for creating instances of the annotation.
   */
  public static class Factory extends AnnotationLiteral<NoView> 
    implements NoView {

    private Factory() {
    }

    public static NoView get() {
      return new Factory();
    }
  }

}

Now we can take a quick look at our Message bean, this is slightly more complicated than my previous example to showing filtering of subgraphs in a very simple form. As I said before the class is annotated with a NoView annotation at the root - this should mean that the privateData is never returned to the client as it is not specifically annotated.

import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
@NoView
public class Message {


  private String privateData;
  @SummaryView
  private String summary;
  @NormalView
  private String message;
  @DetailedView
  private String subtext;
  @DetailedView
  private SubMessage submessage;

  public Message() {
    summary = "Some simple summary";
    message = "This is indeed the message";
    subtext = "This is the deep and meaningful subtext";
    submessage = new SubMessage();
    privateData = "The fox is flying tonight";
  }


  // Getters and setters not shown
}


public class SubMessage {

  private String message;

  public SubMessage() {
    message = "Some sub messages";
  }

  // Getters and setters not shown
}

As noted before there is no code in the resource class to deal with filtering - I considered this to be a cross cutting concern so I have abstracted this into a WriterInterceptor. Note the exception thrown if a entity is used that doesn't have the NoView annotation on it.

import java.io.IOException;

import java.lang.annotation.Annotation;

import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.Set;

import javax.ws.rs.ServerErrorException;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.HttpHeaders;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.ext.Provider;
import javax.ws.rs.ext.WriterInterceptor;
import javax.ws.rs.ext.WriterInterceptorContext;

@Provider
public class ViewWriteInterceptor implements WriterInterceptor {


  private HttpHeaders httpHeaders;

  public ViewWriteInterceptor(@Context HttpHeaders httpHeaders) {
    this.httpHeaders = httpHeaders;
  }


  @Override
  public void aroundWriteTo(WriterInterceptorContext writerInterceptorContext) 
    throws IOException,
           WebApplicationException {

    // I assume this case will never happen, just to be sure
    if (writerInterceptorContext.getEntity() == null) {

      writerInterceptorContext.proceed();
      return;
    }
    else
    {
      Class<?> entityType = writerInterceptorContext.getEntity()
          .getClass();
      String entityTypeString = entityType.getName();
      
      // Ignore any Jersey system classes, for example wadl
      //
      if (entityType == String.class  || entityType.isArray() 
        || entityTypeString.startsWith("com.sun") 
        || entityTypeString.startsWith("org.glassfish")) {
        writerInterceptorContext.proceed();
        return;
      }
      
      // Fail if the class doesn't have the default NoView annotation 
      // this prevents any unannotated fields from showing up
      //
      else if (!entityType.isAnnotationPresent(NoView.class)) {
        throw new ServerErrorException("Entity type should be tagged with @NoView annotation " + entityType, Response.Status.INTERNAL_SERVER_ERROR);
        
      }
    }

    // Get hold of the return media type:
    //

    MediaType mt = writerInterceptorContext.getMediaType();
    String level = mt.getParameters().get("level");

    // Get the annotations and modify as required
    //

    Set<Annotation> current = new LinkedHashSet<>();
    current.addAll(Arrays.asList(
      writerInterceptorContext.getAnnotations()));

    switch (level != null ? level : "") {
      default:
      case "detailed":
        current.add(com.example.annotation.DetailedView.Factory.get());
      case "normal":
        current.add(com.example.annotation.NormalView.Factory.get());
      case "summary":
        current.add(com.example.annotation.SummaryView.Factory.get());

    }

    writerInterceptorContext.setAnnotations(
      current.toArray(new Annotation[current.size()]));

    //

    writerInterceptorContext.proceed();
  }
}

Finally you have to enable the EntityFilterFeature manually, to do this you can simple register it in your Application class

import java.lang.annotation.Annotation;

import javax.ws.rs.ApplicationPath;

import org.glassfish.jersey.message.filtering.EntityFilteringFeature;
import org.glassfish.jersey.server.ResourceConfig;

@ApplicationPath("/resources/")
public class SelectableApplication extends ResourceConfig {
  public SelectableApplication() {

    packages("...");

    // Set entity-filtering scope via configuration.
    property(EntityFilteringFeature.ENTITY_FILTERING_SCOPE, 
    new Annotation[] {
      NormalView.Factory.get(), DetailedView.Factory.get(), 
      NoView.Factory.get(), SummaryView.Factory.get()
    });
    register(EntityFilteringFeature.class);
  }


}

Once you have this all up and running the application will respond as before:

GET .../hello Accept application/json; level=detailed or application/json
{
  "message" : "This is indeed the message",
  "submessage" : {
    "message" : "Some sub messages"
  },
  "subtext" : "This is the deep and meaningful subtext",
  "summary" : "Some simple summary"
}

GET .../hello Accept application/json; level=normal
{
  "message" : "This is indeed the message",
  "summary" : "Some simple summary"
}


GET .../hello Accept application/json; level=summary
{
  "summary" : "Some simple summary"
}

This is feel is a better alternative to using the MOXy annotations directly - using custom annotations should have to much easier to port your application to over implementation even if you have to provide you own filter. Finally it is worth also exploring the Jersey extension to this that allows Role based filtering which I can see as being useful in a security aspect.

Selecting level of detail returned by varying the content type

So I have been playing with the various ways of generating JSON in Jersey this week and I have been looking at solutions to the problem of returning different levels of detail depending on the client requirements. Here is one solution that uses the MoXY JAX-B provider in Jersey 2.x.

Consider this very simple hello world class:

@Path("hello")
public class SelectableHello {
    
    @GET
    @Produces({"application/json; level=detailed", "application/json; level=summary", "application/json; level=normal"})
    public Message hello() {
        return new Message();
    }
}

The Message class is annotated with some special annotations from MOXy that allows you to specify different partial object graphs. So my very simple message class looks like the following with three different levels defined:

import javax.xml.bind.annotation.XmlRootElement;

import org.eclipse.persistence.oxm.annotations.XmlNamedAttributeNode;
import org.eclipse.persistence.oxm.annotations.XmlNamedObjectGraph;
import org.eclipse.persistence.oxm.annotations.XmlNamedObjectGraphs;


@XmlNamedObjectGraphs({
  @XmlNamedObjectGraph(name = "summary", 
    attributeNodes = { 
      @XmlNamedAttributeNode("summary") }),
  @XmlNamedObjectGraph(name = "normal",
    attributeNodes =
    { @XmlNamedAttributeNode("summary"), 
      @XmlNamedAttributeNode("message") }),
  @XmlNamedObjectGraph(name = "detailed",
    attributeNodes =
    { @XmlNamedAttributeNode("summary"), 
      @XmlNamedAttributeNode("message"),
      @XmlNamedAttributeNode("subtext") })
  })
@XmlRootElement
public class Message {

  private String summary, message, subtext;

  public Message() {
    summary = "Some simple summary";
    message = "This is indeed the message";
    subtext = "This is the deep and meaningful subtext";
  }

  public void setSummary(String summary) {
    this.summary = summary;
  }

  public String getSummary() {
    return summary;
  }

  public void setMessage(String message) {
    this.message = message;
  }

  public String getMessage() {
    return message;
  }

  public void setSubtext(String subtext) {
    this.subtext = subtext;
  }

  public String getSubtext() {
    return subtext;
  }


}

Then it is a relatively easy step to define a JAX-RS ContextResolver that returns a suitably configured MoxyJsonConfig depending on the mime type. The code in here is a little bit ropey as it assumes that the first item in the accept list is the correct one.

import javax.ws.rs.core.Context;

import javax.ws.rs.core.Context;
import javax.ws.rs.core.HttpHeaders;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.ext.ContextResolver;
import javax.ws.rs.ext.Provider;

import org.eclipse.persistence.jaxb.MarshallerProperties;

import org.glassfish.jersey.moxy.json.MoxyJsonConfig;

@Provider
public class MoxyJsonConfigResolver implements ContextResolver<MoxyJsonConfig> {

    HttpHeaders httpHeaders;


    @Context
    public void setHeaders(HttpHeaders httpHeaders) {
        this.httpHeaders = httpHeaders;
    }

    @Override
    public MoxyJsonConfig getContext(Class<?> c) {

        // Assume we are going to match the first accept
        //
        MediaType responseType = httpHeaders.getAcceptableMediaTypes().get(0);

        MoxyJsonConfig config = new MoxyJsonConfig();
        String level = responseType.getParameters().get("level");
        config.getMarshallerProperties().put(MarshallerProperties.OBJECT_GRAPH, level != null ? level : "detailed");
        return config;
    }
}

So once this little application is running, you can see the output for different mime types. In our case the default for "application/json" is "detailed" but I can see the argument in many cases for "normal" to be the default.

GET .../hello Accept application/json; level=detailed or application/json

{
    "message" : "This is indeed the message",
    "subtext" : "This is the deep and meaningful subtext",
    "summary" : "Some simple summary"
}

GET .../hello Accept application/json; level=normal

{
    "message" : "This is indeed the message",
    "summary" : "Some simple summary"
}

GET .../hello Accept application/json; level=summary

{
    "summary" : "Some simple summary"
}

This does work, but as you can see the annotations could be prettier, the next blog post will be an example using EntityFilteringFeature of Jersey. This feature builds on top of the MOXy functionality above while allowing a custom annotation scheme. This to my mind is a lot easier to work with.

Post-hoc tracing using a debugger

Once nice little features of most debuggers that I have been exercising recently is the ability to log information on a breakpoint. This can be a really useful was to understand code without having to modify it is involve byte code modification.

Let's consider this very trivial and inefficient implementation of a function to return the n'th number in the Fibonacci sequence.

public class Fib {



    public long fib(long number) {
        return number < 1 ? 0 :    // Breakpoint here
            number < 2 ? 1 : fib(number - 2) + fib(number - 1);
    }




    public static void main(String[] args) {

        Fib fib = new Fib();
        System.out.println(fib.fib(10L));

    }

}

Now we add a simple breakpoint, I am going to modify it slightly so that it doesn't halt; but it will log a simple expression which gives us the current value of number

This gives you the following output for our rather inefficient code:

ebugger connected to local process.
Source breakpoint: Fib.java:11, evaluate(number)=10 (long)
Source breakpoint: Fib.java:11, evaluate(number)=8 (long)
Source breakpoint: Fib.java:11, evaluate(number)=6 (long)
Source breakpoint: Fib.java:11, evaluate(number)=4 (long)
Source breakpoint: Fib.java:11, evaluate(number)=2 (long)
Source breakpoint: Fib.java:11, evaluate(number)=0 (long)
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
Source breakpoint: Fib.java:11, evaluate(number)=3 (long)
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
Source breakpoint: Fib.java:11, evaluate(number)=2 (long)
... sometime later
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
Source breakpoint: Fib.java:11, evaluate(number)=2 (long)
Source breakpoint: Fib.java:11, evaluate(number)=0 (long)
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
55

So what we have done here is to add trace post-hoc, you might not even have access to the source and still be able to get useful information out of what the code is doing.

Now the obvious take away from the above example is that we are computing the same value over and over again. So here is another version of the code which instead uses a Map to store previously computed values in the sequence. Note also it is more of a pain to add trace to this code than in the previous case without making the Lambda less pretty.

import java.util.HashMap;
import java.util.Map;

public class Fib {


    Map<Long, Long> data = new HashMap<>();
    {
        data.put(1L, 1L);
    }


    public long fib(long number) {
        return data.computeIfAbsent(number,
                      n -> n < 1 ? 0 : fib(n - 2) + fib(n - 1)); // Breakpoint here
    }




    public static void main(String[] args) {

        Fib fib = new Fib();
        System.out.println(fib.fib(10L));

    }

}

It is worth noting two things here, first of all your breakpoint log expression should be evaluating the value of the Lambda parameter n and that the second is that you should always put your expression on a new line so that future developer can easily breakpoint it. (Not matter now much smaller/ prettier/ clever it would look if you put the code on one).

So the trace output is now as follows, much better as each value is only computed the once.

Debugger connected to local process.
Source breakpoint: Fib.java:17, evaluate(n)=10
Source breakpoint: Fib.java:17, evaluate(n)=8
Source breakpoint: Fib.java:17, evaluate(n)=6
Source breakpoint: Fib.java:17, evaluate(n)=4
Source breakpoint: Fib.java:17, evaluate(n)=2
Source breakpoint: Fib.java:17, evaluate(n)=0
Source breakpoint: Fib.java:17, evaluate(n)=3
Source breakpoint: Fib.java:17, evaluate(n)=5
Source breakpoint: Fib.java:17, evaluate(n)=7
Source breakpoint: Fib.java:17, evaluate(n)=9
55

The screen grabs in this blog are from Jdeveloper; but similar features are available in Intelij, Netbeans and if you are a bit crafty you can get something similar in Eclipse.

Lambda'ery WebSocket code (from UKTECH13 presentation)

At UKTECH13 I was asked to post the source code to the WebSocket used in my presentation, primarily because it was using JDK 8 constructs that were unfamiliar to many. One of the very nice things about the changes to the languages and the supporting library changes is the lack of if statements in the code.

I would note that in a real world chat application that it unlikely different rooms would have different resource URIs but for the purposes of the presentation this made sense.

package websocket;

import static java.util.Collections.emptySet;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.CopyOnWriteArraySet;

import javax.websocket.OnClose;
import javax.websocket.OnError;
import javax.websocket.OnMessage;
import javax.websocket.OnOpen;
import javax.websocket.Session;
import javax.websocket.server.PathParam;
import javax.websocket.server.ServerEndpoint;

@ServerEndpoint(value = "/chat/{room}")
public class ChatService {

  private static final Set<Session> EMPTY_ROOM = emptySet();

  static final ConcurrentMap<String, Set<Session>> rooms =
    new ConcurrentHashMap<>(); 


  @OnOpen
  public void onOpen(Session peer, @PathParam("room") String room) {
    rooms.computeIfAbsent(room,
                          s -> new CopyOnWriteArraySet<Session>()).add(peer);
  }

  @OnClose
  public void onClose(Session peer, @PathParam("room") String room) {

    rooms.getOrDefault(room, EMPTY_ROOM).remove(peer);
    
  }

  @OnError
  public void onError(Session peer, Throwable th,
                      @PathParam("room") String room) {
    System.out.println("Peer error " + room + " " + th);
  }


  @OnMessage
  public void message(String message, Session peer,
                      @PathParam("room") String room) {

    // Send a message to all peers in a room who are not the current
    // peer and are still open. Send the message asynchronously to ensure
    // that the first client is not hung up. 

    rooms.getOrDefault(room, EMPTY_ROOM).parallelStream()
         .filter(s -> s != peer && s.isOpen())
         .forEach(s -> s.getAsyncRemote().sendObject(message));
  };

}

One of the problem with the above design is that there is no error logging when an invalid room is used. This could be useful to diagnose errors. Not wanting to use any conditional statements you could use an Optional object:

import java.util.Optional;


  @OnClose
  public void onClose(Session peer, @PathParam("room") String room) {

    Optional.ofNullable(rooms.get(room))
      .orElseThrow(() -> new IllegalStateException("Cannot find room " + room))
      .remove(peer);
    
  }

Of course you might want this on your method objects, so you can use default methods to create a mixin for this on all your Map objects with a trivial subclass.

import java.util.Optional;


  public interface OptionalMap<K,V> extends ConcurrentMap<K,V> {
     public default Optional<V> find(K key) {
        return Optional.ofNullable(get(key));
     }
  }

  public static class OptionalHashMap<K,V> extends ConcurrentHashMap<K,V>
   implements OptionalMap<K,V> {
       
  }
    

  static final OptionalMap<String, Set<Session>> rooms =
    new OptionalHashMap<>(); 


  @OnClose
  public void onClose(Session peer, @PathParam("room") String room) {

    rooms.find(room)
      .orElseThrow(() -> new IllegalStateException("Cannot find room " + room))
      .remove(peer);
    
  }

Whilst working on my presentation it became apparent that it was also possible to use the "openSessions" and "getUserProperties" method to store discrimination data against the Session. I don't have enough experience yet to say which is the better design for a particular case.

package websocket;


import javax.websocket.OnClose;
import javax.websocket.OnError;
import javax.websocket.OnMessage;
import javax.websocket.OnOpen;
import javax.websocket.Session;
import javax.websocket.server.PathParam;
import javax.websocket.server.ServerEndpoint;

@ServerEndpoint(value = "/chat/{room}")
public class ChatService {

  private static final String ROOM_PROPERTY = "ROOM";


  @OnOpen
  public void onOpen(Session peer, @PathParam("room") String room) {
    peer.getUserProperties().put(ROOM_PROPERTY, room);
  }

  @OnClose
  public void onClose(Session peer, @PathParam("room") String room) {
    
    // No need to tidy up and data is store against peer
  }

  @OnError
  public void onError(Session peer, Throwable th,
                      @PathParam("room") String room) {
    System.out.println("Peer error " + room + " " + th);
  }


  @OnMessage
  public void message(String message, Session peer,
                      @PathParam("room") String room) {

    peer.getOpenSessions().parallelStream()
         .filter(s -> room.equals(s.getUserProperties().get(ROOM_PROPERTY)))
         .filter(s -> s != peer && s.isOpen())
         .forEach(s -> s.getAsyncRemote().sendObject(message));
  };

}

Lambda, will it serialize?

So I have been ponder an enhancement required on the Tyrus project that would allow a user to broadcast to a subset of client connected to a URL across a cluster of machines. There are various way of doing this; but since I was playing with JDK 8 this problem definitely looked like a nail.

To this end I created a simple unit test class that would take my filter, serialise it to disk, read it back and in then execute it. It had a instance field "VALUE" that we could use to reference directly or indirectly to find out what would cause the serialisation to fail.

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.NotSerializableException;
import java.io.ObjectInput;
import java.io.ObjectInputStream;
import java.io.ObjectOutput;
import java.io.ObjectOutputStream;
import java.io.Serializable;

import java.util.function.Predicate;

import org.junit.Test;

public class SerializablePredicateFilterTest {

  public String VALUE = "Bob";

  public interface SerializablePredicate<T> extends Predicate<T>, Serializable {

  }


  public <T> void filter(SerializablePredicate<T> sp, T value) throws IOException, ClassNotFoundException {


    sp.getClass().isLocalClass();

    File tempFile = File.createTempFile("labmda", "set");


    try (ObjectOutput oo = new ObjectOutputStream(new FileOutputStream(tempFile))) {
      oo.writeObject(sp);
    }


    try (ObjectInput oi = new ObjectInputStream(new FileInputStream(tempFile))) {
      SerializablePredicate<T> p = (SerializablePredicate<T>) oi.readObject();

      System.out.println(p.test(value));
    }


  }

}

So just to calibrate lets make sure that an anonymous inner class will fail, because it will always contain a reference to enclosing object....

  @Test(expected = NotSerializableException.class)
  public void testAnonymousDirect() throws IOException, ClassNotFoundException {

    String value = VALUE;


    filter(new SerializablePredicate<String>() {

      @Override
      public boolean test(String t) {
        return value.length() > t.length();
      }
    }, "Bob");

  }

The same is true for local classes, what you don't use local classes?

  @Test(expected = NotSerializableException.class)
  public void testLocalClass() throws IOException, ClassNotFoundException {

    class LocalPredicate implements SerializablePredicate<String> {
      @Override
      public boolean test(String t) {
        // TODO Implement this method
        return false;
      }
    }

    filter(new LocalPredicate(), "Bobby");

  }

So a standalone class will of course work, in this case a nested class for convenience.

  public static class LengthPredicate implements SerializablePredicate<String> {

    private String value;


    public LengthPredicate(String value) {
      super();
      this.value = value;
    }

    public void setValue(String value) {
      this.value = value;
    }

    public String getValue() {
      return value;
    }

    @Override
    public boolean test(String t) {
      // TODO Implement this method
      return false;
    }
  }


  @Test
  public void testStaticInnerClass() throws IOException, ClassNotFoundException {

    filter(new LengthPredicate(VALUE), "Bobby");

  }

So lets get down with JDK 8, it turns out that my first try also fails but it does confirm that the serialisation is quite happy to take a Lambda in general.

  @Test(expected = NotSerializableException.class)
  public void testLambdaDirect() throws IOException, ClassNotFoundException {


    filter((String s) -> VALUE.length() > s.length(), "Bobby");

  }

A slight modification to copy the value into a effectively final attributes, and voila the lambda is now serialised and retrieved properly.

  @Test
  public void testLambdaInDirect() throws IOException, ClassNotFoundException {

    String value = VALUE;

    filter((String s) -> value.length() > s.length(), "Bobby");

  }

And of course if the value is a simple method parameter it also works fine.

  @Test
  public void testLambdaParameter() throws IOException, ClassNotFoundException {

    invokeWithParameter(VALUE);

  }

  private void invokeWithParameter(String value) throws java.lang.ClassNotFoundException, java.io.IOException {
    filter((String s) -> value.length() > s.length(), "Bobby");
  }

So the answer is yes, you can get it to serialise if you are a bit careful.

Write an auto-debugger to catch Exceptions during test execution

Previously I have stated that there are some exceptions you would always want to keep an debugger breakpoint on for. This help prevents code rotting away without you noticing - sometimes masking a different problem.

If you take this seriously then it is a good idea to extend this idea to you automated testing; but coming up with a comprehensive solution is not entirely trivial. You could just start with a try/catch but that won't capture exceptions on other threads. You could also do something using AOP; but depending on the framework you are not guaranteed to catch everything also it does mean you are testing with slightly different code which will worry some.

A few days ago I came across this blog entry on how to write your own debugger, and I wondered if it was possible for java process to debug itself. Turn out you can and here is the code I came up with as part of this little thought experiment.

The first part of the class just contains something fairly hacky code to guess what the port that would be required to connect back to the same VM based on the start up parameters. It might be possible to use the Attach mechanism to start the debugger; but I didn't seen an obvious way to get it to work. Then there are just a couple of factory methods that take a list of exceptions to look out for.

package com.kingsfleet.debug;

import com.sun.jdi.Bootstrap;
import com.sun.jdi.ReferenceType;
import com.sun.jdi.VirtualMachine;
import com.sun.jdi.connect.AttachingConnector;
import com.sun.jdi.connect.Connector;
import com.sun.jdi.connect.IllegalConnectorArgumentsException;
import com.sun.jdi.event.ClassPrepareEvent;
import com.sun.jdi.event.Event;
import com.sun.jdi.event.EventQueue;
import com.sun.jdi.event.EventSet;
import com.sun.jdi.event.ExceptionEvent;
import com.sun.jdi.event.VMDeathEvent;
import com.sun.jdi.event.VMDisconnectEvent;
import com.sun.jdi.request.ClassPrepareRequest;
import com.sun.jdi.request.EventRequest;
import com.sun.jdi.request.ExceptionRequest;

import java.io.IOException;

import java.lang.management.ManagementFactory;
import java.lang.management.RuntimeMXBean;

import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;


public class ExceptionDebugger implements AutoCloseable {




   public static int getDebuggerPort() {
       // Try to work out what port we need to connect to

       RuntimeMXBean runtime = ManagementFactory.getRuntimeMXBean();
       List<String> inputArguments = runtime.getInputArguments();
       int port = -1;
       boolean isjdwp = false;

       for (String next : inputArguments) {
           if (next.startsWith("-agentlib:jdwp=")) {
               isjdwp = true;
               String parameterString = next.substring("-agentlib:jdwp=".length());
               String[] parameters = parameterString.split(",");
               for (String parameter : parameters) {
                   if (parameter.startsWith("address")) {
                       int portDelimeter = parameter.lastIndexOf(":");
                       if (portDelimeter != -1) {
                           port = Integer.parseInt(parameter.substring(portDelimeter + 1));
                       } else {
                           port = Integer.parseInt(parameter.split("=")[1]);
                       }
                   }
               }
           }
       }
       return port;
   }


   public static ExceptionDebugger connect(final String... exceptions) throws InterruptedException {
       return connect(getDebuggerPort(),exceptions);
   }

   public static ExceptionDebugger connect(final int port, final String... exceptions) throws InterruptedException {

       ExceptionDebugger ed = new ExceptionDebugger(port, exceptions);

       return ed;
   }

The constructor create a simple daemon thread that starts the connection back to the VM. It is very important to do this is a separate thread otherwise obviously the VM will grind to a halt when we hit a breakpoint. It is a good idea to make sure that code in that thread doesn't throw one of the exceptions - for the moment I am just hoping for the best.

Finally the code just maintains a list of the banned exceptions, if you had a bit more time it should be possible to store the stack trace where the exception occurred.

// 
   // Instance variables

   private final CountDownLatch startupLatch = new CountDownLatch(1);
   private final CountDownLatch shutdownLatch = new CountDownLatch(1);

   private final Set<String> set = Collections.synchronizedSet(new HashSet<String>());
   private final int port;
   private final String exceptions[];
   private Thread debugger;
   private volatile boolean shutdown = false;


   //
   // Object construction and methods
   //




   private ExceptionDebugger(final int port, final String... exceptions) throws InterruptedException {

       this.port = port;
       this.exceptions = exceptions;

       debugger = new Thread(new Runnable() {

           @Override
           public void run() {
               try {
                   connect();
               } catch (Exception ex) {
                   ex.printStackTrace();
               }
           }
       }, "Self debugging");
       debugger.setDaemon(true); // Don't hold the VM open
       debugger.start();

       // Make sure the debugger has connected
       if (!startupLatch.await(1, TimeUnit.MINUTES)) {
           throw new IllegalStateException("Didn't connect before timeout");
       }
   }


   @Override
   public void close() throws InterruptedException {
       shutdown = true;
       // Somewhere in JDI the interrupt was being eaten, hence the volatile flag 
       debugger.interrupt();
       shutdownLatch.await();
   }


   /**
    * @return A list of exceptions that were thrown
    */
   public Set<String> getExceptionsViolated() {
       return new HashSet<String>(set);
   }

   /**
    * Clear the list of exceptions violated
    */
   public void clearExceptionsViolated() {
       set.clear();
   }

The main connect method is a fair simple block of code that ensures the connection and configures any initial breakpoints.

//
   // Implementation details
   //


   private void connect() throws java.io.IOException {


       try {
           // Create a virtual machine connection
           VirtualMachine attach = connectToVM();


           try
           {

               // Add prepare and any already loaded exception breakpoints
               createInitialBreakpoints(attach);

               // We can now allow the rest of the work to go on as we have created the breakpoints
               // we required

               startupLatch.countDown();

               // Process the events
               processEvents(attach);
           }
           finally {

               // Disconnect the debugger
               attach.dispose();

               // Give the debugger time to really disconnect
               // before we might reconnect, couldn't find another
               // way to do this

               try {
                   TimeUnit.SECONDS.sleep(1);
               } catch (InterruptedException e) {
                   Thread.currentThread().interrupt();
               }
           }
       } finally {
           // Notify watchers that we have shutdown
           shutdownLatch.countDown();
       }
   }

Connecting back to self is just a process of finding the right attaching connector, in this case Socket although I guess you could use the shared memory transport on some platforms if you modified the code slightly.

private VirtualMachine connectToVM() throws java.io.IOException {

       List<AttachingConnector> attachingConnectors = Bootstrap.virtualMachineManager().attachingConnectors();
       AttachingConnector ac = null;

       found:
       for (AttachingConnector next : attachingConnectors) {
           if (next.name().contains("SocketAttach")) {
               ac = next;
               break;

           }
       }

       Map<String, Connector.Argument> arguments = ac.defaultArguments();
       arguments.get("hostname").setValue("localhost");
       arguments.get("port").setValue(Integer.toString(port));
       arguments.get("timeout").setValue("4000");

       try {
           return ac.attach(arguments);
       } catch (IllegalConnectorArgumentsException e) {
           throw new IOException("Problem connecting to debugger",e);
       }
   }

When you connect the debugger you have no idea as to whether the exceptions you are interested in have been loaded, so you need to register breakpoints for both the point where the classes are prepared and for those that have already been loaded.

Note that the breakpoint is set with a policy only to break the one thread - otherwise for obvious reasons the current VM will grind to a halt if the debugger thread is also put to sleep.

private void createInitialBreakpoints(VirtualMachine attach) {
       // Our first exception is for class loading

       for (String exception : exceptions) {
           ClassPrepareRequest cpr = attach.eventRequestManager().createClassPrepareRequest();
           cpr.addClassFilter(exception);
           cpr.setSuspendPolicy(EventRequest.SUSPEND_EVENT_THREAD);
           cpr.setEnabled(true);
       }

       // Then we can check each in turn to see if it have already been loaded as we might
       // be late to the game, remember classes can be loaded more than once
       //

       for (String exception : exceptions) {
           List<ReferenceType> types = attach.classesByName(exception);
           for (ReferenceType type : types) {
               createExceptionRequest(attach, type);
           }
       }
   }


   private static void createExceptionRequest(VirtualMachine attach, 
                                              ReferenceType refType) {
       ExceptionRequest er = attach.eventRequestManager().createExceptionRequest(
           refType, true, true);
       er.setSuspendPolicy(EventRequest.SUSPEND_EVENT_THREAD);
       er.setEnabled(true);
   }

The event processing loop polls for EventSet instances which contain one or more Event instances. Not all of these events are down to a breakpoint request though so you have to take care to not always call resume on the event set. This is because you might have two event sets in a row, with the code calling resume before you even get to read the second one. This results in missed breakpoints as the code caught up.

For some reason JDI appeared to be eating the interrupted flag, hence the boolean property to stop the loop with the close method from before.

private void processEvents(VirtualMachine attach) {
       // Listen for events

       EventQueue eq = attach.eventQueue();
       eventLoop: while (!Thread.interrupted() && !shutdown) {

           // Poll for event sets, with a short timeout so that we can
           // be interrupted if required
           EventSet eventSet = null;
           try 
           {
               eventSet = eq.remove(500);
           }
           catch (InterruptedException ex) {
               Thread.currentThread().interrupt();
               continue eventLoop;  
           }

           // Just loop again if we have no events
           if (eventSet == null) {
               continue eventLoop;
           }

           //

           boolean resume = false;
           for (Event event : eventSet) {

               EventRequest request = event.request();
               if (request != null) {
                   int eventPolicy = request.suspendPolicy();
                   resume |= eventPolicy != EventRequest.SUSPEND_NONE;
               }

               if (event instanceof VMDeathEvent || event instanceof VMDisconnectEvent) {
                   // This should never happen as the VM will exit before this is called

               } else if (event instanceof ClassPrepareEvent) {

                   // When an instance of the exception class is loaded attach an exception breakpoint
                   ClassPrepareEvent cpe = (ClassPrepareEvent) event;
                   ReferenceType refType = cpe.referenceType();
                   createExceptionRequest(attach, refType);

               } else if (event instanceof ExceptionEvent) {

                   String name = ((ExceptionRequest)event.request()).exception().name();
                   set.add(name);
               }
           }

           // Dangerous to call resume always because not all event suspend the VM
           // and events happen asynchornously.
           if (resume)
               eventSet.resume();
       }
   }

}

So all that remains is a simple test example, since this is JDK 7 and the ExceptionDebugger is AutoCloseable we can do this using the try-with-resources construct as follows. Obviously if doing automated test use the testing framework fixtures of your choice.

public class Target {

   public static void main(String[] args) throws InterruptedException {


       try (ExceptionDebugger ex = ExceptionDebugger.connect(
               NoClassDefFoundError.class.getName())) {

           doSomeWorkThatQuietlyThrowsAnException();

           System.out.println(ex.getExceptionsViolated());
       }


       System.exit(0);
   }


   private static void doSomeWorkThatQuietlyThrowsAnException() {
       // Check to see that break point gets fired

       try {
           Thread t = new Thread(new Runnable() {
                           public void run() {
                               try
                               {
                                   throw new NoClassDefFoundError();
                               }
                               catch (Throwable ex) {

                               }
                           }
                      });
           t.start();
           t.join();
       } catch (Throwable th) {
           // Eat this and don't tell anybody
       }
   }
}

So if you run this class with the following VM parameter, note the suspend=n otherwise the code won't start running, you will find that it can connect back to itself and start running.

-agentlib:jdwp=transport=dt_socket,address=localhost:5656,server=y,suspend=n

This gives you the following output, note the extra debug line from the VM:

Listening for transport dt_socket at address: 5656
[java.lang.NoClassDefFoundError]
Listening for transport dt_socket at address: 5656

As every I would be interested to read if this was something that is useful for people and to help remove any obvious mistakes.