Monday, November 17, 2014

But that's impossible, or finding out that the JIT has broken your code.

Every now and then you look at some code and think that it cannot be possibly be wrong. Once you have ruled out a simple programmer screw up / enemy action in code (Make sure you read Java Puzzlers or similar) or a concurrency issue (Read Java Concurrency or go on Dr Heniz excellent course) you should sit back and take a few days and then starting thinking about whether the JDK is indeed out to get you. I haven't seen one in the wild in my 18 odd years as a Java programmer so it kinda took me by surprise.

If you are running against JDK 8 in a large scale Swing application you might eventually see the following exception, lots of lots of times. (Unless you forgot the lesson learned in my previous blog in your logging code, in which case you might see lots of ArrayOfOutBoundsException)

Caused by: java.lang.NullPointerException 
    at javax.swing.text.GlyphView.getBreakSpot(GlyphView.java:799) 
    at javax.swing.text.GlyphView.getBreakWeight(GlyphView.java:724) 
    at javax.swing.text.FlowView$LogicalView.getPreferredSpan(FlowView.java:733) 
    at javax.swing.text.FlowView.calculateMinorAxisRequirements(FlowView.java:233) 
    at javax.swing.text.ParagraphView.calculateMinorAxisRequirements(ParagraphView.java:717) 
    at javax.swing.text.BoxView.checkRequests(BoxView.java:935) 
    at javax.swing.text.BoxView.getMinimumSpan(BoxView.java:568) 
    at javax.swing.text.BoxView.calculateMinorAxisRequirements(BoxView.java:903) 
    at javax.swing.text.BoxView.checkRequests(BoxView.java:935) 
    at javax.swing.text.BoxView.setSpanOnAxis(BoxView.java:343) 
    at javax.swing.text.BoxView.layout(BoxView.java:708) 
    at javax.swing.text.BoxView.setSize(BoxView.java:397) 
    ...

This error is particularly insidious because it takes around ten minutes to show itself and sometimes not at all. If you look at the code for this class the line in question, starts with "startsFrom = break", only accesses two local variables both of which have been previously referenced in the method.

            Segment s = getText(pstart, pend); 
            s.first(); 
            BreakIterator breaker = getBreaker(); 
            breaker.setText(s); 

            // Backward search should start from end+1 unless there's NO end+1 
            int startFrom = end + (pend > end ? 1 : 0); 
            for (;;) { 
                startFrom = breaker.preceding(s.offset + (startFrom - pstart))   
                          + (pstart - s.offset); 
                if (startFrom > start) { 
                    // The break spot is within the view 
                    bs[ix++] = startFrom; 
                } else { 
                    break; 
                } 
            } 

The most direct way to rule out a JIT error is to disable compilation for just this single method, here is an example; but you can fine more in the documentation for the command line java tool.

javaThing -XX:CompileCommand=exclude,javax/swing/text/GlyphView,getBreakSpot

When this parameter is added, the problem goes away. - since we have ruled out enemy action by code or a concurrency issue we can be more sure this is a JIT issue. Now as part of the bug logging for this I output diagnostics for this single method, and found out that the problem, didn't occur until the method was JITted for a fifth time.

javaThing -XX:CompileCommand=print,javax/swing/text/GlyphView,getBreakSpot

Here is some diagnostic output seen with the above command:

Compiled method (c2)  914078 33142       4       javax.swing.text.GlyphView::getBreakSpot (247 bytes)
 total in heap  [0x00002aaab0749e10,0x00002aaab0750fe0] = 29136
 relocation     [0x00002aaab0749f38,0x00002aaab074a1e8] = 688
 constants      [0x00002aaab074a200,0x00002aaab074a2a0] = 160
 main code      [0x00002aaab074a2a0,0x00002aaab074cde0] = 11072
 stub code      [0x00002aaab074cde0,0x00002aaab074ce40] = 96
 oops           [0x00002aaab074ce40,0x00002aaab074ce58] = 24
 metadata       [0x00002aaab074ce58,0x00002aaab074d058] = 512
 scopes data    [0x00002aaab074d058,0x00002aaab074ea20] = 6600
 scopes pcs     [0x00002aaab074ea20,0x00002aaab0750c50] = 8752
 dependencies   [0x00002aaab0750c50,0x00002aaab0750c80] = 48
 handler table  [0x00002aaab0750c80,0x00002aaab0750e90] = 528
 nul chk table  [0x00002aaab0750e90,0x00002aaab0750fe0] = 336
OopMapSet contains 113 OopMaps
#0 
OopMap{[8]=Oop [32]=Oop [40]=Oop off=892}
#1 
OopMap{[32]=Oop [40]=Oop off=960}
#2 
OopMap{[32]=Oop [40]=Oop off=980}
#3 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=1048}
#4 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=1084}
#5 
OopMap{[0]=Oop [24]=Oop [48]=Oop [56]=Oop [80]=Oop off=2500}
#6 
OopMap{rbx=Oop rdi=Oop [32]=Oop [40]=Oop [112]=Oop off=2533}
#7 
OopMap{rbx=Oop rdi=Oop r14=Oop [32]=Oop [112]=Oop off=3081}
#8 
OopMap{rbx=Oop rdi=Oop r14=Oop [32]=Oop [40]=Oop [112]=Oop off=3190}
#9 
OopMap{[8]=Oop [32]=Oop [40]=Oop off=4408}
#10 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=4640}
#11 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=5232}
#12 
OopMap{rbp=Oop [0]=NarrowOop [32]=Oop off=5364}
#13 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=5408}
#14 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop off=5436}
#15 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop off=5468}
#16 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop off=5524}
#17 
OopMap{rbp=Oop [32]=Oop [40]=Oop [48]=Oop [88]=Oop off=5552}
#18 
OopMap{[32]=Oop [40]=Oop [48]=Oop [64]=Oop [72]=Derived_oop_[64] [112]=Oop off=5608}
#19 
OopMap{[8]=Oop [32]=Oop off=5680}
#20 
OopMap{rbp=Oop off=5720}
#21 
OopMap{rbp=Oop off=5752}
#22 
OopMap{rbp=Oop [24]=NarrowOop [28]=NarrowOop [32]=Oop [40]=Oop [48]=Oop [56]=Oop [64]=Oop [88]=Oop off=5812}
#23 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop [88]=Oop off=5960}
#24 
OopMap{[0]=Oop [24]=Oop [48]=Oop [56]=Oop [72]=Oop [88]=NarrowOop off=6056}
#25 
OopMap{[40]=Oop off=6088}
#26 
OopMap{[0]=Oop off=6120}
#27 
OopMap{[8]=Oop [24]=Oop [56]=Oop [72]=Oop [112]=Oop off=6216}
#28 
OopMap{[0]=Oop [32]=NarrowOop [40]=Oop off=6284}
#29 
OopMap{rbp=Oop [16]=Oop [40]=Oop [64]=Oop [112]=Oop off=6384}
#30 
OopMap{[0]=Oop off=6412}
#31 
OopMap{[0]=Oop [16]=Oop [32]=NarrowOop [40]=Oop [48]=Oop off=6488}
#32 
OopMap{rbp=Oop [16]=Oop [40]=Oop [48]=Oop off=6560}
#33 
OopMap{[32]=Oop [40]=Oop [48]=Oop [64]=Oop [112]=Oop off=6608}
#34 
OopMap{[8]=Oop [28]=NarrowOop [32]=Oop [40]=Oop [48]=Oop off=6768}
#35 
OopMap{rbp=NarrowOop [0]=Oop [16]=Oop [32]=Oop [40]=NarrowOop off=6860}
#36 
OopMap{[0]=Oop [16]=Oop [32]=NarrowOop [40]=Oop [48]=Oop off=6988}
#37 
OopMap{rbp=Oop [32]=Oop off=7024}
#38 
OopMap{rbp=NarrowOop [0]=Oop [24]=Oop [32]=Oop off=7260}
#39 
OopMap{rbp=NarrowOop [0]=Oop [24]=Oop [32]=Oop off=7344}
#40 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [60]=NarrowOop [64]=Oop off=7452}
#41 
OopMap{rbp=NarrowOop [32]=Oop off=7476}
#42 
OopMap{rbp=NarrowOop [0]=Oop off=7524}
#43 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=7588}
#44 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=7616}
#45 
OopMap{[32]=Oop [40]=Oop [48]=Oop off=7632}
#46 
OopMap{rbp=NarrowOop [32]=Oop off=7676}
#47 
OopMap{rbp=NarrowOop [0]=Oop off=7724}
#48 
OopMap{[0]=Oop [16]=Oop [28]=NarrowOop [40]=Oop [48]=Oop [56]=NarrowOop [64]=Oop off=7868}
#49 
OopMap{[8]=Oop [28]=NarrowOop [32]=Oop [40]=Oop [48]=Oop [56]=Oop off=7916}
#50 
OopMap{rbp=Oop [16]=Oop [24]=NarrowOop off=8016}
#51 
OopMap{rbp=Oop [16]=Oop [28]=NarrowOop off=8080}
#52 
OopMap{rbp=NarrowOop [0]=Oop [24]=Oop [32]=Oop off=8152}
#53 
OopMap{rbp=Oop [8]=NarrowOop off=8212}
#54 
OopMap{rbp=NarrowOop [32]=Oop off=8236}
#55 
OopMap{rbp=Oop [16]=NarrowOop off=8272}
#56 
OopMap{rbp=NarrowOop [0]=Oop off=8320}
#57 
OopMap{rbp=Oop [12]=NarrowOop off=8360}
#58 
OopMap{rbp=NarrowOop [32]=Oop off=8400}
#59 
OopMap{rbp=Oop [12]=NarrowOop off=8460}
#60 
OopMap{rbp=NarrowOop [0]=Oop off=8508}
#61 
OopMap{rbp=Oop [24]=NarrowOop [40]=Oop off=8572}
#62 
OopMap{rbp=Oop off=8600}
#63 
OopMap{rbp=Oop [8]=Oop [28]=NarrowOop off=8640}
#64 
OopMap{rbp=Oop [8]=Oop [20]=NarrowOop [112]=Oop off=8704}
#65 
OopMap{rbp=Oop [16]=Oop [24]=Oop [48]=Oop off=8788}
#66 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=8912}
#67 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9036}
#68 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9160}
#69 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9284}
#70 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9408}
#71 
OopMap{rbp=Oop [16]=Oop [24]=Oop [40]=Oop [64]=Oop off=9532}
#72 
OopMap{off=9556}
#73 
OopMap{off=9580}
#74 
OopMap{off=9604}
#75 
OopMap{[112]=Oop off=9628}
#76 
OopMap{rbp=Oop [8]=Oop [24]=Oop [32]=NarrowOop off=9696}
#77 
OopMap{rbp=Oop [8]=Oop [24]=NarrowOop off=9760}
#78 
OopMap{off=9784}
#79 
OopMap{off=9812}
#80 
OopMap{off=9836}
#81 
OopMap{off=9860}
#82 
OopMap{off=9884}
#83 
OopMap{off=9908}
#84 
OopMap{off=9932}
#85 
OopMap{off=9956}
#86 
OopMap{off=9980}
#87 
OopMap{off=10004}
#88 
OopMap{off=10028}
#89 
OopMap{rbp=Oop [16]=Oop [28]=NarrowOop off=10092}
#90 
OopMap{rbp=Oop [16]=Oop [24]=Oop [48]=Oop off=10176}
#91 
OopMap{off=10200}
#92 
OopMap{off=10224}
#93 
OopMap{off=10248}
#94 
OopMap{off=10272}
#95 
OopMap{off=10296}
#96 
OopMap{off=10320}
#97 
OopMap{off=10344}
#98 
OopMap{off=10368}
#99 
OopMap{off=10392}
#100 
OopMap{off=10416}
#101 
OopMap{off=10440}
#102 
OopMap{off=10464}
#103 
OopMap{off=10488}
#104 
OopMap{off=10512}
#105 
OopMap{off=10536}
#106 
OopMap{off=10560}
#107 
OopMap{off=10584}
#108 
OopMap{off=10608}
#109 
OopMap{off=10632}
#110 
OopMap{off=10656}
#111 
OopMap{off=10680}
#112 
OopMap{off=11028}
java.lang.NullPointerException
 at javax.swing.text.GlyphView.getBreakSpot(GlyphView.java:799)
 at javax.swing.text.GlyphView.getBreakWeight(GlyphView.java:724)
 at javax.swing.text.html.InlineView.getBreakWeight(InlineView.java:150)
 at javax.swing.text.FlowView$LogicalView.getPreferredSpan(FlowView.java:733)
 at javax.swing.text.FlowView.calculateMinorAxisRequirements(FlowView.java:233)
 at javax.swing.text.ParagraphView.calculateMinorAxisRequirements(ParagraphView.java:717)
 at javax.swing.text.html.ParagraphView.calculateMinorAxisRequirements(ParagraphView.java:157)
 at javax.swing.text.BoxView.checkRequests(BoxView.java:935)
 at javax.swing.text.BoxView.getMinimumSpan(BoxView.java:568)
 at javax.swing.text.html.ParagraphView.getMinimumSpan(ParagraphView.java:270)
 at javax.swing.text.BoxView.calculateMinorAxisRequirements(BoxView.java:903)

Now I am still working with the JDK team for a fix for this one; but I do feel I have discovered a useful set of tools for providing some evidence that the JIT compiler is causing my bad day. And more importantly I have a workaround so I can run my tests until this is resolved.

Thursday, November 6, 2014

Missing stack traces for repeated exceptions (reminder)

A long while ago a optimisation was added to the JVM so that if the same exception is thrown again and again and again a single instance of the Exception is created without the stack trace filled in in order to increase performance. This is an excellent idea unless you are trying to diagnose a problem and you have missed the original error.

If you forgot about this optimisation you send the afternoon looking at the following log output and weeping slightly. (In my defence I have a little one in the house hence the fuzzy brain and lack of blogging action this year)

java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
...
java.lang.Exception: Uncaught exceptions during test
 at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/common-adf/CreateNewCustomAppAndProjectWithName.xml:25)
 at oracle.jdevstudio-testware-tests.level0.testADFcMATS(/.../work/mw9111/jdeveloper/jdev/extensions/oracle.jdevstudio-testware-tests/abbot/level0/testADFcMATS.xml:94)
 at oracle.abbot.JDevScriptFixture.runTest(JDevScriptFixture.java:555)
 at junit.framework.TestCase.runBare(TestCase.java:134)
 at junit.framework.TestResult$1.protect(TestResult.java:110)
 at junit.framework.TestResult.runProtected(TestResult.java:128)
 at junit.framework.TestResult.run(TestResult.java:113)
 at junit.framework.TestCase.run(TestCase.java:124)
 at junit.framework.TestSuite.runTest(TestSuite.java:243)
 at junit.framework.TestSuite.run(TestSuite.java:238)
 at junit.textui.TestRunner.doRun(TestRunner.java:116)
 at junit.textui.TestRunner.doRun(TestRunner.java:109)
 at oracle.abbot.AbbotRunner.run(AbbotRunner.java:614)
 at oracle.abbot.AbbotAddin$IdeAbbotRunner.run(AbbotAddin.java:634)
 at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArrayIndexOutOfBoundsException

[Crickets]

It turns out that you can turn off this optimisation with a simple flag:

java -XX:-OmitStackTraceInFastThrow ....

In my particular case the actual exception causing this trouble was a NPE from the GlyphView code in JDK8. (One that is being caused by a glitch in hotspot it seems) But that in turn was causing the AIOOBE in some logging code clouding the issue even more.

This in particular is a good flag to add by default when running your automated tests, particularly in combination with the stack trace length override I have talked about before.

Wednesday, May 21, 2014

Reading and writing JAX-RS Link objects

JAX-RS contains a rather nice handy representation of the a Link that can be serialised with and adapter into XML and JSON, unfortunately there is a bug in the JAX-RS spec that means that the standard adapter provided is missing the setters required to deserialise the Link later on.

This isn't a problem if you are using the JAX-B RI as it appears to be more relaxed than the standard; but it will be a problem for other implementations. There is a further bug if you are using MOXy, aka EclipseLink, to produce either JSON or XML that it will fail and just call toString() because it doesn't like the type adapter being an inner class of the class that is being adapted. (Bug TBC)


Direction JAXB RI + JAX-RS Adapter MOXy + JAX-RS Adapter MOXy + Outer Adapter MOXy + Outer Adapter + Setters MOXy + Inner Adapter (Different class)
Marshalling Yes No Yes Yes Yes
Unmarshalling Yes No No Yes No


The workaround is to create a new adapter as a top level class in order to replace the one provided by the standard. Just quickly it would look something like this:

import java.util.HashMap;
import java.util.Map;

import javax.ws.rs.core.Link;

import javax.xml.bind.annotation.adapters.XmlAdapter;
import javax.xml.namespace.QName;

public class LinkAdapter
    extends XmlAdapter {

    public LinkAdapter() {
    }

    public Link unmarshal(LinkJaxb p1) {
        
        Link.Builder builder = Link.fromUri(p1.getUri());
        for (Map.Entry<QName, Object> entry : p1.getParams().entrySet()) {
            builder.param(entry.getKey().getLocalPart(), entry.getValue().toString());
        }
        return builder.build();
    }

    public LinkJaxb marshal(Link p1) {
        
        Map<QName, Object> params = new HashMap<>();
        for (Map.Entry<String,String> entry : p1.getParams().entrySet()) {
            params.put(new QName("", entry.getKey()), entry.getValue());
        }
        return new LinkJaxb(p1.getUri(), params);
    }
}

With a simple POJO to go with it.

import java.net.URI;

import java.util.HashMap;
import java.util.Map;

import javax.xml.bind.annotation.XmlAnyAttribute;
import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.namespace.QName;

public class LinkJaxb  {

    private URI uri;
    private Map<QName, Object> params;


    public LinkJaxb() {
        this (null, null);
    }

    public LinkJaxb(URI uri) {
        this(uri, null);
    }

    public LinkJaxb(URI uri, Map<QName, Object> map) {
        
        this.uri = uri;
        this.params = map!=null ? map : new HashMap<QName, Object>();
        
    }



    @XmlAttribute(name = "href")
    public URI getUri() { 
        return uri;
    }

    @XmlAnyAttribute
    public Map<QName, Object> getParams() { 
        return params;
    }

    public void setUri(URI uri) {
        this.uri = uri;
    }

    public void setParams(Map<QName, Object> params) {
        this.params = params;
    }

}


With this you can read and write the Link objects to and from XML, and JSON with MOXy, to your hearts content.

Tuesday, May 20, 2014

Declarative Linking in Jersey 2.9 and up

A couple of weeks ago A couple of months ago I was looking how I was going to engineers new REST API for an Oracle Cloud project. Once of the things I had planned to do was to use the declarative link injection created in Jersey 1.x by Marc Hadley. Sadly this hadn't been forwarded ported yet, so a quick chat to the project lead and I took on the small medium sized job of bringing the code up to date.

One of the things that has changed in the new version is that in JAX-RS 2.0 there is a Link object so rather than being able to only inject String and URI you can also inject the correct rel attributes. This has means that the existing annotations coded by Marc have been merged into once simple set of annotations for both Link headers and for injected properties.

This functionality is available now, along with a simple example. The original version of the feature that I committed has some serious limitations that are described later, you will need a version of Jersey post 2.8 or you can build a 2.9-SNAPSHOT image that contains my changes currently to implement the example in this blog.

This blog looks at using this new API to provide simple injection for a collections API. One of the common patterns in RESTful services, in particular those based on JSON, is to have an array of structural links at the top level of the structure. For the purposes of this blog I am going to follow the form of the Collection+JSON hypermedia type.

{ "collection" :
  {
    "version" : "1.0",
    "href" : "http://example.org/friends/?offset=10&limit=10",
    
    "links" : [
      {"rel" : "create", "href" : "http://example.org/friends/"}
      {"rel" : "next", "href" : "http://example.org/friends/?offset=20&limit=10"}
      {"rel" : "previous", "href" : "http://example.org/friends/?offset=0&limit=10"}
    ],
   
    "items" : [
       ...
    ]
  }
}

So I can inject the links in the following form, not there is a bunch of boiler plate missing here for clarity. This is not the tidiest code; but in a later cycle it should be possible to simply them somewhat. The design currently uses EL to access properties - this has the advantage of making it possible to write back values as you can represent properties. I can understand it is disliked by some; but I am not sure if I see any value in moving to JavaScript at the moment. Also don't be put of by the @Xml annotations, I am using MOXy for JSON generation - this isn't an XML only thing.

{


  @XmlTransient
  private int limit, offset; // Getters for these

  @XmlTransient
  private int modelLimit; // Getters for these


  @InjectLink(
            resource = ItemsResource.class,
            method = "query",
            style = Style.ABSOLUTE,
            bindings = {@Binding(name = "offset", value="${instance.offset}"),
                @Binding(name = "limit", value="${instance.limit}")
            },
            rel = "self"
  )
  @XmlElement(name="link")
  private String href;

  @InjectLinks({
    @InjectLink(
          resource = ItemsResource.class,
          style = Style.ABSOLUTE,
          method = "query",
          condition = "${instance.offset + instance.limit < instance.modelLimit}",
          bindings = {
            @Binding(name = "offset", value = "${instance.offset + instance.limit}"),
            @Binding(name = "limit", value = "${instance.limit}")
          },
          rel = "next"
    ),
    @InjectLink(
          resource = ItemsResource.class,
          style = Style.ABSOLUTE,
          method = "query",
          condition = "${instance.offset - instance.limit >= 0}",
          bindings = {
            @Binding(name = "offset", value = "${instance.offset - instance.limit}"),
            @Binding(name = "limit", value = "${instance.limit}")
          },
          rel = "prev"
  )})
  @XmlElement(name="link")
  @XmlElementWrapper(name = "links")
  @XmlJavaTypeAdapter(Link.JaxbAdapter.class)
  List<Link> links;

  ....
}


The original porting of the declarative linking code that exists in version Jersey before 2.8 had very naive code with regards working out what the URI should be for a particular resource, it couldn't deal with any resources that were not at the root of the application, nor would it cope with query parameters which are so important when dealing with collections.

In theory there could be more than one URI for a particular resource class; but this code does need to assume a 1:1 mapping, the current implementation contains a simple algorithm that walks the Jersey meta-model to try to work out the structure, is this doesn't work in your can you can simple provide another implementation of ResourceMappingContext.

Some may question why should I use these ugly annotations when it might be easier just to inject the URI myself? Well the reason is to provide metadata that other tools can use. One of my next jobs is to extend this work to generate the hypermedia extensions and for this I need the above metadata. (Waiting on a pull request being approved before I can really get into it).

Finally it is worth noting that the paging model has its own problems which become apparent if you think of a REST collection as some kind of array that you can safely page over. Concurrent updates along with the lack of state mean that the client can never be sure they have the complete model and should expect to see some items more than once as the model is updated. Cursor or linking based schemes should be considered instead which is yet another good reminder as to why you should always treat the URI as opaque - the sever might need to changes its structure in the future. But that is a entirely different blog for another day.....

Update: See the following blog for some workarounds when reading and writing Link objects.

Tuesday, April 15, 2014

Quick, and a bit dirty, JSON Schema generation with MOXy 2.5.1

So I am working on a new REST API for an upcoming Oracle cloud service these days so one of the things I needed was the ability to automatically generate a JSON Schema for the bean in my model. I am using MOXy to generate the JSON from POJO and as of version 2.5.1 of EclipseLink it now has the ability to generate a JSON Schema from the bean model.

There will be a more formal solution integrated into Jersey 2.x at a future date; but this solution will do at the moment if you want to play around with this.

So the first class we need to put in place is a model processor, very much and internal Jersey class, that allows us to amend the resource model with extra methods and resources. To each resource in the model we can add the JsonSchemaHandler which does the hard work of generating a new schema. Since this is a simple POC there is no caching going on here, please be aware of this if you are going to use this in production code.

import com.google.common.collect.Lists;

import example.Bean;

import java.io.IOException;
import java.io.StringWriter;

import java.text.SimpleDateFormat;

import java.util.Date;
import java.util.List;

import javax.inject.Inject;

import javax.ws.rs.HttpMethod;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.core.Configuration;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

import javax.xml.bind.JAXBException;
import javax.xml.bind.SchemaOutputResolver;
import javax.xml.transform.Result;
import javax.xml.transform.stream.StreamResult;

import org.eclipse.persistence.jaxb.JAXBContext;

import org.glassfish.jersey.process.Inflector;
import org.glassfish.jersey.server.ExtendedUriInfo;
import org.glassfish.jersey.server.model.ModelProcessor;
import org.glassfish.jersey.server.model.ResourceMethod;
import org.glassfish.jersey.server.model.ResourceModel;
import org.glassfish.jersey.server.model.RuntimeResource;
import org.glassfish.jersey.server.model.internal.ModelProcessorUtil;
import org.glassfish.jersey.server.wadl.internal.WadlResource;

public class JsonSchemaModelProcessor implements ModelProcessor {

  private static final MediaType JSON_SCHEMA_TYPE = 
    MediaType.valueOf("application/schema+json");
  private final List<ModelProcessorUtil.Method> methodList;


  public JsonSchemaModelProcessor() {
    methodList = Lists.newArrayList();
    methodList.add(new ModelProcessorUtil.Method("$schema", HttpMethod.GET, 
      MediaType.WILDCARD_TYPE, JSON_SCHEMA_TYPE,
      JsonSchemaHandler.class));
  }

  @Override
  public ResourceModel processResourceModel(ResourceModel resourceModel, 
      Configuration configuration) {
    return ModelProcessorUtil.enhanceResourceModel(resourceModel, true, methodList, 
      true).build();
  }

  @Override
  public ResourceModel processSubResource(ResourceModel resourceModel, 
      Configuration configuration) {
    return ModelProcessorUtil.enhanceResourceModel(resourceModel, true, methodList, 
      true).build();
  }


  public static class JsonSchemaHandler 
    implements Inflector<ContainerRequestContext, Response> {

    private final String lastModified = new SimpleDateFormat(WadlResource.HTTPDATEFORMAT).format(new Date());

    @Inject
    private ExtendedUriInfo extendedUriInfo;

    @Override
    public Response apply(ContainerRequestContext containerRequestContext) {

      // Find the resource that we are decorating, then work out the
      // return type on the first GET

      List<RuntimeResource> ms = extendedUriInfo.getMatchedRuntimeResources();
      List<ResourceMethod> rms = ms.get(1).getResourceMethods();
      Class responseType = null;
      found:
      for (ResourceMethod rm : rms) {
        if ("GET".equals(rm.getHttpMethod())) {
          responseType = (Class) rm.getInvocable().getResponseType();
          break found;
        }
      }

      if (responseType == null) {
        throw new WebApplicationException("Cannot resolve type for schema generation");
      }

      //
      try {
        JAXBContext context = (JAXBContext) JAXBContext.newInstance(responseType);

        StringWriter sw = new StringWriter();
        final StreamResult sr = new StreamResult(sw);

        context.generateJsonSchema(new SchemaOutputResolver() {
          @Override
          public Result createOutput(String namespaceUri, String suggestedFileName) 
              throws IOException {
            return sr;
          }
        }, responseType);


        return Response.ok().type(JSON_SCHEMA_TYPE)
          .header("Last-modified", lastModified)
          .entity(sw.toString()).build();
      } catch (JAXBException jaxb) {
        throw new WebApplicationException(jaxb);
      }
    }
  }


}

Note the very simple heuristic in the JsonSchemaHandler code it assumes that for each resource there is a 1:1 mapping to a single JSON Schema element. This of course might not be true for your particular application.

Now that we have the schema generated in a know location we need to tell the client about it, the first thing we will do is to make sure that there is a suitable link header when the user invokes OPTIONS on a particular resource:

import java.io.IOException;

import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerResponseContext;
import javax.ws.rs.container.ContainerResponseFilter;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.Link;
import javax.ws.rs.core.UriInfo;

public class JsonSchemaResponseFilter implements ContainerResponseFilter {

  @Context
  private UriInfo uriInfo;

  @Override
  public void filter(ContainerRequestContext containerRequestContext,
                     ContainerResponseContext containerResponseContext) throws IOException {


    String method = containerRequestContext.getMethod();
    if ("OPTIONS".equals(method)) {

      Link schemaUriLink =
        Link.fromUriBuilder(uriInfo.getRequestUriBuilder()
          .path("$schema")).rel("describedBy").build();

      containerResponseContext.getHeaders().add("Link", schemaUriLink);
    }
  }
}

Since this is JAX-RS 2.x we are working with we of course are going bundle all the bit together into a feature:

import javax.ws.rs.core.Feature;
import javax.ws.rs.core.FeatureContext;

public class JsonSchemaFeature implements Feature {

  @Override
  public boolean configure(FeatureContext featureContext) {

    if (!featureContext.getConfiguration().isRegistered(JsonSchemaModelProcessor.class)) {
      featureContext.register(JsonSchemaModelProcessor.class);
      featureContext.register(JsonSchemaResponseFilter.class);
      return true;
    }
    return false;
  }
}

I am not going to show my entire set of POJO classes; but just quickly this is the Resource class with the @GET method required by the schema generation code:

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

@Path("/bean")
public class BeanResource {
    
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public Bean getBean() {
        return new Bean();
    }
}

And finally here is what you see if you perform a GET on a resource:

GET .../resources/bean
Content-Type: application/json

{
  "message" : "hello",
  "other" : {
    "message" : "OtherBean"
  },
  "strings" : [
    "one",
    "two",
    "three",
    "four"
  ]
}

And OPTIONS:

OPTIONS .../resources/bean
Content-Type: text/plain
Link: <http://.../resources/bean/$schema>; rel="describedBy"

GET, OPTIONS, HEAD

And finally if you resolve the schema resource:

GET .../resources/bean/$schema
Content-Type: application/schema+json

{
  "$schema" : "http://json-schema.org/draft-04/schema#",
  "title" : "example.Bean",
  "type" : "object",
  "properties" : {
    "message" : {
      "type" : "string"
    },
    "other" : {
      "$ref" : "#/definitions/OtherBean"
    },
    "strings" : {
      "type" : "array",
      "items" : {
        "type" : "string"
      }
    }
  },
  "additionalProperties" : false,
  "definitions" : {
    "OtherBean" : {
      "type" : "object",
      "properties" : {
        "message" : {
          "type" : "string"
        }
      },
      "additionalProperties" : false
    }
  }
}

There is a quite a bit of work to do here, in particular generating the hypermedia extensions based on the declarative linking annotations that I forward ported into Jersey 2.x a little while back. But it does point towards a solution and we get to exercise a variety of solutions to get something working now.

Friday, February 7, 2014

Transparent PATCH support in JAX-RS 2.0

The PATCH method is one the the less well loved HTTP methods simple because until recently there really wasn't a standard PATCH format. This has been standardized for JSON for a while now so there are quite a few libraries that will do the heavy lifting for you. For the purposes of this blog I am going to use json-patch although it would be easy to adapt this particular implementation to the patch library of your choice.

A per normal lets get the resource and bean classes out of the way. In this example code we have a simple resource that knows how to return the original object and one that allows you to perform the PATCH method. Note that the patch method just accepts the bean object, this is because of some magic we are going to do in a little bit to pre-process the patch.

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

@Path("service")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class Service {

  @GET
  public Bean get() {
    return new Bean(true);
  }

  @PATCH
  @Consumes("application/json-patch+json")
  public Bean patch(Bean input) {
    System.out.println(input.getMessage() + "  " + input.getTitle());
    return input;
  }

}


import java.util.ArrayList;
import java.util.List;

public class Bean {

  private String title = "title";
  private String message = "message";
  private List<String> list = new ArrayList<String>();

  public Bean() {
    this(false);
  }

  public Bean(boolean init) {
    if (init) {
      title = "title";
      message = "message";
      list.add("one");
      list.add("two");
    }
  }


  public void setList(List list) {
    this.list = list;
  }

  public List getList() {
    return list;
  }

  public void setTitle(String title) {
    this.title = title;
  }

  public String getTitle() {
    return title;
  }

  public void setMessage(String message) {
    this.message = message;
  }

  public String getMessage() {
    return message;
  }

}

So the @PATCH annotation is something we have to create for this example, luckily JAX-RS contains a extension meta-annotation for this purpose. We are also going to use @NameBinding as this example is using JAX-RS 2.0 so we can connect up our filter in a moment.

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import javax.ws.rs.HttpMethod;
import javax.ws.rs.NameBinding;


@Target({ ElementType.METHOD, ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
@HttpMethod("PATCH")
@Documented
@NameBinding
public @interface PATCH {
}

So here is the implementation of the ReaderInterceptor that will process the incoming stream and replace it with the patched version. Note that the class is annotated with @PATCH also in order to make the @NamedBinding magic work and also that there is a lot of error handling that is missing as this is a simple POC.

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

import com.github.fge.jsonpatch.JsonPatch;
import com.github.fge.jsonpatch.JsonPatchException;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;

import java.lang.annotation.Annotation;
import java.lang.reflect.Method;

import javax.ws.rs.GET;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedHashMap;
import javax.ws.rs.core.UriInfo;
import javax.ws.rs.ext.MessageBodyWriter;
import javax.ws.rs.ext.Provider;
import javax.ws.rs.ext.ReaderInterceptor;
import javax.ws.rs.ext.ReaderInterceptorContext;

import org.glassfish.jersey.message.MessageBodyWorkers;

@Provider
@PATCH
public class PatchReader implements ReaderInterceptor {
  private UriInfo info;
  private MessageBodyWorkers workers;

  @Context
  public void setInfo(UriInfo info) {
    this.info = info;
  }

  @Context
  public void setWorkers(MessageBodyWorkers workers) {
    this.workers = workers;
  }

  @Override
  public Object aroundReadFrom(
    ReaderInterceptorContext readerInterceptorContext) 
    throws IOException,
           WebApplicationException {

    // Get the resource we are being called on, 
    // and find the GET method
    Object resource = info.getMatchedResources().get(0);

    Method found = null;
    for (Method next : resource.getClass().getMethods()) {
      if (next.getAnnotation(GET.class) != null) {
        found = next;
        break;
      }
    }


    if (found != null) {

      // Invoke the get method to get the state we are trying to patch
      //
      Object bean;
      try {
        bean = found.invoke(resource);
      } catch (Exception e) {
        throw new WebApplicationException(e);
      }
      
      // Convert this object to a an aray of bytes 
      ByteArrayOutputStream baos = new ByteArrayOutputStream();
      MessageBodyWriter<? super Object> bodyWriter =
        workers.getMessageBodyWriter(Object.class, bean.getClass(), 
          new Annotation[0], MediaType.APPLICATION_JSON_TYPE);

      bodyWriter.writeTo(bean, bean.getClass(), bean.getClass(), 
          new Annotation[0], MediaType.APPLICATION_JSON_TYPE,
          new MultivaluedHashMap<String, Object>(), baos);


      // Use the Jackson 2.x classes to convert both the incoming patch  
      // and the current state of the object into a JsonNode / JsonPatch
      ObjectMapper mapper = new ObjectMapper();
      JsonNode serverState = mapper.readValue(baos.toByteArray(), 
        JsonNode.class);
      JsonNode patchAsNode = mapper.readValue(
         readerInterceptorContext.getInputStream(), 
        JsonNode.class);
      JsonPatch patch = JsonPatch.fromJson(patchAsNode);

      try {
        // Apply the patch
        JsonNode result = patch.apply(serverState);

        // Stream the result & modify the stream on the readerInterceptor
        ByteArrayOutputStream resultAsByteArray = 
          new ByteArrayOutputStream();
        mapper.writeValue(resultAsByteArray, result);
        readerInterceptorContext.setInputStream(
          new ByteArrayInputStream(
            resultAsByteArray.toByteArray()));

        // Pass control back to the Jersey code
        return readerInterceptorContext.proceed();


      } catch (JsonPatchException e) {
        throw new WebApplicationException(
          Response.status(500).type("text/plain").entity(e.getMessage()).build());
      }

    } else {
      throw new IllegalArgumentException("No matching GET method on resource");
    }


  }
}

So once you have this deployed you can start playing with the data, so the original message is:

{
  "list" : [
    "one",
    "two"
  ],
  "message" : "message",
  "title" : "title"
}

So if you apply the following patch, the result returned is:

[
  {
    "op" : "replace",
    "path" : "/message",
    "value" : "otherMessage"
  },
  {
    "op" : "add",
    "path" : "/list/-",
    "value" : "three"
  }
]


{
  "list" : [
    "one",
    "two",
    "three"
  ],
  "message" : "otherMessage",
  "title" : "title"
}

This example shows it is relatively trivial to add PATCH support to your classes by following a simple coding pattern and using a simple Annotation. In this way PATCH support becomes trivial as the implementation can just delegate to your existing PUT method.

Update: Mirsolav Fuksa from the Jersey team reminded me that in order for this implementation to comply with the PATCH RFC it should provide the Accept-Patch header when the client performs an OPTIONS request. You can do this with a simple CotnainerResponseFilter:

import java.io.IOException;

import java.util.Collections;

import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerResponseContext;
import javax.ws.rs.container.ContainerResponseFilter;
import javax.ws.rs.ext.Provider;

@Provider
public class OptionsAcceptHeader implements ContainerResponseFilter {

  @Override
  public void filter(ContainerRequestContext requestContext,
                     ContainerResponseContext responseContext) throws IOException {

    if ("OPTIONS".equals(requestContext.getMethod())) {
      if (responseContext.getHeaderString("Accept-Patch")==null) {
        responseContext.getHeaders().put(
          "Accept-Patch", Collections.<Object>singletonList("application/json-patch+json"));  
      }
    }
  }
}

Thursday, February 6, 2014

wadl2java 1.6 released

Just an incremental release with a cluster of bug fixes:

  • Upgrade to jsonschema2pojo 0.4.0
  • Fixed bugs where generation would fail due to duplicated class names caused by similar resource names
  • Upgrade Jersey 2-ALPHA deps to 2.5.1 some small changes in the APIs
  • Fix for XSL for correct return on onClick hander, allows better integration with IDE's
  • Interface of MethodNode.getSupportedOutput to support latest WADL model
  • Method parameter were not being shown consistently up the tree when using the XSLT
  • Reserved words were not being escaped when generating the client
  • Non-standard lines in the XSL causing failure on other non oracle transformers
  • WADL XSL now generates expandable / collapsable sections
Most of the work in this release was done by Michael Bachand who recently left Oracle, thanks very much for all of this efforts in this. (And of course Pavel for doing the release for me)

Wednesday, February 5, 2014

Using JSON-P, aka JSR 353 with Jersey 1.x, or indeed any JAX-RS 1.x

I have been playing with JSON-P a little bit recently and one of the thing I wanted to try as making it work with Jersey 1.x. (Jersey 2.x has this functionality built in).

I had hoped to use the org.glassfish:jsonp-jaxp component that comes from the json-p project; but it depends on the JAX-RS 2.0 API. With a bit of friendly badgering of the ever helpful Pavel Bucek, a version was published that would work with JAX-RS 1.x implementations.

If you are a maven user this would be as simple as adding the following dependencies:

<dependency>
      <groupId>org.glassfish</groupId>
      <artifactId>javax.json</artifactId>
      <version>1.0.4</version>
    </dependency>
    <dependency>
      <groupId>org.glassfish</groupId>
      <artifactId>jsonp-jaxrs-1x</artifactId>
      <version>1.0</version>
    </dependency>

Otherwise you can easily just download the two files from here and here and add them to your project.

Then you can consume and produce JsonStructure (JsonObject and JsonArray) elements to your hearts content, using the helper methods from the previously mentioned post:

@Path("/hello")
public class Hello {

  @GET
  @Produces(MediaType.APPLICATION_JSON)
  public JsonStructure hello() {

    return JsonObjectBuilder builder =
      ob()
       .add("hello", "world")
       .add("fark", ".com")
       .add("fish", ob()
         .add("child", "child"))
       .add("array", ab()
         .add("one")
         .add("two")
         .add(ob()
           .add("boolean",
           true))).build();
  }


}



Thursday, January 30, 2014

Making your JSON-P (JSR-353) code slightly prettier

The JSON-P API as described in JSR-353 is a limited API for working with JSON; but at the basic level it will do the job. It is possible to add a few utility methods that can make your code, in my eyes, prettier.

The first think that annoyed me was the use of Json.createObjectBuilder() and Json.createArrayBuilder() when trying to construct a JSONObject.

So lets create a nice helper class with some very short method names for both:

public static JsonObjectBuilder ob() {
  return Json.createObjectBuilder();
}

public static JsonArrayBuilder ab() {
  return Json.createArrayBuilder();
}

This makes the creation of object just that little bit less wordy:

JsonObject object = ob()
  .add("hello", "world")
  .add("fark", ".com")
  .add("fish", ob()
    .add("child", "child"))
  .add("array", ab()
    .add("one")
    .add("two")
    .add(ob()
      .add("boolean",true)))
  .build();

The second problem is accessing element in a JsonObject can be wordy; but it is relatively easy to knock up a method that would allow some simply XPath like accessor:

public static <T extends JsonValue> T get(JsonStructure structure, 
  String path, Class<T> type) {

  String segments[] = path.split("/");
  JsonValue currentValue = structure;
  for (String segment : segments) {

    if (segment.length() == 0) {
      continue;
    }

    if (currentValue instanceof JsonObject) {
      JsonObject currentObject = (JsonObject) currentValue;
      currentValue = currentObject.get(segment);
    } else if (currentValue instanceof JsonArray) {
      if (segment.startsWith("[") && segment.endsWith("]")) {
        int index = Integer.parseInt(segment.substring(1, segment.length() - 1));
        currentValue = ((JsonArray) currentValue).get(index);
      } else {
        throw new IllegalArgumentException("Array type requires key of the form [n]");
      }
    } else {
      throw new IllegalStateException("Value types are not decomposible" 
        + currentValue.getValueType());
    }

  }

  return type.cast(currentValue);

}


// Example to get hold of a string value

System.out.println(get(object, "/fish/child", JsonString.class));


Of course even this is a little bit untidy as you have to care about the internal JsonValue types which is a pain; and deal with possibly null pointers without the aid of Optional. It doesn't take much to wrap these up though.

public static String getString(JsonStructure structure, String path) {
  JsonString value = get(structure, path, JsonString.class);
  return value != null ? value.getString() : null;
}

public static boolean is(JsonStructure structure, String path) {
  JsonValue value = get(structure, path, JsonValue.class);
  return value != null ? value == JsonValue.TRUE : false;
}

public static boolean isNull(JsonStructure structure, String path) {
  JsonValue value = get(structure, path, JsonValue.class);
  return value != null ? value == JsonValue.NULL : false;
}

public static BigInteger getBigDecimal(JsonStructure structure, String path) {
  JsonNumber value = get(structure, path, JsonNumber.class);
  return value != null ? value.bigIntegerValue() : null;
}

public static int getInt(JsonStructure structure, String path) {
  JsonNumber value = get(structure, path, JsonNumber.class);
  return value != null ? value.intValue() : null;
}

public static JsonArray getArray(JsonStructure structure, String path) {
  return get(structure, path, JsonArray.class);
}

public static JsonObject getObject(JsonStructure structure, String path) {
  return get(structure, path, JsonObject.class);
}

This means you can write more direct code such as:

if (is(object, "/array/[2]/boolean")) {
  System.out.println(getString(object, "/fish/child"));
}

Sometimes it only take a few statically imported methods to make a API more useful / easier to read.

Wednesday, January 29, 2014

Selecting level of detail returned by varying the content type, part II

In my previous entry, we looked at using the feature of MOXy to control the level of data output for a particular entity. This post looks at an abstraction provided by Jersey 2.x that allows you to define a custom set of annotations to have the same effect.

As before we have an almost trivial resource that returns an object that Jersey will covert to JSON for us, note that for the moment there is nothing in this code to do the filtering - I am not going to pass in annotations to the Response object as in the Jersey examples:

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

@Path("hello")
public class SelectableHello {


  @GET
  @Produces({ "application/json; level=detailed", "application/json; level=summary", "application/json; level=normal" })
  public Message hello() {

    return new Message();

  }
}

In my design I am going to define four annotations: NoView, SummaryView, NormalView and DetailedView. All root objects have to implement the NoView annotation to prevent un-annotated fields from being exposed - you might not feel this is necessary in your design. All of these classes look the same so I am going to only show one. Note that the factory method creating a AnnotationLiteral has to be used in preference to a factory that would create a dynamic proxy to have the same effect. There is code in 2.5 that will ignore any annotation implemented by a java.lang.reflect.Proxy object, this includes any annotations you may have retrieved from a class. I am working on submitting a fix for this.

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import javax.enterprise.util.AnnotationLiteral;

import org.glassfish.jersey.message.filtering.EntityFiltering;

@Target({ ElementType.TYPE, 
  ElementType.METHOD, ElementType.FIELD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@EntityFiltering
public @interface NoView {

  /**
   * Factory class for creating instances of the annotation.
   */
  public static class Factory extends AnnotationLiteral<NoView> 
    implements NoView {

    private Factory() {
    }

    public static NoView get() {
      return new Factory();
    }
  }

}

Now we can take a quick look at our Message bean, this is slightly more complicated than my previous example to showing filtering of subgraphs in a very simple form. As I said before the class is annotated with a NoView annotation at the root - this should mean that the privateData is never returned to the client as it is not specifically annotated.

import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
@NoView
public class Message {


  private String privateData;
  @SummaryView
  private String summary;
  @NormalView
  private String message;
  @DetailedView
  private String subtext;
  @DetailedView
  private SubMessage submessage;

  public Message() {
    summary = "Some simple summary";
    message = "This is indeed the message";
    subtext = "This is the deep and meaningful subtext";
    submessage = new SubMessage();
    privateData = "The fox is flying tonight";
  }


  // Getters and setters not shown
}


public class SubMessage {

  private String message;

  public SubMessage() {
    message = "Some sub messages";
  }

  // Getters and setters not shown
}


As noted before there is no code in the resource class to deal with filtering - I considered this to be a cross cutting concern so I have abstracted this into a WriterInterceptor. Note the exception thrown if a entity is used that doesn't have the NoView annotation on it.

import java.io.IOException;

import java.lang.annotation.Annotation;

import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.Set;

import javax.ws.rs.ServerErrorException;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.HttpHeaders;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import javax.ws.rs.ext.Provider;
import javax.ws.rs.ext.WriterInterceptor;
import javax.ws.rs.ext.WriterInterceptorContext;

@Provider
public class ViewWriteInterceptor implements WriterInterceptor {


  private HttpHeaders httpHeaders;

  public ViewWriteInterceptor(@Context HttpHeaders httpHeaders) {
    this.httpHeaders = httpHeaders;
  }


  @Override
  public void aroundWriteTo(WriterInterceptorContext writerInterceptorContext) 
    throws IOException,
           WebApplicationException {

    // I assume this case will never happen, just to be sure
    if (writerInterceptorContext.getEntity() == null) {

      writerInterceptorContext.proceed();
      return;
    }
    else
    {
      Class<?> entityType = writerInterceptorContext.getEntity()
          .getClass();
      String entityTypeString = entityType.getName();
      
      // Ignore any Jersey system classes, for example wadl
      //
      if (entityType == String.class  || entityType.isArray() 
        || entityTypeString.startsWith("com.sun") 
        || entityTypeString.startsWith("org.glassfish")) {
        writerInterceptorContext.proceed();
        return;
      }
      
      // Fail if the class doesn't have the default NoView annotation 
      // this prevents any unannotated fields from showing up
      //
      else if (!entityType.isAnnotationPresent(NoView.class)) {
        throw new ServerErrorException("Entity type should be tagged with @NoView annotation " + entityType, Response.Status.INTERNAL_SERVER_ERROR);
        
      }
    }

    // Get hold of the return media type:
    //

    MediaType mt = writerInterceptorContext.getMediaType();
    String level = mt.getParameters().get("level");

    // Get the annotations and modify as required
    //

    Set<Annotation> current = new LinkedHashSet<>();
    current.addAll(Arrays.asList(
      writerInterceptorContext.getAnnotations()));

    switch (level != null ? level : "") {
      default:
      case "detailed":
        current.add(com.example.annotation.DetailedView.Factory.get());
      case "normal":
        current.add(com.example.annotation.NormalView.Factory.get());
      case "summary":
        current.add(com.example.annotation.SummaryView.Factory.get());

    }

    writerInterceptorContext.setAnnotations(
      current.toArray(new Annotation[current.size()]));

    //

    writerInterceptorContext.proceed();
  }
}

Finally you have to enable the EntityFilterFeature manually, to do this you can simple register it in your Application class

import java.lang.annotation.Annotation;

import javax.ws.rs.ApplicationPath;

import org.glassfish.jersey.message.filtering.EntityFilteringFeature;
import org.glassfish.jersey.server.ResourceConfig;

@ApplicationPath("/resources/")
public class SelectableApplication extends ResourceConfig {
  public SelectableApplication() {

    packages("...");

    // Set entity-filtering scope via configuration.
    property(EntityFilteringFeature.ENTITY_FILTERING_SCOPE, 
    new Annotation[] {
      NormalView.Factory.get(), DetailedView.Factory.get(), 
      NoView.Factory.get(), SummaryView.Factory.get()
    });
    register(EntityFilteringFeature.class);
  }


}

Once you have this all up and running the application will respond as before:

GET .../hello Accept application/json; level=detailed or application/json
{
  "message" : "This is indeed the message",
  "submessage" : {
    "message" : "Some sub messages"
  },
  "subtext" : "This is the deep and meaningful subtext",
  "summary" : "Some simple summary"
}

GET .../hello Accept application/json; level=normal
{
  "message" : "This is indeed the message",
  "summary" : "Some simple summary"
}


GET .../hello Accept application/json; level=summary
{
  "summary" : "Some simple summary"
}

This is feel is a better alternative to using the MOXy annotations directly - using custom annotations should have to much easier to port your application to over implementation even if you have to provide you own filter. Finally it is worth also exploring the Jersey extension to this that allows Role based filtering which I can see as being useful in a security aspect.

Tuesday, January 28, 2014

Selecting level of detail returned by varying the content type

So I have been playing with the various ways of generating JSON in Jersey this week and I have been looking at solutions to the problem of returning different levels of detail depending on the client requirements. Here is one solution that uses the MoXY JAX-B provider in Jersey 2.x.

Consider this very simple hello world class:

@Path("hello")
public class SelectableHello {
    
    @GET
    @Produces({"application/json; level=detailed", "application/json; level=summary", "application/json; level=normal"})
    public Message hello() {
        return new Message();
    }
}

The Message class is annotated with some special annotations from MOXy that allows you to specify different partial object graphs. So my very simple message class looks like the following with three different levels defined:

import javax.xml.bind.annotation.XmlRootElement;

import org.eclipse.persistence.oxm.annotations.XmlNamedAttributeNode;
import org.eclipse.persistence.oxm.annotations.XmlNamedObjectGraph;
import org.eclipse.persistence.oxm.annotations.XmlNamedObjectGraphs;


@XmlNamedObjectGraphs({
  @XmlNamedObjectGraph(name = "summary", 
    attributeNodes = { 
      @XmlNamedAttributeNode("summary") }),
  @XmlNamedObjectGraph(name = "normal",
    attributeNodes =
    { @XmlNamedAttributeNode("summary"), 
      @XmlNamedAttributeNode("message") }),
  @XmlNamedObjectGraph(name = "detailed",
    attributeNodes =
    { @XmlNamedAttributeNode("summary"), 
      @XmlNamedAttributeNode("message"),
      @XmlNamedAttributeNode("subtext") })
  })
@XmlRootElement
public class Message {

  private String summary, message, subtext;

  public Message() {
    summary = "Some simple summary";
    message = "This is indeed the message";
    subtext = "This is the deep and meaningful subtext";
  }

  public void setSummary(String summary) {
    this.summary = summary;
  }

  public String getSummary() {
    return summary;
  }

  public void setMessage(String message) {
    this.message = message;
  }

  public String getMessage() {
    return message;
  }

  public void setSubtext(String subtext) {
    this.subtext = subtext;
  }

  public String getSubtext() {
    return subtext;
  }


}

Then it is a relatively easy step to define a JAX-RS ContextResolver that returns a suitably configured MoxyJsonConfig depending on the mime type. The code in here is a little bit ropey as it assumes that the first item in the accept list is the correct one.

import javax.ws.rs.core.Context;

import javax.ws.rs.core.Context;
import javax.ws.rs.core.HttpHeaders;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.ext.ContextResolver;
import javax.ws.rs.ext.Provider;

import org.eclipse.persistence.jaxb.MarshallerProperties;

import org.glassfish.jersey.moxy.json.MoxyJsonConfig;

@Provider
public class MoxyJsonConfigResolver implements ContextResolver<MoxyJsonConfig> {

    HttpHeaders httpHeaders;


    @Context
    public void setHeaders(HttpHeaders httpHeaders) {
        this.httpHeaders = httpHeaders;
    }

    @Override
    public MoxyJsonConfig getContext(Class<?> c) {

        // Assume we are going to match the first accept
        //
        MediaType responseType = httpHeaders.getAcceptableMediaTypes().get(0);

        MoxyJsonConfig config = new MoxyJsonConfig();
        String level = responseType.getParameters().get("level");
        config.getMarshallerProperties().put(MarshallerProperties.OBJECT_GRAPH, level != null ? level : "detailed");
        return config;
    }
}

So once this little application is running, you can see the output for different mime types. In our case the default for "application/json" is "detailed" but I can see the argument in many cases for "normal" to be the default.

GET .../hello Accept application/json; level=detailed or application/json

{
    "message" : "This is indeed the message",
    "subtext" : "This is the deep and meaningful subtext",
    "summary" : "Some simple summary"
}

GET .../hello Accept application/json; level=normal

{
    "message" : "This is indeed the message",
    "summary" : "Some simple summary"
}

GET .../hello Accept application/json; level=summary

{
    "summary" : "Some simple summary"
}


This does work, but as you can see the annotations could be prettier, the next blog post will be an example using EntityFilteringFeature of Jersey. This feature builds on top of the MOXy functionality above while allowing a custom annotation scheme. This to my mind is a lot easier to work with.

Tuesday, January 7, 2014

Post-hoc tracing using a debugger

Once nice little features of most debuggers that I have been exercising recently is the ability to log information on a breakpoint. This can be a really useful was to understand code without having to modify it is involve byte code modification.


Let's consider this very trivial and inefficient implementation of a function to return the n'th number in the Fibonacci sequence.


public class Fib {



    public long fib(long number) {
        return number < 1 ? 0 :    // Breakpoint here
            number < 2 ? 1 : fib(number - 2) + fib(number - 1);
    }




    public static void main(String[] args) {

        Fib fib = new Fib();
        System.out.println(fib.fib(10L));

    }

}

Now we add a simple breakpoint, I am going to modify it slightly so that it doesn't halt; but it will log a simple expression which gives us the current value of number

This gives you the following output for our rather inefficient code:

ebugger connected to local process.
Source breakpoint: Fib.java:11, evaluate(number)=10 (long)
Source breakpoint: Fib.java:11, evaluate(number)=8 (long)
Source breakpoint: Fib.java:11, evaluate(number)=6 (long)
Source breakpoint: Fib.java:11, evaluate(number)=4 (long)
Source breakpoint: Fib.java:11, evaluate(number)=2 (long)
Source breakpoint: Fib.java:11, evaluate(number)=0 (long)
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
Source breakpoint: Fib.java:11, evaluate(number)=3 (long)
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
Source breakpoint: Fib.java:11, evaluate(number)=2 (long)
... sometime later
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
Source breakpoint: Fib.java:11, evaluate(number)=2 (long)
Source breakpoint: Fib.java:11, evaluate(number)=0 (long)
Source breakpoint: Fib.java:11, evaluate(number)=1 (long)
55

So what we have done here is to add trace post-hoc, you might not even have access to the source and still be able to get useful information out of what the code is doing.

Now the obvious take away from the above example is that we are computing the same value over and over again. So here is another version of the code which instead uses a Map to store previously computed values in the sequence. Note also it is more of a pain to add trace to this code than in the previous case without making the Lambda less pretty.

import java.util.HashMap;
import java.util.Map;

public class Fib {


    Map<Long, Long> data = new HashMap<>();
    {
        data.put(1L, 1L);
    }


    public long fib(long number) {
        return data.computeIfAbsent(number,
                      n -> n < 1 ? 0 : fib(n - 2) + fib(n - 1)); // Breakpoint here
    }




    public static void main(String[] args) {

        Fib fib = new Fib();
        System.out.println(fib.fib(10L));

    }

}

It is worth noting two things here, first of all your breakpoint log expression should be evaluating the value of the Lambda parameter n and that the second is that you should always put your expression on a new line so that future developer can easily breakpoint it. (Not matter now much smaller/ prettier/ clever it would look if you put the code on one).

So the trace output is now as follows, much better as each value is only computed the once.

Debugger connected to local process.
Source breakpoint: Fib.java:17, evaluate(n)=10
Source breakpoint: Fib.java:17, evaluate(n)=8
Source breakpoint: Fib.java:17, evaluate(n)=6
Source breakpoint: Fib.java:17, evaluate(n)=4
Source breakpoint: Fib.java:17, evaluate(n)=2
Source breakpoint: Fib.java:17, evaluate(n)=0
Source breakpoint: Fib.java:17, evaluate(n)=3
Source breakpoint: Fib.java:17, evaluate(n)=5
Source breakpoint: Fib.java:17, evaluate(n)=7
Source breakpoint: Fib.java:17, evaluate(n)=9
55

The screen grabs in this blog are from Jdeveloper; but similar features are available in Intelij, Netbeans and if you are a bit crafty you can get something similar in Eclipse.