Validating annotations at compile time, example using JAX-RS

I am in the process of re-reading Effective Java now that I have gotten around to buying the second edition. I always enjoy reading anything that Bloch puts out and I always learn something new. I was working my way through item 35 "Prefer Annotations to Naming conventions" when I noticed the following statemen that was talking about validating the annotations:

"... It would be nice if the compiler could enforce this restriction, but it can't. There are limits to how much error checking the compiler can do...."

Now it is normally very hard find something that you think that Mr Bloch has got wrong, and also be right; but I think this validation is very possible. Recently I have been looking at Project Lombok which does interesting things with the annotation processor. The general idea behind the annotation processors is that they give the ability to generate new code; but it occurred to me that an annotation processor can just check source files for errors. This allows you to extend the java compiler to do interesting non trivial annotation validation.

Rather than deal with the simple @Test example in the book, I have a solution for that one if anybody in interested, lets instead look at a real world examples from JAX-RS web services:

package restannotationtotest;

import javax.ws.rs.GET;
import javax.ws.rs.QueryParam;

public class ExampleResource {

    @GET
    public String goodGetNoParam() {
        return "Hello";
    }

    @GET
    public String goodGetParam(@QueryParam("name")String name) {
        return "Hello " + name;
    }
    
    // This annotation will fail at deploy time    
    @GET
    public String badGet(String name) {
        return "Hello " + name;
    }
}

The last method will fail at deploy time as a HTTP GET request cannot have a method body as implied by having a method parameter that is not otherwise consumed by the framework. This is a mistake I kept on making when I started with Jersey so I though it was worth starting with. So our first goal is to flag this last method up at compile time as being in error.

In order for an annotation processor to work you need a jar file with an entry in the META-INF/services path called javax.annotation.processing.Processor. This contain a list of fully qualified processor class names in plain text. In some tools you might find that the javax.annotation.processing.Processor file is not copied to the classpath as you need to add ".Processor" to the list of file extensions copied at compile time. You also of course need a class that implements the Processor interface so my project looks like this:

First of all lets look at the basic house keeping parts of the JaxRSValidator class without worry about the meat of the class. We extend the AbstractProcessor rather than implementing the Processor interface both to gain some basic functionality and to future proof the code against interface evolution. We could have used the @SupportedAnnotationTypes annotation rather than implement the corresponding method; but for later it was handy to have a list of class literals. As you can see there is not much to the configuration:

package com.kingsfleet.rs;


import java.lang.annotation.Annotation;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

import javax.annotation.Resource;
import javax.annotation.processing.AbstractProcessor;
import javax.annotation.processing.Messager;
import javax.annotation.processing.ProcessingEnvironment;
import javax.annotation.processing.RoundEnvironment;
import javax.annotation.processing.SupportedSourceVersion;

import javax.lang.model.SourceVersion;
import javax.lang.model.element.Element;
import javax.lang.model.element.ElementKind;
import javax.lang.model.element.ExecutableElement;
import javax.lang.model.element.TypeElement;
import javax.lang.model.element.VariableElement;
import javax.lang.model.util.ElementScanner6;
import javax.lang.model.util.Types;

import javax.tools.Diagnostic;

import javax.ws.rs.CookieParam;
import javax.ws.rs.DELETE;
import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.HEAD;
import javax.ws.rs.HeaderParam;
import javax.ws.rs.MatrixParam;
import javax.ws.rs.OPTIONS;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;


@SupportedSourceVersion(SourceVersion.RELEASE_6)
public class JaxRSValidator extends AbstractProcessor {

    private static final List<Class<? extends Annotation>> PARAMETER_ANNOTATIONS
                     = Arrays.asList(
                           CookieParam.class,
                           FormParam.class,
                           HeaderParam.class,
                           MatrixParam.class,
                           PathParam.class,
                           QueryParam.class, 
                           Context.class,
                           Resource.class);
     
    private static final List<Class<? extends Annotation>> METHOD_ANNOTATIONS
                    = Arrays.asList(
                          GET.class,
                          POST.class,
                          PUT.class,
                          DELETE.class,
                          HEAD.class,
                          OPTIONS.class);

    private static final Set<String> MATCHING_ANNOTATIONS_AS_STRING;
    static {
        Set<String> set = new HashSet<String>();
        for (Class a : METHOD_ANNOTATIONS) {
            set.add(a.getName());
        }
        // We care about path as well
        //
        set.add(Path.class.getName());
        //
        MATCHING_ANNOTATIONS_AS_STRING = Collections.unmodifiableSet(
                                           set);
    }


    @Override
    public Set<String> getSupportedAnnotationTypes() {
        return MATCHING_ANNOTATIONS_AS_STRING;
    }

  
    // Implementation of processor
    //

}

The actual implementation of the processor is in the process method, you will notice that for most annotation processors all the work is done using a visitor or the concrete scanner implementations.

The scanner we have implemented looks for an ExecutableElement that is of type "Method". The javax.lang.model is a little bit obscure in its terminology so it did take me a little bit of time to work out that this is the correct structure. Another limitation is that you cannot drill down into the finer details of the code structure without casting this a class from com.sun.tools.javac.code.Symbol: with attendant maintenance issues of using a com.sun.* package. Fortunately for the validation I want to do I can stick with the public APIs. Hopefully a future version of Java will expand on this API.

It is a relatively simple matter after that to process each method in turn looking for a one with the @GET annotation that would suggest a message body of some kind.

    private static ElementScanner6<Void, ProcessingEnvironment> SCANNER =
        new ElementScanner6<Void, ProcessingEnvironment>() {
        @Override
        public Void visitExecutable(ExecutableElement e,
                                    ProcessingEnvironment processingEnv) {

            final Messager log = processingEnv.getMessager();
            final Types types = processingEnv.getTypeUtils();

            // Make sure for a GET all parameters are mapped to
            // to something sensible
            //

            if (e.getKind() == ElementKind.METHOD) {
                
                // GET no body
                verifyNoBodyForGET(e, log);
            }
            return null;
        }


        /**
         * Check that if we have a GET we should have no body. (Should
         *   also process OPTIONS and others)
         */
        private void verifyNoBodyForGET(ExecutableElement e,
                                        final Messager log) {
            if (e.getAnnotation(GET.class) != null) {
                
                // For each parameter check for the standard annotations
                found : for (VariableElement ve : e.getParameters()) {
                    for (Class<? extends Annotation> c : PARAMETER_ANNOTATIONS) {
                        if (ve.getAnnotation(c)!=null) {
                            break found;
                        }
                    }
                    
                    log.printMessage(Diagnostic.Kind.ERROR, 
                                     "Parameters on a @GET cannot be mapped to a request body, try one of the @*Param annotations",
                                     e);
                }
            }
        }
    };

    public boolean process(Set<? extends TypeElement> annotations,
                           RoundEnvironment roundEnv) {

        for (TypeElement annotation : annotations) {
            for (Element e : roundEnv.getElementsAnnotatedWith(annotation)) {
                SCANNER.scan(e, processingEnv);
            }
        }

        // Allow other processors in
        return false;
    }

It is important when you use the log.printMessage(...) method to include the Element as the final parameter as this allows tooling to correctly display the error/warning message location. So it is a simple matter to build this project into a jar file using the tool of your choice and then have it on the the classpath when you build ExampleResource that we defined earlier. (JDeveloper users note my previous post on Lombok on running javac "Out of Process" to get this working). Depending on your tool you should get an error message that looks something like this:

Lets look at a more complicated example from Jersey to build on our code. In this example we need the parameters in the @Path annotation to match @PathParam parameters on the matching method. In this case here are two that are fine and two that might fail at some point later on. Neither of the problems can be picked up by the compiler.

package restannotationtotest;

import javax.ws.rs.Path;
import javax.ws.rs.PathParam;

public class PathResource {
    
    // Fine
    @Path("{param}")
    public ExampleResource getOneParam(@PathParam("param") String param) {
        return null;
    }

    // Fine
    @Path("{param}/someotherText/{param1}")
    public ExampleResource getOneParam(@PathParam("param") String param, @PathParam("param1") String param1) {
        return null;
    }

    // Suspect
    @Path("{param}")
    public ExampleResource getUnusedParam() {
        return null;
    }
    
    // Definitely broken
    @Path("{param}")
    public ExampleResource getMissMatch(@PathParam("paramMissMatch") String param) {
        return null;
    }
}

Our original code already visits all the methods so we need to simply extend the code to check that the parameters match entries on the @Path. We can also check for a zero length @PathParam argument which again is something that compiler can't do for free.

    private static ElementScanner6<Void, ProcessingEnvironment> SCANNER =
        new ElementScanner6<Void, ProcessingEnvironment>() {
        @Override
        public Void visitExecutable(ExecutableElement e,
                                    ProcessingEnvironment processingEnv) {

            final Messager log = processingEnv.getMessager();
            final Types types = processingEnv.getTypeUtils();

            // Make sure for a GET all parameters are mapped to
            // to something sensible
            //

            if (e.getKind() == ElementKind.METHOD) {
                
                // GET no body
                verifyNoBodyForGET(e, log);
                
                // Try to match path param to @Path
                verifyPathParamMatches(e, log);
            }
            return null;
        }

        /**
         * Check that if we have path param we have all the matching
         * path elements consumed.
         */
        private void verifyPathParamMatches(ExecutableElement e,
                                            final Messager log) {
            
            // Verify that we have a method that has resource
            //
            
            Path p = e.getAnnotation(Path.class);
            if (p!=null && p.value()!=null) {
                
                // Hack the resources out of the string, verify
                // path parameters, TODO write regex
                //
                List<String> resources = new ArrayList<String>();
                String path = p.value();
                final String[] splitByOpen = path.split("\\{");
                for (String bit : splitByOpen) {
                    String moreBits[] = bit.split("}");
                    if (moreBits.length >= 1 && moreBits[0].length() !=0) {
                        resources.add(moreBits[0]);
                    }
                }
                
                // If we have resource try to find path params to match
                if (resources.size() > 0) {
                    found : for (VariableElement ve : e.getParameters()) {

                        PathParam pp = ve.getAnnotation(PathParam.class);
                        String mappedPath = pp.value();
                        if (mappedPath==null || mappedPath.length()==0) {
                            log.printMessage(Diagnostic.Kind.ERROR, 
                                             "Missing or empty value",
                                             ve);
                        }
                        else if (!resources.contains(mappedPath)) {
                            log.printMessage(Diagnostic.Kind.WARNING, 
                                             "Value " + mappedPath + " doesn't map to path",
                                             ve);
                        }
                        else {
                            // Make this as processed
                            resources.remove(mappedPath);
                        }
                    }
                    
                    if (resources.size() > 0) {
                        log.printMessage(Diagnostic.Kind.WARNING, 
                                         "Unmapped path parameters " + resources.toString(),
                                         e);
                    }
                }
            }
        }

        /**
         * Check that if we have a GET we should have no body. (Should
         *   also process OPTIONS and others)
         */
        private void verifyNoBodyForGET(ExecutableElement e,
                                        final Messager log) {
            ...
        }
    };

Again you can simple compile the project with ExampleResource and PathResources using your favorite build tool and you should see something like:

Note the second method gets two warnings, the line numbers are the same.

So this contrary to the original statement it is possible to get the compilers to perform quite complex validation of annotations. I wonder if we could convince some of the annotation based JEE projects and libraries to come with a validation jar file that contains a matching processor to validate against the spec. It would save a lot of confusion. Tools developers like myself would also be able to harness the same code in code editors to provide in line error feedback in a consistent way. Interesting stuff.