Showing posts with label json-schema. Show all posts
Showing posts with label json-schema. Show all posts

Tuesday, April 15, 2014

Quick, and a bit dirty, JSON Schema generation with MOXy 2.5.1

So I am working on a new REST API for an upcoming Oracle cloud service these days so one of the things I needed was the ability to automatically generate a JSON Schema for the bean in my model. I am using MOXy to generate the JSON from POJO and as of version 2.5.1 of EclipseLink it now has the ability to generate a JSON Schema from the bean model.

There will be a more formal solution integrated into Jersey 2.x at a future date; but this solution will do at the moment if you want to play around with this.

So the first class we need to put in place is a model processor, very much and internal Jersey class, that allows us to amend the resource model with extra methods and resources. To each resource in the model we can add the JsonSchemaHandler which does the hard work of generating a new schema. Since this is a simple POC there is no caching going on here, please be aware of this if you are going to use this in production code.

import com.google.common.collect.Lists;

import example.Bean;

import java.io.IOException;
import java.io.StringWriter;

import java.text.SimpleDateFormat;

import java.util.Date;
import java.util.List;

import javax.inject.Inject;

import javax.ws.rs.HttpMethod;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.core.Configuration;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

import javax.xml.bind.JAXBException;
import javax.xml.bind.SchemaOutputResolver;
import javax.xml.transform.Result;
import javax.xml.transform.stream.StreamResult;

import org.eclipse.persistence.jaxb.JAXBContext;

import org.glassfish.jersey.process.Inflector;
import org.glassfish.jersey.server.ExtendedUriInfo;
import org.glassfish.jersey.server.model.ModelProcessor;
import org.glassfish.jersey.server.model.ResourceMethod;
import org.glassfish.jersey.server.model.ResourceModel;
import org.glassfish.jersey.server.model.RuntimeResource;
import org.glassfish.jersey.server.model.internal.ModelProcessorUtil;
import org.glassfish.jersey.server.wadl.internal.WadlResource;

public class JsonSchemaModelProcessor implements ModelProcessor {

  private static final MediaType JSON_SCHEMA_TYPE = 
    MediaType.valueOf("application/schema+json");
  private final List<ModelProcessorUtil.Method> methodList;


  public JsonSchemaModelProcessor() {
    methodList = Lists.newArrayList();
    methodList.add(new ModelProcessorUtil.Method("$schema", HttpMethod.GET, 
      MediaType.WILDCARD_TYPE, JSON_SCHEMA_TYPE,
      JsonSchemaHandler.class));
  }

  @Override
  public ResourceModel processResourceModel(ResourceModel resourceModel, 
      Configuration configuration) {
    return ModelProcessorUtil.enhanceResourceModel(resourceModel, true, methodList, 
      true).build();
  }

  @Override
  public ResourceModel processSubResource(ResourceModel resourceModel, 
      Configuration configuration) {
    return ModelProcessorUtil.enhanceResourceModel(resourceModel, true, methodList, 
      true).build();
  }


  public static class JsonSchemaHandler 
    implements Inflector<ContainerRequestContext, Response> {

    private final String lastModified = new SimpleDateFormat(WadlResource.HTTPDATEFORMAT).format(new Date());

    @Inject
    private ExtendedUriInfo extendedUriInfo;

    @Override
    public Response apply(ContainerRequestContext containerRequestContext) {

      // Find the resource that we are decorating, then work out the
      // return type on the first GET

      List<RuntimeResource> ms = extendedUriInfo.getMatchedRuntimeResources();
      List<ResourceMethod> rms = ms.get(1).getResourceMethods();
      Class responseType = null;
      found:
      for (ResourceMethod rm : rms) {
        if ("GET".equals(rm.getHttpMethod())) {
          responseType = (Class) rm.getInvocable().getResponseType();
          break found;
        }
      }

      if (responseType == null) {
        throw new WebApplicationException("Cannot resolve type for schema generation");
      }

      //
      try {
        JAXBContext context = (JAXBContext) JAXBContext.newInstance(responseType);

        StringWriter sw = new StringWriter();
        final StreamResult sr = new StreamResult(sw);

        context.generateJsonSchema(new SchemaOutputResolver() {
          @Override
          public Result createOutput(String namespaceUri, String suggestedFileName) 
              throws IOException {
            return sr;
          }
        }, responseType);


        return Response.ok().type(JSON_SCHEMA_TYPE)
          .header("Last-modified", lastModified)
          .entity(sw.toString()).build();
      } catch (JAXBException jaxb) {
        throw new WebApplicationException(jaxb);
      }
    }
  }


}

Note the very simple heuristic in the JsonSchemaHandler code it assumes that for each resource there is a 1:1 mapping to a single JSON Schema element. This of course might not be true for your particular application.

Now that we have the schema generated in a know location we need to tell the client about it, the first thing we will do is to make sure that there is a suitable link header when the user invokes OPTIONS on a particular resource:

import java.io.IOException;

import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerResponseContext;
import javax.ws.rs.container.ContainerResponseFilter;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.Link;
import javax.ws.rs.core.UriInfo;

public class JsonSchemaResponseFilter implements ContainerResponseFilter {

  @Context
  private UriInfo uriInfo;

  @Override
  public void filter(ContainerRequestContext containerRequestContext,
                     ContainerResponseContext containerResponseContext) throws IOException {


    String method = containerRequestContext.getMethod();
    if ("OPTIONS".equals(method)) {

      Link schemaUriLink =
        Link.fromUriBuilder(uriInfo.getRequestUriBuilder()
          .path("$schema")).rel("describedBy").build();

      containerResponseContext.getHeaders().add("Link", schemaUriLink);
    }
  }
}

Since this is JAX-RS 2.x we are working with we of course are going bundle all the bit together into a feature:

import javax.ws.rs.core.Feature;
import javax.ws.rs.core.FeatureContext;

public class JsonSchemaFeature implements Feature {

  @Override
  public boolean configure(FeatureContext featureContext) {

    if (!featureContext.getConfiguration().isRegistered(JsonSchemaModelProcessor.class)) {
      featureContext.register(JsonSchemaModelProcessor.class);
      featureContext.register(JsonSchemaResponseFilter.class);
      return true;
    }
    return false;
  }
}

I am not going to show my entire set of POJO classes; but just quickly this is the Resource class with the @GET method required by the schema generation code:

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

@Path("/bean")
public class BeanResource {
    
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public Bean getBean() {
        return new Bean();
    }
}

And finally here is what you see if you perform a GET on a resource:

GET .../resources/bean
Content-Type: application/json

{
  "message" : "hello",
  "other" : {
    "message" : "OtherBean"
  },
  "strings" : [
    "one",
    "two",
    "three",
    "four"
  ]
}

And OPTIONS:

OPTIONS .../resources/bean
Content-Type: text/plain
Link: <http://.../resources/bean/$schema>; rel="describedBy"

GET, OPTIONS, HEAD

And finally if you resolve the schema resource:

GET .../resources/bean/$schema
Content-Type: application/schema+json

{
  "$schema" : "http://json-schema.org/draft-04/schema#",
  "title" : "example.Bean",
  "type" : "object",
  "properties" : {
    "message" : {
      "type" : "string"
    },
    "other" : {
      "$ref" : "#/definitions/OtherBean"
    },
    "strings" : {
      "type" : "array",
      "items" : {
        "type" : "string"
      }
    }
  },
  "additionalProperties" : false,
  "definitions" : {
    "OtherBean" : {
      "type" : "object",
      "properties" : {
        "message" : {
          "type" : "string"
        }
      },
      "additionalProperties" : false
    }
  }
}

There is a quite a bit of work to do here, in particular generating the hypermedia extensions based on the declarative linking annotations that I forward ported into Jersey 2.x a little while back. But it does point towards a solution and we get to exercise a variety of solutions to get something working now.

Monday, November 19, 2012

JSON-Schema generation in Jersey

So in my previous post I talked about a proposal to allow the use of JSON-Schema in a WADL, this post looks at how to get this working with a recently build of Jersey. You are going to have to download / reference 1.16SNAPSHOT until 1.16 is released.

If you are using Maven this should be quite straight forward to update your dependencies assuming you already have jersey and jersey-json. You are just going to need to add a dependency on the "jersey-wadl-json-schema" artefact from the "com.sun.jersey.contribs" group to get the new feature.

If you are outside of Maven the easiest thing to do would be to download the latest jersey-archive and then the jersey-wadl-json-schema jar. How you deploy these is tool specific, but if you are using WLS then here are some specific notes on how to upgrade the version of Jersey.

Once you have this working, you need to create a WadlGeneratorConfig class in order to enable this new grammar generation:

package jersey;

import com.sun.jersey.api.wadl.config.WadlGeneratorConfig;
import com.sun.jersey.api.wadl.config.WadlGeneratorDescription;
import com.sun.jersey.wadl.generators.json.WadlGeneratorJSONGrammarGenerator;

import java.util.List;

public class JsonGeneratorConfig extends WadlGeneratorConfig {
 
    @Override
    public List configure() {
        return generator(WadlGeneratorJSONGrammarGenerator.class).descriptions();
    }
}

This can then be registered in a variety of ways, here is an example using a servlet init param. Note also that to make this example simple that we are using the Jersey POJO mapping; but whilst writing this blog I noticed that this setting affected the format of the JSON version of the WADL in case you try this.

<?xml version = '1.0' encoding = 'ISO-8859-1'?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">
  <servlet>
    <servlet-name>jersey</servlet-name>
    <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
    <init-param>
      <param-name>com.sun.jersey.config.property.WadlGeneratorConfig</param-name>
      <param-value>jersey.JsonGeneratorConfig</param-value>
    </init-param>
    <init-param>
      <param-name>com.sun.jersey.api.json.POJOMappingFeature</param-name>
      <param-value>true</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>jersey</servlet-name>
    <url-pattern>/resources/*</url-pattern>
  </servlet-mapping>
</web-app>

So I put together a really simple echo service, just to check this is all working:

package jersey;

import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;

@Path("/echo")
public class EchoResource {
     

    @GET
    @Produces("application/json")
    public EchoBean echo() {
        EchoBean bean = new EchoBean();
        bean.setMessage("Hello");
        return bean;
    }

    @POST
    @Consumes("application/json")
    @Produces("application/json")
    public EchoBean echo(EchoBean echo) {
        return echo;
    }
    
}

and

package jersey;

public class EchoBean {
    public EchoBean() {
        super();
    }
    
    private String message;

    public void setMessage(String message) {
        this.message = message;
    }

    public String getMessage() {
        return message;
    }


}

This very simple example results in the following WADL with the JSON-Schema elements referenced:

<?xml version = '1.0' encoding = 'UTF-8'?>
<ns0:application xmlns:ns0="http://wadl.dev.java.net/2009/02">
   <ns0:doc xmlns:ns1="http://jersey.java.net/" ns1:generatedBy="Jersey: 1.16-SNAPSHOT 11/19/2012 12:59 AM"/>
   <ns0:grammars/>
   <ns0:resources base="http://localhost:7103/Jersey/resources/">
      <ns0:resource path="/echo">
         <ns0:method id="echo" name="GET">
            <ns0:response>
               <ns0:representation mediaType="application/json" xmlns:ns2="http://wadl.dev.java.net/2009/02/json-schema" ns2:describedby="application.wadl/echoBean"/>
            </ns0:response>
         </ns0:method>
         <ns0:method id="echo" name="POST">
            <ns0:request>
               <ns0:representation mediaType="application/json" xmlns:ns3="http://wadl.dev.java.net/2009/02/json-schema" ns3:describedby="application.wadl/echoBean"/>
            </ns0:request>
            <ns0:response>
               <ns0:representation mediaType="application/json" xmlns:ns4="http://wadl.dev.java.net/2009/02/json-schema" ns4:describedby="application.wadl/echoBean"/>
            </ns0:response>
         </ns0:method>
      </ns0:resource>
   </ns0:resources>
</ns0:application>

The URI application.wadl/echoBean contains this simple JSON-Schema definition:

{
    "type" : "object",
    "properties" : {
        "message" : {
            "type" : "string"
        }
    },
    "name" : "echoBean"
}

Now there are a number of limitations in the current design, not least that the generated schema doesn't take into account any notation settings. But I thought this would be enough to provoke feedback on whether this feature would be useful generally. There appears to be a growing interest in JSON-Schema both around the net and internally to Oracle, so it would be interesting to see whether this description becomes more common.

Friday, November 16, 2012

JSON-Schema in WADL

In between other jobs I have been recently been reviewing the WADL specification with a view to fixing some documentation problems with a view to producing an updated version. One of the things that because apparent was the lack of any grammar support for languages other than XML - yes you can use a mapping from JSON<->XML Schema but this would be less than pleasant for a JSON purist.

So I began to look at how one would go about attaching a JSON-Schema grammar of a JSON document in a WADL description of a service. This isn't a specification yet; but a proposal of how it might work consistently.

Now I work with Jersey mostly, so lets consider what Jersey will current generate for a service that returns both XML and JSON. So the service here is implemented using the JAX-B binding so they both use a similar structure as defined by the XML-Schema reference by the include.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<application xmlns="http://wadl.dev.java.net/2009/02">
    <doc xmlns:jersey="http://jersey.java.net/" jersey:generatedBy="Jersey: 1.16-SNAPSHOT 10/26/2012 09:28 AM"/>
    <grammars>
        <include href="xsd0.xsd">
            <doc title="Generated" xml:lang="en"/>
        </include>
    </grammars>
    <resources base="http://localhost/">
        <resource path="/root">
            <method id="hello" name="PUT">
                <request>
                    <representation xmlns:m="urn:message" element="m:requestMessage"  mediaType="application/json" />
                    <representation xmlns:m="urn:message" element="m:requestMessage" mediaType="application/xml" />
                </request>
                <response>
                    <representation xmlns:m="urn:message" element="m:responseMessage" mediaType="application/json"/>
                    <representation xmlns:m="urn:message" element="m:responseMessage" mediaType="application/xml" />
                </response>
            </method>
        </resource>
    </resources>
</application> 

So the first thing we considered was re-using the existing element property, which is defined as a QName, on the representation element to reference an imported JSON-Schema. It is shown here both with and another an arbitrary namespace to it can be told apart from XML elements without a namespace.

<grammars>
        <include href="xsd0.xsd" />
        <include href="application.wadl/responseMessage" />
    </grammars>


    <representation element="responseMessage" mediaType="application/json"/>

Or
    xmlns:json="http://wadl.dev.java.net/2009/02/json" 

    <representation 
        element="json:responseMessage" mediaType="application/json" />


The problem is that the JSON-Schema specification as it stands doesn't have a concept of a "name" property, so each JSON-Schema is uniquely identified by it's URI. Also from my read of the specification each JSON-Schema contains the definition for at most one document - not the multiple types / documents that can be contained in XML-Schema.

So the next best suggestion would be to just use the "filename" part of the URI as a proxy for the URI; but of course that won't necessarily be unique. I could see for example the US government and Yahoo both publishing there own "address" micro format.

The better solution to this problem is to introduce a new attribute, luckily the WADL spec was designed with this in mind, that is of type URI that can be used to directly reference the JSON-Schema definitions. So rather than the direct import in the previous example we have a URI property on the element itself. The "describedby" attribute name comes from the JSON-Schema proposal and is consistent with the rel used on atom links in the spec.

xmlns:json="http://wadl.dev.java.net/2009/02/json-schema" 
    xmlns:m="urn:message" 


    <grammars>
        <include href="xsd0.xsd" />
    </grammars>

    <representation 
        mediaType="application/json"
        element="m:responseMessage" 
        json:describedby="application.wadl/responseMessage" />


The has the secondary advantage in that this format is backwardly compatible with tooling that was relying on the XML-Schema grammar. Although this is probably only of interesting to people who work in tooling / testing tools like myself.

Once you have the JSON-Schema definition then some users are going to want to do away with the XML all together, so finally here is a simple mapping of the WADL to a JSON document that contains just the JSON-Schema information. It has been suggested by Sergey Breyozkin the JSON mapping would only show the json grammars and I am coming around to that way of thinking. I would be interested to hear of a usecase for the JSON mapping that would want access to the XML Schema.

{
   "doc":{
      "@generatedBy":"Jersey: 1.16-SNAPSHOT 10/26/2012 09:28 AM"
   },
   "resources":{
      "@base":"http://localhost/",
      "resource":{
         "@path":"/root",
         "method":{
            "@id":"hello",
            "@name":"PUT",
            "request":{
               "representation":[
                  {
                     "@mediaType":"application/json",
                     "@describedby":"application.wadl/requestMessage"
                  }
               ]
            },
            "response":{
               "representation":[
                  {
                     "@mediaType":"application/json",
                     "@describedby":"application.wadl/responseMessage"
                  }
               ]
            }
         }
      }
   }
}

I am currently using the mime type of "application/vnd.sun.wadl+json" for this mapping to be consistent with the default WADL mime type. I suspect we would want to change this in the future; but it will do for starters.

So this is all very interesting but you can't play with it unless you have an example implementation. I have something working for both the server side and for a Java client generator in Jersey and wadl2java respectively and that will be the topic of my next post. I have been working with Pavel Bucek on the Jersey team on these implementations and the WADL proposal, thanks very much to him for putting up with me.