A little script to update job descriptions on Matrix jobs

We have started to use Matrix jobs in Hudson to try and reduce the amount of clutter on the front page; but we have run into problems in that the job description plugin we used to add the LABEL name to the job id only works for the subordinate tasks. This leaves the top level job without a description.

There are of course other ways you can do this; but since I was playing with the new REST api yesterday, I am going to use this. You could of course use the old RESTy API or indeed the command line tool to perform similar feats. This week I mostly just have a hammer.

Before you use the proxy classes you need to first create a client with the right security credentials configured:

   public static Client createClient() {
       Client client = Client.create();
       client.addFilter(new HTTPBasicAuthFilter("xxxxx@oracle.com","xxxxxxx"));
       return client;
   }

The first part of the job is going to be to create proxies for both the parent project and of the child variant:

   public static void main(String[] args) {

       if (args.length!=2) {
           System.err.println("Expecting two parameters project and variant");
       }

       String project = args[0];
       String variant = project + "%2F" + args[1]; 

       System.out.printf("Will copy missing description from %s to %s\n", variant, project);

       // Create a Jersey client
       Client client = createClient();

       // Create a suitable proxy using the client we configured
       Projects hudsonProjects = projects(client);

       // Handles to the right resources
       Projects.ProjectNameBuilds projectBuilds = hudsonProjects.projectNameBuilds(project);
       Projects.ProjectNameBuilds variantBuilds = hudsonProjects.projectNameBuilds(variant);

Then we can make a quick map of the variant build number to descriptions by getting the first build object:

       // Copy the descriptions out from the variant
       Map<Integer, String> variantToDescription = new HashMap<Integer,String>();
       BuildsDTO variantBuildsDTO = variantBuilds.getAsApplicationXml(BuildsDTO.class);
       for (BuildDTO build : variantBuildsDTO.getBuild()) {
           if (build.getDescription()!=null)
           {
               variantToDescription.put(
                   build.getNumber(), 
                   build.getDescription());
           }
       }

Now it turns out that the current version of the API doesn't allow you to update build descriptions; but we can easily read them using this API and then update them using a simple form POST using the same client.

       // Update the main project descriptions
       BuildsDTO projectBuildsDTO = projectBuilds.getAsApplicationXml(BuildsDTO.class);
       for (BuildDTO build : projectBuildsDTO.getBuild()) {

           String description = build.getDescription();
           // Update description if it has not already been set
           if (description == null || description.length()==0) {

               String variantDesc = variantToDescription.get(build.getNumber());
               if (variantDesc!=null && variantDesc.length()!=0) {

                   // We need to set the description; but nothing in the REST API
                   // so we can use the url property to perform a form submit that will
                   // update the description for us.

                   MultivaluedMap map = new Form();
                   map.add("description", variantDesc);
                   map.add("Submit", "Submit");
                   WebResource buildResource = client.resource(
                       build.getUrl() + "submitDescription");
                   ClientResponse response = buildResource.type(MediaType.APPLICATION_FORM_URLENCODED_TYPE)
                       .post(ClientResponse.class, map);
                   System.out.printf("Updating build %d with description %s gave response %d\n",
                                     build.getNumber(), variantDesc, response.getStatus());

               }
           }
       }

Getting hold of the schemas for the Hudson REST API, and generating a client

I have been playing with the newish REST-like interface in Hudson 2.1.2 and I wanted to generate a client; but as it only uses Jersey 1.5 so it doesn't contain my fix to publish and connect them up directly. Luckily you can just down load these files from the github repository. You ideally want to add these to the wadl description of the service so download the root WADL using a tool such as wget from http://hudson.my.com/rest/application.wadl and then you can add the required imports.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<application xmlns="http://research.sun.com/wadl/2006/10">
    <doc xmlns:jersey="http://jersey.dev.java.net/" jersey:generatedBy="Jersey: 1.5 01/14/2011 12:36 PM"/>
    
    <grammars>
      <include href="build.xsd" />
      <include href="common.xsd" />
      <include href="fault.xsd" />
      <include href="project.xsd" />
    </grammars>
    
    <resources base="http://hudson/rest/">

    ....

Now you can either use the schemas as is and make sure you have annox on your class path when you generate the model classes, or you can find and replace jaxb:extensionBindingPrefixes="xjc annox" with jaxb:extensionBindingPrefixes="xjc".

This will give you a wadl that will generate the interface and the model classes in the same step; we will have to wait for later version of Hudson to connect the types up to the methods. For the moment though if you have a quick look at the WADL it is relatively obvious which types are for which.

/**
 *
 */
@Generated(value =
           { "wadl|file:/tmp/workspaceWorkDir5662539174912846886.abbot/Hudson/Hudson-WADL/src/wadl/application.wadl",
             "run|ca8e07f8-00d8-40ab-bde3-b15cee244091" }, comments = "wadl2java, http://wadl.java.net",
           date = "2011-10-17T14:31:17.818+01:00")
public class Hudson_Rest {


    public static void main(String[] args) {

        // Write out all projects status
        Hudson_Rest.Projects hudsonProjects = projects();
        ProjectsDTO projects = hudsonProjects.getAsApplicationXml(ProjectsDTO.class);
        for (ProjectDTO project : projects.getProject()) {
            System.out.printf("++++ %s +++++\n", project.getName());
            System.out.println(project.getHealth().getDescription());
        }

        // Query the enablement of one particular project
        Hudson_Rest.Projects.ProjectName hudsonAbbotSf = hudsonProjects.projectName("abbot_SF");
        ProjectDTO abbot_SF = hudsonAbbotSf.getAsApplicationJson(ProjectDTO.class);
        System.out.println("Abbot_SF is enabled " + abbot_SF.isEnabled());
   }

   ...
}

So this is a simple read operation, I will look at writing back through this API which will involve configuring the client for authentication in a later installment.

A few simple command line operations on zip files every java developer should know

Java developer spend a lot of time working with zip files it always surprises me that many reach for winzip when some really quite simple command line operation will do. These should all work fine on Unix or using Cygwin, I have consciously used unzip rather than jar as it is more readily available.

First is a quite simple question, does this file file contain a particular file?

unzip -l Example.jar | grep LookingFor.java
or the slightly prettier version with jar:
jar -tf Example.jar | grep LookingFor.java

Second is that you want to look at the content of a file in the zip.

unzip -pq Example.jar META-INF/MANIFEST.MF 

Thirdly you sometimes need to search a bunch of jar files looking for a particular class file: (Thanks a lot to Mark Warner for optimising this for me)

find . -name "*.jar" -type f -exec sh -c 'jar -tf {} | grep "ProtocolHandlerT3"' \; -exec echo ^^^^^^^ in {} \;

Finally the last operation is that you want to be able to compare two zip files, if you have a some UI tools for this then great but this little script will give you a quick command line diff:

#!/bin/bash

FIRST_DIR=`mktemp -d`
SECOND_DIR=`mktemp -d`

echo Unzipping $1 to ${FIRST_DIR}
unzip -q $1 -d ${FIRST_DIR}
echo Unzipping $2 to ${SECOND_DIR}
unzip -q $2 -d ${SECOND_DIR}

diff -r ${FIRST_DIR} ${SECOND_DIR} 

rm -rf ${FIRST_DIR}
rm -rf ${SECOND_DIR}