Friday, November 30, 2018

In JDK 9 (and well 8) and above everything can be a stream.

In JDK 8 we finally got to use streams and everything was good apart from the times when the API you used couldn't produce a stream. Then you ended up writing a wrapper class method that allowed you to convert an iterator into a Stream, because you missed streams.

public static <T> Stream<T> asStream(Iterator<T> it) {
    return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it,
        Spliterator.IMMUTABLE | Spliterator.ORDERED),false);
}

Now there are methods to programmatically generate stream in the case of iterate and generate but both of these generate an infinite stream whereas in most cases you really wanted to adapt an existing interface into a finite Stream.

This was resolved nicely in JDK 9 with the introduction of a new form of iterate method that allows you to provide a predicate to signal the end of the stream.

In the examples below I am going to use a predicate that continues until you get a null entry to the stream, I will leave it up to the reader to come up with more imaginative uses for predicate. In this simple example I am using the getCause method of Throwable to move us along a linked list of errors. Note how little code this would take when compared with a pre stream version.

// Simple linked list
//
Exception e = new Exception("one");
Exception e2 = new Exception("two",e);
Exception e3 = new Exception("three", e2);

Stream.iterate(e3, Objects::nonNull, Throwable::getCause)

    // Output the messages in turn
    .map(Throwable::getMessage)
    .forEach(System.out::println);

The second example converts a ReferenceQueue into a Stream so that we can easily drain its contents for processing as required. This code is a little bit different because the container is different from the object be worked on, so we provide the seed and the next value using the same method, This returns null when the queue becomes empty.

ReferenceQueue<Thing> queue = new ReferenceQueue<>();

// Make some things and then collect them
WeakReference one = new WeakReference<Thing>(new Thing(), queue);
WeakReference two = new WeakReference<Thing>(new Thing(), queue);
System.gc(); System.gc(); System.gc(); System.gc(); System.gc();

Stream.<Reference<? extends Thing>>iterate(
    queue.poll(), Objects::nonNull, v -> queue.poll())


    .forEach(System.out::println);

The third example shows a walk over a Node tree, note the nested stream iterator to work back up the list when we have worked to the end of a leaf.

Node root = doc.getDocumentElement();

Stream.iterate(
    root,
    Objects::nonNull,
    v -> {
        if (v.getFirstChild()!=null) {
            return v.getFirstChild();
        }

        if (v.getNextSibling()!=null) {
            return v.getNextSibling();
        }

        return Stream.iterate(v, Objects::nonNull, Node::getParentNode)
            .filter(node -> node.getNextSibling()!=null)
            .map(Node::getNextSibling).findFirst().orElse(null);
    })

    .map(Node::getNodeName)
    .forEach(System.out::println);


So with a little bit of mental gymnastics it is possible to transform most legacy APIs into a nice clean Stream, so you can ignore those nasty old fashioned for loops. And if you are stuck in JDK 8 then it is quite easy to put together a similar function using the asStream from before:

public static<T> Stream<T> iterateFinite(T seed, Predicate<? super T> hasNext, UnaryOperator<T> next) {

    return asStream(new Iterator<>() {

        T current = seed;

        @Override
        public boolean hasNext() {
            return hasNext.test(current);
        }

        @Override
        public T next() {
            if (current == null) {
                throw new NoSuchElementException();
            }
            try {
                return current;
            } finally {
                current = next.apply(current);
            }
        }
    });
}


Tuesday, November 27, 2018

await in turn by mistake

I really enjoy working with the new async functions but it is really easy to set up a situation where code that could be running in parallel is force to run in sequence. Consider this simple invocation of a javascript function that takes two values returned from other async functions:

   const combination = await combine(await value(1), await value(2));

The problem here is that unless the JS environment does some optimisation for you, the actions contained in the two called to await are performed in series.

function action() {
   return new Promise(resolve => {
       setTimeout(resolve, 5000)
   })
}

async function value(v) {
   console.log('started ' + v);
   return action('broken ' + v).then(()=> {
      console.log("finished " + v);
      return "ok "  + v
   })
}

async function test() {
 console.log(await value('1') + " " + await value('2'));
}
test();

This will result in an output that look something like this, and will take about ten seconds:

(index):39 started 1
(index):39 finished 1
(index):37 started 2
(index):39 finished 2
(index):44 ok 1 ok 2

So there are a number of way to re-write this code in order to ensure that the original processes are all started at the same time, I think the second is the best form, just wish they had put some syntactic sugar so we could do away with Promise.all references.

async function test() {
  // Worst
  const three = value('3');
  const four = value('4');
  console.log(await three + " " + await four);

  // Better
  const [five, six] = await Promise.all([value('5'), value('6')]; 
  console.log(five + " " + six));

  // Better?
  console.log(...await Promise.all([value('7'), value('8')]))
}
test();

So you would expect a test output to be similar to this, with each block taking about the minimum 5 seconds.

(index):37 started 3
(index):37 started 4
(index):39 finished 3
(index):39 finished 4
(index):47 ok 3 ok 4
(index):37 started 5
(index):37 started 6
(index):39 finished 5
(index):39 finished 6
(index):48 ok 5 ok 6
(index):37 started 7
(index):37 started 8
(index):39 finished 7
(index):39 finished 8
(index):48 ok 7 ok 8

It is really easy to make the same mistake when writing a for loop for example by waiting on each item in turn

async function test() {

  const list = ['1', '2', '3'];
  const result = [];
  for (const item of list) {
     result.push(await value(item));
  }

  console.log(result);
}

For most operation that involve Promise and loop you will normally need map/Promise.all at some point. This version should complete in around the minimum 5 seconds.

async function test() {

  const list = ['1', '2', '3'];
  const result = await Promise.all(list.map(value));

  console.log(result);
}

Friday, October 5, 2018

Understanding ordering with JavaScript async methods

So I challenged someone in a code review to prove that there code that made use of async functions wouldn't be susceptible to a race condition. To that end I came up with a very trivial code example to demonstrate the issue, worth trying to write down the output and line orderings before you read on.

let list;

async function clearList () {
    list = []; // A
}

async function processList (processList) {
   await clearList(); // B
   list = list.concat(processList); // C
}

processList([1,2,3]); // D
processList([4,5,6]) // E
   .then(() => {
      console.dir(list); // F
   }) 

So the two questions here are what is the output of this code and what order do the lines of code get executed in. Now the point of the code was that it would show that the output is [1,2,3,4,5,6] because of the race condition; but the actual ordering of the execution of the lines of code I had wrong.

My assumption was that you would not enter the async method directly and instead be queued to be performed on a later clock tick. This gave me an execution flow of D,E,B,B,A,A,C,C,F which I was happy with until my coworker Millan Kuchtiak pointed out I was entirely incorrectly.

It turns out that at least in Chrome and Safari that code doesn't get transferred queue until you reach the first await call. So actually when you run the code through with a debugger the flow is D,B,A,E,B,A,C,C,F. This make sense from a performance point of view, some async methods might never need the change of context, so just in time async.

To summarise an async method is synchronous up until the first await

Wednesday, August 8, 2018

Trusting a self signed certificate in Oracle Visual Builder Cloud

When you are setting up a development environment you sometimes need to use self signed certificates or a separate developer CA, in order to use this in VBCS you will need to make sure that these certificates are trusted by the kss://system/trust store and have the latest 18.3.3 patches.

To do this you need to head to /em aka "Fusion Middleware Control" for your instance and navigate to the keystore page for the domain.



You then need to navigate to the system/trust keystore and select manage:



The simply import your certificate, it need to be in .pem format so you might need to do some conversion depending on what you have lying about (Note if you dragged and dropped the certificate from Chrome you can just convert using openssl x509 -inform der -in certificate.cer -out certificate.pem):


This should now be trusted, you may have to restart the Breeze war in order to see the changes take effect.

Wednesday, January 31, 2018

Quick Debugger Tip, where was that class loaded from.

Often when it is Friday afternoon and you want to go home early you come across an intractable class loading issue. You might grep your application server home, examine all the domains you can find. (I was looking at weblogic) And still you cannot find out where the class is coming from.

Now you can use the debugger in your IDE of choice to get hold of the class loader; but in this case the parent class loader had upwards of 300 jar files in it so this was going to be a pain to be sure. So it occurred there was a quicker way, just use the standard method getResource to ask the class loader which jar the file was coming from:


Turned out someone has installed SOA on this particular weblogic install, and all the relevant files were in a different directory root: this is why my original grep didn't find the jar files.

Thursday, June 15, 2017

Interpreting wpad.dat using jrunscript from JDK 8 to work out the right proxy.

In many environments you find that the correct proxy can only be fetched by parsing http://wpad/wpad.dat, which is a pain because it is actually a JavaScript file. Working out the right proxy to call for say a unix environment is a bit of a fiddle particularly if you company is huge and requires different proxies for different countries. JDK 8 to the rescue, as you can run a simple script in the Nashorn environment to get the suggested proxy for one host and use that for everything:

jrunscript -e "`curl -o - -s http://wpad/wpad.dat` function isPlainHostName(hostname) { return hostname.indexOf('.')==-1;}; var proxyList = FindProxyForURL('http://github.com','github.com'); var firstProxy = proxyList.split(';')[0].trim(); print('DIRECT'.equals(firstProxy) ?  '' : 'http://' + firstProxy.split(' ')[1]);"

Depending on your local environment you might have to implement one or more functions from the list that your wpad.dat expects, in our case I needed to implement isPlainHost.

The last part of the script just picks the first proxy name, as the function returns a list of PROXY is suggested order. The raw output looks like this:

PROXY london.example.com:80; PROXY sf.example.com:80; DIRECT;

So we just pick the first one with http as the prefix, as some tools such as docker are fussy about this, or returns an empty string if DIRECT is recommended.

http://london.example.com:80

Tuesday, March 14, 2017

The "debugger" reserved word in JavaScript

This is a handy little tip for cases where you just can't get your debugger to start in the right context or the framework you are using doesn't give you a clear path to know when you file might be loaded. (For example ABCS, or Application Builder Cloud Service, that I am currently working on has this problem)

In this case just add the debugger statement in your javascript and voila when run in a browser the debugger is popped up.

   someCode().then(value => {
       debugger;
   });


Always pays to read the reserved word list when learning a new language, you never know what you might find.