Testing time-sensitive logic in Java


It's astounding. Time is... fleeting (The Rocky Horror Picture Show)

Testing time-sensitive business logic is essentially about being able to change the current time in our tests - somehow - and then checking how this affects the behaviour of the domain object being tested.

The primitive and brute-force way to do this is to manipulate the computer system clock by manually changing the current time prior to each test...  Crucially this approach does not lend itself to running as part of an automated test suite, for obvious reasons.

The other (better) way is use two different clocks: the production code can rely on the system clock while the tests code will depend on a custom clock, i.e is a clock which can be setup to return any particular time as the current time. Usually this custom clock will expose methods to advance/rewind the clock to specific points in time. 


interface clock{
   public DateTime now();
}

class SystemClock implements Clock{
   public DateTime now(){
     return new DateTime();
   }
}

class CustomClock implements Clock{
    private DateTime now;
    public CustomClock(DateTime now){
       this.now = now;
    }
    public DateTime now(){
      return now;
    }
    public void tick(){
      now = now.plusDays(1);
    }
}

Both clocks realize the "now()" method defined in the Clock interface. The difference being that the now() method from SystemClock is a simple wrapper around a new instance of a Joda dateTime instance, while the now() method from the CustomClock returns a dateTime attribute which can be modified through the tick() method to make time pass faster :)

The custom clock will be injected as a dependency of the testing code and the system clock as a dependency of the production code. For a (somewhat contrived) example of how this plays out check out: https://github.com/eleco/customclock

First steps with... Cucumber JVM

Cucumber is a tool used to support behaviour driven development. Originally written in Ruby there is now a JVM version, called, quite logically, Cucumber-JVM

The basic idea (very summarized...) is to write acceptance tests for a new feature, together with the product owner, before the code is written. Then run the tests, see the tests fail, implement the missing behaviour, re-run the tests etc.. until the tests pass. 

The key objective here is to involve the product owners as much as possible in writing the tests, which from experience can be tricky as they do not generally have a technical background. So it's important for an acceptance tests framework to generate tests with a syntax which is a close as possible to a natural language. 

Cucumber achieves this quite well, see below for an example Cucumber script (click to zoom in)


The right panel defines the tests scripts to execute, easily understandable by non-technical people. No messing around with HTML either, a big win compared with alternative frameworks such as Fitnesse or Concordion.

The left panel maps the tests scripts to their associated junit tests. Full code source for this example is at: https://github.com/eleco/bdd


Cucumber outputs the tests results in a nicely-formatted page like so.





Deserializing/serializing JSON with GWT

A fairly typical use case: a GWT server-side component serializes java objects into Json, to be consumed by GWT clients.

Server-side serialization is actually very easy thanks to the google-gson library. It's literally a one-liner:
String json =new Gson().toJson(myObject);

On the client side things are a little bit more complicated. Gson (or any other library with features which are not supported on a GWT client such as reflection, dynamic class-loading, multithreading... ) cannot be used.

One possible alternative is to use autobeans

To deserialize a Person class
class Person {
   private String name;
   public String getName();
   public void setName(String s);
}

1. define an interface for the class to deserialize
interface IPerson {
   public String getName();
   public void setName(String s);
]

2. Mark the class to deserialize as implementing the above interface
class Person implements IPerson {...}

3. Define the interface extending AutoBeanFactory
interface Beanery extends AutoBeanFactory{  
   AutoBean <IPerson> createBean();
}

4. Instantiate the bean factory and deserialize
 
Beanery beanFactory = GWT.create(Beanery.class);
IPerson person = AutoBeanCodex.decode(beanFactory, IPerson.class, json).as();


Infinite scroll with GWT

The classical pagination pattern leads the user to click on numbered icons to navigate in a long list, each icon being associated with a different page in the list. This is how you browse google search results for instance. This is optimal when moving from page 1 to page 999, but not so much when moving from page to page in sequential order, which is the most frequent use case.

The alternative is the "infinite scroll" technique. Here the application detects when the user scrolls down to the bottom of the list, and automatically adds the result of the next page to the list. Scrolling can go on this way for as long as results are available to be added to the list, hence the term "infinite scroll". Obviously it's infinite in theory only... but I guess "very long scroll" doesnt have quite the same ring to it.

Building an infinite scroll component withb GWT is easy.... because it's already been done. Check out the GWT showcase  for an example. Or have a look at my side-project javadevjobs.com .

The key is in the ShowMorePagerPanel class (source code available from the showcase page) which listens for scroll events and  increases the display range when the it detects that the scrollbar has nearly reached its bottom position. This triggers in turn a rangeChangeEvent which acts as cue for the dataProvider to go and fetch more records from the database.











volatile piggybacking


Volatilty piggybacking is the (dubious ?) technique which attributes volatile-like semantics to non-volatile variables.

The new memory model , from java 5, states that "a write to a volatile field (§8.3.1.4) happens-before every subsequent read of that field".

i.e writing to a volatile field creates a memory fence, which will flush the data held in memory cache, so that anything visible to the thread writing to a volatile field becomes visible to any other thread reading that same field. 

The "anything" in the sentence above might be a non-volatile variable - which will end up being visible to all threads in the same way the volatile which initially triggered the memory fence is. Thus the former piggybacks on the memory flush triggered by the later.


Examples of this technique in core jdk classes are hard to come by (ie. I havent found any...) . Possibly a reflection on how fragile that technique is. 


First steps with GWT Bootstrap


Gwt-Bootstrap is the port for Gwt of the Twitter Bootstrap framework. Twitter Bootstrap  defines a set of javascript and css components which are used to kickstart the development of websites (at least on the client-side).

The idea is to make it easy to develop a reasonably-good looking website without too much effort. Although of course a truly polished result will require additional customisation work on top of the framework. Twitter bootstrap also provides advanced features out-of-the-box, eg. responsive design (the components size is automatically adjusted function of the resolution of the device they're being drawn on to). 


How to use.

The best way to learn is to download the Gwt-bootstrap sources from Github and study the examples provided... but to sum up:

- Grab the GWT-bootstrap jar , version 2.0.3.0-SNAPSHOT at the time of writing.

- Add the jar to your project build path (menu File->Properties->Java Build Path in Eclipse)

- In the gwt.xml config file add a reference to the gwt-bootstrap library.
 <inherits name ="com.github.gwtbootstrap.Bootstrap"/>

- Assuming you're using UI binder then add the following namespace to the <ui:UiBinder> element.
xmlns:b="urn:import:com.github.gwtbootstrap.client.ui"

- the bootstrap components are now ready to be used:
<b:heading size="2">Hello World</b:heading>
    

Downsides.

- Gwt-Bootstrap it's still a work in progress. Not all Gwt widgets have been ported yet, eg.CellTable is missing at the time of writing.

- Most sites built with bootstrap tend to look a little bit similar. Greyish tones, the top navigation bar and the "Hero" unit underneath are the usual dead giveaways.


Examples.

Builtwithbootstrap has an extensive collection of websites leveraging Twitter bootstrap.

For Gwt-bootstrap specific websites - the main reference is the gwt-bootstrap showcase.


Edit 1: my side-project also runs gwt-bootstrap. Check it out.

Edit 2: one of the GWT-bootstrap committer copies most of the above post, without attribution. Now - as the saying goes imitation is the most sincere form of flattery but still that's another point in the downsides section - dubious ethics from (some) of the developers on this project.





Comparing csv files with the linux shell


My heart sinks whenever I witness people resorting to Excel to compare large files. It's kind of ok when the files to be compared are below 10K rows in size... anything bigger than that and the time it takes to select the rows to compare (and the comparison in itself) becomes too much of a frustration.

The alternative is to use the command line of course.

To compare two csv files file1.csv and file2.csv, columns to columns, on a Linux operating system.

step 1. copy all lines containing the string to search from the first input file
grep 'searchString' file1.csv > f1.csv


step 2. extract the relevant columns (here columns 3,4 and 5)
cut -d',' -f3,4,5 f1.csv  > cols_f1.csv


step 3. sort on the 2nd column (for example)
sort -k2  -t"," cols_f1.csv > sorted_f1.csv


step 4. remove duplicates
uniq sorted_f1.csv > uniq_f1.csv




Quite a bit of typing here... and that's only to extract the columns from the first file. Fortunately all of these commands can be piped.

steps 1,2,3 and 4 for the second file.

grep 'searchString' file2.csv | cut -d',' -f3,4,5 | sort -k2 -t"," | uniq > uniq_f2.csv



And finally the last step.  Show all lines unique to file1, all lines unique to file2 and all lines common to both files, arranged in a 3-columns output

comm uniq_f1.csv uniq_f2.csv

The Guava Joiner - an example


Google Guava ships a nice utility to transform a list of strings into a single string, with invidual elements separated by a separator character.

import com.google.common.base.Joiner;
List words = newArrayList("abc","def","ghj");
System.out.println(Joiner.on(",").join(words));

is a replacement for the rather verbose:

List words = newArrayList("abc","def","ghj");
StringBuilder sb = new StringBuilder();
Iterator iterator = words.iterator();
if (iterator.hasNext()) {
   sb.append(iterator.next());
   while (iterator.hasNext()) {
      sb.append(","); 
      sb.append(iterator.next());
   }
}
System.out.println(sb);

Unsprung! moving away from Spring DI


Advantages of Spring (when used for dependency injection):

- separates configuration from the code. Thus the wiring of the application can be modified without recompiling, in theory. Not sure how often this happens in practice.

... and the cons:

-   Xml config files are easy to get wrong (no compile time checks obviously)
- Annotations spread through out the code are not terribly visible
- Error messages thrown by the framework can be cryptic at times

There is a simpler alternative. Do away with the Spring container and inject the dependencies manually. All that is needed is a class to inject the dependency, a context which creates the appropriate dependency, and a bit of application code to wire the two together.


A DbReader uses a constructor-injected datasource to retrieve database results.
class DbReader{

  DataSource dataSource;

   DbReader (DataSource dataSource){
     this.dataSource= dataSource;
   }

   public  Object fetch(){
    //use the data source to execute a database query
   }

A Context  holds a reference to a data source (mock or real)
class Context {

  DataSource dataSource;

  static Context liveContext(){
     //get production db data source
     DataSource dataSource = ...;
     return new Context(dataSource);
  }

  static Context mockContext(){
     // get in memory test db data source (or mock)
     DataSource dataSource = ...; 
     return new Context(dataSource);
  }

  Context(DataSource dataSource){
      this.dataSource = dataSource;
  }

  DataSource getDataSource(){
     return dataSource;
  }

}


Project classes inject the datasource associated with a live context in the DbReader constructor.

   public static void main(String args[]){
      Context ctx = Context.liveContext();
      //fetch data from a prod database
      Object result = new DbReader(ctx.getDataSource()).fetch();
   }


while integration tests inject a mock datasource from a mock context.:

   @Test
   public void Test(){
     Context ctx = Context.mockContext();
     //fetch data from  a mock or in-memory db
     Object result = new DbReader(ctx.getDataSource()).fetch();
     //assert that result is as expected...
    }


Simple, easy to understand (and to debug), compile-time checks in place, no messing around with xml configuration files (or annotations).

...and it's even simpler with the Unsprung project which can help generate the Context class above from a Spring configuration file.

Test infrastructure, part 3 - Matching with Hamcrest

A word on Hamcrest, a library which integrates with JUnit to improve test readability.

When checking if two objects obj1 and obj2 are equals, the "classic" JUnit way is to write something like:

import org.junit.Test;

class Test

   @Test
   public void equalsTest(){  
  
      assertEquals(ob1, obj2);
      //is obj1 the expected or actual result ? 
      //not clear without resorting to the javadoc 
   }
}

with Hamcrest - the test reads (almost) like plain ordinary english and it is easier to see that obj1 is the actual result and obj2 is the expected result:

import org.junit.Test;
import static org.hamcrest.MatcherAssert.assertThat; 
import static org.hamcrest.Matchers.is;

class Test{

   @Test
   public void equalsTest(){
      //asserts that the actual result is what's expected.
      assertThat (obj1, is(obj2));
   }
}


The above illustrates the use of the is() matcher, there are many other matchers to choose from... Just to name a few:

//hasEntry matcher
Map aMap = new HashMap ();
amap.put("test",123);
assertThat (aMap, hasEntry("test",123);

//greaterThan matcher
assertThat(stuff.size(), greaterThan(0));

//combining is and not matchers
assertThat (stuff.size(), is(not(0)))x

//assert that an array contains certain elements
String [] array1 =....;
assertThat (array1, allOf(hasItemInArray("element1"), hasItemInArray("element2")));

Test infrastructure, part 2 - randomizing data

The builder created in part 1 initialises the domain object under test with predefined values:

 id {123} and name {defaultName}

public class DomainObjectBuilder{
   private Integer id="123";
   private String name="defaultName";

   public DomainObject build(){
      return new CustomObject(id,name);
   }
   ....
}

The objects thus constructed by the builder are then used in tests. So far so good. Sometimes though we want these tests to work a little harder, and prove that they can run ok with different input values. The methods below can be used to that end, building on the RandomStringUtils and RandomUtils methods from Apache Commons.

eg. to randomise a string by adding a 2 digits to the end of it.
public String randomizedString(String aString){
   return aString+RandomStringUtils.randomAlphaNumeric(2);
}
and the builder becomes (assuming the randomizedString method is statically imported)
public class DomainObjectBuilder{
   private Integer id=randomizedString("123");
   private String name=randomizedString("defaultName");

   public DomainObject build(){
      return new CustomObject(id,name);
   }
   ....
}
Each invocation of the DomainOBjectBuilder.build() method will now return domain objects with slightly different values. Of course it's not only strings which can be randomized. The same principles applies to enums:
public static <T extends Enum> T randomizedEnum (class <T> enumClass){
   T[] enumConstants = enumClass.getEnumConstants();
  return enumConstants[RandomUtils.nextInt(enumConstants.length);
}
... and to lists.
public static <T>  T randomElementFromList(List aList){
   return aList.get(RandomUtils.nextInt(aList.size()));   
}

Test infrastructure, part 1 - building the builders

The basic brick of a test infrastructure is the ability to easily instantiate and set properties on the domain object under test. This can be achieved with a builder object, as per below. This builder exposes a fluent interface (in the chaining of methods withId and withName) to improve readibility.


Given a domain object such as

public class DomainObject{

   public DomainObject (Integer id, String name){
     this.id = id;
     this.name = name;
   }

   public Object doSomething(){
      ...
   }
}

then the associate builder will be:
public class DomainObjectBuilder{
   private Integer id="123";
   private String name="defaultName";

   public DomainObject build(){
      return new CustomObject(id,name);
   }

   public DomainObjectBuilder withId(Integer anId){
      this.id = anId;
      return this;
   }

   public DomainObjectBuilder withName(String name){
      this.name = name;
      return name;
   }
}
and a test for a DomainObject will look like:

DomainObject anObject = new DomainObjectBuilder()
.withId("345").withName("someTest").build();
assertThat (anObject.doSomething, is(anExpectedResult));

An alternative approach would be to do without a builder and simply add setters on the domain object, setters which would be invoked during the tests. The major drawback of this technique is that adding setters breaks immutability.

Why I (finally) moved from Eclipse to Intellij



I had to keep both Eclipse and Intellij running side-by-side for a while, because I could not peel away from Eclipse, my editor of choice for years. A few days later and Intellij has eclipsed Eclipse (pardon the pun). Here' s a few reasons why:


- I't s easy to switch between implementation and test (CTRL+shift+T) in Intellij, there's no equivalent in Eclipse.

- When searching for the usages of a variable/method/class Intellij scopes the search by test files, production files (or both).  In Eclipse you would have to manually specify the full path to search in each case.

- In general Intellij is pretty smart when it comes to refactoring. for instance when renaming a class attribute  the name of the getter method for that class attribute will  be renamed accordingly. 

- Smart code completion: Intellij will make much better suggestions as to what expression type is expected based on the context than Eclipse.

- automatically resolution of import static.
  
- Paste from history

- Use regex in search

- On the downside Intellij is a tad slower than Eclipse and tends to lock up for 30 secs or so once or twice a day.

So overall there's no major functionality or killer feature offered by Intellij that Eclipse cannot replicate. But the difference is in the many little details which are often better thought out in Intellij, resulting in a better experience (and increased productivity).


Regexp use for text transformation with Intellij

Given a snippet of code such as:
put("item123", x);
put("item456", x);
....
put("item888", x);


                
To retrieve the content in between quotes:
item123
item456
...
item888


Then this can be achieved using the in-place find and replace functionality offered by Intellij.
(obviously a manual edit would also work, but might not be as efficient if the block of code is repeated 1000 times).


1. search for the regex put\("


2. replace the selection with an empty field, this yields:

item123", x);
item456", x);
....
item888", x);


3. search for the regex \".+  (select everything extending from the first quote to the end of the line)


4.replace the selection with an empty string yields: 

item123
item456
....
item888