Thursday, January 05, 2006

Unit Testing Serialization Evolution

Say for example I use session replication in my web application. If I deploy some new code to one server in the cluster to flush out any remaining bugs before a full deployment, I want the old servers to fail over to the new server. And if I decide to take that new server down and implement some more changes, I want its session state to fail over to one of the old servers that are still running. This means the serialized state of objects on the session needs to be compatible in both directions. Here's another example. I'm a JVM vendor. I need to make sure that my implementation of HashMap can deserialize state from an old implementation or another vendor's implementation and vice versa. How do you test this? Cross your fingers? Serialization code can get pretty complex when you start considering compatibility. If you only need backward compatability, it's easy. Check out the old code, write some serialized state to a file, and then write tests that deserialize it using the new code. But what do you do when you need compatability in both directions? The following method spoofs the class name when serializing an object enabling you to deserialize it as a different type:
  public static <S> S serializeAndDeserialize(Object o, 
      Class<S> spoofedType)
      throws IOException {
    final String oldName = o.getClass().getName();
    final String newName = spoofedType.getName();
    ByteArrayOutputStream bout = new ByteArrayOutputStream();
    ObjectOutputStream oout = new ObjectOutputStream(bout) {
      public void writeUTF(String s) throws IOException {
        super.writeUTF(s == oldName ? newName : s);
    ByteArrayInputStream bin = new ByteArrayInputStream(
    ObjectInputStream oin = new ObjectInputStream(bin);
    try {
      return spoofedType.cast(oin.readObject());
    } catch (ClassNotFoundException e) {
      throw new RuntimeException(e);
If our class name is Foo, we can copy the old version into our test directory and rename it OldFoo. Now we can create tests that create an OldFoo and serialize and deserialize it using our new method into a Foo and vice versa. Now we can evolve classes with both ease and confidence.


Blogger swankjesse said...

In my project, we actually use a binary dump of a serialized Object to verify compatibility:

public void testVersion20051003() throws IOException, ClassNotFoundException {
byte[] serializedBytes = new byte[] { .... };
Object expected = ....
Object deserialized = GlazedListsTests.fromBytes(serializedBytes);
assertEquals(expected, deserialized);

9:00 AM  
Blogger Bob Lee said...

Yeah, I think a lot of people already do that. Like I said, "If you only need backward compatability, it's easy. Check out the old code, write some serialized state to a file, and then write tests that deserialize it using the new code."

This doesn't work for testing compatability in both directions though. And producing the binary dump for each case can become a pain. If you decide you want to go back and change some state, you have to check out the old code and reproduce the binary dump. With my approach, you simply change the code that creates the old object and rerun your test.

9:42 AM  
Blogger brian said...

But are we not imposing a restriction that the evolved version should essentially retain the variables, their types and order as present in the old version ?

11:35 PM  
Blogger Bob Lee said...

Brian, we definitely are not. This comes in handy when you make changes that break default compatibility and have to write and test code that manually maintains compatibility, i.e. using ObjectStreamField, GetField, PutField, optional data, etc. (BTW, ordering doesn't impact compatibility.)

11:42 PM  

Post a Comment

Links to this post:

Create a Link

<< Home