Comparing 2 images for equality

<accesscontrol>Main:MyGroup</accesscontrol>

Introduction

At the time of writing this page, we are testing the affection of changing stylesheet manually. This is not a good way of testing, because there are more than 2000 settings in stylesheet, and testers simply can't re-test over and over again. So, we come to a decision of making some automatic testing tool that can assert the correctness of diagrams.

However, there are charts that are generated as images in report system. Comparing 2 images pixel by pixel is not a bad idea in terms of at least having something that make sure our system is still working. But the image comparing have limitation in pointing out exactly the differences (in label, in color etc...)

Then it comes to SVG image format as a good candidate. SVG stands for Scalable Vector Graphic which stores image in XML format. Thus, comparing 2 images becoming comparing 2 XML files. SVG image can be viewed using IE with a plugin from Adobe call SVGViewer

Note: there is a limitation with SVG support in Nevron, at the time of writing (05.2009) CatGlobe system is still using a very old version of Neveron (Q4-2007) which does not support for SVG very well, almost diagrams do not work well in SVG mode. However, the newest version (Q1-2009) supports SVG rather well. SVG format in itself contains a limitation that it cannot display 3D images. As conclusion, only develop testing with this article with Q1-2009 or newer version of Nevron, and only test on 2D chart.

Nevron chart and SVG

Generating chart image to SVG with Nevron chart control (the main control that we are using for generating chart image) is as easy as below code;

            NSvgImageFormat svgImageFormat = new NSvgImageFormat();
            svgImageFormat.Compress = false;
            nChartControl1.CreateImage(svgImageFormat).SaveToFile(@"D:\svn" + DateTime.Now.ToString("ddMMyyyy_hhmmss")+".svg", new NSvgImageFormat());

XML Comparision

Microsoft provides free tools called XMLDiffPatch for comparing/synchronizing 2 XMLs.

In this download, you can also find an XmlDiffPatch.View which process the diff result and generate HTML report. Which looks something like this

NUnit XML Assertion

The implementation of the XML assertion can be found under CatGlobe\Components\CatGlobe.NUnit.Addins.

There are a couple of methods for asserting the equality of XML (stream, fragment, file) as below:

   public class XmlAssert
   {
      /// <summary>
      /// Assert the 2 XML stream for equality.
      /// </summary>
      /// <param name="expectedXmlStream">Expected XML Stream</param>
      /// <param name="actualXmlStream">Actual XML Stream</param>
      public static void AreEqual(Stream expectedXmlStream, Stream actualXmlStream);

      /// <summary>
      /// Assert the 2 XML stream for equality.
      /// </summary>
      /// <param name="expectedXmlStream">Expected XML Stream</param>
      /// <param name="actualXmlStream">Actual XML Stream</param>
      /// <param name="message">Customer display message when the 2 streams are not equal</param>
      /// <param name="args">Arguments use for formatting the <paramref name="message"/></param>
      public static void AreEqual(Stream expectedXmlStream, Stream actualXmlStream, string message, params object[] args);

      /// <summary>
      /// Assert the 2 XML fragments for equality.
      /// </summary>
      /// <param name="expectedXmlStream">Expected XML fragment</param>
      /// <param name="actualXmlStream">Actual XML fragment</param>
      public static void AreEqual(string expectedXml, string actualXml);

      /// <summary>
      /// Assert the 2 XML fragments for equality.
      /// </summary>
      /// <param name="expectedXmlStream">Expected XML fragment</param>
      /// <param name="actualXmlStream">Actual XML fragment</param>
      /// <param name="message">Customer display message when the 2 fragments are not equal</param>
      /// <param name="args">Arguments use for formatting the <paramref name="message"/></param>
      public static void AreEqual(string expectedXml, string actualXml, string message, params object[] args);

      /// <summary>
      /// Assert the 2 XML file for equality.
      /// </summary>
      /// <param name="expectedXml">Expected XML file</param>
      /// <param name="actualXml">Actual XML file</param>
      public static void AreEqualXmlFile(string expectedXml, string actualXml);

      /// <summary>
      /// Assert the 2 XML file for equality.
      /// </summary>
      /// <param name="expectedXml">Expected XML file</param>
      /// <param name="actualXml">Actual XML file</param>
      /// <param name="message">Customer display message when the 2 files are not equal</param>
      /// <param name="args">Arguments use for formatting the <paramref name="message"/></param>
      public static void AreEqualXmlFile(string expectedXml, string actualXml, string message, params object[] args);
   }

The asserts are used as any other normal assert methods. See code below:

         XmlAssert.AreEqual("<test />", "<test>fasdfd</test>");
         XmlAssert.AreEqualXmlFile(@"D:\svn09042009_045701.svg", @"D:\svn09042009_045701_new.svg");

Of course, references to assemblies (CatGlobe.NUnit.Addins.dll, XmlDiffPatch.dll, XmlDiffPath.View.dll) must be added to the unit test project. The assertion result is displayed on screen like this:

The HTML report is generate using XmlDiffPatch.View.dll would looking something like the first image in this article. There is a setting in .config file for shutting down the report generation (in case, we just want to know if there is an error and don't care about what exactly are the errors). This can save some speed for running the unit test.

      <!-- For XML Comparision -->
      <add key="GenerateXMLComparisionReport" value="false" />

Possible optimization

For HTML report, there are possible optimization as

1. Add jump to next different.
2. Display line number.
3. Skip same parts with some text "Skip xxxx lines that are the same." to reduce HTML file size and speed up comparision.