Difference between revisions of "New Report Design - 2009"

From Catglobe Wiki
Jump to: navigation, search
 
(34 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
<accesscontrol>Main:MyGroup</accesscontrol>
 +
[[Category:Miscellaneous]]
 
== Introduction ==
 
== Introduction ==
−
The new report design targets at;
+
[[Image:ReportEngine.jpg]]
−
* Remove XML processing during report generation
 
−
* Separate information to different layers for increasing re-usability
 
−
* Eliminate code duplication
 
−
* Move some calculation to diagram level instead of letting the CGScript engine do all the works
 
  
−
<i><u>''' This page is written at the time of implementing ''VN2560REP - Implement single case of CDG'', thus it focuses on Cross-Diagram and table chart only. And the design can be found here: \\catproc\Share\CatGlobe Teams\R & D\Documents\Designs\CatGlobe.Report.VN2560REP.eap.'''</u></i>
+
The components/modules that are using Report engine are: Dashboard, Monitor, Tabulation Script, Report Viewer, Diagram Viewer. These modules send request of generation to report engine including:
 +
* A data cache which contains all source data
 +
* Stylesheets (including external and inline stylesheet) to apply
 +
* Kind of presentation that the report should be exported as (pdf, excel, ppt, word, html, image)
 +
* Other kinds of settings
  
−
== General Report Generation Process ==
+
The report engine takes the request, make calculation using the Interpreter, and then export the result into desired format.
  
−
[[Image:Diagram_-_Generating_from_HttpRequest.gif]]
+
A report consists of elements like: diagrams, images, text blocks, pages, paragraphs. The most important element is Diagram, which can be discriminated based on:
  
−
* The input for the whole process is a ''DiagramInfo'' which contain all necessary information for generating a diagram
+
* Data
−
* From the ''DiagramInfo'', through the ''Diagram'' class, the ''Chart'' object is built. ''Chart'' object contain physical data needed for generating a chart image/table
+
** Cross Diagram
−
* As the last step, the ''Chart'' object become input to a ''ChartBuilder'' together with layout information (content width, stylesheet) the chart image/table is generated to different desired formats (pdf, ppt, html, excel)
+
** Standard Diagram
 +
** Tracking Diagram
 +
** Campaign Diagram
  
−
We can consider the processing as a 3 phases processes in which the result of the first 2 phases can be stored as XML format for later use. More precisely, the classes ''DiagramInfo'' and ''Chart'' are '''SERIALIABLE'''.
+
* Presentation type
 +
** Table chart
 +
** Image chart which include many kinds of chart like: pie, bar, line, funnel, area, radar etc.
  
−
== Detail Design ==
+
== Why a new system? ==
−
In detail there are 2 different kinds of diagram: table and image. However, for the sake of using, the detail must be hind from developers. However, always keep in mind that there are 2 kinds of diagrams which have different processing.
 
  
−
=== Phase 1: Generate ''DiagramInfo'' ===
+
=== Current system drawbacks ===
 +
* Heavy XML processing
 +
* Heavy code duplication
 +
* Inconsistent generation process
  
−
[[Image:Diagram_-_Build_DiagramInfo_from_request.gif]]
+
Thus it makes the cost of fixing bugs and maintain the old system greater than the cost of making an new system.
  
−
For each kind of diagram (Cross, Campaign, Standard, Tracking), there is a correspondence info class. But all of them inherit from the parent ''DiagramInfo'' class which contains common information like ''DiagramType'' and ''DiagramFilterInfo''.
+
=== New system goals ===
 +
* Remove the drawbacks of current system
 +
* Side by side working with the old system
 +
* Fully unit testing
 +
* Unit/Integration tests can be run automatically
 +
* Testing API must be simple enough for testers to join writing test cases
 +
* Report API must be keep as simplest as possible
 +
* Design documents must be officially registered on wiki page
  
−
''DiagramInfo'' also contains information about layout of a diagram. For example: which series should be displayed - ''SeriesDisplayType'', or what kind of chart to be layout (Table, Point, BarLine). For table layout, it needs more specific information which is provided by class ''TableChartLayoutInfo''.
+
== Approach to new system ==
 +
Steps are defined for approaching the new system, which are described below:
  
−
=== Phase 2: Generate ''Chart'' ===
+
* Take the smallest possible case for prototyping (CrossDiagram with one axis for both row and column)
 +
* Design the core functions to server the single case with full testing
 +
* Approach in depth to:
 +
** Tabulation Script
 +
** Diagram Viewer
 +
** StyleSheet
 +
** Different presentation types
 +
* Move on with Report and Report Viewer
 +
* Move on with Monitor
 +
* Move on with Dashboard
 +
* Move on with data calculation optimization
 +
* Performance refining => change design and test cases
 +
* Full implementation for:
 +
** Cross Diagram
 +
** Other diagrams
 +
** All presentation types
 +
* Replace the old system's gui components
  
−
[[Image:Diagram_-_Content_generation.gif]]
+
== Design details ==
 +
* [[Report unit testing system]]
 +
* [[Diagram generation process]]
 +
* [[DiagramInfo]]
 +
* [[Tabulation Script]]
 +
* [[Tabulation Script for setting inline style to Table Chart]]
 +
* [[Diagram viewer control]]
 +
* [[Cross diagram settings control]]
 +
* [[Report viewer control]]
 +
* [[Integrating with Dashboard]]
 +
* [[Integrating with Monitor]]
 +
* [[Generic object visitor pattern]]
 +
* [[Generic XML de-/serialization]]
 +
* [[DiagramInfo de-/serialization]]
 +
* [[Table chart builder]]
 +
* [[Integrating Ajax.Pro and MS AJAX.NET Json Serialization]]
 +
* [[Full CrossDigramImplementation]]
 +
* [[TrackingDiagram]]
 +
* [[Support for IFKA cross diagram layout]]
 +
* [[XmlVisitor]]
 +
* [[Pdf table layout]]
 +
* [[Make pdf generation process in parallel mode]]
  
−
At the center of ''Chart'' generation is the abstract ''Diagram'' class. It receives ''DiagramInfo'' as input and generates expression for specific kind of Diagram (Cross, Campaign, Tracking, Standard). Then it passes the expressions to the ''Calculator'' for evaluating. The ''Calculator'' class has CGScript expression calculated by the ''Interpreter'', so does the Significance calculation but with the ''Significance'' class from Data. The ''Calculator'' itself handles simple calculation like ''sum'', ''percentage'', ''constant''.
+
== Presentations ==
 +
* [http://wiki.catglobe.com/images/5/51/NewReportSystem.ppt NewReportSystem.ppt]
 +
* [http://wiki.catglobe.com/images/8/87/NewReportSystem_-_Unit_Test.ppt NewReportSystem - Unit Test.ppt]
 +
* [http://wiki.catglobe.com/images/e/ef/NewReportSystem_-_Table_chart_builder.ppt NewReportSystem - Table chart builder.ppt]
 +
* [http://wiki.catglobe.com/images/5/5d/NewReportSystem_-_Diagram_generation.ppt NewReportSystem - Diagram generation.ppt]
  
−
The result of this phase is a ''Chart'' object which contains a ''2-dimensions'' array of ''ChartSeries''. In turn, ''ChartSeries'' contains a series of ''DataItem''. Each data item contains a value (normally an ''double'' value or an ''Exception'') and some metadata like ''legend'' and ''SignificanceInfo''
+
== Test cases ==
 +
* [[Test case - Diagram viewer page]]
  
−
=== Phase 3: Generate concrete objects (TO BE CHANGED) ===
+
== Lesson learn ==
−
Concrete objects can be considered ''Aspose.Pdf.Table'', ''Aspose.Ppt.Table'', ''System.Web.UI.Control.HtmlTable'' or simple ''System.Drawing.Image''. There concrete objects are useful in their own context. So we can consider this process as moving information from a generic context to a specific one.
+
This section lists all the:
 +
* ''Mistakes''
 +
* ''Useful knowledge''
 +
that we have encountered during implementation.
  
−
[[Image:Diagram_-_Builders.gif]]
+
* [[Code that MUST be improved]]
 +
* [http://wiki.catglobe.com/images/9/90/Transferring_Array_object_among_different_javascript_context.ppt Transferring Array object among different javascript context]
 +
* [http://wiki.catglobe.com/images/b/bf/Associated_array_vs_MS.AJAX_NET_vs_for_each_loop.ppt Associated array vs MS.AJAX NET vs for each loop]
  
−
At the center of the generation, is the abstract class ''ChartBuilder''. As stated before, there are 2 kinds for chart table and image, thus there are 2 sub-classes of ''ChartBuilder'': ''TableChartBuilder'' and ''NevronChartBuilder''. This page will focus on ''TableChartBuilder'' only.
+
== Current implementation status ==
−
 
+
{| border="1"
−
The ''ChartBuilder'' takes an ''ChartBuilderInfo'' object as an input and generate a concrete object base on that input. The input contains:
+
|-
−
* A ''Chart'' object which contain data for generation
+
| '''Modules'''  
−
* A ''PresentationType'' which is used to determined what kind of concrete object to generate
+
| '''Done'''  
−
* A ''StyleSheet'' which format the concrete object
+
| '''To be done'''  
−
 
+
| '''Time estimation'''
−
The value of ''DataItem'' must be formatted before copying to concrete object. There a are currently 2 formatters corresponding to 2 types of values (double and exception).
+
|-
−
 
+
| CrossDiagram
−
''TableChartBuilder'' also need another formatter to decorate the significance information.
+
|
−
 
+
* Rather stable with lot of unit tests
−
The result of a ''TableChartBuilder'' is a ''Table'' object. The ''Table'' class is the abstraction of all kind of concrete table. Using this abstraction we can eliminate the code duplication copying to different kinds of table.
+
|
−
 
+
* Missing feature (VN2950REP - CrossDiagram - Support missing features in 5.8):
−
<u>'''''Table'' Class'''</u>
+
** Weight
−
 
+
** Left header
−
[[Image:Diagram_-_Table_adaptors.gif]]
+
** Show error
−
 
+
|
−
''Table'' as a normal table contains a 2-dimensions list of ''Cell''. It also contains methods/properties for getting ''Cell'', ''RowCount'', ''ColumnCount''. It is designed to be expanded as needed, means that at first no memory is allocated only when needed. The ''Cell'' class also contains common information of a cell like ''Value'', ''RowSpan'', ''ColumnSpan'', ''RowIndex, ''ColumnIndex''.
+
''TODO''
−
 
+
|-
−
To avoid code duplication on a common scenario of data copying: loop through all cells and copy to concrete object we invent the ''ITableAdaptor'' interface. Each kind of concrete object has an ''adaptor'' to our ''Table'', through which data of ''Table'' can be cloned seamlessly.
+
| Nevron (image) charts
−
 
+
|
−
Whenever, a row/cell is created/deleted or a cell is changed in value, index, span, the attached 'adaptors' would be signal. In turn, the adaptors change the concrete objects correspondingly. After filling data to ''Table'' we have the concrete object ready-to-use without no further copying.
+
* Bar/Line/Pie/Radar charts have been implemented for single case of CrossDiagram => However the design is not good enough !
−
 
+
|
−
== Tabulation Script For generating Diagram ==
+
* A new design and implement will be done (VN2894REP - Reimplement NevronBuilder)
 +
|
 +
''TODO''
 +
|-
 +
| Table charts
 +
|
 +
* Rather complete
 +
|
 +
* Support exporting to PDF with nice layout (VN2808REP - Make PDF output look nice without fiddling with stylesheet)
 +
* More request from the customer (VN2813REP - Conditional Formatting)
 +
|
 +
''TODO''
 +
|-
 +
|
 +
Stylesheet
 +
|
 +
* The stylesheet is kept as is intentionally in order to keep changing at the minimal level
 +
|
 +
* Redefine the hierarchy of stylesheet classes
 +
|
 +
''TODO''
 +
|-
 +
| Tabulation Script
 +
|
 +
* Rather complete
 +
|
 +
* Add missing features (VN2951REP - TabulationScript - Support all PDF related scripts)
 +
|
 +
''TODO''
 +
|-
 +
| Web controls
 +
|
 +
* GUI for crossdiagram is nice enough
 +
|
 +
* More advance implementation with real Skin support (VN2895REP - Support skin for Setting control)
 +
* Increase the ratio of unit testing on GUI
 +
|
 
''TODO''
 
''TODO''
−
 
+
|-
−
== Tabulation Script for setting inline Stylesheet ==
+
| Unit tests
 +
|
 +
* Rather complete for crossdiagram
 +
* 3D image assertion => ''at last resort compare byte to byte, another approach is comparing by calculating hash value''
 +
|
 +
* Integrate with CruiseControl for running everyday
 +
* Web-testing for testing controls
 +
* Ppt/Excel assertion => ''Use new XML format of Office'' (VN2779REP - Upgrade Slide component)
 +
* Configuration settings for hard-coded values
 +
|
 
''TODO''
 
''TODO''
−
 
+
|-
−
 
+
| Optimization
−
== Diagram Viewer Control ==
+
|
 +
* Stylesheet rendering is being optimized for using with AJAX
 +
|
 +
* Code profiling must be done => minor change in design
 +
* Performance counter must be implemented
 +
* Generation time must less than current system
 +
|
 
''TODO''
 
''TODO''
 +
|}
  
 
== Document revisions ==
 
== Document revisions ==
Line 82: Line 197:
 
| '''Changed By'''  
 
| '''Changed By'''  
 
| '''Description'''
 
| '''Description'''
 +
| '''Svn revision'''
 
|-
 
|-
 
| 0.1  
 
| 0.1  
Line 87: Line 203:
 
| Nguyen Trung Chinh  
 
| Nguyen Trung Chinh  
 
| Create the first version for working with a single case of CDG
 
| Create the first version for working with a single case of CDG
 +
| NA
 +
|-
 +
| 0.1
 +
| 20.07.2009
 +
| Nguyen Trung Chinh
 +
| Refine the document
 +
| NA
 +
|-
 +
| 0.2
 +
| 30.07.2009
 +
| Nguyen Trung Chinh
 +
| Finish version 0.2
 +
| 54885
 +
|-
 +
| 0.3
 +
| 27.10.2009
 +
| Nguyen Trung Chinh
 +
| Update implementation status
 +
| 57470
 +
|-
 +
| 0.4
 +
| 31.12.2009
 +
| Nguyen Trung Chinh
 +
| Update status and new projects
 +
| 59167
 
|}
 
|}

Latest revision as of 04:10, 18 October 2013

<accesscontrol>Main:MyGroup</accesscontrol>

Introduction

ReportEngine.jpg

The components/modules that are using Report engine are: Dashboard, Monitor, Tabulation Script, Report Viewer, Diagram Viewer. These modules send request of generation to report engine including:

  • A data cache which contains all source data
  • Stylesheets (including external and inline stylesheet) to apply
  • Kind of presentation that the report should be exported as (pdf, excel, ppt, word, html, image)
  • Other kinds of settings

The report engine takes the request, make calculation using the Interpreter, and then export the result into desired format.

A report consists of elements like: diagrams, images, text blocks, pages, paragraphs. The most important element is Diagram, which can be discriminated based on:

  • Data
    • Cross Diagram
    • Standard Diagram
    • Tracking Diagram
    • Campaign Diagram
  • Presentation type
    • Table chart
    • Image chart which include many kinds of chart like: pie, bar, line, funnel, area, radar etc.

Why a new system?

Current system drawbacks

  • Heavy XML processing
  • Heavy code duplication
  • Inconsistent generation process

Thus it makes the cost of fixing bugs and maintain the old system greater than the cost of making an new system.

New system goals

  • Remove the drawbacks of current system
  • Side by side working with the old system
  • Fully unit testing
  • Unit/Integration tests can be run automatically
  • Testing API must be simple enough for testers to join writing test cases
  • Report API must be keep as simplest as possible
  • Design documents must be officially registered on wiki page

Approach to new system

Steps are defined for approaching the new system, which are described below:

  • Take the smallest possible case for prototyping (CrossDiagram with one axis for both row and column)
  • Design the core functions to server the single case with full testing
  • Approach in depth to:
    • Tabulation Script
    • Diagram Viewer
    • StyleSheet
    • Different presentation types
  • Move on with Report and Report Viewer
  • Move on with Monitor
  • Move on with Dashboard
  • Move on with data calculation optimization
  • Performance refining => change design and test cases
  • Full implementation for:
    • Cross Diagram
    • Other diagrams
    • All presentation types
  • Replace the old system's gui components

Design details

Presentations

Test cases

Lesson learn

This section lists all the:

  • Mistakes
  • Useful knowledge

that we have encountered during implementation.

Current implementation status

Modules Done To be done Time estimation
CrossDiagram
  • Rather stable with lot of unit tests
  • Missing feature (VN2950REP - CrossDiagram - Support missing features in 5.8):
    • Weight
    • Left header
    • Show error

TODO

Nevron (image) charts
  • Bar/Line/Pie/Radar charts have been implemented for single case of CrossDiagram => However the design is not good enough !
  • A new design and implement will be done (VN2894REP - Reimplement NevronBuilder)

TODO

Table charts
  • Rather complete
  • Support exporting to PDF with nice layout (VN2808REP - Make PDF output look nice without fiddling with stylesheet)
  • More request from the customer (VN2813REP - Conditional Formatting)

TODO

Stylesheet

  • The stylesheet is kept as is intentionally in order to keep changing at the minimal level
  • Redefine the hierarchy of stylesheet classes

TODO

Tabulation Script
  • Rather complete
  • Add missing features (VN2951REP - TabulationScript - Support all PDF related scripts)

TODO

Web controls
  • GUI for crossdiagram is nice enough
  • More advance implementation with real Skin support (VN2895REP - Support skin for Setting control)
  • Increase the ratio of unit testing on GUI

TODO

Unit tests
  • Rather complete for crossdiagram
  • 3D image assertion => at last resort compare byte to byte, another approach is comparing by calculating hash value
  • Integrate with CruiseControl for running everyday
  • Web-testing for testing controls
  • Ppt/Excel assertion => Use new XML format of Office (VN2779REP - Upgrade Slide component)
  • Configuration settings for hard-coded values

TODO

Optimization
  • Stylesheet rendering is being optimized for using with AJAX
  • Code profiling must be done => minor change in design
  • Performance counter must be implemented
  • Generation time must less than current system

TODO

Document revisions

Version No. Date Changed By Description Svn revision
0.1 27.04.2009 Nguyen Trung Chinh Create the first version for working with a single case of CDG NA
0.1 20.07.2009 Nguyen Trung Chinh Refine the document NA
0.2 30.07.2009 Nguyen Trung Chinh Finish version 0.2 54885
0.3 27.10.2009 Nguyen Trung Chinh Update implementation status 57470
0.4 31.12.2009 Nguyen Trung Chinh Update status and new projects 59167
Retrieved from "http://wiki.catglobe.com/index.php?title=New_Report_Design_-_2009&oldid=24722"