Data cache service

Introduction

When working on solutions for clients it is not always enough to report on the collected data, in some cases the client also wants to get access to the raw data directly.

Normally this can be handled by exporting the data, but in some cases the client is looking for an automated approach, where they can develop a data consumer, thus we have created a service from which we provide access to data from data caches.

The raw data has also shown to be useful when setting up dashboards, where access to the raw data from an RCS for instance allows us to make very advanced solutions such as our CRM solution.

Service APIs

The data cache service consists of two APIs; an external API allowing external applications to retrieve data from a data cache through SOAP calls, and an internal API that gives access to data from a data cache through JavaScript, which can be used from dashboards for example.

Requests using the internal API happens in a stateful context contrary to the external API where requests happen in a stateless context. Thus the primary difference between the two APIs is that the external API requires username and password each time a request is made.

The user requesting data from a data cache must have minimum observer access to the data cache specification, no matter if the external or internal API is used.

Data structures

When data from a data cache is returned to the requester it is represented by a simple object structure.

The DataCacheDto package the data returned from the data cache, with meta data to map column names to data.

• ColumnMap is an array of key-value pairs which contains one element for each column in the returned data table
  • Key refers to the column name
  • Value refers to the column index starting from 0
• Rows is an array of DataCacheRowDto objects which contains one element for each row retrieved from the data cache
  • Data refers to the array of data, one element per column

Internal API

// Gets the number of rows of a data cache.
int GetRowCount (
   int dcsResourceId)

// Gets all rows from a data cache, with data from a selection of columns.
DataCacheDto GetData (
   string[] columns,
   int dcsResourceId)

// Gets all rows from a data cache, with data from a selection of columns.
DataCacheDto GetDataSorted (
   string[] columns,
   int dcsResourceId,
   string[] orderByColumns,
   DataCacheSortOrderEnum sortOrder)

// Gets all rows from a data cache starting from a row index and returning
// rows as requested.
DataCacheDto GetDataRows (
   string[] columns,
   int dcsResourceId,
   int rowIndex,
   int rowCount)

// Gets all rows from a data cache starting from a row index and returning
// rows as requested.
DataCacheDto GetDataRowsSorted (
   string[] columns,
   int dcsResourceId,
   string[] orderByColumns,
   DataCacheSortOrderEnum sortOrder,
   int rowIndex,
   int rowCount)

// Gets the top N rows from a data cache, with data from a selection of columns,
// ordered by some column. Rows with no values will be excluded, meaning that 
// the data cache returned might contain fewer rows, than indicated by N.
DataCacheDto GetDataTop (
   int top,
   string[] columns,
   int dcsResourceId)

// Gets the top N rows from a data cache, with data from a selection of columns,
// ordered by some column. Rows with no values will be excluded, meaning that
// the data cache returned might contain fewer rows, than indicated by N.
DataCacheDto GetDataTopSorted (
   int top,
   string[] columns,
   int dcsResourceId,
   string[] orderByColumns,
   DataCacheSortOrderEnum sortOrder)

// Gets all rows from a data cache, with data from a selection of columns,
// belonging to a selection of users.
DataCacheDto GetDataForUsers (
   string[] columns,
   int dcsResourceId,
   int[] users)

// Gets all rows from a data cache, with data from a selection of columns,
// belonging to a selection of users.
DataCacheDto GetDataForUsersSorted (
   string[] columns,
   int dcsResourceId,
   int[] users,
   string[] orderByColumns,
   DataCacheSortOrderEnum sortOrder)

// Gets all rows from a data cache,with data from a selection of columns,
// belonging to a logged in user.
DataCacheDto GetDataForLoggedInUser (
   string[] columns,
   int dcsResourceId,
   Guid dataViewUser)

External API

Signature	Description
int GetRowCount (string username, string password, int dcsResourceId)	Gets the number of rows of a data cache.
DataCacheDto GetData (string username, string password, string[] columns, int dcsResourceId)	Gets all rows from a data cache, with data from a selection of columns.
DataCacheDto GetDataSorted (string username, string password, string[] columns, int dcsResourceId, string[] orderByColumns, DataCacheSortOrderEnum sortOrder)	Gets all rows from a data cache, with data from a selection of columns.
DataCacheDto GetDataRows (string username, string password, string[] columns, int dcsResourceId, int rowIndex, int rowCount)	Gets all rows from a data cache starting from a row index and returning rows as requested.
DataCacheDto GetDataRowsSorted (string username, string password, string[] columns, int dcsResourceId, string[] orderByColumns, DataCacheSortOrderEnum sortOrder, int rowIndex, int rowCount)	Gets all rows from a data cache starting from a row index and returning rows as requested.
DataCacheDto GetDataTop (string username, string password, int top, string[] columns, int dcsResourceId)	Gets the top N rows from a data cache, with data from a selection of columns, ordered by some column. Rows with no values will be excluded, meaning that the data cache returned might contain fewer rows, than indicated by N.
DataCacheDto GetDataTopSorted (string username, string password, int top, string[] columns, int dcsResourceId, string[] orderByColumns, DataCacheSortOrderEnum sortOrder)	Gets the top N rows from a data cache, with data from a selection of columns, ordered by some column. Rows with no values will be excluded, meaning that the data cache returned might contain fewer rows, than indicated by N.
DataCacheDto GetDataForUsers (string username, string password, string[] columns, int dcsResourceId, int[] users)	Gets all rows from a data cache, with data from a selection of columns, belonging to a selection of users.
DataCacheDto GetDataForUsersSorted (string username, string password, string[] columns, int dcsResourceId, int[] users, string[] orderByColumns, DataCacheSortOrderEnum sortOrder)	Gets all rows from a data cache, with data from a selection of columns, belonging to a selection of users.

Examples

Accessing a project resource data cache

We have created a search on the standard project resource list which we want to utilize from a dashboard.

In order to do so we must first create a data cache.

Note down the resource id of the newly created RCS, we will need that for the dashboard.

Create a new dashboard, save it, and go to the Layout tab, change the view to HTML and paste in the script below:

<div id=contents>&nbsp;</div>

<script type=text/javascript>
// Creating a variable for the web-site's root path like below
// makes it easier to move the script between different sites
var rootPath = window.location.protocol+'//'+window.location.host;

// A simple table renderer implemented using jQuery
function renderTable(data)
{
   // Create table and header row
   var t = $('<table border="1">');  
   var tr = $('<tr>').appendTo(t);
   for (var c in data.ColumnMap)
   {
      $('<th>')
         .html(data.ColumnMap[c].Key)
         .appendTo(tr);
   }

   // Add data rows
   for (var r = 0; r < data.Rows.length; r++)
   {
      var tr = $('<tr>').appendTo(t);
      var row = data.Rows[r];
      for (var c = 0; c < row.Data.length; c++)
      {
         $('<td>')
            .html((row.Data[c] == null) ? '&nbsp;' : row.Data[c])
            .appendTo(tr);
      }
   }
   
   // Return the jQuery table object
   return t;
}

$(document).ready(function()
{
   // Load the JavaScript API for the internal data cache service
   $.getScript(rootPath+'/Services/Internal/DataCacheService.svc/js',
   function()
   {
      // Columns to retrieve from the data cache
      var columns =
      [
         'Id',
         'Name',
         'Path',
         'ProjectCode',
         'PlannedStartDate',
         'PlannedEndDate'
      ];
      
      // Data cache to retrieve the data from
      var dcsResourceId = 34807128;
      
      // Start retrieving data from row 0
      var rowIndex = 0;
      
      // Get 25 rows from the data cache
      var rowCount = 25;
      
      // Create an instance of the internal data cache service API
      var dataCacheService = new Catglobe.Web.Services.Internal.IDataCacheServiceInternal();
      
      // Get data from the data cache
      dataCacheService.GetDataRows(
         columns,
         dcsResourceId,
         rowIndex,
         rowCount,
         // When call to the service returns successfully
         function(data)
         {
            // We receive data as a DataCacheDto object
            // which we pass on to the table renderer
            renderTable(data).appendTo($('#contents'));
         },
         // When call to the service returns failure
         function(error)
         {
            // We announce the error received from the service
            alert(error.get_message());
         });
   });
});
</script>

To access the same data cache from another site or web application written in PHP for example, we must use the external API for the service to provide user credentials, an example from PHP is shown below:

<?php
// Set the header to return plain text
header('Content-Type: text/plain');

// Create an instance of the soap client, with reference to the external API
$client = new SoapClient('http://supernova.catglobe.com/services/external/DataCacheService.svc?wsdl');

// Configure an anonymous object to pass necessary parameters to the service
$obj->username = 'XXX';
$obj->password = 'XXX';

// Only select the top 5
$obj->top = 5;

// Columns to retrieve from the data cache
$obj->columns = array
(
   'Id',
   'Name',
   'Path',
   'ProjectCode',
   'PlannedStartDate',
   'PlannedEndDate'
);

// Data cache to retrieve the data from
$obj->dcsResourceId = 34807128;

// Get the top 5 from the data cache
$result = $client->GetDataTop($obj);

// Output the result
print_r($result);
?>