Tracking monitoring

Introduction

• daily management and tracking overview reports
• fixing quotas/changing quota periods
o quota standards for the Automated Tracking Script
• setting up tracking management of new tracking questionnaires
o group standards
• Setting up a tracking for external sample supply
• Managing a tracking with external sample supply
• Setting up tracking management of new tracking questionnaires
• Group standards
• Selecting sample


Daily tracking management
All trackings must be under supervision Monday – Friday. Usually every tracking must be open and complete interviews as evenly as possible every one of these week days, before the weekly target is reached. Furthermore, it is of the utmost importance that the weekly target is reached Sunday at the absolute latest. Reaching the weekly target Friday is preferred.

To a tracking manager, a tracking consists of the following relevant items
• a questionnaire
o with quotas
o with a bulkmail
• a set of groups, usually the following:
o a mailout group
o a samplestatus group
o an exclude group
• a work flow (an “Automated Tracking Script”)
• a scheduled cattask

This is at the moment all supervised with a master document, the “Tracking Overview”, which is used for supervising and creating tracking reports.

Regardless of who is handling the trackings (in Denmark or in Vietnam), the workflows’ scheduled cattasks should always be set to run between 10:00 and 11:00 Danish time. There are usually some changes to be made to some of the tracking questionnaires, and sometimes we do not know before Monday morning at 09:00-09:45. In that case, we need time to be able to set the respective tracking questionnaires on “paused” status, so that the respective scheduled cattask will not execute sample.

When the last tracking cattask has run to end, the daily tracking managent can begin. This would be 10 minutes after the time of the last scheduled tracking cattask.

The “tracking overview” is copied into a “tracking overview reports” folder, and is renamed to “week XX weekday tracking overview” (for instance “week 51 Tuesday tracking overview”). This folder will then hold historical data of the progress and handling of all trackings, which can be used for error correction, optimization, and gaining experience and knowledge. I have gained a lot by this method. Also it allows others to pick up where the work was left.

Open the copied tracking overview report for today.

Check your e-mail inbox, preferably you have made a folder specifically for tracking status reports automatically sent by the automated tracking scripts (work flows). Here you should see an unread mail for each tracking, containing information about how many mail invites has been sent out, what targets are for the week, and what is the current status of the tracking interviews. The three things needed to be done are 1) check that the target in todays tracking overview report is the same as in the status report mails for each individual tracking, and 2) type in the “Current” value for each tracking in the tracking overview reports “Current” column. You should then have an overview of how many interviews are missing for each tracking in numbers along with the completion percentage of the tracking in the “progress” column to the right. 3) double-check that all

Optimization: It would be an advantage to create an automated mail that contains the “current” status of all the trackings week quota at minimum, to be sent to the tracking management group at least twice a day. This will replace having to open each individual tracking questionnaire to follow the progress.

Mondays: Tracking adjustments and corrections
Monday before 9:30 Danish time, it is important to check with the Production Manager in Denmark whether or not there are changes scheduled to be made to the tracking questionnaires for the current week. If there are changes to be made, ensure that the trackings for maintenance were indeed fully completed last week.
Always check the last status report from Sunday, to check if all trackings that were not completed Friday have been completed by Sunday. If some trackings were not completed Sunday by the time of the status report, manually open each of these questionnaires to check if last weeks quotas were met.

If they were, set the respective tracking questionnaires status on “paused”, so that the respective scheduled cattask will not execute sample before the changes to the tracking questionnaire have been made.
If they were not, this is bad. Contact the Danish Production Manager (Produktionschef), for a solution. Normally the missing interviews must be completed early Monday, before the changes to the tracking questionnaire can be made. This is a bad solution, since it greatly affects quality and procedure, as well as panellists, who cannot enter the survey shortly after being invited to it, due to the fact that the questionnaire cannot be open whilst potential changes are being made. Furthermore, it takes up an amount of data engineer time, and it is methodically wrong.

When changes have been made, tracking management will receive a “go” for the respective trackings from the Production Manager.
- Then change the tracking questionnaire status to “open” via the questionaire list (Tools  Surveys  Questionnaires)
- go to “work flow” (Tools  Administration  Work flow)
- open the respective tracking work flow, and go to the “Script” tab
- Click “Run” (do not click Run batch). The Automated Tracking Script will then send out first batch, instead of waiting untill tomorrow for the scheduled cattask to execute, thus improving tracking data quality.


Fixing quotas/changing quota periods
Quotas in the tracking questionnaire are used by the Automated Tracking Script to adjust the amount of mail invites sent to each segment of a target group of panellists, in accordance to the current amount of successful interviews. Quotas define the final amount of intervews succeeded in each individual segment of the target group, within the given quota period.
At times, the quotas must be changed or adjusted, either by wish of the Client, or for maintenance purposes. The quotas should never be changed without prior consent of the project leader.

In order for the automated tracking script to successfully address the specified quotas, quota label naming must be consistant.
- The week quotas must be labelled “YYYYWW” (example: “200913” for week 13 in the year 2009)
All remaining subquota labels must consist of a definition of time for the quotas to be active, followed by the quota name itself, like so:
- "YYYY WW - YYYY WW: segment name"
(even when the quota period is one week, in example "2009 21 - 2009 21: Males 15-25")
furthermore recommend always using segment name standards:
- “Sex AA-AA” in example: “Males 25-34” or “Females 55-99”
or in the rarer occasions where three suffixes are used in the Automated tracking Script to more precisely control the sample selection:
- “Sex AA-AA region xx” example “Males 15-25 region Hovedstaden”
Notice that these standards for quota names (not the entire quota labels) counts for mailout group rule labels just as well.


Setting up a tracking for external sample supply
All questionnaires and trackings created from the CGtemplates already have the relevant questions and scripts for redirect implemented. So when we need external sample supply, there are only three things that needs to be done:

1) A sample cost & feasibility check request must be sent to at least one Panel Partner. A supplier is then chosen, from cost, quality and feasibility etc.

Research Now and Toluna have been favourites in the last half of the year 2008, Research Now especially in the nordic countries. Which Panel Partner is best at what kind of request in quality and price, takes an amount of experience to know. There are too many factors to take into consideration to give the proper insight here.

Help: The document "International Panel Overview.xls" is useful for quickly finding out which panel partners have panels in certain countries, and of what sizes the respective panels are (These numbers are practically worth nothing, without proper experience and inside knowledge of the respective companies, most are directly far from true).
The document "Sample Request Form.xls" is used for remembering filling out all necessary information to the Panel Partners when requesting samle cost and feasibility.

2) Receive and insert exit links from the Panel Partner.

Exit links ensure that the Panel Partners panellists are returned to their own site upon completion (succession, screenout or quotafull) of the questionnaire, thus giving the Panel Partner the option of following the progress of completion of sample, or just registering the counts for all three end statuses. This depends on their own respective research platforms.

How to: The questions in the questionnaires made from the CGTemplate (questionnaire templates) needed to insert exit links into are named "SUCCEEDED_PP", "QUOTAFULL_PP" and "SCREENOUT_PP". Insert the exit links in the respective question scripts under the Questionnaire Editor --> Properties --> Edit question properties. Remember to insert the right exit links under the right end questions, and to insert the exit links in the script for the current Panel Partner! You insert the exit link (exit URL) by replacing the text INSERTXXXXXXXXXXRETURNURL inside the quotation marks with the given exit URL (still leaving the quotaion marks in the script.

3) Generate a Test link and a Live link with the correct link parameters, and send it to the Panel Partner.

*Remember that it is good custom to test the Test link yourself, appending a user id you either get from the Panel Partner, or that you know have a structure that their system will accept, and remember to inform them of the user id's you tested with. Easy examples could be "66666666" or "11111111". Assuming that the questionnaire itself has already previously been tested, you must focus on testing if the return links work, and do indeed redirect you to the Panel Partners end screens, for both Quotafull (if possible), screenout, and successfull (generally known as "completed").

Ensure that the Panel Partner does not use the Live link for testing, as live tests take time to locate in Catglobe, and convert to test interviews. Test links are exactly the same as live links, with the two exception that a test link can be used multiple times, and has a "test" link parameter, that automatically converts what is in fact really a live interview to a test interview at the end of the survey.

How to
• in the questionnaire list, locate and highlight the relevant questionnaire
• click "Actions" --> "Information"
• copy the entire "Anonymous survey hyperlink" (the light green url)
• paste it into the worksheet of the current project in the "Tracking Overview.xls" master document (so that this information remains in all following Tracking Overview Reports
• Now, to get the Test link edit the anonymous survey hyperlink like so (always assume case sensitivity):
o change the link parameter "&n=1" to "&n=2" (always without the quotation marks)
o insert the Bypasscookie parameter at the end, like this "&b=1" (this bypasses a cookie, and lets you use the link for testing multiple times).
o Insert the Test parameter "&Test=1" (converts the otherwise live interview to a test interview instead)
o insert the Panel_Provider parameter "&PP=?". To know exactly what to write instead of the question mark, open the questionnaire with the Questionnaire Editor, and go to the question "Panel_Provider". Here you see a list of panel partner names. Find the relevant panel partner and type in the name exactly as it is written. The parameter could for instance be "&PP=Toluna"
o insert the Panel_Provider_User_ID. This is a specific link parameter for each panel partner. To know exactly what to write, open the questionnaire with the Questionnaire Editor, and go to the question "Panel_Provider_User_ID". Find the relevant panel partner user id parameter and type it in exactly as it is written. The parameter could for instance be "&TolunaID="
• Now you have a Test link that could for instance look like this: "http://mycatinet.catglobe.com/Login.aspx?r=93c1b14e-f989-4c26-b180-82fba92c375b&n=2&b=1&Test=1&PP=Toluna&TolunaID=". The Panel Partner must insert an existing ID to be sure to be returned to their exit links when testing.
• Now, to get the Live link edit the Test link like so:
o remove the Test parameter (so the interview remains a live interview)
o remove the Bypasscookie parameter (so the link can be used only once)
• Now you have a corresponding Live link that would look something like this: "http://mycatinet.catglobe.com/Login.aspx?r=93c1b14e-f989-4c26-b180-82fba92c375b&n=2&PP=Toluna&TolunaID=". The Panel Partner must be instructed to append their panellist user id at the end of the live link, right after the "=" mark (without quotation marks)

If just one parameter is wrong, it is likely to have bad consequenses. The interviews might seemingly succeed with valid data, but really be test interviews, or we might lose all id's from the Panel Partner. This would mean they are unable to incentivise, and that we have no proof of the amount of completes they have delivered.