This tool compares the percentage change in a performance measure to the same measure either over the same time period one year earlier (controlling for possible seasonal effects), or a user specified time period, for two different groups (a treatment group and control group). The test performed to make the comparison is Welch's two sample t-test*.
Since the measures can involve using over a year's worth of historic data, it is important to have sufficient historic data available that is prior to the start of the test. If this condition is not met, then an error will occur.
This tool uses the R programming language. Go to Options > Download Predictive Tools to install R and the packages used by the R Tool.
C anchor: A Controls data stream that contains a field with a unique identifier or each control unit that allows the control units to be identified in the performance data (described below) and a field that gives the identifier of the treatment unit to which each control unit is assigned.
T anchor: A Treatments data stream that contains a field with a unique identifier for each treatment unit that allows the treatment units to be identified in the performance data (described below).
P anchor: A Performance data stream that contains a field with a performance measure (e.g., sales, visits) of interest to be compared between the treatment and control groups, a date field identifying the period each value of the performance measure to which the value pertains (this allows for the creation of the correct comparison time periods), and a unit identifier field that allows the treatment and control units to be properly identified.
Controls: Use the drop downs to:
Treatments: Use the drop down to Select the field with the Treatments unit identifier. For best results, this field should be some sort of character based type such as String or V-String.
Performance Data: Use the drop downs to specify:
Test Start Date: The date the test started, selected using a calendar interface.
Test End Date: The date the test ended, selected using a calendar interface.
Set the dates of the comparison period: Checking this box allows the user to specify a custom period to use for comparison purposes by selecting the start and end dates of the comparison using a calendar interface. By default, the comparison period is taken to be the same time period one year earlier.
The test name: This information is used to identify the test the analysis pertains to in the report.
Provide any additional information about the test: Any other relevant information that makes sense to include in the report, such as the DMA(s) in which the test was carried out, additional information about characteristics of the test, and so on.
Optional alternative name for the performance measure: This enables the performance measure to be identified in the report in terms that will be easier for readers to understand (e.g., "Customer Traffic" instead of "T_Cust_Day").
The required or target percentage growth level (a value of 0.0 implies not target): This number is likely to be based on the incremental increase needed in revenues needed to achieve breakeven or a minimal target needed target to justify the roll out of the treatment program to the relevant population.
Save Dashboard: Specify the file location to save the interactive report (I) output.
The format used to display dates in the line plot comparison: Specify the Date format to use in the interactive report (I) output.
Graph Resolution: Specify the image resolution. Lower resolution creates a smaller file and is best for viewing on a monitor. Higher resolution creates a larger file with much better print quality.
O anchor: Includes a textual summary of the test results, basic summary statistics of the test results, a number of visualizations of the test results, and a table of the results of a Welch's two sample (the treatment and control samples) t-test of the difference in the percentage change for the performance measure of interest.
E anchor: The Extended output is a data table consisting of the values used to populate the Dot Plot found in the O and I outputs.
G anchor: The Grouped Data output is a data table consisting of the values used to populate the Time Comparison Plot found in the O and I outputs.
I anchor: The Interactive Dashboard is HTML that displays in your browser. You can interact with the visualizations by clicking on the different graphic elements to reveal more information, values, metrics and analytics.
*en.wikipedia.org/wiki/Welch's_t_test
©2018 Alteryx, Inc., all rights reserved. Allocate®, Alteryx®, Guzzler®, and Solocast® are registered trademarks of Alteryx, Inc.