Standard Report: Back to Back Triangle Test Results

Standard Report: Back to Back Triangle Test Results

Overview

The Standard report for Back to Back Triangle test data (replications on the same two samples) can provide the following analysis:
  1. Sensory discrimination (exact) analysis. This analysis is a statistical non-inferiority test (which in this case is a binomial test of the lower tail) and it aligns with the ASTM standard E3009-23a. You can choose between discrimination, equivalence, and equivalence for reps analysis.

  2. Discrimination analysis (a Difference and/or a Similarity test). This analysis follows the formulas published in the Sensory evaluation techniques by Meilgaard et al. This analysis has been included in Compusense since this test type was originally implemented in Compusense.

If a Triangle related comment question was asked, you will also see a list of comments for Correct and Incorrect responses. 

Generate the Report

There are four steps in the Create report area:

1. Select report type : Select Standard Report. If necessary, filter the data before generating the report.


2. Select options : Choose whether you wish to run the Sensory discrimination (exact) analysis or Discrimination analysis.
  1. If you choose the Sensory discrimination (exact) analysis, choose from the following analysis options: 
    1. Difference test. Select if your objective is to determine whether the samples differ. 

      Enter the Alpha you wish to use for the test. The default is 0.05.

      Difference thresholds:
      Update the d' (d prime) by typing directly in the box. The default is 0. You will notice that the Pd and Pc values will automatically recalculate.

      If you update the Pd (the default is 0), then the d' and Pc values will automatically recalculate. Likewise, if you update the Pc value (the default is 0.3333), the d' and Pd values will automatically recalculate.

    2. Equivalence test. Select if your objective is to determine whether the samples are practically the same within acceptable limits. The alpha and the Pd threshold can be set right below the next option, the Equivalence test for replications option.

    3. Equivalence test for replications. Select if your objective is to determine whether the replicated samples are practically the same within acceptable limits. 

      Enter the Alpha you wish to use for the test(s). The default is 0.05.

      Equivalence thresholds: 
      1. d'. The d-prime value is applicable to the Equivalence test only. The default is 1.25. Update as necessary. You will notice that the Pd and Pc values will automatically recalculate.

      2. Pd. The Proportion of distinguishers value is applicable to both the Equivalence test and Equivalence test for replications. The default is 0.19. If you update the Pd, then the d' and Pc values will automatically recalculate. 

      3. Pc. The Proportion correct value is applicable to the Equivalence test only. The default is 0.46. If you update the Pc value, the d' and Pd values will automatically recalculate. 

  2. In the Discrimination analysis select to run a Difference test and/or Similarity test. The Alpha and Beta levels can be updated as necessary. The default for is 0.1. Analysis is based on the Binomial distribution.

3. Select questions : Select which questions you wish to include in the report. 

4. Select export type : Choose the export type that best suits your reporting needs. 


Report and Sample Details

 
The Standard report for Back to back Triangle question data includes the following information about the report itself and about the samples:
  1. Test Name. The name you specified during the test setup in the Overview tab.

  2. Number of Evaluations. The number of evaluations in the test. This will include complete and partial results. Use the Filters feature to control what is included in the analysis.

  3. Test Completion Date. The date when your test was set to Complete in the Run test tab. No date will be shown if the test is not set to Complete. It is recommended to set tests to Complete when no further data collection is expected.

  4. Samples Table. This table provides a summary of all your sample information as specified during the test setup in the Overview tab.


    Sensory Discrimination (Exact) Analysis

    Difference Test

    The Difference Test will provide a count of correct and incorrect responses as well as a statement of significance. 

    1. Above the table, the following information is displayed:
      1. The threshold values specified by the user before generating the report.

      2. Statement on how conclusions are made based on the selected alpha. The confidence % is the only value that will change in this statement depending on the Alpha value specified before the report was generated.

    2. The table displays the following details:
      1. p-value: The p-value for the triangle test.

      2. Responses Required: The minimum number of correct responses necessary to reach the conclusion of Different.

      3. Conclusion: A conclusion is made based on whether the difference between samples is significantly larger than the difference threshold.

      4. Triads. This workflow focuses on a back to back Triangle where multiple triads of the same two samples were evaluated. In our example, the same two samples were evaluated twice (two reps).
        1. Triad #: The triad the analysis is shown for.

        2. Pooled : When two samples are evaluated back to back (true replicates), the N is increased by multiplying the actual N by the number of tests. Each test is treated as if done by different panelists even though they were seen by the same panelists.

        3. Both Triads : When two samples are evaluated back to back (true replicates) you will see how many panelists answered both Triangle tests correctly. The probability of panelists correctly identifying the odd sample for both tests by chance decreases.

    Back to back Triangle test data on 4+ samples, will generate separate set of tables for each pair of samples.

    Equivalence Test 

    An equivalence test is a statistical method used to demonstrate that two samples are practically the same within acceptable limits. Unlike traditional hypothesis tests, which assume equality and look for differences, an equivalence test starts by assuming the two are different and seeks to prove they are “close enough.” 


    1. Above the table, the following information is displayed:
      1. The threshold values specified by the user before generating the report.

      2. Statement on how conclusions are made based on the selected alpha. The confidence % is the only value that will change in this statement depending on the Alpha value specified before the report was generated.

    2. The table displays the following details:
      1. p-value: The p-value for each triangle test.

      2. Responses Required: The maximum number of correct responses permitted to reach the conclusion of Equivalent.

      3. Conclusion: A conclusion is made based on whether the difference between samples is significantly larger than the equivalence threshold.

      4. Triads: 
        1. Triad #: The triad the analysis is shown for.

        2. Pooled : When two samples are evaluated back to back (true replicates), the N is increased by multiplying the actual N by the number of tests. Each test is treated as if done by different panelists even though they were seen by the same panelists.

        3. Both Triads : When two samples are evaluated back to back (true replicates) you will see how many panelists answered both Triangle tests correctly. The probability of panelists correctly identifying the odd sample for both tests by chance decreases.

    Back to back Triangle test data on 4+ samples, will generate separate set of tables for each triad.


    Equivalence test for replications

    The Equivalence via sensory discrimination (exact) equivalence test for replications analysis in Compusense is based on Meyners, M., Carr, B. T., & Kunert, J. (2023). Equivalence and non‐inferiority tests using replicated discrimination and preference data. Journal of Sensory Studies, 38(6), e12882. https://doi.org/10.1111/joss.12882

    This table will be included in the report only if selected in the options before generating the report and if the test was set up to evaluate replications of the same 2 samples.


    1. Above the table, the following information is displayed:
      1. Pd. The Proportion of distinguishers value specified by the user before generating the report.
         
      2. Alpha. The Alpha value specified by the user before generating the report.

      3. X. The number of correct responses across all reps.

      4. n. The total number of responses included in the analysis for this question. 

      5. k. The number of replications of the two samples.

    2. The table displays the following details:
      1. Sample names. The names of the samples evaluated in replicates.

      2. Responses Required: The maximum number of correct responses permitted to reach the conclusion of Equivalent.

      3. p-value: The p-value for each triangle test.

      4. Conclusion: A conclusion is made based on whether the difference between samples is significantly larger than the equivalence threshold.



    Supporting tables

    Proportion Correct (Pc):


    1. N: Total number of evaluations included in the report.

    2. Correct: Count of correct responses (where the odd sample was identified correctly).

    3. Incorrect: Count of incorrect responses (where the odd sample was not identified correctly).

    4. Estimate Pc: Estimated proportion correct.

      Formula: Pc = c / n

      Where:
      c = number of correct responses.

      n = total number of evaluations.

    5. SE: Standard error of the estimate of the proportion of momentary discriminators.

    6. LCL: Lower confidence limit at the Alpha specified before generating the report.

    7. UCL: Upper confidence limit.

    8. Triads: 
      1. Triad #: The triad the analysis is shown for.

      2. Pooled : When two samples are evaluated back to back (true replicates), the N is increased by multiplying the actual N by the number of tests. Each test is treated as if done by different panelists even though they were seen by the same panelists.

      3. Both Triads : When two samples are evaluated back to back (true replicates) you will see how many panelists answered both Triangle tests correctly. The probability of panelists correctly identifying the odd sample for both tests by chance decreases.

    Back to back Triangle test data on 4+ samples, will generate separate set of tables for each triad.


    Proportion of momentary discriminators (Pd):

    1. Estimate Pd: Estimated proportion of distinguishers.

    2. SE: Standard error of the estimate of the proportion of momentary discriminators.

    3. LCL: Lower confidence limit at the Alpha specified before generating the report.

    4. UCL: Upper confidence limit.

    5. Triads: 
      1. Triad #: The triad the analysis is shown for.

      2. Pooled : When two samples are evaluated back to back (true replicates), the N is increased by multiplying the actual N by the number of tests. Each test is treated as if done by different panelists even though they were seen by the same panelists.

      3. Both Triads : When two samples are evaluated back to back (true replicates) you will see how many panelists answered both Triangle tests correctly. The probability of panelists correctly identifying the odd sample for both tests by chance decreases.

    Back to back Triangle test data on 4+ samples, will generate separate set of tables for each triad.


    d prime:

    1. d' (read d prime): The measure of the extent of the overlap of panelist perception. It is the distance between the means of the two distributions measured in terms of their standard deviation.

      The larger the d’ value, the samples are more different. Where the d' value is 0 it means that the samples are indistinguishable. A d’ of 1 or less might be considered small and a d’ of 2 or more might be considered large, but it depends on various factors.

      This calculation is always done treating each response as independent, as if the test is unreplicated. For example, d' on a back-to-back test with 50 assessors who provide 2 responses, calculation is done as if there are 100 assessors who provide 1 response each.

    2. LCL: Lower confidence limit at the Alpha specified before generating the report.

    3. UCL: Upper confidence limit.

    4. Triads: 
      1. Triad #: The triad the analysis is shown for.

      2. Pooled : When two samples are evaluated back to back (true replicates), the N is increased by multiplying the actual N by the number of tests. Each test is treated as if done by different panelists even though they were seen by the same panelists.

      3. Both Triads : When two samples are evaluated back to back (true replicates) you will see how many panelists answered both Triangle tests correctly. The probability of panelists correctly identifying the odd sample for both tests by chance decreases.

    Back to back Triangle test data on 4+ samples, will generate separate set of tables for each triad.



    Discrimination Analysis

    Difference Test



    This table provides the results of the Difference test. 
    1. Chance : The probability of panelists correctly selecting the odd sample by chance.
    1. Number of Evaluations (N) : The number of evaluations in the test. This will include Complete and Partial Results. 
    1. Correct/Incorrect Count : The count of Correct and Incorrect Responses. 
    1. d':  The measure of the extent of the overlap of panelist perception. It is the distance between the means of the two distributions in terms of their standard deviation. The larger the d' value, the larger the difference between products.
    1. p-value : The probability of obtaining the observed test statistic or one that is more extreme under the null hypothesis. 
    1. Decision of Significance : A statement of whether or not the samples are significantly different based on the Alpha selected when creating the Report.
    1. Triad Number : The triad the analysis is shown for.
    1. Pooled : When two samples are evaluated back to back, the N is increased by multiplying the actual N by the number of tests. Each test is treated as if done by different panelists even though they were seen by the same panelists.
    1. Both Triads : When two samples are evaluated back to back (true replicates) you will see how many panelists answered both Triangle tests correctly. The probability of panelists correctly identifying the odd sample for both tests by chance decreases.

    Similarity Test

    The Similarity Test will provide a count of Correct responses as well as a statement on whether or not your samples are Similar or Not Similar. 


    This table provides the results of the Similarity Test.
    1. Number of Evaluations (N) : The number of evaluations in the test. This will include Complete and Partial Results.
    1. Correct : The count of Correct responses.
    1. Responses Required : The number of corrects responses required to reach a conclusion of Not Similar.
    1. Conclusion : A statement of Similar or Not Similar. Samples are Similar when the number of correct responses is smaller than the responses required. Samples are Not Similar when the number of responses is equal to or greater than the responses required.
    1. Pc (estimate) : Proportion correct.
    1. Pd (estimate) : Proportion of distinguishers.
    1. Std error (Pd) : Standard error of the estimate of the proportion of (momentary) discriminators.
    1. Pmin, Lower confidence limit (=0.1) : Formula: Pmin = Pd + Zalpha * Std error (Pd). The default Alpha value is 0,10, this value can be changed when creating the Report.
    1. Pmax, Upper confidence limit (=0.1) : Formular: Pmax=Pd + Zbeta * Std error (Pd). The default Beta value is 0.10, this value can be changed when creating the Report.
    1. Triad Number : The triad the analysis is shown for.
    1. Pooled : When two samples are evaluated back to back, the N is increased by multiplying the actual N by the number of tests. Each test is treated as if done by different panelists even though they were seen by the same panelists.
    1. Both Triads : When two samples are evaluated back to back (true replicates) you will see how many panelists answered both Triangle tests correctly. The probability of panelists correctly identifying the odd sample for both tests by chance decreases.


    Additional Triangle Analysis

    If your test included a follow up Triangle related Comment or a Triangle related Choose question, your Standard report will provide you with Triangle related Comments or Triangle related Choose data, depending on what question type you added into your test. Let's review what each follow up question provides in the Standard report.

    Comment




    If your Triangle test includes a Triangle related comment question, panelists; comments will appear in one of two tables; Correct response related comments or Incorrect response related comments.

    1. Sample Number : The sample the panelists selected.
    1. SS# : The sample set number used to identify the panelist.
    1. Triad : The triad the panelist seen.

    2. Panelist Code: If registered panelists evaluated the samples, their Usernames (also known as Panelist codes in subscriptions created prior to 2022) will be displayed in this column.
    1. Result : The panelists' comments.


    Graphs



    The data from a single and back to back Triangles can also be graphed. To generate the graph:
    1. In the Results area, click the Graphs tab. If necessary, click Filters to filter specific sample sets out.

    2. In the left sliding pane, select the Triangle question.

    3. Update any of the parameters needed in the right sliding pane. Don't forget to scroll down to see all the different settings on the right.

    4. Export the graph, or take a screenshot to include it in your reports.

    PowerPoint Report

    Compusense offers two types of PowerPoint Reports to provide visual representation of the panelists' responses, and the statistical conclusion:
    1. Custom built PowerPoint Report. Generate a custom built PowerPoint Report with your own logo, including the information and analysis you need, and in the order you need it.


    2. Build in PowerPoint Report. Generate a built in PowerPoint Report on your Triangle data to get a 

    Idea
    If you would like to generate PowerPoint Reports with your corporate colours in them, please contact our Support Team and they will assist you with the next steps.

    Data



    You can use the Data tab within your test, in the Results area, for a quick and easy insight into panelist performance within individual tests (rather than across multiple tests as described below). The example screenshot above was taken in a back to back Triangle test.


    Panelist Performance Report

    Run a Panelist Performance report in the Panelists library to see how your panelists have performed over multiple Triangle tests over time, allowing you to discover patterns or non-differentiators. The workflow for the report will be available soon.


      • Related Articles

      • Standard Report: Single Triangle Test Results

        Overview The Standard Report for Single Triangle test data can provide the following analysis: Sensory discrimination (exact) analysis. This analysis is a statistical non-inferiority test (which in this case is a binomial test of the lower tail) and ...

      • Standard Report: Tetrad Test Results

        Overview Analysis for the Tetrad data in Compusense uses a one-tailed binomial test. Tetrad Analysis Options When generating the Tetrad Standard Report, it is important to select the options that meet your analysis objectives. Go to Results > Reports ...

      • Triangle Test

        Overview Triangle test is a discrimination test type. Determine if there is a difference between products by asking panelists to select the different sample out of three samples, of which two are the same. This method is commonly used with employee ...

      • Standard Report Review

        Overview The Standard Report provides a breakdown of results by attributes (where applicable) and samples, as well as the analysis that was selected while generating the report. Question Compatibility The report is compatible with the following ...

      • PowerPoint Report: Triangle Test Results

        Overview The PowerPoint Report in Triangle tests provides a visual representation of the correct/incorrect responses as a percentage, as well as a Decision of Significance. For the full page view of this workflow, please click on the icon with an ...