Designs distributed with the software contain the minimum number of sample sets required to accomplish positional and pairwise balance where applicable. Custom designs may or may not be balanced.
A sample set is the group of samples presented to a panelist.
The design determines which samples are presented and the order of their
presentation within each sample set.
The
Samples & design
area of a test automatically multiplies
the
Base design
sample sets to match the specified total
number of sample sets in the test.
Different test types have different best practices for minimum number of results recommended to obtain statistical analysis. Please refer to sensory and statistics literature for your test type recommendations.
In addition to considering those recommendations, we also recommend to set the total number of sample sets in your tests to be higher than the expected number of panelists. For example, if for your
Triangle test you have scheduled 40 panelists, we recommend the total number of sample sets to be at least 50 or higher. Extra sample sets are useful to have in case additional panelists show up, or for any other unforeseen situations. Extra sample sets enable you to use them if needed, and they do not have a negative impact on the data collection and analysis if not used. On the other hand, if you do not have enough sample sets, it can have negative impact on the data collection, and consequently on the analysis.
If required, in the
Samples & design tab of your test, you can
shuffle the sample sets
to meet your testing objectives.
Shuffle Sample Set Order
By applying the
Shuffle Sample Set Order
option
within a test
, the order of
sample sets from a
Base design
gets rearranged, but the sample order within each sample set remains as originally set in the
Base design
.
For example, you might have a design for 4 samples that has 24 sample sets (24 unique orders of 4 samples) in the
Base design
. Each sample set is equivalent to an order of samples each person (panelist) receives.
The
Base design
always has the sample set 1 first with its 4 samples in a specific order. For example, 1,4,2,3. The sample set 2 has the sample order 2,1,3,4, etc.
In many cases, when adding a
Base design
to a test, keeping the sample sets as they are in the
Base design
will be just fine. The sample set 1 will be taken by the first panelist, sample set 2 will be taken by the second panelist, etc.
However, when you are using the same panelists for testing, e.g. employees, to eliminate a possibility that they might learn the sample order after a period of time, it is useful to shuffle the sample sets in your tests. For example, if you normally use
Sample sets pre-assigned to panelists
option in the
Logistics tab, frequent panelists could eventually learn that their sample order is always the same. This is because the position of their panelist record in the
Panelists
list in a test equals a specific sample set, which equals a specific order of samples every time.
For
example the first panelist in your list will always be assigned sample set 1.
When you enable the
Shuffle Sample Set Order
feature in a test, the sample set 1 in the
Base design can be served to the 5th panelist, for example; the sample set 2 in the
Base design can be served to the 18th panelist, etc. The order of samples within each of those sample sets does not change from what they are in the
Base design
, but the position of the sample sets in the
Base design
changes.
TI-R
Time Intensity Reliability (TI-R) is the absolute average of a set of standard deviations. The standard deviations (sd) are the variability at each sampling point in a set of repeated
time intensity trials. The number of points depends on the sampling rate used and on the duration of the time intensity response of the panelists.
The analysis of time intensity reliability has an important role in the development, refinement, and maintenance of time-intensity panelists. During training, panelists can be evaluated based upon their T-IR scores (the lower the T-IR scores, the more reliable and less variable the panelist). Panels can be refined by eliminating panelists who have consistently high T-IR scores, and by setting the decrement in T-IR scores as a goal in panel training.
Source: "An objective numerical method of assessing reliability of time intensity panellists"
K. Bloom, L. M. Duizer, C. J. Findlay, Journal of Sensory Studies, 1995, 10(3), 285-294.