Multiphase experiments (also Multi-phase experiments)

Brien (2017c) reviews multiphase experiments, giving an expository example that introduces the terminology, outlines their development and the areas in which they have been applied, describes how they might be designed and analyzed in practice and provides illustrative examples.

Areas in which multiphase experiments have been used

The following is a list of the areas in which multiphase experiments have been applied, as reported in the literature. For each area, some examples have been listed, primarily those that have occurred in the publications of Brien and Bailey or were cited by them. A more complete list of the experiments is given in Brien (2017c), where the features of an example from each area are detailed.

Plant pathology experiments
A tobacco mosaic virus experiment (McIntyre, 1955) in which both phases were laboratory phases.
Greenhouse/growth chamber experiments
A horticultural experiment in which both phases occurred in a greenhouse (Preece, 1991).
Product storage experiments
Potato storage experiment (Brien and Bailey, 2006, Example 13), Milk sensory experiment (Wood, Williams and Speed, 1988, second example), Apple storage experiment (Wilkinson et al, 2008).
Sensory evaluation phase experiments
In the following examples, the second phase is a sensory evaluation of produce from the first phase: Simple sensory experiment (Brien, 1983; Brien and Bailey, 2006, Example 1), Wine sensory experiments (Brien, May and Mayo, 1987), Milk sensory experiment (Wood, Williams and Speed, 1988, second example), Meatloaf tasting experiment, Knitted socks experiment (Brien and Bailey, 2006, Example 14), Nonorthogonal sensory experiment (Brien and Bailey, 2006, Example 15; 2009, Example 2).
Gene-expression studies
Kerr (2003) noted that the microarray experiments can be the measurement phase of a two-phase experiment and Jarrett and Ruggiero (2008) recognize their two-phase nature.
Human experiments
Athlete training experiments (Brien, Harch, Correll and Bailey, 2011; Brien, 2017c).
Laboratory analysis of crop experiments
An early experiment of this type was a cotton fibre experiment (Cox, 1958, Example 3.2; Brien and Bailey, 2006, Example 4) in which fibre from the field phase was tested for strength in the laboratory phases. Smith et al. (2001) discussed the design and analysis of wheat experiments that involved a field and a milling phase, and in which not all units from the field phase are processed in the milling phase. Cullis et al. (2003) described the design and analysis of a three-phase experiment that involved a field phase in which barley lines were grown, a malting phase in which barley malts were produced and a measurement phase in which several traits were determined. Again not all units from one phase continue on to subsequent phases. Smith et al. (2006) give general principles for the design of such experiments, while Smith et al., 2011 discuss the use of partial replication and sampling in later phases and Smith et al., 2015 incorporate the use of compositing of samples. Structure-balanced examples of experiments in which the first phase is a field experiment and the second phase is a laboratory experiment are a wheat experiment (Brien and Bailey, 2006, Example 9; 2009, Example 5), the duplicated wheat experiment and a corn seed germination experiment (Brien and Bailey, 2006, Example 12; 2010, Example 5).

Experiments with a second phase in the laboratory

A very common kind of multiphase experiment is those that involve a later laboratory phases. Brien and Bailey (2006, Examples 1, 4, 9, 12, 14 and 15, and Figure 7) are examples of experiments involving a first phase followed by a laboratory phases. The produce from field and other experiments is often processed in a laboratory, which results in two-phase experiments. Here, the laboratory assessment is to be interpreted broadly as a phase in which further processing, measurement, testing and so on are performed, even if, strictly speaking, a laboratory is not involved. Laboratory phases occur when there is to be the measurement of chemical attributes using equipment such as spectrometers, gas chromatographs or pH meters, testing using laboratory equipment, or the production of processed products such as wine, bread and malt that are subsequently assessed often by an expert panel. For some experiments both phases occur in the laboratory, such as in food processing when there is a phase in which mixtures are prepared, and a processing phase to produce the final product. Clinical trials can result in two phases, clinical and laboratory, when specimens from patients are processed in a laboratory. It would appear that such experiments occur frequently in the agriculture and other biological sciences. They all fit into the class of multiphase experiments, the simplest being two-phase experiments. Most of the examples published in the literature employ only one of the simplest types of multiple allocation and they are often unbalanced.

Brien, C.J., Harch, B.D., Correll, R.L. and Bailey, R.A. (2011) present a general discussion of the design and analysis of two-phase experiments in which the second phase is a laboratory phases and in which orthogonal designs have been used. Brien (2018) extends this to nonorthogonal designs. They provide a set of principles and laws for designing such experiments

Principles and laws for designing multiphase experiments

The following principles and laws have been adapted from Brien, C.J., Harch, B.D., Correll, R.L. and Bailey, R.A. (2011) and Brien (2018).

Principles

  1. Evaluate designs with decomposition/skeleton analysis-of-variance tables: Formulate the decomposition table or, if possible the skeleton-ANOVA table, using the factor-allocation diagram for an experiment, irrespective of whether its data is to be analysed by ANOVA.
  2. Fundamentals: A good experimental design employs: replication to provide a measure of random error and sufficient to achieve adequate precision; randomization to avoid systematic effects and other biases; and, where appropriate, blocking (or local control) to reduce variation among experimental units.
  3. Minimize variance: Block the units into groups if it seems that the units within the groups will be more homogeneous than set of units taken as a whole; when groups are formed, assign treatments to the units within the groups so that the contribution of group variation to the variance of the estimates of treatment effects is reduced as far as is possible.
  4. Split-units: Confound some treatment sources with unit sources for which greater variation is expected if some treatment factors (i) require larger units than others, (ii) are expected to have a larger effect, or (iii) are of less interest than others.
  5. Simplicity desirable: Whenever possible, in choosing a design to assign first-phase units to laboratory units, allocate first-phase unit factors that have treatments assigned to them so that sources associated with these factors are confounded with a single laboratory-unit source.
  6. Preplan all: If possible, plan all phases of an experiment before commencing it.
  7. Allocate all and randomize in laboratory: The laboratory-phase design should always allocate all the first-phase unit factors, as well as any laboratory treatments, to the laboratory units, using randomization wherever possible.
  8. Big with big: Confound big first-phase unit sources that have no treatment sources confounded with them, with potentially big second-phase unit sources.
  9. Use pseudofactors: Use pseudofactors to split sources, when necessary, to keep track of all factors in the experiment or to produce structure-balanced designs.
  10. Compensating across phases: If treatments are confounded with a large source of unit variation in the first phase, then consider confounding this source with a smaller source of variation in the laboratory phase.
  11. Laboratory replication: Replicated measurement of first-phase units is not required, but is highly desirable when uncontrolled variation in the laboratory phase is large relative to the first phase. It is also needed if the relative magnitudes of field and laboratory variation are to be assessed.
  12. Laboratory treatments: To minimize the variance of the estimates of laboratory treatment effects, confound them with sources to which only small components of laboratory variation contribute. When also confounding with first-phase unit sources, they too should be as small as possible.
  13. Design selection attributes: In designing a comparative experiment, consider the A-optimality, balance and available Residual degrees of freedom of potential designs; whether a resolved design has advantages over a more efficient unresolved design
  14. Balance when cannot be orthogonal: If an orthogonal design for a phase is impossible then the next best thing is a structure-balanced design. In such a phase, if it includes laboratory replication that requires portions of the products from the previous phase such that Multiphase Law 2 (Equal numbers of objects) pertains, then omitting the portions in constructing a design for the phase can permit a structure-balanced design. For designs that include portions, first-order balance is acceptable.
  15. Orthogonal later-phase designs advantageous: If possible, use an orthogonal design for the second, and any subsequent, phase, preferably one that does not require pseudofactors to make it orthogonal.
  16. Unbalanced appropriate: While experimental designs that are at least first-order balanced have desirable properties, unbalanced designs are on occasion appropriate.
  17. Nonrandomization: Sources cannot be randomized when there are practical restrictions, such as the timing of events in the different phases. Further, units sources from earlier phases that have random effects that are not independently distributed should not be randomized in later phases.

Laws

  1. DF never increase: The degrees of freedom for sources from a previous phase can never be increased as a result of the design for a subsequent phase. It is possible that the design splits a source from a previous phase into two or more sources, each with fewer degrees of freedom than the original source.
  2. Equal numbers of objects: In an experiment with three or more tiers, for the combined design to be structure balanced when the numbers of objects for two consecutive tiers are equal then, for the allocated tier of the pair, (i) the combined allocation of any tiers to it must be structure balanced and, (ii) its structure, possibly refined, for instance by including pseudofactors, must be orthogonal in relation to the recipient tier of the pair. Orthogonal or not, for the recipient tier of the pair, there are no Residual sources for its sources and its sources are said to be exhausted by those from the allocated tier.
    Corollary 1: Structure balance impossible:
    An implication of Multiphase law 2 (Equal numbers of objects) is that, if the numbers of objects for two consecutive tiers are equal and the structure for the allocated tier is not orthogonal to that of the recipient tier, then the multiphase design using them cannot be structure balanced. It may be first-order balanced.
  3. Inestimable components: Not all canonical (co)variance) components can be estimated when either (i) the numbers of objects for two consecutive tiers are equal, or (ii) the source for the term corresponding to a canonical components is exhausted by sources that are only confounded with it (Bailey and Brien, 2016, Section 8.2.1). For canonical-components models, only the sum of the canonical components for the identity terms are estimable when (i) applies.
  4. Combined structure balance: When the design for a multiphase experiment involves a chain of allocations and the design for each phase is structure balanced, which includes orthogonal, then the complete design is structure balanced, provided Multiphase corollary 1 (Structure balance impossible) does not apply. When the combined design is structure balanced, the (canonical) efficiency factors of the component designs multiply so that the efficiency factor for a source is:
    For composed allocations:
    The product of its efficiency factor with the product of those for sources, from subsequent phases, with which it is confounded.
    For allocated-inclusive allocations:
    As for composed allocations, except that the efficiency factors for a component design are those for modified sets of allocated and recipient sources formed from the original sets of sources by including, into the relevant set, sources for appropriate pseudofactors.
    Further, the sum of the efficiency factors for each source that has been allocated is one.
  5. Combined A-optimality: Combining two phases using an A-optimal design in each phase may not produce an A-optimal combined design. In particular, even when a two-phase design that combines two phases using two A-optimal, structure-balanced designs in a chain of allocations is structure-balanced, it may not be the A-optimal design amongst all possible two-phase designs for the anticipated model.