Analysis of multitiered experiments

Two methods that can be used to analyse the data are mixed-model estimation and analysis of variance.

Mixed-model estimation

Mixed-model estimation, based on Residual or Restricted Maximum Likelihood (REML) is widely available. It is particularly applicable for nonorthogonal experiments and those with pseudofactors, because combined estimates will be obtained and the specification of pseudofactors can usually be avoided. It is strongly advocated that the mixed model is randomization-based.

However, it has the disadvantage that, when random terms are totally confounded by fixed effects, they need to be removed for the fitting algorithms to work. In addition, when unrandomized random terms from the first randomization are totally confounded by unrandomized random terms from the second randomization, then some must be omitted. For example, either Sessions or Blocks must be ommitted from the mixed model for the meatloaf tasting experiments. Consequently, the model supplied to a procedure may not be the full randomization-based mixed model for the data, but a model of convenience. Also, convergence problems may be experienced when some random terms are near zero or have low degrees of freedom. In addition, low degrees of freedom cause difficulties because of the asymptotic approximations on which the analysis is based.

Analysis of variance

The AMTIER procedure (Brien and Payne, 2006) is purpose-built to perform the analysis of variance of multitiered experiments that can be specified with three formulae and are structure balanced; it uses the algorithm described by Brien and Payne (1999). If a response variable is not supplied, it generates a skeleton ANOVA table.

To perform an analysis using standard analysis of variance or regression software, it will be necessary to determine the set of sources that will produce the set of sums of squares that correctly partitions the total sums of squares . That is, the derivation of the analysis of variance table will have to be done manually and the sums of squares added to it once they have been obtained using a package. In practice this is only feasible for orthogonal designs

For example, three structure formulae were used in deriving the decomposition tables for the meatloaf tasting experiments illustrating the proposed procedure. However, although there are problems as discussed below, this analysis could be obtained with the following single structure formula:

Sessions / (Panellist + Time-orders) + Rosemary * Irradiation * Blocks

This formula will provide all the sums of squares that are required in the ANOVA table, corresponding to the decomposition tables for this example. The interactions of Rosemary and Irradiation with Blocks will provide the Residual sums of squares once they have been summed.

Problems with using a single formula

Does not display confounding
With a single structure formula each contrast is associated with only a single term and so there is no indication of the multiple sources associated with some of the contrasts as a result of the confounding arising from the randomization. For example in the full decomposition table derived from three structure formula it is clear that the one degree of freedom associated with Rosemary is confounded with Meatloaves[Blocks] which in turn is confounded with Panellists#Time-orders[Sessions]. With a single formula there is only the term Rosemary associated with this one degree of freedom.
Does not correctly identify sources of variation
It is very important to understand that we do not regard the Block interactions with Rosemary and Irradiation as indicating actual sources of variation for which we are including terms in the model. Rather they are merely convenient way of obtaining the sums if squares. Summing these interactions is one way of obtaining the Residual for Meatloaves[Blocks]. However, the term in the model is Meatloaves[Blocks], not Blocks#Rosemary + Blocks#Irradiation + Blocks#Rosemary#Irradiation. So we have a term for variability between meatloaves within a block in the model and we are assuming that there is no interaction between blocks and rosemary and irradiation treatments.
Cannot obtain full analysis of nonorthogonal experiments
As explained by Brien and Payne (1999) one obtains the correct intrablock analysis with a single strucutre formula. However, to obtain the interblock analysis, for use in recombination of information, requires the use of multiple stucture formulae