[Up][Contents][Index]

Agreement

Agreement indices

It is necessary to define an index which gives a good measure of how well any posterior distribution agrees with the prior distribution. It seems intuitive that any such distribution should be unity (100%) if there is no alteration in the distribution and that the index the index should fall off in proportion to the overlap of the likelihood distribution with the posterior distribution. Such behaviour is in fact provided by the function defined here for individual distributions. Let the likelihood distribution be p(t) and the posterior distribution be p'(t) an agreement index can then be defined:

which is a simple overlap integral between the two distributions. We will come back to the subject of the threshold for accepting the agreement as good - this turns out to be about 60% for most purposes.

Likelihood indices

To see if a probability distribution is likely to combine well with the group of other distributions we can define an overlap integral similar to that for the agreement. Assuming that the likelihood distribution of interest is p(t) and the combination of all of the other distributions is r(t) the likelihood index is defined (for this program) as:

Overall agreement

To calculate on overall pseudo-Bayes-Factor, B, for the model, one might simply multiply together all of the agreement indices for the individual distributions (B=A1 A2 A3 ... An). However, to make this figure easier to use a modification of this definition has been used here. The rationale is this: since the agreement indices Ai will average about 1, their logarithms will tend to average about zero (this is not strictly true but a reasonable first order assumption); assuming these deviations are all random, ln(B) will tend deviate from zero as a random walk; the scale of any such deviation will therefore tend to be proportional to the square root of n.

The most useful definition for the overall agreement is therefore found to be

Variations from 100% will have the same significance as they do for the individual agreements.

With the exception of the power term, this is then a pseudo Bayes-factor (see for example chapter 9 of Gilks et al 1996 and the agreement indices Ai are factors of this term. The Bayes factor here is being used to compare the constrained model to the entirely unconstrained model. The power term merely provides the convenience of a suitable acceptance cutoff which is independant of the total number of terms (see below).

This overall agreement function has some interesting properties. The first of these can be found by considering the particular case of combinations of probability distributions (here performed with Combine and D_Sequence): in such cases the errors are not independent as all of the comparisons are made with the same posterior distribution which has an error which decreases with square root of n. The special case of combinations of gaussian distributions (generated with C_Date) gives identical results to the direct combinations of gaussians (using C_Combine) and so it seems reasonable that the threshold for acceptance of the combination should be the same as the chi squared test normally performed. It turns out (and this can be verified by trying groups of values) that the threshold for Aoverall which corresponds to the chi squared test at 5% is equal to:

At this threshold, we can then calculate the logarithmic average of the individual agreement indices that make this up. This is given by:

These results are tabulated here for some values of n: 

________________________

  n     An(%)    A'n(%)
________________________
  1     70.7     70.7
  2     50.0     61.3
  3     40.8     59.6
  4     35.4     59.5
  5     31.6     59.8
  6     28.9     60.2
  7     26.7     60.7
  8     25.0     61.3
  9     23.6     61.8
 10     22.4     62.3
 15     18.3     64.5
 20     15.8     66.2
 25     14.1     67.6
 30     12.9     68.8
 40     11.2     70.7
 50     10.0     72.2
 60      9.1     73.4
 80      7.9     75.3
100      7.1     76.7
________________________
From this table it can be seen that for most purposes, where the number of constraints is small, a reasonable value of the agreement i of a single constrained distribution, given by A'n, is approximately:

A'c = 60%

This is then taken as the threshold of acceptance for the individual agreement indices.

Aoverall was defined to be An index based on this, whose significance would be independent of n. For this reason, A'c is also always used by OxCal as the threshold for Aoverall when the errors are non-correlated. When the errors are correlated (as for combinations and wiggle matches), An is used instead.

The mathematical formulation here is not entirely rigorous, and given the nature of the problem this is probably inevitable. However, these agreement indices do give a good working indication of when a statistical model is inconsistent with the age measurements used.