Normalization: Difference between revisions

From Cybis Wiki
Jump to navigation Jump to search
 
(11 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''Normalization''' is, within the context of the [[CDendro]] program, the process of transforming (preparing) data before a [[Correlation Analysis]].
'''Normalization''' is, within the context of the [[CDendro]] program, the process of transforming (preparing) data before a [[Correlation analysis]].
Doing correlation analysis directly on untransformed ring width data is not a way to achieve proper cross dating results. Neither
Doing correlation analysis directly on untransformed ring width data is not a way to achieve proper cross dating results. Neither
does a [[Detrending transformation]] lead to data that works with the correlation analysis.
does a [[Detrending transformation]] lead to data that works with the correlation analysis.
Line 31: Line 31:


===Hollstein normalization===
===Hollstein normalization===
Ernst Hollstein introduced the concept of "Wuchswert" (growth value) where he basically divided this years ring width by  
Ernst Hollstein introduced the concept of "Wuchswert"  
<ref>Ernst Hollstein: "Mitteleuropäische Eichenchronologie". Verlag Philipp von Zabern, Mainz am Rhein, 1980, ISBN 3-8053-00964. Page 14. </ref>
(growth value) where he basically divided this years ring width by  
the ring width of the previous year. When you plot such a curve of ring width quotes you will often get a number of high peaks  
the ring width of the previous year. When you plot such a curve of ring width quotes you will often get a number of high peaks  
which will make the crossdating analysis difficult.
which will make the crossdating analysis difficult.
Line 37: Line 39:
logarithm by hand as Hollstein certainly did is not difficult if you use a semi-logarithmic paper for that convertion.)
logarithm by hand as Hollstein certainly did is not difficult if you use a semi-logarithmic paper for that convertion.)
:<math>dw(y) = \ln {\frac{w(y)}{w(y-1)}}</math>
:<math>dw(y) = \ln {\frac{w(y)}{w(y-1)}}</math>
This can also be written as:
:<math>dw(y) = \ln {w(y)} - \ln {w(y-1)}</math>


===Baillie/Pilcher normalization===
===Baillie/Pilcher normalization===
The Baillie/Pilcher transformation divides this years (current) ring width with the mean value of the surrounding five ring widths
The Baillie/Pilcher transformation (presented as early as in 1973) divides this years (current) ring width by the mean value of the surrounding five ring widths
including the current ring. Then the natural logarithm is taken on that value as with the Hollstein transformation.
including the current ring. Then the natural logarithm is taken on that value as with the Hollstein transformation.
(Referens till Ba/Pi-artikel)
<ref>Baillie, M. G. L. and J. R. Pilcher A simple crossdating program for tree-ring research.
[http://www.treeringsociety.org/ Tree-Ring Society] [http://arizona.openrepository.com/arizona/handle/10150/260029 Tree-Ring Bulletin 33:7-14]''</ref>
:<math>dw(y) = \ln {\frac{5 w(y)}{w(y+2)+w(y+1)+w(y)+w(y-1)+w(y-2)}}</math>


====A weakness with the Hollstein and Baillie/Pilcher transformations====
When just "plain t-values", without any further discussions or explanations, are mentioned in literature or reports, it is often the t-values according to Baillie/Pilcher normalization which is referred to.
 
===A weakness with the Hollstein and Baillie/Pilcher transformations===
When there is one or more very very narrow ring widths in two curves being compared, then the
When there is one or more very very narrow ring widths in two curves being compared, then the
correlation analysis will be fooled by the "overall shape" of the curves where the effect of the narrow rings will dominate the view.  
correlation analysis will be fooled by the "overall shape" of the curves where the effect of the narrow rings will dominate the view.  
Normally this does not happen with actual measurement data, though if e.g. a missing ring is replaced by a very small number this may happen.
Normally this does not happen with actual measurement data, though if e.g. a missing ring is replaced by a very small number this may happen.
It may also happen if data created with the [[Corridor method]] is offset to make all values positive and then used as if it was  
It may also happen if data created with the [[Corridor method]] is offset to make all values positive (and the smallest value is made very small) and then the data is used as if it was ring width data.
ring width data.
See also [http://www.cybis.se/forfun/dendro/foolbynarrow/index.htm How to get fooled by your normalization method and some too narrow ring widths]


===Besancon Index E normalization===
===Besancon Index E normalization===
Line 73: Line 82:
==Notes==
==Notes==
<references/>
<references/>
[[Category:Methods]] [[Category:CDendro]]

Latest revision as of 10:45, 13 April 2016

Normalization is, within the context of the CDendro program, the process of transforming (preparing) data before a Correlation analysis. Doing correlation analysis directly on untransformed ring width data is not a way to achieve proper cross dating results. Neither does a Detrending transformation lead to data that works with the correlation analysis.

Proportion of last two years growth

The default transformation of CDendro, is to divide the ring width of the current year with the sum of the ring widths during the last two years. This will give a series of numbers between 0 and 1, i.e.

where y is a year number like 2009, 2008, 2007 etc. and w(y) is the ring width of year y.

I.e. this transformation calculates for each year, that years proportion of ring width growth during the last two years. Within CDendro this transformation is named "Proportion of last two years growth" or "P2Yrs". The effect of the P2Yrs transformation is that low frequency data, i.e. long time variations of growth rate, is removed from the curves to be crossdated. Another property of the transformation is that the resulting curve has no extreme peaks which is essential not to fool the correlation analysis.

The P2Yrs transformation was first put into the predecessor of CDendro, a DOS based program of 1995-1997. The P2Yrs transformation is indeed related to earlier transformations used for crossdating like the Hollstein and Baillie/Pilcher transformations which give almost equally good results but are mathematically somewhat more complicated and not as conceptually clear as the P2Yrs transformation.

Extensive testing [1] has shown that the P2Yrs transformation gives good crossdating results - i.e. a correlation analysis on P2Yrs data will usually sort out the correct match between two curves.

Other moving average transformations

The P2Yrs transformation could be considered a member of a class of "moving average transformations", where the current ring width is divided with a mean value of surrounding ring widths considered to be inside a moving frame. Parameters of this class would then be

  • the width of that frame (divisor frame length),
  • where that frame starts in relation to the current ring (divisor frame offset),
  • whether the current ring is included in the average
  • whether we should take the natural logarithm of the result of the division.

Within CDendro there is a "Toolbox for normalization algorithm based on sliding frame" where you can experiment with variations of the normalization algorithm.

Hollstein normalization

Ernst Hollstein introduced the concept of "Wuchswert" [2] (growth value) where he basically divided this years ring width by the ring width of the previous year. When you plot such a curve of ring width quotes you will often get a number of high peaks which will make the crossdating analysis difficult. Apparently to avoid this problem, Hollstein took the natural logarithm of each quote to get his "Wuchswert". (Taking the logarithm by hand as Hollstein certainly did is not difficult if you use a semi-logarithmic paper for that convertion.)

This can also be written as:

Baillie/Pilcher normalization

The Baillie/Pilcher transformation (presented as early as in 1973) divides this years (current) ring width by the mean value of the surrounding five ring widths including the current ring. Then the natural logarithm is taken on that value as with the Hollstein transformation. [3]

When just "plain t-values", without any further discussions or explanations, are mentioned in literature or reports, it is often the t-values according to Baillie/Pilcher normalization which is referred to.

A weakness with the Hollstein and Baillie/Pilcher transformations

When there is one or more very very narrow ring widths in two curves being compared, then the correlation analysis will be fooled by the "overall shape" of the curves where the effect of the narrow rings will dominate the view. Normally this does not happen with actual measurement data, though if e.g. a missing ring is replaced by a very small number this may happen. It may also happen if data created with the Corridor method is offset to make all values positive (and the smallest value is made very small) and then the data is used as if it was ring width data. See also How to get fooled by your normalization method and some too narrow ring widths

Besancon Index E normalization

The frame length is 7 and centred around the current ring width (w). The current ring width does not take part in the mean value calculation. The minimum and the maximum values within the frame are excluded from the mean value (m). The normalized value normV is calculated as normV = w / m

Logarithm may be optionally applied in two different ways:

A. Log on normal value: normV = log(normV) if normV = 0 then normV = 0.001 to avoid misunderstanding this value as a zero ring

B. Besancon variant of applying logarithm:
valueForLog = 100 * ( w / m - 1 )
If abs(valueForLog) < 0.001 Then valueForLog = 0.001
If valueForLog > 0 then normV = log(valueForLog) else normV = - log(-valueForLog)
If abs(normV) < 0.001 Then normV = 0.001

The Besancon variant seems to give very high standard deviation values when normalized values for a reference curve (a sum-curve) are created as mean values of the normalized values of the collection members.

Notes

  1. Torbjörn Axelson and Lars-Åke Larsson: What is a good TTest value
  2. Ernst Hollstein: "Mitteleuropäische Eichenchronologie". Verlag Philipp von Zabern, Mainz am Rhein, 1980, ISBN 3-8053-00964. Page 14.
  3. Baillie, M. G. L. and J. R. Pilcher A simple crossdating program for tree-ring research. Tree-Ring Society Tree-Ring Bulletin 33:7-14