Partitioning Diversity

[pauljost]

The Journal of Ecology's editor opened a 30 page forum to debate on my brother Lou's papers on proper usage of diversity indices.  The editor also made the announcement that my brother's 2007 paper may be the greatest advancement in measuring diversity for ecology since Whittaker's paper 50 years ago!!!  My brother, Lou, is really excited about it!  I've attached the editor's comments and like to brag about my brother a bit...

Regards,

Paul Jost

[edfrank]

Paul,

Congratulations to your brother.  This is fantastic news!  I hope someone will send e copies of the comments when they appear in print.  Thanks for posting this.

Ed

[pauljost]

For those that are interested, here are all my brother's papers on diversity:

               

                       

EntropyAndDiversity_Jost2006.pdf

                                               

(159.53 KiB) Downloaded 50 times

               

               

               

                       

PartitioningDiversityIntoIndependentComponents_Jost2007.pdf

                                               

(157.07 KiB) Downloaded 49 times

               

               

               

                       

PartitioningDiversityIntoIndependentComponents_Append_A.pdf

                                               

(91.73 KiB) Downloaded 51 times

               

               

               

                       

PartitioningDiversityerrata.pdf

                                               

(68.43 KiB) Downloaded 40 times

               

               

               

                       

Biometrics070539Final2008.pdf

                                               

(132.95 KiB) Downloaded 46 times

               

               

               

                       

Jost2008GstMolecularEcologyBlackwell.pdf

                                               

(190.63 KiB) Downloaded 44 times

               

               

               

                       

JostMismeasuringBiologicalDiversity.pdf

                                               

(160.25 KiB) Downloaded 46 times

               

               

               

                       

Jost2009ReplyRymanLeimarMolEc.pdf

                                               

(135.62 KiB) Downloaded 45 times

               

               

               

                       

Jost2010DiversityAndEvenness.pdf

                                               

(283.08 KiB) Downloaded 47 times

               

               

               

                       

Jost(2010)Ecology.pdf

                                               

(106.52 KiB) Downloaded 46 times

               

               

               

                       

Jostetal2010DiversityAndDistributions.pdf

                                               

(262.02 KiB) Downloaded 46 times

               

               

There are more to come.  He has just finished a book chapter on the topic of similarity measures.  An article coauthored by Anne Chao, Chun-Ho Chiu, and my brother, Lou Jost, on phylogenetic diversity will also come out in a few months in the Proceedings of the Royal Society B.

[edfrank]

Paul,

Thanks for posting these papers here.  The second paper in the list  PartitioningDiversityIntoIndependentComponents_Jost2007.pdf    is the paper that provides the theoretical basis for the Ecology Forum.  I believe you, or somebody passed along a copy to me previously.  I certainly remember the unpleasent math in the paper.  The concepts are presented in a straight forward and easy way to understand even if someone has trouble following the mathematical derivations involved.  I am sure that some of these concepts can be applied to some of the hieght studies we are doing here in ENTS.  You should invite your borther to join.  I am sure he has lots of free time when not dong field work in Ecuador and otherwise working on his research.  :)

Ed Frank

[dbhguru]

Paul,

    Congratulations to your brother Lou and to the entire Jost family. What an accomplishment! You have spoken about Lou from time to time and we've all been anxious to hear more, but I doubt many of us realized just how distinguished Lou is. Again, congratulations.

Bob

[Don]

Paul-
Clearly Lou is in the forefront on this.  
I did scan most of the kindly offered supporting papers, for non-math infused rationale, and have come to a conclusion that may or may not align with his...
But first, coming from a forester's perspective, where sampling and statistics forms our objective conclusions from performed research, I have to make a comment here.  It's my impression that we differ from forest ecologists/ecologists at least in this manner.  Forest researchers will randomly select plots to statistically sample a given experimental block (itself subject to levels of randomness), perform measurements, and with analysis make conclusions about the parts and the whole.
Forest ecologists on the other hand, seem to walk into an area that to them seems to be an ideal example of what it is they are trying to define, and take measures of that 'community' so as to qualitatively summarize what it is they thought they'd seen to begin with.

That said, and I'm sure some toes have inadvertently been stepped on, having read the offered forum, the attached papers, I'd have to say that Whittaker had at least one thing right, that seemed to get lost with the fooforall that followed.  Inherent to Whittaker's view, is the suggestion that the differences between local and regional communities is some measure of habitat gradient differences as you go from one to the other.  Now without specifics being made anywhere I was able to comprehend, discussions of measures of richness and evenness being dependent, to me is a direct function to habitat gradient measures.

Hope I've made some amount of sense and not embarrassed myself!
-Don

[edfrank]

Don,

You wrote:

It's my impression that we differ from forest ecologists/ecologists at least in this manner. Forest researchers will randomly select plots to statistically sample a given experimental block (itself subject to levels of randomness), perform measurements, and with analysis make conclusions about the parts and the whole.  Forest ecologists on the other hand, seem to walk into an area that to them seems to be an ideal example of what it is they are trying to define, and take measures of that 'community' so as to qualitatively summarize what it is they thought they'd seen to begin with.  

This is an interesting observation.  I am neither a forester, nor a forest ecologist, but I do have some knowledge of sampling and experimental design so I have no particular axes to grind.  I don't think your observation is really a fair appraisal of the situation.  There are problems with both approaches

If the analysis is based upon statistical analysis of random plots, that analysis is only correct to the degree in which the random sampling is representative of the whole.  Is what you sampled in the random sampling typical or not?  If an atypical of anomalous area or feature is included in one of the random plots, that feature will be over emphasized in the statistical analysis.  If there is a large number of plots, these anomalous samples will be minimized by the weight of data from other sample plots.  If there is only a few plots, then the data will be skewed.  One example of this I have come across was an analysis of tree populations in the island in the Allegheny River Island Wilderness.  One report, (I would need to look up the specifics), listed butternut as one of the major species on the islands representing a high percentage of the tree population.  In fact from personal experience on these islands, I have found that butternut is present, but only sporadically and in patches.  The random transect or plot happened to include one of these patches of butternut and the statistical analysis of the sampling led to an erroneous conclusion about the relative percentage the butternut in the population.  Random plots work best where there are large numbers of samples, but can misrepresent the actual ground truth if only a limited number of plots are analyzed.

In the second scenario there is no doubt that selection of a site can lead to people finding what they want to find.  But this is not always the case.  In site selection, if done properly, there should have been developed a set of criteria for what sites to select.  These can be developed by some random test plots, by general qualitative surveys of the entire area.  The criteria would include what major characteristics or features are typically present, and what features are present but uncommon.  Then a site site would be selected for detailed sampling and analysis based upon this set of selection criteria.  This would include more detail than in the preliminary overview of the area. The site selected would include all of the major features found to be typical in the preliminary surveys, and minimize the atypical elements.  The quality of the analysis would be vary based on how well the selection criteria was able to characterize a typical area within the forest.  If only a limited number of sites are sampled, this has a potential to provide a better characterization of a typical site than does a random sampling, and it will allow a more detailed analysis focusing on one site rather than many sites.  But it also has the potential to add observer bias as well.  So neither methodology is without problems.  

Ed

[Lee Frelich]

Ed, Don:

Yes there are problems with both (sampling a predefined area versus 'random' samples').  I do either or both under different circumstances.  Often I use a stratified random or stratified systematic sample, which sometimes has intermediate characteristics.

With regard to 'random' samples, in a forest setting random samples do not guarantee independence of the samples, and systematic samples (e.g. set out on a grid) are more accurate for most purposes, and are actually more independent for the same average distance between plots.  You  have to remember that there is no statistical test that requires a random sample, they require independent samples (even non parametric methods), and we make the assumption that random samples gives independence, and get so stick on random samples that we forget that independence of samples is what we are after. Since forests are patchy, one can measure the level of spatial dependence by distance, and then choose to place plots far enough apart so that the variables of interest one one plot have no influence on the others nearby--then the samples are independent and a systematic sample will be easier to carry out and more accurate.

Then, also, don't forget the tradeoff between statistical significance and ecological significance. With 1000 plots, one can establish statistically significant differences that are tiny and ecologically/biologically trivial.  People (even scientists, but not me) do this all the time, and publish meaningless results.

Lee

[dbhguru]

Lee,

   I'm glad to see you weigh in on this one. Sampling methodology is pretty darned complicated. I have been unimpressed with the flood of foolishly designed studies that don't prove anything, let alone what their supposed to prove. I saw that over and over in DOD. The social sciences are loaded with poorly designed studies. That comes as no surprise. However, I am disappointed that ecologists fall into some of the same traps.

Bob

[pauljost]

I have a brief update.  Because of Lou's recent achievements (http://www.loujost.com,http://www.ecominga.net our family has just been told that Lou has been awarded the virtual equivalent of an alumni of the year award for 2011 from Lawrence University, http://www.lawrence.edu.  It has not been formally announced, but is described as follows:

"The George B. Walter '36 Service to Society Award
This award is presented to recognize alumni of Lawrence University or Milwaukee-Downer College who best exemplify the ideals of a liberal education through socially useful service in their community, the nation, and/or the world. This award honors George B. Walter '36, alumnus, faculty member, educator, and dean of men, whose work at the college and beyond promoted his conviction that every individual can and should make a positive difference in her or his world."

Regards,

Paul