tag:blogger.com,1999:blog-7805174.post4901648686024775767..comments2023-10-19T03:06:44.613-07:00Comments on AMac: Stoat's First Debate on the Use of the Lake Korttajarvi (Tiljander) Proxies by Mann et al (2008)AMachttp://www.blogger.com/profile/08872008617279528583noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-7805174.post-82392995193193011682010-02-15T14:41:53.538-08:002010-02-15T14:41:53.538-08:00--- concluding ---
There are doubtlessly ways for...--- concluding ---<br /><br />There are doubtlessly ways for Mann et al to argue this point differently from how I've presented it. By all means, they should do so. When they submit their corrected version of the paper as a belated response to McIntyre and McKitrick's 2009 Comment. <br /><br />But the extended Groundhog Day arguments about "No correction needed" on the upside-down use of the Tiljander proxies serve to excuse Mann et al from facing up to their responsibilities as scientists. The Methods should be correct, certainly as regards errors of the magnitude of using uncalibratable proxies upside-down. The Discussion should squarely discuss the data that Mann et al generated and presented.<br /><br />They should not be using PNAS as a vanity press.AMachttps://www.blogger.com/profile/08872008617279528583noreply@blogger.comtag:blogger.com,1999:blog-7805174.post-23910525713998375322010-02-15T14:38:53.907-08:002010-02-15T14:38:53.907-08:00--- continuing ---
With hindsight, readers have i...--- continuing ---<br /><br />With hindsight, readers have important information that these authors lacked in 2008. We know that the Tiljander proxies are about the worst possible data to add into a temperature reconstruction. Since they are uncalibratable, all four are grossly miscalibrated to the instrumental record. The "skill" test that was done by splitting the 1850-1995 period in half gave encouraging results that we now know were entirely spurious. Worse, two and probably three of the proxies were added in to the reconstruction algorithim in an upside-down orientation. Whatever "warmer" information they contain was misread as "colder", and vice versa.<br /><br />So, with this insight, we can see that SF8a is best seen as a test of a null hypothesis. <br /><br />Null Hypothesis: "The combination of Mann 08's proxy selection protocol and computational algorithm does not produce a valid temperature reconstruction."<br /><br />"We have derived a paleotemperature reconstruction that we believe has merit ("NH CPS minus 7"). If we pollute the input data with spurious information to generate a pseudo-reconstruction ("NH CPS"), the pseudo-reconstruction will deviate greatly from the genuine one. If it does not, that demonstrates that the algorithm we used to construct the genuine reconstruction is refractory to corrupt, meaningless input. In that case, we must accept the null hypothesis."<br /><br />Alas, the null hypothesis <i>should</i> be accepted.<br /><br />--- continues ---AMachttps://www.blogger.com/profile/08872008617279528583noreply@blogger.comtag:blogger.com,1999:blog-7805174.post-1032100128455720612010-02-15T14:18:34.258-08:002010-02-15T14:18:34.258-08:00Lazar, that's a great comment, and thanks for ...Lazar, that's a great comment, and thanks for the tip on TCO's remarks. <br /><br />Thinking on your prior writing helped me to clarify what I see as a central problem with Tiljander. I know it's annoying to be faced with addressing a sprawling critique. So here's a focused one. What with time constraints, food for thought at least for now.<br /><br />I think that the Tiljander proxies represent one thing in Mann '08, as the authors saw it. However, once the authors acknowledge the errors in their use of these proxies, it becomes possible to see that the authors' assumptions were wrong. Tiljander's proxies are, instead, important in a very different way. <br /><br />The frame of reference of Mann '08's authors was:<br /><br />"We have a good system for extracting temperature signals out of proxies to generate paleoclimate reconstructions. Let's go well beyond bristlecones and search for other proxies. In the case of the Tiljander proxies: are they valid? Do the concerns of the original authors disqualify them? Well -- we'll do the analysis with and without Tiljander (Supplemental Figure 8a). The reconstruction with Tiljander are nearly identical or very similar to those without (<b>*</b>). We conclude that the Tiljander proxies tell a story that is consistent with those told by the other proxies we've employed."<br /><br />(<b>*</b>) Eyeballing the different Supplemental Figure 8a's:<br /><br />Original SF8a, "NH CPS" (green line) is virtually identical with "NH CPS minus 7" (black line).<br /><br />Once-revised SF8a, "NH CPS" (green line) is virtually identical with "NH CPS minus 7" (black line), except 1020-1100, 1120-1200, 1350-1390, and 1630-1640.<br /><br />Twice-revised SF8a, "NH CPS" (black line) is virtually identical with "NH CPS minus 7" (green line), except 1610 and 1640-1680.<br /><br />--- continues ---AMachttps://www.blogger.com/profile/08872008617279528583noreply@blogger.comtag:blogger.com,1999:blog-7805174.post-51887492203802419362010-02-15T12:34:15.135-08:002010-02-15T12:34:15.135-08:00Another new issue;
The "red noise" crit...Another new issue;<br /><br /><i>The "red noise" criticism of Roman M and Jean S (Climate Audit) and Jeff Id (the Air Vent) is that Mann's procedure will generate hockey stick graphs from such input.</i><br /><br />I don't know the precise nature of the criticisms. So I don't know if what I say will be of any use or relevance. A relatively high likelihood of obtaining a hockey-stick shape from red noise is unavoidable if you're correlating with the instrumental record. The trend in the target series is matched whilst everything outside tends to flatline. So such a criticism (as I've presented it) is true, but it is also somewhat facile. Cross-validation is important. You can generate a hockey-stick shape with red noise, sure. Can you get it to pass validation? Note that the 'blade' is not flat in Mann et al., there are peaks and troughs where we expect them to be from energy-balance models and other studies. <br /><br /><a href="http://www.mpimet.mpg.de/fileadmin/staff/claussenmartin/publications/bauer_al_1000_grl_03.pdf" rel="nofollow">Bauer, E., M. Claussen, V. Brovkin, and A. Huenerbein (2003)</a>, Assessing climate forcings of the Earth system for the past millennium, Geophys. Res. Lett, 30(6), 1276, doi:10.1029/2002GL016639.<br /><br /><a href="http://dusk.geo.orst.edu/prosem/GEO518_Panel01_Oerlemans_2005.pdf" rel="nofollow">Oerlemans, J. (2005)</a>, Extracting a climate signal from 169 glacier records, Science, 308(5722), 675, doi:10.1126/science.1107046.<br /><br /><i>So these discussions have a circular quality. X is right, well you proved that X is wrong, if I were Mann I would fix X, Mann should fix X, come to think of it his figure claims X doesn't make a difference, Mann refused to fix X, thus No Correction Needed, in other words X is fine as is, Mann need not fix X.</i><br /><br />I think, from an scientist's 'I want to understand nature' point of view, it is sufficient to demonstrate that an alleged error can only have trivial effects on the output of an algorithm. Acknowledging the nature of the allegations is important again but is another issue. How notification is issued is another (procedural) issue again. I think all these issues are being mixed up in the various responses.<br /><br />Anyway, thanks for engaging.<br /><br />* If you're into McIntyre's criticisms of Mann's various output, it's worthwhile reading TCO's comments at CA.Lazarnoreply@blogger.comtag:blogger.com,1999:blog-7805174.post-16669710323466796492010-02-15T12:26:19.270-08:002010-02-15T12:26:19.270-08:00AMac,
My time is severely strained so this may be...AMac,<br /><br />My time is severely strained so this may be my last response. As my friend TCO (*) might say, it would be better if you disaggregated these issues; the stuff you write on procedural issues, e.g. notification and documentation of changes, from; <i>new</i> methodological issues you raise, from; the actual effects on the given reconstruction of including/excluding Tiljander proxies. These are seperate issues.<br /><br /><i>I do not agree that "The alleged error has little effect on results and conclusions." First, the version of Figure 8a (from memory) that you linked to is not the original</i><br /><br />This (the fact of a change to the published materials) is very far from my meaning of effect on results and conclusions. Results are the output of the algorithm, which are little changed after excluding upside-down Tiljander proxies. The conclusions include;<br /><br /><i>We find that the hemispheric-scale warmth of the past decade for the NH is likely anomalous in the context of not just the past 1,000 years, as suggested in previous work, but longer. This conclusion appears to hold for at least the past 1,300 years</i><br /><br />... the 1000-year conclusion remains unchanged. You may argue that the 1300-year conclusion needs to be revised, <i>if you also exclude tree-ring proxies</i>. Maybe. So the inclusion of an updated graph shows to me... that the conclusions are (relatively) unchanged.<br /><br /><i>that is in the SI at the PNAS website (last I checked). He's changed it twice since publication. Altering stuff without properly documenting it: this is poor practice that has the effect of hobbling critics.</i><br /><br />This may be true or not, but it is a seperate issue. The following are also seperate issues;<br /><br /><i>How many proxies were used to calculate each step? [...] Does each of the, say, 25 proxies contribute 4% to the signal? Or are a small number crucial, with the rest effectively serving as bystanders? [...] A simple and to me likely explanation is Confirmation Bias. Mann et al set out to identify a novel set of proxies to prove that the previously-studied tree-ring proxies told the right paleoclimate story</i><br /><br />This last is in my view highly speculative, I do not find evidence for the claim.<br /><br />--continued--Lazarnoreply@blogger.comtag:blogger.com,1999:blog-7805174.post-40263520570539735392010-02-14T20:02:47.669-08:002010-02-14T20:02:47.669-08:00-- continuing --
A simple and to me likely explan...-- continuing --<br /><br />A simple and to me likely explanation is Confirmation Bias. Mann et al set out to identify a novel set of proxies to prove that the previously-studied tree-ring proxies told the right paleoclimate story: the hockey stick story. They identified candidates, qualified them, ran the analysis, found the expected pattern, and published.<br /><br />But "hockey stick" might not be the correct story. There are other outcomes to consider. <br /><br />Examples:<br /><br />a. MWP temperatures might have been as high as today's, and proper analysis of relevant proxies would show that.<br /><br />b. Available proxies may be so noisy that a meaningful temperature history cannot be teased out of them.<br /><br />A null hypothesis would be, "there was no temperature trend, 0 AD - 1850 AD." So: if one takes a set of synthetically-generated 2,000 year pseudoproxies and runs them through the algorithm, does it return this no-trend result? The "red noise" criticism of Roman M and Jean S (Climate Audit) and Jeff Id (the Air Vent) is that Mann's procedure will generate hockey stick graphs from such input. I have not done such analyses myself (I'm far from fluent in R and other languages). Given the track record of these three, I give a lot of weight to their criticism. I'm not aware that Mann has addressed the point.<br /><br />When I got into looking at Mann et al (2008), it didn't occur to me that people would defend the obvious mistake of retaining uncalibratable upside-down proxies. With these crude points not out of contention, I haven't returned to any of the subtler arguments for some time (till just now, at your prompting). <br /><br />Nick Stokes offers, "I would discard it [the upside-down proxies]." In his Response to McIntosh and McKitrick, <i>Mann does not</i>.<br /><br />Gavin Schmidt goes further, ending his analysis of Mann et al (2008) with "No correction needed." Nick Stokes endorses Gavin's remark.<br /><br />So these discussions have a circular quality. X is right, well you proved that X is wrong, if I were Mann I would fix X, Mann should fix X, come to think of it his figure claims X doesn't make a difference, Mann refused to fix X, thus No Correction Needed, in other words X is fine as is, Mann need not fix X.<br /><br />Let's see (1) the authors acknowledge their mistakes on Tiljander instead of scorning critics for noticing their errors ("bizarre"). Then let's see (2) the authors correct their results so that they aren't contaminated by the obvious errors. At that point we can (3) go on to constructively discuss the merits and shortcomings of the repaired paper. <br /><br />That would be preferable to this Tiljander Proxy remake of Groundhog Day.<br /><br />Lazar, thanks again for writing out your thoughts. Sorry I didn't get to all of the points you raised.AMachttps://www.blogger.com/profile/08872008617279528583noreply@blogger.comtag:blogger.com,1999:blog-7805174.post-58076719200426729802010-02-14T19:51:40.133-08:002010-02-14T19:51:40.133-08:00Lazar, thanks for the comments. A couple of respo...Lazar, thanks for the comments. A couple of responses.<br /><br />I meant "lethal to Mann et al" to mean lethal to the paper in its current form. The uses of the Tiljander proxies are wrong. Mann can certainly correct the Methods and Results. He'd have to do that before we could see what that revised paper looked like.<br /><br />I do not agree that "The alleged error has little effect on results and conclusions." First, the version of Figure 8a (from memory) that you linked to is not the original that is in the SI at the PNAS website (last I checked). He's changed it twice since publication. Altering stuff without properly documenting it: this is poor practice that has the effect of hobbling critics.<br /><br />Second, it's very difficult to understand what these figures show.<br /><br />How many proxies were used to calculate each step? At each step, which proxies ended up being heavily weighted? Does each of the, say, 25 proxies contribute 4% to the signal? Or are a small number crucial, with the rest effectively serving as bystanders?<br /><br />Consider the case of the (NH CPS and NH CPS minus-7) figure that you link. Let's engage our knowledge that all four of the Tiljander proxies were wildly miscalibrated, and that two (probably three) of the four Tiljander proxies are also inverted. The Tiljander series thus contribute a great deal of bad information.<br /><br />Yet the CPS minus 7 trace (green line) is virtually superimposable over the original CPS trace (black line) with the exception of two spots between 1600 and 1720. In the as-published version, the superimposability is almost complete.<br /><br />Taking out very-bad information and ending up with virtually the same reconstruction. Two ways to accomplish this: (1) If you gave the bad information almost no weight in the first place, or (2) If confirmation bias has led you to construct an algorithm that returns more-or-less the same hockey stick trace, over a wide range of inputs.<br /><br />Who has looked at at the inputs and outputs of this paper and claims to understand what's going on here? For the major figures, how many proxies go into the curve for each century's output? How are they weighted? Would the substitution of "red noise" pseudoproxies for the actual proxies yield a trendless curve, or just another hockey stick?<br /><br />-- continues --AMachttps://www.blogger.com/profile/08872008617279528583noreply@blogger.comtag:blogger.com,1999:blog-7805174.post-14365847170292844172010-02-14T14:16:13.944-08:002010-02-14T14:16:13.944-08:00AMac,
I've scanned your comments about Tiljan...AMac,<br /><br />I've scanned your comments about Tiljander, so, as briefly as I can...<br /><br />I'm unimpressed with the canned 'response' that multiple regression is insensitive to the sign of input series. Criticisms obviously relate to the signs on regression coefficients running contrary to a priori physical understanding. The effect probably ought to have been screened (yeah, 1000+ proxies makes that a lot of work, life is tough). I would bet on McIntyre being right, though I wouldn't bet much money. More generally, I've come to dislike the 'throw it all in and see what comes out of the soup'-type correlational analysis. OTOH Mann et al. did cross-validate and ran an impressive number of sensitivity tests.<br /><br />The alleged error has <a href="http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/NHcps_no7_v_orig_Nov2009.pdf" rel="nofollow">little effect</a> on results and conclusions. I don't understand why you think that it is<br /><br /><i>lethal to Mann et al (PNAS, 2008)</i><br /><br /><i>All</i> models, <i>all</i> data, and <i>all</i> methodologies are flawed. Do the flaws matter? This comes back to effect sizes.<br /><br />'Does Mann et al. contain correlations which are physically implausible but do not substantially effect the results?', is not an interesting question from a science point of view. What are the physical signals contained in the Tiljander proxies?, is an interesting question. Why do they correlate with temperature?, is another.<br /><br /><i>Pro-AGW-Consensus argument that a data series cannot be used upside-down</i><br /><br />I don't understand how that is meant to be 'pro-consensus'. 'The consensus' is a list of broadly similar, widely shared beliefs that are held by climate scientists based on their interpretation of the scientific literature. I cannot think of any consensus statement being significantly effected by the use of Tiljander proxies in one (or two) studies.<br /><br /><i>it strikes me that leaders of the AGW Consensus have an overarching</i><br /><br />There are no leaders. The consensus is not a social group.<br /><br /><i>If the varve XRD proxy is inconsequential, why include it at all?</i><br /><br />That was not known a priori. The corollary is, if it is inconsequential, why remove it?<br /><br /><i>The publication [...] ought to be) extremely embarrassing to the high-impact PNAS and its editors and peer reviewers</i><br /><br />Peer review is not expected to check signs on regression coefficients, and generally doesn't.<br /><br /><i>Those who defend the indefensible--because of whatever mix of ignorance, team loyalty, hubris, and confirmation bias--forfeit the presumption of trust on related issues.<br /><br />That's not just Nick Stokes (Lucia's, yesterday) and Gavin Schmidt. It's many others as well (including Bob Grumbine, who commented earlier on this thread). Nobody is dissenting from the "party line."</i><br /><br />Nick Stokes states;<br /><br /><i>Mann [...] came out with a proxy that Tiljander reasonably argues is the wrong way around [...] I would discard it.</i><br /><br />Which are opinions I share. We both agree with the scientific consensus. What more would you desire/expect from us?<br /><br />Robert Grumbine was taking apart, in his usual <a href="http://moregrumbinescience.blogspot.com/2009/10/sound-and-fury-at-wuwt.html" rel="nofollow">educational</a> style, a boneheaded claim by Watts, as well as urging <i>caution</i> with regard to other claims being made. I have no prolem with either.<br /><br />So, I'm not really sure what or whom "the party line" is meant to represent. Are some being overly defensive of Mann et al.? Probably. Are others greatly overhyping the importance of alleged issues? Again, probably.Lazarnoreply@blogger.com