Posts
One of the joys of blogging for me is interacting with people I'd never have met otherwise. My posts on the Lancet study of Iraqi deaths (background here) have afforded many opportunities for that. One in particular prompts this post.
On Friday I received an e-mail from Sean Gourley, a physicist at the University of Oxford and Royal Holloway, University of London, who has just co-authored a critical review of the Lancet study. He has graciously allowed me to report on his findings. His fellow researchers on this project are Neil Johnson, also in the Oxford Dept of Physics, and Michael Spagat of the Dept. of Economics, Royal Holloway, University of London.
As I pointed out in this post, the Lancet survey included only residents of urban areas, thus introducing significant bias into the results. Mr Gourley and his co-researchers argue that the survey methodology also excludes many urban residents, making bias problems even worse. The problem is what they call “main street bias”.
The Lancet surveyors selected clusters by randomly choosing administrative units within Iraq’s Governorates in proportion to population. Then:
The third stage consisted of random selection of a main street within the administrative unit from a list of all main streets. A residential street was then randomly selected from a list of residential streets crossing the main street. On the residential street, houses were numbered and a start household was randomly selected. From this start household, the team proceeded to the adjacent residence until 40 households were surveyed.
Only residential streets crossing a “main street” were eligible for selection. Urban areas typically contain residential streets that do not cross a main street; but the methodology ruled them out. Such streets could never be selected for surveying.
The map below, sent by Mr Gourley, shows a section of Oxford, UK. (The traffic circle near the top left corner is just across a short bridge from Magdalen College at the end of High Street, so it is only a few minutes’ walk from the centre of Oxford.) Three main streets are marked by the three black arrows; each street that does not cross one of them is marked by a red arrow. So, if the Lancet methodology were to be implemented in this section of Oxford, there would appear to be hundreds of households who could never be selected for surveying.
Generally speaking, armed conflict is more common in or near main streets than in side streets. Certainly, given typical traffic patterns, conflict on main streets is likely to endanger more people. So, excluding streets that do not cross main streets would tend to result in overestimation of casualties. Thus, "main street bias".
The crucial question becomes: How exactly did the Lancet surveyors define main streets? That question was put to lead author Gilbert Burnham, epidemiologist at Johns Hopkins, by the Oxford-Royal Holloway researchers. According to an article in Science (behind a subscriber wall, but Sean Gourley sent me a copy), Prof Burnham had two different, and apparently contradictory, answers.
Burnham counters that such streets were included and that the methods section of the published paper is oversimplified. He also told Science that he does not know exactly how the Iraqi team conducted its survey; the details about neighborhoods surveyed were destroyed “in case they fell into the wrong hands and could increase the risks to residents.”
Every time I read a Lancet co-author defend that article, it just gets worse. If Prof Burnham doesn’t “know exactly how the Iraqi team conducted its survey”, how can he know whether the methodological description is oversimplified or not?
Not only that, he admits that data have already been destroyed. To call this bad statistical practice is putting it mildly. Statisticians I know could be reprimanded or even lose their jobs if they destroyed data only a few months after a survey, especially one they knew ahead of time would generate public controversy. In my experience, it is standard procedure to store all survey materials in a secure location for an absolute minimum of three years—and, in practice, usually longer.
Speaking of secure locations, it sounds like the Iraqi surveyors didn’t have one. If they really had no safe place to store completed surveys, they should not have gone out and gathered the data—and not just because of the potential consequences for interviewees if confidential information is leaked. No: the real issue here is the professionalism of the surveyors. Professional surveyors and statisticians take whatever steps are necessary ahead of time to ensure that confidentiality will be protected. If they couldn’t do that, they had no business going into the field in the first place.
Now that essential information has been destroyed, there is no way of verifying Burnham’s claim that all streets, not only those crossing main streets, were included in the sample frame. Failing to ensure that data, analysis, and results can be independently verified is another indication of unprofessional statistical practice.
I’m not the only one who’s irritated that the controversy over the Lancet article’s methodology has turned into a circus.
Michael Spagat, an economist at Royal Holloway, University of London, who specializes in civil conflicts, says the scientific community should call for an in-depth investigation into the researchers’ procedures. “It is almost a crime to let it go unchallenged,” adds [Neil] Johnson.
Fred Kaplan at Slate has also had difficulty getting a straight answer from Gilbert Burnham about his study. Mr Kaplan concludes:
It sounds as if he's saying he didn't destroy the data because they never existed in the first place. If that's the case, how does Burnham know whether his instructions on methodology were followed at all? How can anyone verify the findings? And this is a peer-reviewed article. Who were these peers? And what did they review?
I, too, would be very happy to see a thorough evaluation by independent experts—including statistical methodologists, not just the epidemiologists who seem to be running this little show. The only problem is that essential background information has been destroyed—or was never collected in the first place—so it may already be too late for that.
Previous related posts:
- Lancet study of Iraqi deaths is statistically unsound and unreliable
- Lancet researchers ignored superior study on Iraqi deaths
UPDATE (5 Dec.): Sean Gourley and his colleagues have come up with a specification of the extent of main street bias.
[...] “Main street bias” in Lancet study [...]
There may very well be main street bias in the Lancet article…its hard to say as it depends on the geometry of the street layout.
What is certain is there is no main street bias when the streets are laid out rectangularly since the house is equally likely to be anywhere along the street.
There is however a RESIDENTIAL STREET bias given that all interviews were restricted to households on residential streets.
So we have one CERTAIN bias which leads to underestimates and one CONDITIONAL bias that depends on street geometry which leads to overestimates. Did Johnson cherry-pick the Oxford section to get the desired answer?
Having lived in Oxford for a year, I can assure you that the Oxford section shown is typical, not just of Oxford, but of European towns and cities in general. So, the question is: How are Iraqi towns and cities laid out?
The thing is: for the “conditional” bias to avoided, every street in the town or city must be laid out in a rectangular pattern. A quick search turned up this map of Baghdad and it doesn’t look good for that requirement. (Although, admittedly, not every street is shown.)
The thing is: for the “conditional” bias to avoided, every street in the town or city must be laid out in a rectangular pattern.
Yep but if the departures from rectalinearity are small, the bias will be small and the unconditional bias will dominate. I suppose one could do numerical experiments.
“So we have one CERTAIN bias which leads to underestimates and one CONDITIONAL bias that depends on street geometry which leads to overestimates.”
That’s incorrect for several reasons. You assume the “main street” effect of greater risk of violence applies only to those streets Roberts termed “main.” But what if the increased risk applies both to “main” streats and to the principal residential streets? Then it may be the case that even the “residential” streets surveyed are high risk.
Also, Roberts seems to have claimed to Oxford correspondents that main streets were indeed included. If so, that means only smaller residential streets were not included, making underestimation likely.
All this is the result of what seems a poor method of selecting the starting point.
Roberts should, if professional, be in a position to show the exact methdology and to be able to verify that his methods of selecting starting point were valid and unbiased. Can he do that?
Then it may be the case that even the “residential” streets surveyed are high risk.
And more likely, it may not be.
Seems? Did he or didn’t he?
“And more likely, it may not be.” Pure speculation.
“Seems? Did he or didn‚Äôt he?” According to the Oxford critics. The ambiguity is not their fault but Roberts’; as a matter of principle, such matters should have been specified clearly in the study.
The ILCS used methods ensuring not only that all households were independent, but also that starting points were random. The earlier Johns-Hopkins study sacrificed independence of households within cluster, but did use a more defensible method (GPS) for choosing starting points. This last Johns-Hopkins study sacrifices BOTH independence of households and randomness of starting points, according to the methods stated in the report.
Part of professionalism consists of stating openly the weaknesses in one’s own method and analysing the effects. Choosing starting points only on major residential streets (thus excluding main streets and minor residential streets) means only a subset of the universe was surveyed; it’s inexcusable unless the authors can present evidence of that sub-set being representative of the whole.
Now, if the methodology did include main and minor streets (as the Ocford critics say Roberts told them), then why didn’t the study say so?
If the Oxford critics are being truthful (about Roberts’ claim of proper street inclusion), then the study mis-states the method.
If the Oxford critics aren’t truthful, then the flaw in the study remains.
The ambiguity lends weight to the suggestion that Johns-Hopkins let the local team do as it wished without much oversight or even knowledge of its methods. Such knowledge and oversight should have been present.
Yet another ambiguity: The net reports of Roberts’ communications with the Oxford critics vary, some stating that main streets were indeed represented, some stating that both main and minor residential streets were represented. I frankly wish the John-Hopkins team would just publicly and clearly expand upon their method of street selection, and either properly justify it (which would be fine), or else admit it flawed.
The world seems to have divided into anti-Iraq-war people excusing or rationalising the study’s sloppiness, or pro-Iraq-war people finding (sometimes legitimate) flaws.
Though anti-Iraq-war, I simply find the study’s science inexcusably sloppy in many ways (as I’ve detailed in another thread).
Two relevant posts (on another blog) about street patterns in Baghdad and Basra:
http://scienceblogs.com/deltoid/2006/10/science_on_lancet_study.php#comment-246531
http://scienceblogs.com/deltoid/2006/10/science_on_lancet_study.php#comment-246548
And more likely, it may not be. Pure speculation.
As was your claim about ‘violent’ residential streets. The whole point was is that as soon as the ‘main street model’ was criticized, you modified it by adding ’some’ violent residential streets.’ Sure you can do that – but resulting models are increasingly speculative.
Part of professionalism consists of stating openly the weaknesses in one’s own method and analysing the effects. Choosing starting points only on major residential streets (thus excluding main streets and minor residential streets) means only a subset of the universe was surveyed; it’s inexcusable unless the authors can present evidence of that sub-set being representative of the whole.
You seem to be accept that the Lancet methodology is flawed compared to similar studies. I find it odd that you appeal to non-professionals for that judgement.
“The world seems to have divided into anti-Iraq-war people excusing or rationalising the study‚Äôs sloppiness, or pro-Iraq-war people finding (sometimes legitimate) flaws.”
More accurately, the world has been divided into pro-war, anti-empidemiologists and the rest.
You seem to be accept that the Lancet methodology is flawed compared to similar studies. I find it odd that you appeal to non-professionals for that judgement.
Several professionals have argued that the methodology is seriously flawed. See, e.g., posts on this blog—including the one you’re commenting on.
‘The Oxford critics?’ By ‘professionals’ I meant epidemiologists experienced in conducting these kind of polls and knowledgeable about what is and what isn’t possible. Not some physicists who think that knowing a little statistics makes them qualified epidemiologists.
First, I don’t see any reason why epidemiologists are any more qualified to deal with surveys of violence than market researchers (who deal with sampling surveys daily) or physicists or anyone else who deals with statistics. (Yes, modern physics involves stats daily.)
Second, I find intellectually offensive the idea that “they have a particular title so anyone else’s comments are invalid.”
Third, in my younger years I handled enough surveys (and the concomitant stat issues) to recognise flaws in methodology when I see them — and choosing to sample only a subset of street type is ridiculous, as is allowing the field team to substitute non-random starting points at their whim.
It is not unreasonable to claim that the current wartime conditions necessitate less-than-ideal survey methods, but that’s not what the authors argued; they simply deny the flaws. A more professional approach would be to acknowledge the compromises in the methodology required by war, and then to seriously try to estimate the effects of those compromises on the results.
“As was your claim about ‚Äòviolent‚Äô residential streets. The whole point was is that as soon as the ‚Äòmain street model‚Äô was criticized, you modified it by adding ‚Äôsome‚Äô violent residential streets.‚Äô Sure you can do that – but resulting models are increasingly speculative.”
What “main street model”????? A few lines in an e-mail or letter to Lancet are not a model; they are only an issue being raised of a not-necessarily-representative sub-sample.
The point to be learnt from the Oxford group, no matter the name chosen or the way they expressed it, is quite simply that choosing starting points non-randomly from only a subset of the population (those on a certain type of street) renders the study invalid unless one can show that subset to have similar statistics to the whole. That cannot be assumed the case if the distribution of violence over size or type of street is not uniform — regardless of the rules one uses to classify streets. A subset just cannot be assumed representative of the whole.
“You seem to be accept that the Lancet methodology is flawed compared to similar studies. I find it odd that you appeal to non-professionals for that judgement.”
What are you talking about? I certainly have not “appealed” to anyone, having never said “the methodology is flawed because Oxford’s Sean Gourley said so.” A priori, I don’t accept anything positive or negative about the Lancet methodology; I judge for myself.
But if you do wish to rely on “professionals,” why do you ignore the fact that Jon Pederson, another Iraq survey designer and head of Norway’s Fafo Institute for Applied Social Science, has said publicly that the Lancet study is flawed and over-estimates? The Lancet study itself terms Pederson “the highly regarded Norwegian researcher.”
I’ve seen an Australian open letter, by doctors belonging to the “Medical Association for the Prevention of War,” claiming the methodology sound and regurgitating the study’s results — but it’s hardly clear whether they’ve considered the technical criticism; they seem more interested in having Howard take the matter seriously. Further, they falsely claim the methodology the same as Darfur/Congo surveys. Finally, the fact that the group — like the Lancet’s Horton and Hopkins’ Roberts — openly state their anti-war agenda does not help to convince me that their judgments of the study are untainted by politics. (Yes, criticism from the right is definitely tainted — which is why I’ve simply judged for myself after absorbing the comments of StatGuy and other scientists.)
FYI, here’s the Australian letter:
http://www.theage.com.au/news/opinion/the-iraq-deaths-study-was-valid-and-correct/2006/10/20/1160851135985.html?page=fullpage
I’m politically anti-war; Pederson is neutral politically; and we both have seen no satisfactory answer to the study’s flaws.
Because they are familiar with the difficulties involve in surveying catastrophes, unlike ‘market ‘researchers’.
Second, I find intellectually offensive the idea that “they have a particular title so anyone else’s comments are invalid.
I’m not. I’m merely pointing out that it is unwise to dismiss the opinion of experts. I doubt you’d do that if you needed brain surgery.
“The point to be learnt from the Oxford group, no matter the name chosen……”
No, the point to be learnt is that as soon I pointed out a flaw in the ‘main street ‘model’ you merely changed it by making it more complicated.
” or the way they expressed it, is quite simply that choosing starting points non-randomly from only a subset of the population (those on a certain type of street) renders the study invalid unless one can show that subset to have similar statistics to the whole.”
Pointing out that something has a flaw may render it inexact but it doesn’t follow that it isn’t the best estimate out there. The epidemiologists opinion is useful in that regard.
Finally I find it strange that the critics of the Lancet article pretend that every flaw is fatal even as the accept the Iraq Body Count numbers based on press reports…even though it is WELL-KNOWN (at least in epidemiological circles) that press reports lead to huge undercounts often by a factor of 10 or more.
Tom W,
Maybe you missed it, but I’m a professional statistician. I’ve been crunching numbers for a living for 18 years now. I didn’t mention it on this post, but I did near the top of this post and it’s also in my “about” box.
The methodological flaws discussed in that earlier post and in this one are indeed fatal. If the survey was not done properly, its results are unreliable.
StatGuy, this whole affair of the Lancet Iraq study has been an education for me in the rather despicable character of man as political animal. As I’ve said, I opposed the Iraqi intervention, but think it intellectually dishonest to twist science to fit ideology.
The shock is that almost everyone commenting decides ideology first and insists on forcing facts to fit ideology. I now understand how extremist movements and conspiracy theories can take root; human decisions seem to be emotional — facts be damned.
Beyond the flaws, the final straw was Figure 4. The Figure 4 chart is absurd for anyone with even a basic mathematical education — asserting the study confirmed by the same “trend” of a curve and its derivative on the same graph — complete and utter smoke. Worse, suppose two observers measure the same stochastic process p(x,t) over time, with one consistently reporting .6*p(x,t) and the other consistently reporting 1.8*p(x,t). The measurements of these two observers will show identical trend though neither is an accurate measurement of the value of p(x,t). In other words, we all know that one curve and another twice its value will have a correlation coefficient of 1.0; the scale factor drops out of both numerator and denominator of the correlation coefficient. Correlation does NOT provide information about scale. Yet, the text TWICE (p 6, p7) argues the study’s results confirmed by similar “trend” to the IBC and DoD shown in Figure 4. How can the peer reviewers have missed such obvious nonsense?
“I‚Äôm merely pointing out that it is unwise to dismiss the opinion of experts.”
Then why do you dismiss Pederson’s, the other “expert” in Iraq surveys cited in the text of the study itself?
Furthermore, I haven’s seen any disinterested “experts” who actually provided answers to the study’s specific technical flaws; they’ve simply expressed their confidence in cluster sampling generally but not resolved the problems in the specific JH design.
“I find it strange that the critics of the Lancet article pretend that every flaw is fatal even as the accept the Iraq Body Count numbers based on press reports.”
I’ve seen no evidence of any Lancet critics with stat background contesting that the IBC methodology under-estimates. The IBC itself does not contest that.
IBC under-estimation provides no information to judge Lancet over-estimation. It is perfectly possible for the one to under-estimate and the other to over-estimate, isn’t it?
“it is WELL-KNOWN (at least in epidemiological circles) that press reports lead to huge undercounts often by a factor of 10 or more”
A strange argument.
1. Have you proof of the factor of error you assert?
2. Have you proof war deaths have the same behaviour as the statistic being measured in your proof?
3. If one can assign a number to the discrepancy factor as you did, why didn’t the Lancet study show their results “confirmed” by the application of that factor to the IBC results?
4. If the discrepancy factor is “large but not known exactly,” then it could just as well be 5 rather than 10. That factor applied to the IBC yields ~250,000 deaths, not the 650,000 touted by the Lancet study. Were the Lancet results 250,000 they’d be much less startling.
Sceptic,
I agree that Figure 4 is highly problematic. With two different scales on the two vertical axes, it would be easy to manipulate the scales to achieve misleading results. A full chapter of Darrell Huff’s classic How To Lie With Statistics is devoted to the problems of graphs. Charts can be useful for illustrative purposes, but they do not substitute for statistical analysis.
Peer reviewers serve a necessary function, but they’re by no means perfect. I’ve read other peer-reviewed articles where I just shake my head and wonder what the reviewer was thinking when he/she approved the publication.
Have you proof of the factor of error you assert?
You apparently haven’t read the articles you are critiquing. For example in the 2006 paper we fin
The citations are given in the paper (I’m assuming the Lancet wouldn’t all them to get away will a bald-faced lie.)
This is also a nice example of how a little knowledge of the literature be useful.
Oops…I messed up the tags…
Have you proof of the factor of error you assert?
You apparently haven’t read the article you are critiquing. According to the 2006 Lancet article
Aside from Bosnia,21 we can find no conflict
situation where passive surveillance recorded more than
20% of the deaths measured by population-based
methods. In several outbreaks, disease and death recorded
by facility-based methods underestimated events by a
factor of ten or more when compared with populationbased
estimates.11,22–25 Between 1960 and 1990, newspaper
accounts of political deaths in Guatemala correctly
reported over 50% of deaths in years of low violence but
less than 5% in years of highest violence.26
The relevant citations are given in the papers.
This example also shows how knowledge of the epidemiological literature can be useful.
I’ve seen no evidence of any Lancet critics with stat background contesting that the IBC methodology under-estimates.
I just gave you the evidence they cited.
IBC under-estimation provides no information to judge Lancet over-estimation. It is perfectly possible for the one to under-estimate and the other to over-estimate, isn’t it?
The point was that the pro-war crowd seems to eagerly accept the CERTAIN biase of the IBC underestimate compared to the speculative bias of the Lancet.
You haven’t read critically. Obviously surveys will have a higher estimate than passive counts, and that’s all I took Burnham’s comment to indicate. But you have no proof of an exact factor (such as the 10 you claim), and Burnham cannot simply “adopt” the factor claimed in other situations which differ from his own:
1. Guatemala hardly had a free press, nor the coverage by reporters to the extent of Iraq. Do you imagine Montt wanted his crimes known?
2. “Facility-based” methods are not the same as counting reports.
3. The figure of 20% is a 5/1 ratio and actually confirms my point to you that you don’t really know whether the factor is 10 or just 5.
4. As the Hopkins citation admits, in Bosnia the factor was less than 5.
Again, if the Hopkins study had shown an estimate of 5 x IBC ~ 250,000, it would not have been so controversial.
One problem with the Hopkins study is its lack of preofessionalism. Burnham didn’t impress by vaguely citing not-necessarily-comparable studies. Had he wanted to be professional, he would have tried to pick comparale ones and analyse the similarities and differences. He’s too obviously grinding an axe, and it’s hurt his credibility.
“The point was that the pro-war crowd seems to eagerly accept the CERTAIN biase of the IBC underestimate compared to the speculative bias of the Lancet.”
I’m not the “pro-war crowd,” and I certainly haven’t suggested accepting the IBC figures as complete. Even the IBC doesn’t do that.
Rgeardless of how you phrase it, I judge the IBC under-estimates while Hopkins over-estimates.
Since you like “experts” so much, I will agree with Norway’s Pedersen. Have YOU read the Hopkins study? It praises Pedersen. Pedersen has said the Hopkins study certainly an over-estimate.
Finally, guesses at multiplicative factors, on the basis of other countries and situations, do little to resolve the technical flaws StatGuy and I have pointed out in the Hopkins study.
Obviously surveys will have a higher estimate than passive counts, and that’s all I took Burnham’s comment to indicate. But you have no proof of an exact factor
An exact factor? I didn’t claim there was an exact factor. To pretend I did is just plain stupid.
The studies cited did not ‘just say’ there was an undercount, they reported that bias can be HUGE particularly when the level of violence is high.
Guatemala hardly had a free press, nor the coverage by reporters to the extent of Iraq. Do you imagine Montt wanted his crimes known?
The Iraqi government is hardly transparent either. In fact they continue to refuse to release figures.
The figure of 20% is a 5/1 ratio and actually confirms my point to you that you don’t really know whether the factor is 10 or just 5.
Proves nothing. I didn’t claim it was an exact factor. The factor obviously depends on circumstances, level of violence, freedom of movement, transparency, all of these factors work against the Iraq press count.
It is interesting that you pretend that the factor is just 5, when in fact is was reported to range between 2 and 20…the latter when the level of violence is high. You really should try to restrain yourself from misrepresenting what was said.
The range 2 and 20 is not inconsistent with the 2004 and 2006 Lancet article which reported deaths higher by a factor of 3 and 15 than the Iraq body count (less if we account for the fact that the Lancet counts all excess deaths (including those due to increased poverty and disease) whereas the Iraq body count reports on war related violent deaths. The higher factor of the 2006 study is also consistent with the fact that the level of violence was also found to increase.
Maybe you missed it, but I’m a professional statistician.
I’ve done a little statistics myself (statistical mechanics, turbulence theory) and I have yet to see anything resembling a statistical argument here. You know – error estimates etc. Simply claiming that ‘the methodology is flawed’, ‘the number of samples is too small’ or ‘the confidence interval is too large’ without providing some kind of analysis is hardly a serious critique. Similarly it is ludicrous to claim a study is invalid because of bias without even bothering to provide an estimate of the size of bias.
Rgeardless of how you phrase it, I judge the IBC under-estimates while Hopkins over-estimates.
It’s obvious that IBC under-estimates. The Lancet involves both positive (e.g. main street bias) and negative biases (e.g. ignoring Falluja, not counting households that are totally wiped out etc.) but despite the fact that you have no clue about the relative size of these, you arbitrarily decide which dominates. This is not good science.
Tom, I think you’re primarily interested in a politcal agenda, and I shall not bother further with you.
Just to goes to show that with statistics, as with knowledge generally, a little is a dangerous thing.
Similarly it is ludicrous to claim a study is invalid because of bias without even bothering to provide an estimate of the size of bias.
What’s ludicrous is thinking that a statistical analysis cannot be shown to be biased without an estimate of the amount of bias. The bias of the Lancet study cannot be estimated because the researchers have not provided a complete and unambiguous description of the methodology and have, by their own admission, either destroyed or never collected essential data necessary for such an estimate.
Your comment means that an estimate arising from a poorly designed sample, inadequate methodology, and sloppy data collection and processing would be impervious to criticism simply because the authors refuse to release information necessary for precise estimation of bias.
Be that as it may, we’ve had lots of opportunities to argue and debate. At this point, it’s clear that no one’s going to change his mind.
On that happy note, comments are closed.
[...] “Main street bias” in Lancet study [...]
[...] A specification of the Lancet study‚Äôs Main street bias By StatGuy The scientists who provided the information behind my post “’Main street bias’ in Lancet study” have now come up with an equation providing a framework for estimation of the extent of bias in the Lancet study on Iraqi deaths. As discussed in that post, the Lancet study published in October included only urban residents of Iraq and, more specifically, only people living on a residential street crossing a “main street”. From page 2 of the pdf version of the October Lancet article: The third stage consisted of random selection of a main street within the administrative unit from a list of all main streets. A residential street was then randomly selected from a list of residential streets crossing the main street. On the residential street, houses were numbered and a start household was randomly selected. From this start household, the team proceeded to the adjacent residence until 40 households were surveyed. [...]
[...] Was 2004 Lancet study correct to toss out Falluja data? By StatGuy I’ve written several posts about the 2006 study of Iraqi deaths, written by Burnham et al, but not about the predecessor study, authored by Roberts et al and published in 2004. (This blog started in August 2005.) Both studies analysed data collected in personal interviews at Iraqi households selected using cluster sample methodology and purported to find that tens of thousands of Iraqis have died because of the US invasion. Both were published in The Lancet. [...]
[...] of my previous blog posts criticising the 2006 study have focused on points 1 and 2. For example, this: The [...]
Lancet study: Serious ethical lapses, data quality problems…
An exhaustive investigation of the methods used in the analysis of Iraqi deaths conducted in 2006 and published by The Lancet has uncovered many serious violations of accepted ethical practices and possible data fabrication.
“Mortality after the …