Thanks largely to
public support for the Release the First Report campaign, Australia’s leading
research institute, NHMRC, recently gave in to public pressure and released its
first report on homeopathy, produced in 2012 but buried until 26 August 2019.
As we always expected, the draft 2012 report is far
more positive than the second NHMRC Australian Report published in 2015, which
has caused widespread damage to homeopathy.
The First Report found “encouraging evidence for the effectiveness of
homeopathy” for five conditions, including otitis media (ear infections),
upper respiratory tract infection in adults, and some side effects of cancer
As Rachel Roberts, HRI Chief Executive, explains, “The first report found some good
evidence that homeopathy works for certain medical conditions. More studies are
needed to confirm and build on these findings, but some evidence is very
different from no evidence.”
against releasing the First Report for 4 years, so this is a huge win for
public accountability, but perhaps even
more importantly, the NHMRC CEO has finally put the record straight about the
second ‘Australian report’ published in 2015.
Publication of NHMRC’s 2015 report triggered
inaccurate headlines around the world – from the UK: “Homeopathy not effective
for treating any condition” (Guardian) to the USA:“1800 studies later
scientists conclude homeopathy doesn’t work”(Smithsonian.com).
NHMRC Chief Executive Prof Anne Kelso’s has now
finally admitted that, “Contrary to some claims, the review did not conclude
that homeopathy was ineffective” (CEO Statement, 26 August 2019).
It is shocking that NHMRC has never corrected this misinterpretation of their report before. When these widely inaccurate headlines first circulated in March 2015 NHMRC stayed silent.
When the story hit the media again in February 2016 they said and did nothing.
Even when the NHMRC 2015 report started to have a real-world impact being used to make decisions against homeopathy in countries all around the world, they failed to step in and clarify what their report had really found – that even when the amount of evidence was drastically reduced by using their totally unjustifiable method created just for this review – the results showed the evidence for most conditions was inconclusive, not negative.
The CEO finally making it clear that their report never said homeopathy doesn’t work is a crucial and welcome first step towards NHMRC undoing the damage they have single-handedly caused to the homeopathy sector worldwide.
NHMRC has finally released their first report on homeopathy, following an overwhelming response to the Release The First Report campaign which saw more than 75,000 people from around the world demand its publication.
As expected, the conclusions of NHMRC’s 2012 report – The
Effectiveness of Homeopathy: an overview review of secondary evidence – are
in stark contrast to their 2015 attempt, with the first review concluding that there
is “encouraging evidence” homeopathy works for five medical conditions.
“For over three years NHMRC have refused to release their 2012 draft report on homeopathy, despite Freedom of Information requests and even requests by members of the Australian Senate. To see this document finally seeing the light of day is a major win for transparency and public accountability in research.”
Rachel Roberts, HRI Chief Executive
NHMRC CEO, Professor Anne Kelso, explained her decision to
release the report by acknowledging the “considerable interest” in this hidden
NHMRC’s sudden decision to release the report, despite
stating to the Australian Senate only a month ago that they had no plans to
release it, comes at a time when the verdict of an Ombudsman investigation into
NHMRC’s Homeopathy Review is pending.
Further analysis of the 293-page first report
The Australian Homoeopathic Association has criticised the Pharmaceutical Society of Australia (PSA) over an open letter to all pharmacies in Australia, instructing them to “cease all activities that encourage the stocking, promotion, recommendation or marketing of homeopathy”.
The PSA letter cites the NHMRC’s flawed second report on Homeopathy. As the AHA indicate in their response, the timing of this action from the PSA is suspect, with an Ombudsman investigation on NHMRC’s alleged scientific misconduct, procedural breaches, bias and conflict of interest rapidly approaching its conclusion.
The Director General of the Central Council for Research in Homoeopathy, an Indian Government research institute, has written directly to Professor Anne Kelso, CEO of NHMRC, over the Australian research institute’s misleading review of Homeopathy.
In his letter, Dr Raj Manchanda lauds the benefits of the ‘cafeteria-style’ integrative healthcare model reaped by the Indian populace, of which homeopathy is a major part. He also expressed concern ‘that Australia’s current approach to reforms on Homoeopathy appears to be contrary to the WHO Traditional Medicines Strategy 2014-2023, leading towards reduced patient access, rather than promoting its use.’
The full letter can be read here and Dr Manchanda explains why the CCRH are supporting the #ReleaseTheFirstReport campaign below.
The Faculty of Medicine, Lille University, France has suspended its Homeopathy course on the basis of a European document quoting the SECOND Australian Report.
La Faculté de Médecine de Lille décide de suspendre son Diplôme d’Université d’homéopathie pour l’année universitaire débutante dans l’attente de la position de la H.A.S. et d’échanges nationaux sur l’encadrement de cette pratique et de son enseignement.
The Diploma course in Homeopathy has been suspended for the 2018/19 academic year while the Faculty awaits the official position of the French health authority on the subject of homeopathy and its teaching.
The French Haute Autorité de Santé is set to make its position clear by February 2019.
Homeopathy is hugely popular in France, with remedies readily available in high-street pharmacies and patients able to claim reimbursement from the social security system for homeopathic treatment. A 2015 study showed that nearly 7 million people receive at least one reimbursement for a homeopathic preparation during a 12-month period.
Time to Act
With yet another government-level decision being influenced by NHMRC’s SECOND report on Homeopathy, the urgency grows for NHMRC to make their FIRST report public.
The debate about homeopathy in France escalated earlier this year, when a group of 124 healthcare practitioners – calling themselves “FakeMed” – signed an open letter published in newspaper Le Figaro, describing homeopaths as “charlatans”, and Homeopathy itself as “dangerous”, “irrational” and “in no way scientific”.
In reaching this conclusion, the group cited the 2017 statement by the European Academies’ Scientific Advisory Council (EASAC), who themselves conducted no new research but cited the NHMRC Australian Report on Homeopathy – under investigation by the Commonwealth Ombudsman for scientific fraud – and the long since discredited 2010 UK Science & Technology Committee report.
Using this to support their position shows a clear lack of understanding of both the evidence base for homeopathy, and the severe flaws in their own arguments.
We know from Freedom of Information documents that NHMRC did their review of evidence on Homeopathy twice. This makes their SECOND report and it’s conclusions that “there is no reliable evidence that Homeopathy is effective for any condition” invalid.
Sign the petition to join thousands of people around the world in demanding that NHMRC #ReleaseTheFirstReport.
IMPORTANT questions asked in the Senate of the National Health & Medical Research Council (NHMRC) over the 2012-2015 review of the evidence on homeopathy (the Homeopathy Review) have not been accurately or fully answered and in some cases not answered at all.
Senator Stirling Griff put a total of 8 targeted questions to the Department of Health regarding the NHMRC’s conduct of the Homeopathy Review, which is currently under investigation by the Commonwealth Ombudsman.
The Homeopathy Review was the first of 16 natural therapies reviews conducted by the NHMRC, reviews that both the Government and Labor Opposition are now using to justify cutting the Private Health Insurance (PHI) Rebate for natural therapies, restricting healthcare choices.
The NHMRC applied the same ‘Overview’ methodology to all of the reviews under its Natural Therapies Review, an unprecedented method which meant that no original research papers were retrieved or assessed and no subject or research experts were consulted for any of the 16 natural therapies targeted. Further, no targeted cost-effectiveness assessment was carried out for any of the therapies, even though the Department of Health informed the public that this was done, with the reviews being used to justify removal of health fund rebate support for ‘Budget savings’. Available evidence points to use of these therapies producing considerable savings to the healthcare budge.
Below, Your Health Your Choice provides a detailed analysis of the NHMRC’s response to one of Senator Griff’s questions – Question on Notice No. 268 – highlighting significant and concerning inaccuracies. We’ll provide analyses of responses to other questions in due course.
Question on Notice no. 268 – studies with less than 150 participants excluded from Homeopathy Review findings:
Senator Stirling Griff asked the Department of Health on 30 May 2018—
“The Homeopathy Review created a rule that trials with less than 150 participants were not ‘reliable’, meaning their results did not “warrant further consideration of their findings” (Information Paper, Appendix C) . Does the NHMRC apply this rule to other trials it funds and/or reviews? If so, where else has this rule been applied and what internationally accepted scientific standards were used to authenticate it?”
The NHMRC has never before applied a rule that the findings of human clinical trials with less than 150 participants are ‘unreliable’. In fact, the NHMRC regularly funds and collaborates on trials with fewer than 150 participants (see Table 1 below).
The NHMRC has never applied this rule when reviewing any other category of health evidence – it was developed just for the Homeopathy Review. There are no NHMRC or any internationally accepted guidelines or standards that authenticate the rule, and it has never been used by any other government agency or research group in Australia or internationally when assessing research evidence. Then why was it used?
The rule directly dismissed the results of 146 out of the 176 studies the NHMRC initially identified – meaning that these trials’ results were not considered as part of the Review’s findings. A further arbitrary rule was also created – that trials had to have a minimum 100% ‘quality rating’ for their results to be considered ‘reliable’. This dismissed the results of a further 25 trials, leaving just 5 trials deemed to be ‘reliable’.
The Review’s published finding of ‘no reliable evidence’ is therefore based on just these 5 trials – a fact not reported in the NHMRC report.
The N=150 rule was formally adopted by the NHMRC Homeopathy Working Committee (HWC) at its face-to-face meeting on 12 July 2013, four months after the contractor (Optum) had already completed assessing the evidence in mid March 2013. The sample size rule was not specified in the original research protocol that was agreed between Optum, the HWC and the Office of NHMRC on 22 December 2012, which was not published.
Answer provided by NHMRC:
“Studies with less than 150 participants were not excluded from consideration in the evidence assessment. The Homeopathy Working Committee (HWC) deemed that, for the purposes of drafting evidence statements, the outcomes of studies with less than 150 participants were not sufficiently powered to reliably inform conclusions on the effectiveness of homeopathy to treat a particular clinical condition.
NHMRC establishes expert working committees, such as the HWC, under section 39 of the NHMRC Act, to advise NHMRC on the methodology and parameters of evidence evaluation when developing health advice and guidelines. This is in line with international best practice for formulating evidence-based health advice. The HWC has expertise in evidence-based medicine, clinical trials and complementary medicine.
There is no ruling as to the number of participants that is appropriate for a trial that applies for funding by NHMRC. Applications are selected for funding through a competitive peer review process. The grant review panel assesses the application against published criteria. Issues such as trial sample size, experimental design and the power are usually assessed under “scientific quality”. If, in the judgement of the expert assessors, the scientific quality is compromised by inadequate trial size, this will be reflected in the score for that assessment criterion.”
Underlined sections in NHMRC’s response are addressed below:
NHMRC’s answers – Fact Check:
1. “There is no ruling as to the number of participants that is appropriate for a trial that applies for funding by NHMRC”:
Senator Griff’s question was, “Does the NHMRC apply [the N<150 trial sample size] rule to other trials it funds and/or reviews? If so, where else has this rule been applied and what internationally accepted scientific standards were used to authenticate it?”
The question asked was not answered.
To answer the question, NHMRC would have had to explain that it routinely funds and collaborates on trials with less than 150 participants (see Table 1 below) and that there is no such thing as a 150 sample size exclusion threshold to ‘reliably inform conclusions’. If these trials are all deemed to be inherently unreliable simply based on a trial size threshold of 150, why are they being funded by NHMRC?
It would also have had to explain that no other evidence review, whether conducted by NHMRC or another research group in Australia or internationally, has ever imposed a blanket dismissal of the findings of all trials of N<150 as having insufficient participants for its results to be considered ‘reliable’.
Further, it would have to explain that no scientific guidelines or standards or guidelines exist to authenticate the rule.
2. “Studies with less than 150 participants were not excluded from consideration”:
The results of all studies with less than 150 participants were, in fact, completely dismissed from any consideration in the Homeopathy Review’s findings and conclusions.
NHMRC’s response correctly points out that all 176 trials in scope of the Review were described and evaluated in the 300 page Optum Overview Report – but this is not relevant to the question Senator Griff actually asked.
Trials that failed to meet the Review’s arbitrary ‘reliability’ criteria were technically ‘included’ in the sense that their existence as part of the evidence base was described in the Optum Overview Report and they were counted as part of the total number of trials originally identified. Howevertheir findings were not described in the Overview Report and their results were not taken into consideration when drafting the evidence statements for each clinical condition, even if the trials showed effectiveness of homeopathy (see NHMRC Information Paper Appendix C, pages 34-35).
Instead evidence statements would simply state, “These studies are of insufficient quality and/or size to warrant further consideration of their findings” and consequently, “there is no reliable evidence that homeopathy is effective.”
In other words, the results of these trials were effectively excluded from the Review’s findings.
In over 900 pages of report documentation, NHMRC did not publicly disclose that the 150 trial sample size and 100% trial quality thresholds together directly dismissed all but 5 of the 176 trials from informing the Review’s finding of ‘no reliable evidence’. Many positive studies rated ‘good’ quality were dismissed from the findings on the basis of the 150 sample size rule.
If the NHMRC reviewed the evidence according to its usual, accepted methods, it would have had to report that of the 176 homeopathy studies included in scope, 88 (50%) reported statistically positive results and only 9 (5%) negative (the rest inconclusive and therefore worthy of further investigation). These findings are strikingly similar to results reported in conventional medical research.
3. “The outcomes of studies with less than 150 participants were not sufficiently powered to reliably inform conclusions”:
This statement is not accurate or truthful, as revealed by documents released under Freedom of Information (FOI) and explained below:
Documents released under FOI reveal that the NHMRC/ Homeopathy Working Committee (HWC) initially created a rule that a trial had to have at least 200 participants for it to be ‘sufficiently powered’. This rule was created in May 2013, two months after the contractor Optum had completed the evidence assessment (NHMRC FOI 2015-16 008 Doc 05 – HWC Teleconference minutes 23 May 2013) and well after the original research protocol had been agreed and finalised in December 2012. Notably the original research protocol, which was never published, did not include any sample size exclusion rule (NHMRC FOI 2014-15-004, Section 52 – Final research protocol for Optum Review, 20 Dec 2012).
FOI documents reveal that on 9 July 2013, NHMRC received independent methodological peer review advice from the Australasian Cochrane Centre (ACC) that challenged the validity of this rule, stating, “It is not clear why a categorisation of adequate and inadequate power was needed (and indeed may be misleading)” and recommending, “It would be preferable not to use the terminology adequate power” (NHMRC FOI 2015-16 008-13 Docs 13 – Australasian Cochrane Centre Methodological Peer Review, 9 July 2013).
In response to the ACC’s expert feedback, on 11-12 July 2013 the NHMRC/ HWC formally abandoned the concept of a trial’s sample size as a measure of whether it was ‘adequately powered’, replacing it instead with a new concept of whether a trial was of “sufficient size to provide meaningful results” (NHMRC FOI 2015-16 008 Doc 06 – HWC face-to-face meeting minutes, 11-12 July 2013). This was now four months after the contractor Optum had completed the evidence assessment and seven months after the research protocol had been agreed and finalised.
At the same time, NHMRC also changed the minimum trial sample size threshold for ‘meaningful results’ from 200 to 150 trial participants. This meant that the results of all trials below this new N=150 threshold were dismissed from consideration in the Review’s findings as not being ‘meaningful’.
The concept of ‘meaningful results’ then underwent further refinement, morphing into the concept of ‘reliable evidence’, which was the final framework used to report the Review’s findings to the public (i.e. ‘no reliable evidence’).
It is particularly noteworthy that none of these midstream manipulations to the research protocol were publicly disclosed in the NHMRC report. Click here for more information about the retrospective, undisclosed changes that were made to the research protocol.
“The responses provided to the Senate are typical of the degree of obfuscation in the NHMRC report”, said Your Health Your Choice spokesperson Ms Petrina Reichman, who also noted they had been provided under Parliamentary privilege.
“Government guidelines state that public servants are required to fulfil their accountability obligations by providing ‘full and accurate’ information to the Parliament and its committees about the factual and technical background to policies and their administration, governed by The Parliamentary Privileges Act 1987”, she said.