Bayliss, P, Hill, P, et al. (2001). "Education for clinical governance." British Journal of Clinical Governance 6(1): 7-8.

For the modernisation of the NHS and successful implementation of clinical governance there must be a new curriculum, with new educational goals for the education of clinicians, managers and consumers. Whilst many elements to that end have been introduced in recent years, a missing element is the study of the system of health care as a system, its properties and risks. The study of safety of and adverse outcomes from error in the ``Quality of Australian health care'', highlighted not only preventable error in individual clinical decision and actions, but more importantly the hidden flaws, the latent errors within the system of health care that can lead to such errors. The study of system error in health care is greatly enhanced by the experience of comparable studies of safety in industry. These issues are explored in postgraduate vocational education and training. Perhaps they should be core curriculum for all undergraduate health profession and management education.

Ioannidis, J, Haidich, A, et al. (2001). "Any casualties in the clash of randomised and observational evidence?" British Medical Journal 322(7291): 879-80.

FULL TEXT
Randomised controlled trials and observational studies are often seen as mutually exclusive, if not opposing, methods of clinical research. Two recent reports, however, identified clinical questions (19 in one report, 1 five in the other 2) where both randomised trials and observational methods had been used to evaluate the same question, and performed a head to head comparison of them. In contrast to the belief that randomised controlled trials are more reliable estimators of how much a treatment works, both reports found that observational studies did not overestimate the size of the treatment effect compared with their randomised counterparts. The authors say that the merits of well designed observational studies may need to be re-evaluated: case-control and cohort studies may need to assume more respect in assessing medical therapies and large-scale observational databases should be better exploited. 1,2 The first claim flies in the face of half a century of thinking, so are these authors right?
The combined results from the two reports indeed show a striking concordance between the estimates obtained with the two research designs. A correlation analysis we performed on their combined databases found that the correlation coefficient between the odds ratio of randomised trials and the odds ratio of observational designs is 0.84 (P < 0.001). This represents excellent concordance (figure). In fact, it is better than that observed when the results of small, randomised trials and their meta-analyses were compared with the results of large randomised trials. 3 To complicate matters, the concordance has been worse when the results of specific large randomised trials on the same topic were compared among themselves. 3 Concato et al further observe that, for the five clinical questions they evaluated, observational studies for each question had very similar odds ratios between themselves, 2 whereas the results of the randomised trials were often very heterogeneous. Popular wisdom has it that a "gold standard" method should give more or less the same results when repeated several times, while a poor method would suffer from lots of variability. So should observational studies be the gold standard instead of randomised trials?
Such a thought would be anathema to most clinical trialists. 4 A closer inspection of the data suggests several caveats. Firstly, in six of 25 comparisons the 95% confidence intervals of the summary effect from observational studies does not include the summary point estimate of the randomised trials. Moreover, in three cases the pooled point estimates are in the opposite direction (one suggests harm, the other benefit); in two more cases one pooled odds ratio estimate is exactly 1.00, and the other documents benefit. So, perhaps concordance is not all that perfect, depending on how one looks at it.
Secondly, variability may be a blessing and not a nuisance. Variable results in randomised trials suggest that these trials have indeed managed to study diverse patient populations and treatment circumstances where the efficacy of a treatment may differ. 5 Observational studies may tend to amalgamate large populations and reach average population-wide effects where there is less variability but where it is also more difficult to discern which patients are likely to benefit from an intervention.
Perhaps more importantly, Benson and Hartz 1 and Concato et al 2 are still dealing with only a very small portion of randomised and observational research. Their sampling failed to capture some prodigious discrepancies between the two methods. Interventions such as â carotene and á tocopherol, which have brought fame to observational epidemiologists, crashed when they were tested in rigorous randomised controlled trials. 6,7 Given the hundreds of thousands of trials and observational studies that have been conducted and are still being conducted, the number of topics studied in the two reports is limited and subject to strong selection biases.
Perhaps the most important bias is that it is only for very selected clinical questions that both designs are concurrently used, and investigators are willing to compare the designs in an even smaller minority. In a continuing effort to compare the merits of the two designs, we have found about 50 topics where both randomised and observational evidence were considered in the same meta-analysis among over 2000 meta-analyses performed in the past 25 years. Despite some overlap, the two types of designs are used in largely different settings.
For interventions that show very large harmful effects in observational studies, randomised trials may be justifiably discouraged and never performed. For interventions that have already shown large beneficial treatment effects in observational trials (risk ratios less than 0.40) the ethics of randomisation may also be questioned. Interventions with modest postulated effects (risk ratios in the range 0.40.0.90) are likely to be targeted by randomised trials; in this setting, observational studies may not be given comparable credit and may be unjustifiably discarded once randomised trials have been performed. Finally, for interventions with very small postulated effects (risk ratios 0.90.1.00) adequately powered randomised trials may be difficult to perform given the sample size requirements, and thus only observational evidence may be generated.
Besides the size of the postulated treatment effect, another important selection force is the frequency of the outcome of interest. Rare yet important outcomes are unlikely to be studied in trials, given the extreme requirements of sample size and follow up. In contrast, when the outcomes of interest are common, trials are convenient.
More empirical evidence is needed on the merits of various research designs. We need more quantitative evidence to understand what exactly each design can tell us and how often and why each design may go wrong. Discarding observational evidence when randomised trials are available is missing an opportunity. Conversely, abandoning plans for randomised trials in favour of quick and dirty observational designs is poor science. The careful comparisons of methods performed by Benson and Hartz 1 and Concato et al 2 can enhance our understanding about their relative merits and we should encourage such comparisons when the use of various clinical research designs is ethically appropriate.
REFERENCES
1. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med 2000; 342: 1878-86.
2. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342: 1887-92.
3. Ioannidis JPA, Cappelleri JC, Lau J. Issues in the comparisons of meta-analysis and large trials. JAMA 1998; 281: 1089-93.
4. Pocock SJ, Elbourne DR. Randomized trials or observational tribulations? N Engl J Med 2000; 342: 1907-9.
5. Lau J, Ioannidis JPA, Schmid CH. Summing up evidence: one answer is not always enough. Lancet 1998; 351: 123-7.
6. Alpha Tocopherol, Beta Carotene Cancer Prevention Study Group. The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. N Engl J Med 1994; 330: 1029-35.
7. Yusuf S, Dagenais G, Pogue J, Bosch J, Sleight P. Vitamin E supplementation and cardiovascular events in high risk patients. The Heart Outcomes Prevention Evaluation Study Investigators. N Engl J Med 2000; 342: 154-60.

Jolly, B and Ho-Ping-Kong, H (1991). "Independent learning: An exploration of student grand rounds." Medical Education 25(4): 334-42.

Grand rounds (GRs) for students were started with the 1987 cohort of first-year clinical students at two University of Toronto teaching hospitals. A qualitative exploratory evaluation, using questionnaires, interviews and observation, of a sample of 78 staff and student participants showed that the aim of the GRs to activate student skills in independent study was achieved but that interaction between students and other clinical and laboratory specialists as a basis for round presentation was less than intended. Students spent an average of 10 hours on independent work. Round presentations were of a high standard. Students' main concerns were the amount of freedom and feedback given. The goals of the programme as perceived by students influenced both the work done and the style of presentation. These outcomes were contrasted with a UK qualitative study of student presentations on ward rounds, and the importance of an appropriate context for independent work highlighted.

National Health and Medical Research Council (1999). How to Review the Evidence: Systematic Identification and Review of the Scientific Literature. Canberra, Commonwealth of Australia: 1-112.

Clinical practice guidelines are systematically developed statements that assist clinicians, consumers and policy makers to make appropriate health care decisions. Such guidelines present statements of 'best practice' based on a thorough evaluation of the evidence from published research studies on the outcomes of treatment or other health care procedures. The methods used for collecting and evaluating evidence, and developing guidelines, can be applied to a wide range of clinical interventions and disciplines, including the use of technology and pharmaceuticals, surgical procedures, screening procedures, lifestyle advice, and so on.
In 1995, recognising the need for a clear and widely accessible guide for groups wishing to develop clinical practice guidelines, the National Health and Medical Research Council (NHMRC) published a booklet to assist groups to develop and implement clinical practice guidelines. In 1999 a revised version of this booklet was published called A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines (NHMRC 1999), which includes an outline of the latest methods for evaluating evidence and developing and disseminating guidelines.
The emerging guideline processes are complex, however, and depend on the integration of a number of activities, from collection and processing of scientific literature to evaluation of the evidence, development of evidence-based recommendations or guidelines, and implementation and dissemination of the guidelines to relevant professionals and consumers. The NHMRC has therefore decided to supplement the information in the guideline development booklet (NHMRC 1999) with a series of handbooks with further information on each of the main stages involved. Experts in each area were contracted to draft the handbooks. An Assessment Panel was convened in June 1999 to oversee production of the series. Membership of the Assessment Panel and the writing group for this handbook are shown at Appendix A.
Each of the handbooks in the series focuses on a different aspect of the guideline development process (review of the literature, evaluation of evidence, dissemination and implementation, consumer publications, economic assessment and so on). This handbook focuses on the vital issue of how to change clinical practice through dissemination and implementation of clinical guidelines or other evidence-based information.
The way in which the guidance provided in this handbook fits into the overall guideline development process recommended by the NHMRC is shown in the flowchart on page vii. Other handbooks that have been produced in this series so far are:
How to Review the Evidence: Systematic Identification and Review of the Scientific Literature
How to Use the Evidence: Assessment and Application of Scientific Evidence
How to Present the Evidence for Consumers: Preparation of Consumer Publications
How to Compare the Costs and Benefits: Evaluation of the Economic Evidence
The series may be expanded in the future to include handbooks about other aspects of the guideline development process, as well as related issues such as reviewing and evaluating evidence for public health issues.

National Health and Medical Research Council (2000). How to Put the Evidence into Practice: Implementation and Dissemination Strategies. Canberra, Commonwealth of Australia: 1-105.

Clinical practice guidelines are systematically developed statements that assist clinicians, consumers and policy makers to make appropriate health care decisions. Such guidelines present statements of 'best practice' based on a thorough evaluation of the evidence from published research studies on the outcomes of treatment or other health care procedures. The methods used for collecting and evaluating evidence, and developing guidelines, can be applied to a wide range of clinical interventions and disciplines, including the use of technology and pharmaceuticals, surgical procedures, screening procedures, lifestyle advice, and so on.
In 1995, recognising the need for a clear and widely accessible guide for groups wishing to develop clinical practice guidelines, the National Health and Medical Research Council (NHMRC) published a booklet to assist groups to develop and implement clinical practice guidelines. In 1999 a revised version of this booklet was published called A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines (NHMRC 1999), which includes an outline of the latest methods for evaluating evidence and developing and disseminating guidelines.
The emerging guideline processes are complex, however, and depend on the integration of a number of activities, from collection and processing of scientific literature to evaluation of the evidence, development of evidence-based recommendations or guidelines, and implementation and dissemination of the guidelines to relevant professionals and consumers. The NHMRC has therefore decided to supplement the information in the guideline development booklet (NHMRC 1999) with a series of handbooks with further information on each of the main stages involved. Experts in each area were contracted to draft the handbooks. An Assessment Panel was convened in June 1999 to oversee production of the series. Membership of the Assessment Panel and the writing group for this handbook are shown at Appendix A.
Each of the handbooks in the series focuses on a different aspect of the guideline development process (review of the literature, evaluation of evidence, dissemination and implementation, consumer publications, economic assessment and so on). This handbook focuses on the vital issue of how to change clinical practice through dissemination and implementation of clinical guidelines or other evidence-based information.
The way in which the guidance provided in this handbook fits into the overall guideline development process recommended by the NHMRC is shown in the flowchart on page vii. Other handbooks that have been produced in this series so far are:
How to Review the Evidence: Systematic Identification and Review of the Scientific Literature
How to Use the Evidence: Assessment and Application of Scientific Evidence
How to Present the Evidence for Consumers: Preparation of Consumer Publications
How to Compare the Costs and Benefits: Evaluation of the Economic Evidence
The series may be expanded in the future to include handbooks about other aspects of the guideline development process, as well as related issues such as reviewing and evaluating evidence for public health issues.

National Health and Medical Research Council (2000). How to Use the Evidence: Assessment and Application of Scientific Evidence. Canberra, Commonwealth of Australia: 1-84.

Clinical practice guidelines are systematically developed statements that assist clinicians, consumers and policy makers to make appropriate health care decisions. Such guidelines present statements of 'best practice' based on a thorough evaluation of the evidence from published research studies on the outcomes of treatment or other health care procedures. The methods used for collecting and evaluating evidence, and developing guidelines, can be applied to a wide range of clinical interventions and disciplines, including the use of technology and pharmaceuticals, surgical procedures, screening procedures, lifestyle advice, and so on.
In 1995, recognising the need for a clear and widely accessible guide for groups wishing to develop clinical practice guidelines, the National Health and Medical Research Council (NHMRC) published a booklet to assist groups to develop and implement clinical practice guidelines. In 1999 a revised version of this booklet was published called A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines (NHMRC 1999), which includes an outline of the latest methods for evaluating evidence and developing and disseminating guidelines.
The emerging guideline processes are complex, however, and depend on the integration of a number of activities, from collection and processing of scientific literature to evaluation of the evidence, development of evidence-based recommendations or guidelines, and implementation and dissemination of the guidelines to relevant professionals and consumers. The NHMRC has therefore decided to supplement the information in the guideline development booklet (NHMRC 1999) with a series of handbooks with further information on each of the main stages involved. Experts in each area were contracted to draft the handbooks. An Assessment Panel was convened in June 1999 to oversee production of the series. Membership of the Assessment Panel and the writing group for this handbook are shown at Appendix A.
Each of the handbooks in the series focuses on a different aspect of the guideline development process (review of the literature, evaluation of evidence, dissemination and implementation, consumer publications, economic assessment and so on). This handbook focuses on the vital issue of how to change clinical practice through dissemination and implementation of clinical guidelines or other evidence-based information.
The way in which the guidance provided in this handbook fits into the overall guideline development process recommended by the NHMRC is shown in the flowchart on page vii. Other handbooks that have been produced in this series so far are:
How to Review the Evidence: Systematic Identification and Review of the Scientific Literature
How to Use the Evidence: Assessment and Application of Scientific Evidence
How to Present the Evidence for Consumers: Preparation of Consumer Publications
How to Compare the Costs and Benefits: Evaluation of the Economic Evidence
The series may be expanded in the future to include handbooks about other aspects of the guideline development process, as well as related issues such as reviewing and evaluating evidence for public health issues.

Wilson, P, Watt, I, et al. (2001). "Survey of medical directors' views and use of the Cochrane Library." British Journal of Clinical Governance 6(1): 34-9.

A postal survey to determine UK medical directors' (n = 491) attitudes on the importance of effectiveness information; their own access, awareness and use of a variety of sources for information on effectiveness, their own use of and views on the perceived usefulness of the Cochrane Library; their attitudes towards their own role in the local application of clinical governance. Respondents reported high levels of awareness and access to a number of sources of clinical/cost effectiveness information. Respondents regularly referred to paper-based publications more than electronic databases (only 10 per cent regularly referring to the Cochrane Library). Respondents felt that the Cochrane Library could be improved by increasing the number and range of topics and clinical areas covered, and by improving access and availability.