4. Supporting Functions

The commitment to consumer outcomes is articulated in the:

“National Mental Health Plan 2003 -2008, outcome 28: Comprehensive implementation and further development of routine consumer outcome measures in mental health, specifically key direction:

• 28.1: Continue to support and develop outcome measurement systems, including full implementation of routine outcome measurement systems, in the mental health sector and for use by other mental health providers and related service sectors.
• 28.2: Establish a national strategy in collaboration the Commonwealth, States and Territories for database development, data analysis (which may include normative comparisons and benchmarking exercises), dissemination and training.
• 28.3: Support the implementation of routine outcome measurement.” (Australian Health Ministers, National Mental Health Plan 2003–2008. Canberra: Australian Government, 2003).

4.1 Documentation

“11.3.8 The assessment is recorded in an individualised clinical record in a timely and accurate manner.” (National Mental Health Working Group, National Standards for Mental Health Services, December 1996)

“ Standard 10: DOCUMENTATIO N AND INFORMATION SYSTEMS - Mental health professionals maintain a high standard of documentation and information systems on clinical interventions and service development, implementation and evaluation to ensure data collection meets clinical, monitoring and evaluation needs.” (National Mental Health Working Group, National Practice Standards for the Mental Health Workforce, September 2002).

The completion of consumer outcomes is fundamentally a clinical activity that supports decision making and enhances the planning, implementation and review of care plans.

Consumer outcomes are governed by the same legislation and considerations as other clinical activities; including privacy and confidentiality of medical records and personal information and freedom of information.

Information system

All states and territories require entry of the consumer outcomes into their electronic databases.

Actual ratings for all the measures are required to be entered into state and territory databases.

Offering of the consumer self-assessment is mandatory at certain collection occasions but voluntary for the consumer to complete. Therefore most electronic databases request the clinician to identify additional items for these measures, such as not offered (and why), offered and refused, offered and completed (entry of actual ratings).

Mental health clinicians are involved in consumer outcomes collection in both the completion of the clinician measures and offering the self-assessment. Consequently clinicians are responsible (regardless of whether they self-enter or delegate to another member of staff) for the accurate and timely entry of consumer outcomes ratings into electronic databases.

The completion and recording of consumer outcomes is a clinical activity, and in an adverse event or through the process of freedom of information, clinicians may be required to justify incomplete, inaccurate or inconsistent consumer outcome records.

There is variability across Australia as to whether administrative staff or clinicians enter the consumer outcomes ratings. It has been demonstrated that where clinicians are familiar with the data system and involved in both data entry and data extraction there is greater likelihood that the ratings will be complete, accurate and utilised.

If clinicians are not responsible for data entry, they need to ensure that the forms are:

• Complete - Do not leave clinician measure items empty as these forms are clinical documents and it is not appropriate for administrative staff to be making a judgement attributing a code for incomplete measures with missing items.
• Legible to ensure accurate data entry.
• Timely.

Top of page

Hard copy file

The recording of consumer outcomes follows an assessment by the clinician. Therefore the clinician ratings should confirm or correspond to the information contained in the consumer’s clinical file.

There should be an entry in the clinical file to indicate when a consumer self-assessment has been offered and the outcome of that offering.

For most services the consumer’s clinical record will be a hard copy file but for services with electronic records this would be the area you write case notes.

When documenting the offering and/or completion, of the consumer self-assessment it is important to detail:

• When - date, and the time taken (if known). If it takes a long time to explain and assist the consumer complete their measure this might be an indication of the level of support required, the impact of their illness or possibly the consumer’s engagement with the service.
• What - the items that (a) indicate areas of concern identified by the consumer, (b) were left blank, and (c) areas that are different from corresponding areas rated by the clinician.
• Action - what you are going to do now? For example present to clinical review, review care plan and associated supports. It is really important that there are clear plans to follow up with consumers after their participation in the self-assessment process, regardless of whether the consumer refused, partially or fully completed.

Refer to 2.2.2 Dialogue

4.2 Policies/procedures

Policies and procedures are valuable for further embedding activities into clinical and corporate practice. They provide platforms for sustainability and have links with accreditation.

Examples

A number of services have developed specific policies and procedures for outcome measurement. Ideally consumer outcomes should be integrated into existing policies and procedures, such as case/clinical review, discharge planning, engagement of consumers etc. In developing policies and procedures around consumer outcomes it is helpful to consider the following:

The Victorian Western Cluster QUATRO project (2005–06) policies and procedures template informed this section.

For more information on the Victorian QUATRO Projects:

www.health.vic.gov.au/mentalhealth/outcomes

Top of page

4.3 Tips and suggestions

Desktop

The simpler it is to find, the more likely it will be used. A great time saver for busy clinicians is to have the relevant consumer outcome links and documents available on their desktop. Suggested links include:

• local database/electronic record
• local reporting tool
• state or territory website - both intranet and internet. Frequently asked questions section and formal communications about the NOCC protocol should be book marked.
• state or territory aggregated reporting (for those with authorisation to access)
• national website - www.mhnocc.org
• national web-based decision support tool – http://wdst.mhnocc.org

Colour coding forms

With consideration and recognition of local protocols and medical record standards, the more engaging and appealing the actual form is the more likely consumers are to complete it. Because there are so many versions of the Strengths and Difficulties Questionnaire (SDQ) it is helpful to colour code the different versions.

For example all initial/entry versions could be on pink paper with follow up versions on blue paper. In addition to colour coding differentiating parents from youth versions could be done by placing a sticker on the youth versions.

Availability of consumer self-assessment forms

Make sure forms are available and accessible by placing copies:

• in interview rooms or clinicians’ offices
• near the car keys, whiteboard, filing cupboards etc
• in staff meeting rooms
• in cars
• at reception desk/counter.

Hang a ’Whose outcome is it anyway? Consumer Self Assessment in Mental Health’ poster near the available copies.

Stapling brochures or information sheets for consumers and carers to the self-assessment form is another way to ensure clinicians have ready access to all the materials required by consumers and carers.

Information packs

Mental health services commonly compile packs with information about available services, consumer and carer rights and responsibilities and complaints processes. If these packs already exist it is really important to ensure that information on consumer outcomes is included. Consumer outcomes are routinely required in mental health services not an add-on or something special.

Therefore it is a good idea to include consumer outcomes in general packs rather than making specific packs for consumer outcomes. If these packs don’t currently exist rather than creating one just for consumer outcomes, create a general pack including consumer outcome information.

Consumer and carer consultants like other mental health staff have an important role in making information packs available to consumers and carers of the service.

Top of page

4.4 Service quality

National Practice Standards for the Mental Health Workforce, specifically:

• “Standard 9: SERVICE PLANNING, DEVELOPMENT AND MANAGEMENT - Mental health professionals develop and acquire skills to enable them to participate in the planning, development, implementation, evaluation and management of mental health services to ensure the delivery of coordinated, continuous and integrated care within the broad range of mainstream health and social services.
• Standard 11: EVALUATION AND RESEARCH Mental health professionals systematically monitor and evaluate their clinical practice, consistent with the National Standards for Mental Health Services and relevant professional standards to ensure the best possible outcomes for consumers, family members and/or carers.” (National Mental Health Working Group, National Practice Standards for the Mental Health Workforce, September 2002).

Where possible, they participate in or conduct research, to promote quality practice and seek funded educational opportunities to become conversant with current research.

“…without information debates are simply matters of difference of opinion - the NOCC data provide a common communication tool and the beginnings of evidence based dialogue...” (AMHOCN, PowerPoint Presentation, December 2006).

The Commonwealth Department of Health and Ageing has provided funding to states and territories since 2005 through the Commonwealth Own Purpose Outlay (COPO) Quality Through Outcomes (QTO). This funding has provided opportunities for concerted efforts on utilisation of consumer outcomes at the individual level for clinical decision making.

Before we can begin to use this information for quality improvement processes we had to establish the collection. As compliance increases and the data quality improves the consumer outcomes present a range of potential service quality applications that warrant further exploration.

Some examples include using consumer outcomes data for resource allocation, and for developing specialist clinical interventions to address identified consumer/carer need. Other ideas include areas for research, such as changes in consumer outcomes in relation to the clinical interventions provided.

National Mental Health Performance Framework

Under the National Mental Health Strategy, states and territories have committed to the development of a national framework to monitor the performance of public mental health services.

The National Mental Health Performance Subcommittee has been established to facilitate the implementation and development of performance measurement and benchmarking in Australia.

For more information visit: www.mhnocc.org/Benchmarking.

Key Performance Indicators (KPI) should report variability, with recognition of the ceiling effect (can’t get better) and the floor effect (can’t get worse). Meaningfulness and simplicity are key features of adopted key performance indicators. Developing indicators is a complex undertaking and the selection criteria for KPI are:

• sensitive to change
• meaningful
• simple to use
• casemix adjustable
• statistically robust.

Currently national KPI includes consumer outcomes in the capability domain as ‘outcome readiness’. That is, there are services capable of routinely collecting consumer outcome measures.

Collection or compliance has been steadily improving though not all services maintain the 85% compliance target. Work continues federally for adoption of consumer outcomes KPI in responsiveness (consumer self-assessment), and effectiveness domains (HoNOS suites).

In addition to the nationally developed KPI suite, states and territories have made significant investments in developing and implementing service level KPI. Funding permitting this developmental area will continue to evolve.

NMHWG Information Strategy Committee Performance Indicator Drafting Group(2005), Key Performance Indicators for Australian Public Mental Health Services, ISC Discussion Paper No.6 Australian Government, Department of Health and Ageing, Canberra

Consumer outcome reporting capability

Once the consumer outcomes measures are completed and entered into the electronic database that information is captured in each of the following levels of reports:

• consumer level reports generated locally for use at the consumer, clinician or team/program level
• aggregate level de-identified consumer data generated by jurisdiction for jurisdiction organisations
• aggregate level de-identified consumer data generated by AMHOCN from jurisdiction submitted data. This data is presented in the following formats:
  • Adherence to the Requirements of the Data Collection Protocols for the National Outcomes and Casemix Collection (NOCC) provided by AMHOCN to each state and territory. www.mhnocc.org/amhocn/Reports/
  • The ‘Decision Support Tool’ (DST) enables users to compare profiles of their consumers against the national aggregate and was originally available as an Access download from the website. Enhancements were made to this and it is now available as a web-based tool http://wdst.mhnocc.org/. All NOCC measures (except Focus of Care in Adult and Older Person) are available. The benefit of DST with the potential for inclusion of clinical prompts would be an opportunity to standardise practice, to prompt clinicians to clinical reference material and practice guidelines. The tool is accessible to everyone so it is important how information is presented. It might be good for consumers to see where they sit relative to others.

There is considerable variation in what is available locally and at the jurisdiction level with some variations in things such reporting parameters and immediacy of the reporting capacity. An overall reporting framework was developed to provide “a conceptual basis for the reporting of analyses of the Mental Health National Outcomes and Casemix Collection (NOCC) Data Sets. It is prepared specifically for those information users who intend to develop and analyse reports from NOCC data collected locally (be it at the jurisdictional or organisational level). The Reporting Framework details the key underlying assumptions used in the development on the National standard reports to date. As such, it is an ‘evolving’ document. As further analyses and reporting is undertaken with the national data, it can be anticipated that there will be refinement to the underlying assumptions.

These refinements will be documented in each release of Standard Reports as well as in subsequent versions of the Reporting Framework.”(www.mhnocc.org).

Example consumer outcomes in service quality

Incorporation

• To what extent are consumer outcomes utilised in program/service evaluation; accreditation; benchmarking and research?
• Have consumer outcomes been considered for inclusion in risk adjustment for KPI?

Compliance

‘Compliance with the National Outcomes and Casemix Collection (NOCC) protocol may identify process issues. Consider the following:

• Is there clarity about who is responsible for ensuring consumer outcomes ratings are entered on electronic databases?
• Are there compliance issues within a team/program?
  • If so, are they limited to a specific collection occasion?
  • What activities might be required to redress this?
• • Are there compliance variations between similar teams/programs?
  • If so, what is this attributed to?

Use of ratings

Consumer outcome measure ratings provide another source of clinical information with the added advantage of being aggregated and reported. Consider the following:

Confirmation or difference in clinical information

• Are the consumer outcomes rating scores confirmed by documentation in clinical file?
• Is there evidence in the clinical file that a consumer self-assessment was offered and/or discussed?
• Are those consumer self-assessment not offered due to acuity consistent with what would be expected in the ratings on the clinician measures?
• Is there evidence in the care plan that the outcome measures have been used to identify areas of need/strength or have they been related to goal setting in some way?

Trends

• What is the incidence of incomplete clinician rated measures (rating 7, 8 or 9)?
  • What is the incidence of incomplete clinician rated consumer outcomes on subsequent collection occasions?
  • If the clinician measures summarise the clinicians assessment, what activities are being considered to improve the clinicians capacity to cover each area in their assessment to enable them to rate each item?
• What is the incidence of ratings of zero or one on the clinician rated measures?
  • Is this consistent with consumer profile for that team/program?
  • If not, what activities might be undertaken to redress this?
• What is the percentage of clinically significant items for HoNOS suites?
  • What might this say about consumers accessing a particular team/program?
  • Is this stable, increasing or decreasing?
  • What additional supports/programs might be required to address the identified areas of difficulty?
  • What support or training might be required for staff on a particular team/program that has a high number of consumers presenting with a particular subset of difficulties?

Top of page