Department of Health

Victorian innovation and reform impact assessment framework

Key messages

  • Evaluation is an essential element of the Workforce innovation and reform roadmap.
  • The Victorian innovation and reform impact assessment framework measures the success of new and innovative health workforce initiatives, their potential for transferability and their alignment with future strategic priorities for the health services sector.
  • The framework is used for all health workforce innovation and reform initiatives in Victoria to inform the decisions of the Health Workforce Reform Implementation Taskforce.

Background to the development of the VIRIAF

In 2011–12 the Department of Health & Human Services commissioned the development of an overarching evaluation framework for use when monitoring and determining the success of health workforce innovation and reform projects. The Victorian innovation and reform impact assessment framework (VIRIAF) enables workforce project evaluations to be undertaken using a standardised approach and is designed to support economic evaluation and financial modelling of project results to inform possible rollout of workforce planning projects.

The evaluation framework was developed to align with the national impact assessment framework (IAF) developed by Health Workforce Australia (HWA).

The VIRIAF

The VIRIAF provides meaningful translation of the IAF principles, concepts and structure to the diverse range of workforce projects undertaken in the Victorian context that vary considerably in scale, scope, setting, rationale and objectives. The VIRIAF is designed to link the high-level national evaluation framework to work happening at the local level in Victoria.

The VIRIAF specifies overarching objectives that are likely to be common across Victorian workforce innovation projects. The seven objectives in the overarching framework are:

  • safety and quality of care – maintain or improve the quality and safety of care provided to clients
  • access to care – improve access to care
  • workforce capacity – improve workforce capacity, utilisation and productivity
  • integrated workforce – develop an integrated workforce through increasing collaboration between agencies, disciplines and the communities to achieve better client outcomes
  • clinician competencies and optimal use of skills – enhance the ability of clinicians to make treatment and assessment decisions (this will achieve the most efficient skill mix and workforce profile relative to the desired outcome
  • workforce satisfaction – enhance workforce training, career pathways, wellness and satisfaction
  • client satisfaction – enhance client satisfaction.

Using these overarching objectives, the VIRIAF can be customised to individual workforce innovation projects for the purposes of ongoing monitoring and future evaluation. The process of developing individual frameworks is outlined below, with more detail and suggestions provided within the VIRIAF document:

  • Step 1 of the evaluation process is to determine project specific objectives based on the overarching Victorian objectives.
  • Step 2 of the process is to identify indicators of success specific to the projects based on efficiency, effectiveness and sustainability.
  • Step 3 is to collect data to substantiate the indicators of success identified in step 2.
  • Step 4 involves assessing appropriateness. This step uses the information collected in steps 1 to 3 to determine whether or not the project objectives have been fulfilled. Appropriateness is measured by assessing the extent to which the following elements have been met:
  • Efficiency: this involves weighing up the tangible inputs and outputs of the project. Examples of inputs may include salary and training costs, while outputs might include change in staff numbers or services delivered. Often, the result of an assessment of efficiency is an indicator of ‘net cost’. Where a ‘net benefit’ can be shown, a favourable assessment of efficiency is established.
  • Effectiveness: this involves the assessment of whether or not the project’s objectives have been met. This is measured against the project-specific indicators outlined in step 2 and the information collected on these indicators in step 3. Indicators of success could include enhanced workforce integration, improved access to care or improved quality and safety of care.
  • Sustainability: this involves determining whether or not the project can continue to meet its objectives in the medium to longer term, analysing the enablers and barriers outlined in step 2 (including, for example, the workforce mix, engagement of stakeholders and clear leadership).
  • Step 5 involves an assessment of feasibility. Feasibility relates to whether or not the project can be extended to a larger scale and informs the decision as to whether a project should be implemented more widely as workforce reform. In this framework, a project would only be assessed for feasibility if it had achieved a (strongly) positive assessment of appropriateness. Assessing feasibility assists to inform departmental planning. This step should be considered in context with other factors such as strategic priorities.
  • Replicability: this refers to the ability of the project to be implemented elsewhere. Relevant considerations in the assessment include clinical settings, geographical regions and other barriers and enablers of replicability such as access to training and availability of funding.
  • Scalability: this refers to the ability of the project to be replicated many times over. Relevant considerations in the assessment may include consumer demand, infrastructure and workforce availability. Two key questions to ask in relation to scalability are:
  • Is it possible that the enablers for scalability could be removed and, if so, what would be the associated impact?
  • Can barriers to greater scalability be removed? If so, how can this be facilitated, and is it worth the associated time and money?
  • Risk: a comprehensive assessment of risk is essential to determining whether or not the project is feasible. Risks encompass those factors that would hinder the successful rollout of the workforce project.

From 2012, all workforce innovation and reform projects supported by the department are required to use the VIRIAF when developing their evaluation approaches.

Reviewed 09 September 2015

Health.vic

Contact details

Health and Human Services Workforce

Was this page helpful?