News / LATEST: Review rejects substantial reference costs changes

There will be no major changes in reference costs in the near future, though there will be greater focus on verifying the quality of data and a shorter timetable for the collection and publication of the information.

The review of reference costs, which was published on 29 July, confirmed that the Audit Commission will audit all reference costs submissions, starting with the 2009/10 reference costs, as part of its payment by results data assurance framework – an extension of the framework already announced by the commission earlier this month. While the perception of reference costs was poor, the Department of Health believed the review has shown this to be a harsh assessment – most data quality problems lay in outpatient and non-admitted care, it said.

Four documents were published – a summary report, together with reports from three workstreams – on the use of reference costs by NHS bodies, uses by the Department and other national organisations and the findings of pilot reviews of the quality of 2008/09 submissions. The Department said the review highlighted its need to examine the feasibility of collecting spell level data, and it acknowledged it should understand the interaction between patient level costing and the production of reference costs. However, for the immediate future the majority of tariffs would be based on reference costs, and improving the quality of this data must be the Department’s priority.

Annual reference cost guidance will be brought forward to December, a validation tool will be developed and, starting with the 2010/11 collection, organisations will only be able to submit their information once. The Department hopes this will encourage them to submit accurate data first time. Reference costs will be published earlier – the 2009/10 publication will be brought forward to December 2010.

The review was carried out before the recent Liberating the NHS white paper and the Department acknowledged some of the uses listed in the report could change. However, it insisted the findings of the review and recommendations on improving data quality remained valid, at least in the short term. Decisions over the collection of reference costs in the long term would be informed by the Department’s information strategy and the review of data collections announced in the white paper.

The review of reference costs submissions, carried out by the Audit Commission, found respondents felt the quality of published reference costs were poor. The commission reviewed reference cost submissions at 16 pilot sites and found the quality of the data varied across the different categories. Trusts’ submission processes were finance driven, but often lacked basic checks to assure the quality of the data, it said. The commission’s findings were similar to its last review of reference costs in 2004 – when it found organisations identified and calculated costs reasonably accurately, but did less well in reporting reliable activity information.

While large sections of the submissions were reliable, especially admitted patient care activity data and total costs, some were less so, including outpatient activity data, non-admitted patient care activity data and cost allocation. The link between reference costs patient level information and costing systems (PLICS) was often not well formed.

Causes of the problems included concerns about the timetable and arrangements for collection; a lack of prescription in the guidance that could cause inconsistencies; and weaknesses in trusts' arrangements for collecting reference costs. Trusts setting up PLICS in particular gave reference costs a lower priority, as they did not make extensive use of detailed national information.

The commission suggested the Department review the timetable and guidance for the collection of reference costs; reconsider arrangements for local and national validation and publish final data in a more timely, accessible and usable format to enable more effective analysis and use by trusts locally. The Department also needed to be clearer about how or if it will use PLICS data nationally.

Trusts could take steps to improve the quality of their data, including sense checking submissions, analysing historical data, reconciling to other sources such as hospital episode statistics and adopting a more corporate approach to collating data rather than relying on individuals in finance departments.

The commission's review of uses by NHS bodies included a survey of health service organisations (PCTs, acute, mental health and ambulance trusts) and held a workshop with SHA representatives. The latter suggested the move to HRG4, and the subsequent increase in the number of HRGs, had made the reference cost collection process too complicated, while lack of clarity in guidance led to inconsistencies. There was concern that, increasingly, trusts were using the Department's verification process as the main sense check of data. In the light of this, the commission said it was unlikely organisations do all they can to ensure the data is right first time.

Some SHA leads suggested trusts might cost-shift into non-PBR areas to support local contracting. While they did not provide examples of this, the commission said that if true this could put question marks over the accuracy of national tariffs and local contract prices.

Despite the perception that reference costs are poor, they were widely used. The commission's survey showed more than 90% of respondents made use of reference costs locally, including benchmarking, contract support and in determining local prices. PCTs used them to underpin programme budgeting information and they were used at a national level to determine tariffs, by regulators and academics.

Virtually all elements of reference costs were used, though different bodies used different elements of the collection. For example, less than 50% of the national collection informs the mandatory national tariff (although it is expected reference costs will be used in the development of new tariffs).

More than50% of those surveyed were either fairly or very dissatisfied with the publication timetable for the national dataset. The commission felt this could be hampering wider use by NHS bodies. While 62% felt the process of collecting reference costs was 'about right', 38% of acute and specialist trusts said it was too complicated.

The Department's PBR team reviewed the use of reference costs by the Department and other stakeholders, such as regulators, academics and think tanks. It said its review confirmed that there was no need to make major changes to the existing scope of the collection and that, with the possible exception of programme budgeting, those who currently use the data will continue to use the data in the foreseeable future.

The Department said reference costs would be required to produce national products, such as the national tariff and productivity indicator in the immediate future. The programme budgeting team’s current pilot of a tariff-based return plus the reference cost-based return in 2009/10 will inform the decision to collect reference costs based programme data beyond 2009/10.

There was some discussion on how reference costs were used to inform the tariff – reference costs are based on finished consultant episodes, while the tariff is based on spells. The Department said collections based on spells would be preferable as this would make the calculation more transparent. A spell-level approach was trialled unsuccessfully during the 2006/07 collection, but stakeholders suggested this could be achieved by having a spell level collection based on a simple aggregate of constituent FCE costs, avoiding the complexities experienced in the 2006/07 collection.