Feature / Data lights the way

In September, the Audit Commission published its latest annual report for the payment by results (PBR) data assurance programme. The report summarises the findings of the commission’s fourth annual audit of data used to underpin PBR payments. This year the main focus was on the reference costs used to set the tariff.

Most trusts’ reference costs submissions were accurate in total. Twelve per cent of trusts had errors that were significant enough for us to view their submission as being inaccurate at a trust level. But the overall level of accuracy means the reference cost index is a reasonably reliable guide to the relative efficiency of a trust.

However, we found the accuracy of individual unit costs varied and was sometimes poor. Correct recording of activity data was the main problem, which led to auditors recommending three-quarters of trusts review how they assigned costs in one or more areas.

The data used to inform the PBR tariff was generally good. It was in 'non-tariff' areas such as community services and chemotherapy where the unit costs were often of poor quality. Under government plans, most or all of the £51bn spent on acute hospital activity, as well as wider community and mental health services, will be subject to a PBR tariff. Tariff currently covers £26bn of acute services.

One trust overstated the activity of a single healthcare resource group (HRG) for high-cost drugs by 55 times – recording activity of

1.2 million when it should have been 23,277. This resulted in a national reference cost of £4. Without the trust’s data, it would have been £521. High-cost drugs are outside tariff.

These errors don’t just have an impact on the ability to set tariff in these areas – there are serious implications for the local NHS. Trusts and primary care trusts often use reference costs in non-tariff areas to inform local contracts. Improving reference cost data isn’t just about getting the national picture right.

We found there were some straightforward solutions trusts could put in place to improve their data. More than half of the trusts audited did not check their submissions against other data sources. Greater use of basic checks, and benchmarking unit costs against those of other providers, will lead to improved data quality.

Better senior leadership within organisations would also help. Directors of finance sign off trust reference costs submissions. In some cases it was hard to see how they had assured themselves about the accuracy of the data. In other cases it was clear the trust knew there were problems with data quality but took no action.

To help address these issues our annual report includes a checklist covering 10 key areas senior hospital managers can use to improve the quality of reference costs submissions (see box).

We have also made the tools we used to target our investigations during the audits available online through our national benchmarker. These will help trusts to check the accuracy of their submissions.

We will update the benchmarker with the latest reference costs data when the Department of Health releases it to the NHS, which is planned for November. You can access the benchmarker online tool, which is freely available to the NHS, from the Audit Commission’s website.

The use of disparate IT systems caused many of the errors in non-tariff services. Fixing this problem is likely to require significant investment. Based on the current timetable, tariffs for 2014/15 will be set using costs in the current financial year (2011/12).

The Department of Health – and Monitor and the NHS Commissioning Board when they get their new responsibilities – should therefore explore other ways in which the tariff can be set in new areas. Reference cost data in non-tariff areas is unlikely to improve quickly enough, even to set prices in 2015/16.

In some services the national spread of costs was wide, but these different costs were a genuine reflection of trust spend on that activity. And alongside variation in costs, there was also significant variation in the hospital setting reported for some services. Activity that was recorded as an inpatient at one organisation was an outpatient at another, and a regular attender at yet another.

A single national price for many non-admitted patient care services, set either through reference costs or by alternative means, would carry serious financial implications for many commissioners and providers.

Such variation could, however, also suggest there is room for greater efficiency, particularly if the different approaches have the same clinical outcomes.

Areas for clarification

The commission also found there were areas of the reference costs guidance that were unclear or open to interpretation, and we have highlighted these to the Department of Health in the report.

When the audit approach was piloted in the previous year, auditors found the implementation of patient level information costing systems (PLICS) had an adverse affect on the quality of reference costs.

However, these problems were not evident during this year’s audits. Instead, where trusts used PLICS well it had introduced a business culture that had improved data quality – for example, by increasing the understanding of the importance of costing throughout the organisation.

Alongside the reference cost audits, auditors undertook inpatient clinical coding and outpatient data audits at trusts that had performed consistently poorly in previous audits. Each review included a previously audited specialty, where we found much improvement.

A key factor in the improvement was that the trusts had now implemented the recommendations from previous audits.

And the findings also demonstrate the benefits of a consistent national assurance programme.

The commission also audited the main independent hospitals that provide acute services to the NHS. Error rates were higher than in the NHS, caused by problems with coding diagnoses, particularly when the patient had co-morbidities. This reflected the focus of the hospitals on interventional procedures rather than management of long-term conditions.

Also, they did not need to code such conditions to receive correct payment under their contracts. If payments for this activity had been made under current PBR rules, the independent sector would have undercharged commissioners.

This year’s report has some powerful findings about the strengths and weaknesses of reference costs and how they can be improved. Managers in the NHS tend to dismiss them. But they give a good guide to a trust’s relative efficiency and to the costs of services currently covered by the tariff.

The report also shows the value of a national assurance programme. Comparing and contrasting performance year on year and between organisations have always been powerful levers for improvement.