Public Audit Committee [Draft]

Meeting date: Thursday, December 15, 2022

Official Report 517KB pdf

---

Section 22 Report: “The 2021/22 audit of National Records of Scotland”

The principal item of business for the committee this morning is to consider the Auditor General for Scotland’s section 22 report “The 2021/22 audit of National Records of Scotland”. I welcome our witnesses this morning. The last time you gave evidence to the committee you did so remotely, so I am very pleased to welcome to the committee room Graeme Samson, who is a senior auditor with Audit Scotland, Dharshi Santhakumaran, who is an audit manager with Audit Scotland and, of course, the Auditor General, Stephen Boyle. You are very welcome. We have some questions to ask you on the section 22 report, but first I invite the Auditor General to make an opening statement.

Good morning, members. I am presenting today’s report on the 2021/22 audit of National Records of Scotland under section 22 of the Public Finance and Accountability (Scotland) Act 2000.

I have prepared this section 22 report to update Parliament on progress since my report on the 2020/21 audit, which highlighted the challenges that NRS was facing with delivery of the census programme and the financial impact of the decision to delay the census to March 2022 because of the Covid 19 pandemic.

Since I last reported, the census went live as planned in February, but achieved a response rate that was lower than the 90 per cent target. NRS decided to extend collection until the end of May, which resulted in an overall response rate of 89 per cent. NRS estimates that the extension will add £6 million to the lifetime cost of the census programme; the Cabinet Secretary for the Constitution, External Affairs and Culture agreed to fund that additional expenditure.

NRS is taking steps to mitigate the impact of the lower-than-expected response rate on the quality and robustness of census outputs, including the decision by the registrar general to establish an international steering group of experts to provide support and guidance. NRS is confident that it is still on track to produce high-quality census outputs, but it will now be more reliant than was originally planned on use of administrative data to support the production of its population estimates. It is important, therefore, that NRS continues to be transparent as it progresses the significant remaining work to conclude the census programme.

Convener, as ever, my colleagues and I look forward to answering the committee’s questions. Thank you.

Thank you very much for that helpful introduction to the report. I will begin by asking a factual question. The report talks about something called the census coverage survey. First of all, could you explain to us what that is?

Yes. I will start; Dharshi Santhakumaran might want to come in and comment further.

We refer to the census coverage survey in the report. It is a survey exercise that takes place after completion of the census. The survey explores returns and the reasons why people did not participate in the census exercise in order, in part, to understand people’s choices and some of the behaviours and data. We highlight in the report that it explores the breakdown of why people said they did not complete the survey. The most significant explanation that was given was that people were too busy, although there were a host of other reasons given for why people did not complete the census.

Before we go into any more detail, convener, I will note an interesting point about overall survey results. The overall return rate for the main census was below target and below the results from the 2011 survey, which is similar to what we see in relation to the following survey results, which are also lower in terms of completion rates and behaviour.

I may have said more than you wished, convener, so I will pause and ask Dharshi whether there is anything she wants to add.

To clarify, I note that the census coverage survey is used in Scotland and in the rest of the United Kingdom. In other countries, a similar method is used to follow up a census. It is, essentially, about filling in gaps to identify the number of households and characteristics of the households and individuals that did not respond to the census. The sample size is roughly 1.5 per cent of the population.

NRS did a smaller separate survey that explored why people did not respond. The coverage survey is part of the standard census methodology that provides more information about the population for census estimates.

The return rates in England, Wales and Northern Ireland were, from memory, about 97 per cent. Did their census coverage survey focus on the reasons why 3 per cent did not respond or did it have a wider role to play in giving more qualitative information about the census returns?

The coverage survey is not so much about the reasons why people did not respond; it is, rather, part of the methodology that provides information about the households that did not complete the census. It is not about the reasons, if that makes sense. The international steering group described three pillars of census methodology: there is the census itself, there is the coverage survey, and there is use of administrative data. Those three things combined are used to produce and to quality assure the population estimates.

I will turn now to administrative data. The committee took evidence almost a year ago from Audit Scotland on the report at that time, which was in advance of the census being carried out. We then took evidence from National Records of Scotland in January of this year. We had exchanges with NRS about administrative data and how it would be used. We were told that it would not be used to fill data gaps, but we have learned since that, given the census return rates, administrative data will probably be used more heavily than was originally anticipated. What is the implication of that for the quality of census data?

It is fair to say that use of administrative data, as Dharshi described, will be more significant in Scotland than was originally planned. That is a result of the survey completion rate that we have settled on: it was 89 per cent in Scotland, which was marginally below the 90 per cent target.

However, within that, there is a bigger story. As we set out in the report, Scotland has 32 local authorities and the completion rate target was met in 30 of them, but there was quite significant regional variation. For example, in Glasgow, the completion rate was 83 per cent; therefore, other approaches must be used in order to give robust and reliable data. Administrative data is a significant part of how we get reliable population estimates, which the survey programme needs for all the very important reasons that sit behind the census.

I will make one last point about administrative data. NRS has provided advice and guidance to support completion of the project in a reliable position, and there is the international steering group that is led by Sir Ian Diamond. Sir Ian gave evidence to the Constitution, Europe, External Affairs and Culture Committee and his judgment was that the overall arrangements that are now in place provide “a solid foundation”—I think that those were his words—for producing robust and reliable population estimates. NRS needs to go through the process during the course of 2023. There is a very important role in that for the Office for Statistics Regulation, which is working with NRS to get to that final point.

The answer to your original question is yes—administrative data will play a more significant part than was originally planned.

To be clear, are there or are there not more risks associated with relying on less primary data and more administrative data?

I am not sure that we are in a position to give a definitive judgment on that. That is primarily for the regulator—the OSR—and the international steering group. They have said that it is not an inappropriate methodology for compilation of robust population estimates, which leads us to say that it is not an inappropriate route for NRS to take. It is fair to say that NRS did not plan to have it this way, but that does not invalidate the overall results. Ultimately, that will be down to the judgment of NRS and the OSR.

You referred to this a few minutes ago. Do you have a view on the target of a 90 per cent return rate as set against the backdrop of there having been a 97 per cent return rate in the other parts of the UK? Does not that target lack ambition, when other parts of the UK managed, in a digital first census, to achieve return rates of 97 per cent 12 months earlier? Why was the ambition only to get a 90 per cent return rate?

NRS is probably better placed to answer, but from our analysis, drawing on the 2011 survey results, Scotland’s completion rate was lower than that in other parts of the UK; it was 94 per cent then. Although you rightly say that England, Wales and Northern Ireland had 97 per cent return rates for the census, we see, in drawing from other sources, a reduction in household survey engagement in returns in surveys.

On where NRS will go to for the next iteration of the survey, it is becoming much more likely that wider sources of data will need to be used—not just in Scotland, but in other jurisdictions—to arrive at population estimates. Bear in mind that the origins of the census methodology go back hundreds of years when there was no alternative to doing door-to-door household surveys; we now have alternatives and a broader suite of data.

Certainly, it is not for statisticians to make the judgment. However, if the OSR’s judgment, along with that of the expert panel, is that that is likely to be the direction of travel, what will matter will be that what is in the report is transparently set out, and that people who use the data and those who scrutinise it in Parliament can have confidence that the overall end position produces reliable estimates.

Do we know, at this stage, whether the next census will be in 10 years or nine years?

We do not, convener. Ultimately, that will be a political choice to be made in conjunction with NRS for the years to come.

I will steer away from politics, then, and ask my final question, at this stage.

You might recall that we had some quite detailed conversations with NRS about its access to administrative data. I think that it is the case that the Office for National Statistics has much more extensive access to HM Revenue and Customs data and Department for Work and Pensions data, for example. It has access to a much broader suite of data as a result of agreements that it had entered into. We were told that those are not data-sharing agreements that were entered into for the sole purpose of the census; they were entered into for other reasons. Nonetheless, it means that the ONS has wider access to much more comprehensive data than National Records of Scotland has.

My recollection is that when we quizzed National Records of Scotland, it said that it was looking to improve access to Scottish data sources in a range of work that it does, but is still at quite an early stage in that. The committee would be interested to learn—if you can help us with this—what progress has been made in respect of the data-sharing agreements that were spoken about in January this year.

09:30  

I am happy to start on that. Perhaps Dharshi Santhakumaran will say more about the gatekeeping and governance arrangements that exist and the progress that NRS is making to access additional data sources.

In high-level terms, NRS has identified a range of administrative data that is, in its view, necessary, having taken guidance from the international steering group. It includes electoral registers, national health service registers, student data and some schools data. That suggests that NRS is clear about what data needs to be sourced in order, ultimately, to get the end result of reliable population estimates.

We are not detecting that barriers are in place, but people need to have confidence that their data is secure and is being used properly and consistently with the purposes for which it was originally intended. Dharshi might have more detail. NRS might be in a position to update publicly and transparently, as it has committed to do.

I will just add that NRS is working with the support of the steering group to get extended access to administrative data for the purposes of the census population estimates. There are two panels to which applications for access to statistical datasets have to go: the public benefit and privacy panel for health and social care, and the statistics public benefit and privacy panel, which controls access to Scottish Government datasets and census data. As I understand it, NRS is hoping to have secured all the necessary permissions by the end of the calendar year. It will also have to come to data-sharing agreements with the organisations that hold the data that it is looking to access, including electoral register data, NHS register data about people who have registered with general practitioners, and school pupil census data.

I am sorry, Dharshi, but I note for the record that today is 15 December. Are you saying that that will happen within the next two weeks, or did you mean that it would happen in the next calendar year?

NRS reported to the census programme board in, I think, September, that it would happen in this calendar year. We have not had an update on that; NRS would be best placed to update you on whether it has been able to secure the permissions.

Okay. The committee will need to consider whether we want to pursue that with NRS.

I invite Willie Coffey to put questions to the witnesses.

Auditor General, before I ask a couple of questions about the work that remains to be done and some questions on the digital aspect of the census, I would like to know whether there is a standard—an industry standard or otherwise—percentage return rate that would be required to get a representative sample of the population in something like a census? What is the percentage figure or is there none?

I am not sure, Mr Coffey. I will turn to colleagues to see whether we know the answer to whether there is a reliable number or a threshold that must be reached. Given that there is variation between the ambitions that are set in different parts of the UK, we can assume that, as with any statistics, there is a margin of confidence for some of those numbers. I will pause and see whether colleagues can provide support on that.

I am not aware of an industry standard, as it were, but we know that NRS aimed for an overall response rate of at least 90 per cent. It hoped for 94 per cent or above, because that is what it got in 2011, but it considered 90 per cent to be the target.

I am curious about that, because we talk about target rates and response rates and they are entirely different things, as we all know. The UK target was not 97 per cent; it was 94 per cent. I am curious about why we think that we are significantly above or below when we do not know what target we are trying to reach for the survey to be valid. I note that in your report, Auditor General, you say that the census target response rate for Scotland’s local authorities was 85 per cent, which was exceeded by the actual response rate. I also note, from the ONS report on the census survey that was carried out in England and Wales, that the target was 80 per cent for local authorities in England and Wales. Why was there a lower target for response rates for England and Wales compared with that of Scotland?

I fear that I might not be able to give you a satisfactory answer to that, Mr Coffey. Variations in methodologies, and how robust they are, are probably questions that NRS and the ONS will be more able to answer. We have looked to take a view about whether, ultimately, having spent approaching £150 million of public money, the ONS achieved the intended outcome of a robust survey that supports population estimates.

I will broaden that out. Although the local authority completion rates were achieved in 30 out of 32 of Scotland’s local authorities, there are still gaps to fill. NRS, together with the regulator, needs to be satisfied that there are robust results. However, regrettably, I am not able to give you a helpful answer on the difference in methodologies between Scotland and England and Wales.

The point that I am trying to make is that, if a target is set lower and the response rate significantly exceeds it, it looks as though the performance is better than might otherwise have been expected. The higher that a target is set, the more difficult it is to do that.

I will move on to another question on your report. As one of your key messages, you say:

“Significant work remains to be done during 2022/23 to ensure that the census delivers robust population estimates and other outputs.”

Will you tell us a wee bit more about what that work involves?

I am happy to do that, and Dharshi Santhakumaran might want to supplement my contribution.

As I touched on briefly in response to the convener, NRS has identified that, overall, it has a gap in the household survey results compared with its targets, as well as regional variations that require it to bring in other sources, primarily that of the administrative data. We understand that it is going through that process at the moment, which will continue over the course of 2023, with further reporting that year and in 2024 before the overall programme is completed and validated with the regulator’s judgment that the population estimates are robust and reliable.

It is not just Audit Scotland that is calling for this programme of work to be done transparently; earlier this year, the OSR did the same, so that the Parliament and the public have a clear understanding of the work that is being undertaken. As we touched on in the paper, the OSR has given a view that the NRS process is becoming more transparent.

The analysis of the results, together with filling in the remaining gaps with the use of administrative data, is a brief summary of what NRS is currently undertaking.

I turn to the digital aspect of the census. Were we hampered in Scotland by the public’s access to digital devices to complete the online survey? Could you start by telling us what the difference was between this census and the previous one? There was a big digital and online component to this census, which was not the case before, so has that been a significant factor?

There are a number of points in there. This was primarily an online survey that households in Scotland chose to complete with appropriate guidance and signals from NRS. There were still many people in Scotland who requested a paper copy; some 600,000 paper copies were issued and half of those were returned. However, it is absolutely the case that Scotland has moved to an almost entirely digital survey.

On your question about whether people were hampered in their ability to complete it due to access issues, we have not analysed that yet. One of our key recommendations in the paper is that NRS, as part of its remaining work and evaluation, goes through the process to understand the choices that people made to complete the survey and why some people did not complete the survey. Through much of our recent reporting, we have spoken with the committee about access inequalities, whether that was one of the key factors behind Scotland having a lower-than-anticipated return rate and what that means in terms of data reliability. It is now a key responsibility for NRS to understand what the reasons were and what barriers there were for household participation in surveys.

Did NRS look at variations in population in relation to the level of access that people have to information technology and skills and so on? I was well aware of that when I ran the cross-party group in digital exclusion. There were huge differences, even within Scotland, of access to digital technology for people. Despite people’s willingness to participate in the online world, there is still an issue about whether some can actually do that. Will the further work try to investigate the portions of the population that could not participate as fully as they might have wished to?

Again, I will start, and Dharshi Santhakumaran might want to come in on this point, as well.

The process that NRS now goes through is really important. I have already mentioned Glasgow a couple of times in my responses. I do not think that we have reached a view about whether there is a direct parallel between deprivation and digital exclusion and participation in the survey in an electronic context, but there are some telling indicators about where deprivation exists in Scotland and participation rates in the survey—digital or otherwise. There will be important lessons for NRS to learn, ultimately. Whether such lessons are applied in however many years’ time for a future census, any population survey that is undertaken has to think about how people can most easily participate in order to get reliable results.

We do not know the exact details of what will be involved in NRS’s evaluation work, but we expect it to look at digital exclusion issues and how it could build on that for future censuses. We know that, during the collection period, particularly in the extension period, it targeted its field force efforts and support efforts to areas where it had identified that the response rate was lower. I hope that that will have also picked up areas where people were digitally excluded or having issues with completing the census online. Obviously, there was a helpline and other means to support people to either complete the census online or get access to paper copies of it.

Anecdotally, one of the things that I picked up around the time of the census was that, for people who did not complete the census in one go, they discovered when they went to log back on that they had to start right from the beginning. I do not know whether that feature was common to the systems that were applied in Wales, Northern Ireland and England or whether it was a deficiency in the Scottish system that, perhaps, drove down the completion rate. I do not know whether you have picked that up.

I had not, convener, but I can well understand why that would be a deterrent for people re-engaging with the survey. It feels like an important area for NRS to investigate as part of the overall validation judgments that it has made on the programme.

09:45  

Auditor General, I would like to go back to something that you started to talk about in, I think, your conversation with the convener. The rest of the UK—England, Wales and so on—had access to other databases, which enabled them effectively to boost the census return levels. Do you have any idea what percentage improvement was achieved because of that?

I will just check my understanding of your question. I am not sure that there is a direct relationship between using administrative data and the overall survey return rate such that someone could say that that boosted the overall return rate. Our understanding is that data is used in parallel to produce ultimately reliable population estimates, as opposed to saying that such use of administrative data would equate to a percentage of return rate in lieu of members of the public not completing the survey. I will just check that my understanding of that is correct.

Yes.

That is our understanding.

It improves the quality of the return, not the quantity.

Correct.

Okay, that is good to know.

In your report, you state that, in order to

“improve the response rate, NRS field force staff offered support and assistance across Scotland”,

and that, in particular, they

“focused support in areas with lower response rates”.

Which areas had lower response rates and what additional support, as referred to, was actually given in those areas? Was it just more of the same, or was there something extra?

We are happy to provide that detail to the extent that we have it. Dharshi Santhakumaran and Graeme Samson may wish to come in with further comments.

I have already mentioned a couple of times that Glasgow was the council area with, at 83 per cent, the lowest return rate in Scotland. What that means—I think that this connects with Mr Coffey’s line of questioning, too—is that where there is lack of access to digital devices or a question of household preference, people were given the option to complete digitally or to complete in person. To support the completion rates, NRS employed a field force of temporary workers to go to people’s doors to remind them of the importance of completing the survey, offering assistance where necessary. That was followed up as required with additional visits. As many of us will recall from surveys in previous decades, there is quite a tradition of field staff going door to door to support and remind people, and to encourage them to complete the survey.

For a place such as Glasgow, do we have any information about whether the return rate for particular ethnic groups was lower than for others? It is quite important that we get that outreach.

Dharshi Santhakumaran might know. Before she comes in, I would add that this is an incredibly important part of the survey validation. It is not just about getting up to a certain percentage of confidence; it is about recognising that the survey results are representative of society. To get to robust population estimates, the results have to be tested against a range of parameters in terms of groups with protected characteristics, which drives funding formulas and the delivery of public services over the following decade.

It is also important that all that thinking is part of the methodology, and equality impact assessments need to feature prominently as part of that assessment. I will pause and check with Dharshi what further information we can add.

We do not have information to hand about response rates among specific population groups, although NRS should be aware of that. Part of its work, certainly during the extension to the collection period as well as subsequently, when it comes to the census coverage survey and the use of administrative data, is about filling in the gaps about the people who did not respond. That means that there is better data for those smaller geographical populations, ethnic groups and so on that are harder to count, if you will. When the OSR is reporting on NRS’s efforts in the new year, presumably it will check that it has that robust data for the smaller populations.

You are saying that the data exists; it is just that it is not available to you at the moment.

I assume that NRS will have the data but currently we do not.

I think that that is right. We do not know the precise position yet. It is a reasonable assumption to make that, within the data and the analysis and evaluation that follow, NRS, with the international steering group and in line with the judgments made by the regulator, will go through that process. All those factors will be combined to reach the judgment that, as a result of the census, Scotland has reliable population estimates. That will happen towards the end of next year and into 2024. Population estimates include all the analysis that you referred to in terms of ethnicity and other characteristics, Mr Beattie.

You referred to Glasgow as an area that had a lower response rate. Are there other areas of significance where similar patterns emerged?

At a headline level, in 30 out of 32 local authorities, Scotland met the target rate of 85 per cent completion that NRS set. However, that tells only part of the story. You mentioned Glasgow, which was below that target, but—this probably follows on from your previous question—that does not necessarily mean that the exercise will produce reliable population estimates. This flows into what comes next in the evaluation. Is there reliable data for different parts of society, whether that is Scottish index of multiple deprivation data, which leads to funding formulas, or data on different ethnic groups, with the associated flow of funding?

That takes us back to the point that the route to get to robust, reliable data involves the survey results plus the administrative data. At the moment, we are reporting where NRS has got to at this stage. There is considerable work to be done over the next year so that NRS can produce that reliable population data by local authority and by different groups across Scotland.

Your report also states that NRS is investigating, or continuing to investigate, the reasons for the lower-than-expected response rate and why it was lower in comparison with rates in other countries. How is it doing that, what progress is it making, and is there a target date?

We are aware of that up to a point—again, colleagues can come in and say a bit more. At an overall level, establishing the international steering group was a positive thing to do because it has given NRS access to experts in the use of data and some of the behaviours of the public as to why they will or will not participate in surveys. In his evidence to the Constitution, Europe, External Affairs and Culture Committee, Sir Ian Diamond talked about some of the change in behaviours and the public’s engagement with surveys, which is lower than it has been historically.

That is all part of the work that NRS is currently undertaking. In our report, we encourage public and transparent reporting of the lessons from the 2021 survey and what those mean not just for the next survey but for future surveys.

It is safe to say that people’s behaviours are changing in terms of their willingness to comply and engage with survey activity. NRS and other survey organisations have a range of tools at their disposal—administrative data as well as online and paper-based surveys—so that their reach is as broad as possible. All that is part of NRS’s work for the year to come.

You have not mentioned anything about timescales or actual progress.

There are two points. The Office for Statistics Regulation is due to publish a report towards the end of 2023 on its work with NRS on population estimates. Then NRS has publicly committed—I think that this was confirmed by the cabinet secretary—to produce a report in 2024 on some of the evaluation lessons learned from this survey. Those are quite important milestones in satisfying the Parliament and the public about the assessment of the survey result.

I will stop in case there is anything that Dharshi Santhakumaran wishes to add.

The full evaluation report is due to be laid before Parliament as the census programme comes to an end, and that should be in 2024. That evaluation is supposed to look at the entire scope of the census, including its design and the operational elements of the census collection period, along with all the subsequent statistical methodology, the processing of the data and the outputs. It should cover the whole programme.

Good morning. Paragraph 12 of your report explains that programme costs are estimated to increase by £6 million in 2022-23. The report goes on to state that

“the actual figure will not be known until the end of 2022/23 and will need to be carefully managed.”

What are the risks if the costs are not carefully managed? Do you have any concerns about that?

I will bring in Graeme Samson to talk about the audit work that we have undertaken in the past year on the spending to date. As we set out in paragraph 12, the costs increased by about £6 million this year, and we have analysed that a little more closely, in terms of there being additional supplier costs of £3 million, £1.7 million for the field staff and then an additional just over £1 million for the census coverage survey.

Risks remain of additional spending on top of the further costs that we discussed with the committee last year as a result of the extension, but I do not think that we are talking about the scale of additional spending that we had considered as a consequence of the planned delay, given the pandemic. Nonetheless, given that the project has cost around £27 million more than originally intended, it matters that these costs are still kept under close review and that they are managed, in order to keep further public spending to a minimum as a result of the project.

I will bring in Graeme Samson in case he wants to share with the committee any of the NRS’s arrangements.

NRS monitors the census spend very regularly, on which it reports to its committees and board. We look at that work as part of the audit in order to test the expenditure on the census and to confirm that it is being closely monitored and scrutinised during the year. NRS has managed to do that in recent years. It has kept very close to budget and is managing that well.

The Auditor General mentioned the extra expenditure of £6 million for 2022-23. NRS was originally given up to £9 million extra but has reduced that, estimating that it will need £6 million. I would say that it is monitoring the expenditure closely, and it will need to continue to do so.

Thank you. Paragraph 15 on page 6 of the report states:

“As the census programme progresses through its later stages, it is important that NRS has knowledge transfer plans in place to build on the skills that it has in-house.”

Do you know what action NRS is taking to ensure that that happens?

Colleagues can update you on whether we have detailed insight as to the specific work that NRS is undertaking. I will bring in Graeme Samson or Dharshi Santhakumaran in, but first I note that we make such comments in a lot of reports, when we talk about IT arrangements in the public sector.

10:00  

Public bodies need to bring in expert services when they are undertaking a project for which they do not require those services day to day—they do not employ them on permanent contracts. A risk that we have seen over the years is that once the design or build phase ends, contractors leave the organisation and the expertise and skills that the public sector has paid significant sums for go with them. The public sector needs to guard against that—although within reason. NRS will not be undertaking the survey year after year, but it is a largely digital organisation, so it is important that the skills and expertise that it has paid for remain in the organisation. We expect that that will be part of its evaluation of the survey. On the steps that it is taking, I ask Graeme Samson whether there is anything further that he can share.

I do not think that I have any further details on that. As the Auditor General says, there is a large use of temporary staff during the census project. We have commented on that issue in annual audit reports in recent years. The nature of the work is that staff are only needed for a short period of time, and NRS will have to look at how that knowledge is transferred. I do not have any more detail around the arrangements.

Good morning, Mr Boyle. I suppose that, as we approach Christmas, we should be reminded that censuses are nothing new; the Romans were conducting them every five years more than 2,000 years ago.

I want to reflect on what happened with the most recent censuses in England and in Scotland. Last week, the ONS published the report “Maximising the quality of Census 2021 population estimates”, which obviously relates to the census south of the border. One of the main conclusions and points that the ONS makes is:

“Our planned flexible approach to collection and well-tested response strategy enabled us to respond to changing circumstances, such as the coronavirus ... pandemic.”

I will focus on the phrase

“planned flexible approach to collection and well-tested response strategy”.

Have you looked at what happened in England to find out why the UK census was more flexible than the Scottish one and how the “well-tested response strategy” in England differed from what took place in Scotland?

We have not done that work, but that is exactly the analysis that NRS will want to reflect on over the next 12 months as part of its evaluation of the completion of Scotland’s 2022 census. NRS will want to consider not just how it went but whether there are opportunities to apply different approaches next time and to learn from other jurisdictions. The fact that it has appointed an international steering group suggests that it is open to bringing that learning to future surveys in Scotland.

I note that NRS is not formally represented on that international steering group. Is there any particular reason why that is the case?

That is potentially quite helpful. Rather than NRS—

Self-auditing.

Yes. Rather than its being seen to have undue influence on the views of experts, it is drawing on independent advice. The fact that the group is led by Sir Ian Diamond, the national statistician, gives it a reach and a credibility to support NRS. It is also important to say that NRS is engaging well with its regulator on the overall population statistics. That all shows that it has a range of sources of expertise to inform judgments about the survey and where it goes next.

The census in England proceeded on the planned timetable, but the census in Scotland was delayed. Have you been able to make any preliminary judgment on whether the decision to delay meant that we improved the data capture rate in Scotland or whether, for some reason, that decision impeded management of the census?

I will reflect on what was said in the previous report. NRS and the Scottish Government took the view that they would not have been able to produce reliable population estimates by undertaking the survey in 2021. In their view, that decision was absolutely necessary so that Scotland could produce reliable population estimates, albeit that the delay was the consequence.

We remember that the committee explored the matter with NRS in a previous evidence session, and the convener has already referred to the committee’s discussion about administrative data and NRS’s methodology compared with that of the ONS. At the risk of labouring the point, I note that NRS is now at the point at which it needs to, over the next 12 months, undertake analysis to learn from and reflect on this survey, and it will then have to consider what that means for the methodology that is used for the next survey. Such surveys take place every 10 or 11 years, and changes in behaviour and technology are taking place at such a pace that, as well as learning the lessons from this survey, the approach will undoubtedly need to be refreshed in the intervening period in anticipation of future significant public surveys in Scotland. I am sure that that will be the case with NRS.

In relation to the use and impact of the Scottish census data, you discussed with Mr Coffey the digital issue and the issue relating to the areas that are hardest to reach, such as those with the highest rates of deprivation. Given that Government bodies need data relating to those areas in order to work out which policy instruments should be deployed, is there now a risk that, unless we focus on the areas that are hard to reach, for which there were lower response rates, we might have a problem in addressing the social and economic needs of those areas?

There is a risk if that does not go well, but it is a key focus. The OSR is very alert to that risk, and addressing it is a key part of its discussions with NRS. As we have touched on already this morning, the issue is not just about getting a particular number of survey returns; it is about the analysis that sits beneath those numbers in relation to different parts of society, which is what, ultimately, produces reliable population estimates. We are clear that NRS, the regulator and the international steering group are focused on that point.

I am looking for clarification on a final point. The report states that the international steering group

“concluded that extending the”

census coverage survey

“would have a negative impact on subsequent stages of the census programme”.

Dharshi Santhakumaran, do you know what those negative impacts were likely to have been?

I think that they relate to timescales and ensuring that outputs can be produced. The aim is to get the first outputs published a year from the end of the collection period. In essence, if the census coverage survey had been extended, that would have had a knock-on effect on the ability to produce outputs.

I have a quick question for clarification. In paragraph 15, you indicate that 42 per cent of overall staff costs relate to temporary staff. The explanation that is given for that is that those staff are needed for the census, which must be a major part, and for specialist IT projects. I am interested in what the split is between the two.

I will check with Graeme Samson whether we have that information at our disposal. If we do not, we can come back to the committee in writing.

I do not readily know what the split is, but we can check whether we have additional information about that.

Thank you.

We might need to follow up with you or NRS on elements that have arisen from the evidence that we have taken. Before I draw the public part of the meeting to a close, I thank our witnesses—Dharshi Santhakumaran, Graeme Samson and the Auditor General, Stephen Boyle—for their evidence.

10:10 Meeting continued in private until 11:24.