The Quality and Reporting of Surveys

Better reporting of surveys would make them more useful to Parliament

Chapter 2, November 2005 Report of the Auditor General

Ottawa, 22 November 2005—Parliamentarians and the public have no way of knowing whether the findings of public opinion surveys commissioned by departments and agencies are reliable, says Sheila Fraser, Auditor General of Canada, in her Report tabled today in the House of Commons. Reports of public opinion survey findings are not accompanied by enough information about survey quality.

"When surveys are used to help government manage and report on its performance, Canadians and parliamentarians need to have confidence in their results," said Ms. Fraser. "We found problems with the reporting to Parliament on the quality and limitations of survey results."

Survey findings are often presented in departmental performance reports, which are key documents used in holding departments accountable for the money they spend. The audit found that the surveys used in 2003–04 performance reports were not accompanied by enough information for readers to be able to assess the reliability of the survey results. Nor was this information generally available in most of the final reports of surveys. If the quality of these surveys is unknown or questionable, then the government risks managing with, and reporting, misleading information.

While individual departments and agencies are responsible for the quality of surveys they commission and pay for, the Public Opinion Research Directorate in Public Works and Government Services Canada has a government-wide leadership role to play in contributing to consistently high quality public opinion surveys. The audit found that the Directorate was not fulfilling all its assigned responsibilities. Also, by not adapting industry standards for government use and not developing benchmarks, the Directorate has missed an important opportunity to contribute to survey quality.

"The Public Opinion Research Directorate is not providing the government-wide leadership required by government policy to promote high quality public opinion surveys across government," said Ms. Fraser.

- 30 -

For more information, please click here.