Systems Under Development Best Practices Symposium

Welcome Speeches

Denis Desautels

Denis Desautels, FCA, served for many years as an advisor to the Auditors General of Canada and Quebec, before he took up the post of Auditor General of Canada on April 1, 1991.

At the time of his appointment, Mr. Desautels was a senior partner in the Montreal Office of Caron Belanger Ernst & Young (formerly Clarkson Gordon).

Mr. Desautels spent the early years of his career in Clarkson Gordon as a computer audit specialist. He was a lecturer on information systems auditing at universities and for professional groups. As managing partner of the Quebec City office he advised the Auditor General of Quebec on the audit of the Public Accounts of Quebec and computer audit methodology used by that office.

The increased interest of our Office in systems under development arises, we can say, directly from concerns expressed by the Public Accounts Committee of the House of Commons. I may confess here to some pushing, or even some encouragement from the Office of the Auditor General. Committee members are really concerned about the apparent cost overruns and the apparent lack of effectiveness in some of the systems that have been implemented.

We made a more complete and direct response to those concerns in our 1992 Report, where I committed our Office to "a program of reviewing the development of major computer systems in the government".

In addition, we've made presentations on this subject to the Treasury Board Information Management Sub-committee known as TIMS, and the Advisory Sub-committee on Information Management (ACIM). These groups have encouraged us to broaden the scope of our inquiries to include the examination of managerial and human resources issues as well as technological issues.

When I mentioned "increased" interest earlier on, I did so specifically because our Office has long had an interest in systems under development. We share with you the same desire to encourage the effective and timely use of technology in order to accomplish the following three objectives: first, to improve the delivery of service to the public; second, to improve the management of the public service; and third, to reduce, to some extent, the deficit through savings.

In essence, we want to encourage you to "reap the benefits of technology" as suggested in my 1992 Report.

In reaping these benefits, we do not want to impede legitimate projects or cause them to be cancelled as a result of our work. At the same time, we are equally concerned with the very large projects which the Government has announced.

For example, the Government has estimated that a significant portion of the $3 billion in annual savings that have been called for, will come from the planned redesign and integration of the Income Security Program. It is important that lessons learned from the past be used to produce more effective systems in the future. We cannot afford to make the same mistakes.

We view our role as helping to avoid mistakes. One of the ways we want to do this is by concentrating on identifying those practices most likely to lead to the selection of projects that will provide the highest net benefits and that, once initiated, will have a good chance of coming in on time and on budget.

From a value-for-money perspective, we want to review the investment in technology to ensure that there is due regard to Economy, Efficiency and Effectiveness. Given both the significant investments planned for hardware, software and communications, and the serious impact on human resources, we have an obligation to ensure that the Crown is receiving value for money.

We also want to ensure that the appropriate data integrity and management controls are considered and implemented early in the system development process. This means that controls and security should be implemented as the system is being developed rather than trying to incorporate them after the system has been implemented, which is a far costlier alternative.

To do this, we must change our traditional audit approach of reviewing systems after they have been completed. We are responding to this need. We must be ready to move much of our work into the development phase of projects. And this must be done on a dynamic basis, rather than on the more traditional static basis.

Our current study on systems under development has two parts. In Part One we're identifying best situations and practices that have been, or are being, used during development and implementation of systems. And while our initial interest is in large systems projects, we're also interested in the application of these concepts to all systems projects.

During the past six months, we've been reviewing the literature and interviewing managers and users, as well as developers, in both the public and the private sectors. We have also sought the advice of experienced senior people on our findings. The highlights of the best practices we found as a result of this work will be presented today and will be published with the results of today's discussions.

In Part Two of our study, we will refine and test our approach and methodology for auditing and monitoring systems. We will then report the results in our 1994 annual Report.

So, where do we go from here? We're taking a very broad approach to this issue, looking at all aspects of systems under development, including human resources management, contracting, contract management and project management.

We value your comments and views very much and want to share with you the critical success factors in systems development projects. These factors will be used to refine our own audit criteria. In our 1994 audits we will benchmark systems against the best practices we've documented. From the results of these audits, we will be able to refine the criteria further. Part of this process will be to determine reporting guidelines as to when and how to report on systems under development, to management and then to Parliament. This symposium will be one step—a big step, I hope—in improving the way auditors can help in these projects.

Finally, I'd like to make some comments on the recently announced government reorganization. One result of this reorganization was the creation of a new function, that of Chief Informatics Officer. The Chief Informatics Officer will now be responsible for developing policy for information management and technology and related telecommunications activities. The CIO, as he's becoming known, will lead a government-wide approach to information technology development and investment with a focus on integrating the internal administrative systems.

We're particularly pleased at the Office of the Auditor General to see that, among other activities, the CIO will be responsible for providing support for the re-engineering of the government's program delivery mechanisms. It's our hope that this will foster improved interdepartmental co-operation and co-ordination in the systems area—a trend that should ultimately benefit taxpayers.

In closing, I'd like to thank all of you again for your participation. I would also like to thank those of you—and I know there are many—who've been interviewed as part of our public and private sector interview process. I hope you'll find the day interesting and informative, and we look forward to hearing your concerns.

Andy Macdonald

Andy Macdonald was appointed Chief Informatics Officer for the Government of Canada in June 1993.

Before joining the Treasury Board Secretariat as CIO, Mr. Macdonald was the Comptroller General of Canada. In this position, he was responsible for financial and management control systems, internal audit, program evaluation and the related professional development activities for the Government of Canada.

Since 1980, he has chaired the federal government's deputy minister Information Management Subcommittee, which has a mandate to provide a senior level perspective on the management issues in information technology.

I am pleased to be here this morning and to participate in this very important seminar. I think it is a good example of the kind of co-operative approach that we can take to address the very important issue of managing systems under development.

We have seen a lot of co-operation in the area of information technology, not only between the office of the external auditor and the internal people of government, but also between departments, other levels of government, as well as between the private sector and government.

For governments, I think the driving force is fiscal necessity. We simply do not have the resources anymore to go our own ways, and the fiscal pressures are intensifying. My prediction is that we will see more and more people who are ready to seek co-operative solutions to new systems development. I think that is important, and I put particular emphasis on the need in the information technology area for an agreed overall framework and a set of criteria that we can all use.

What I hope to see from this seminar and the work that will follow, is a common set of criteria that one can use for IT projects—criteria that will be used by the project managers, as well as by the external auditors. It seems to me that, if we are singing out of the same hymn book, then we have a better chance of at least agreeing as to how the project was discharged, rather than arguing over the criteria that the auditors used in assessing our performance. The latter is definitely the wrong thing to be doing when the auditors are two weeks away from their reporting deadline.

Let me talk about some of the general principles that we have. There are some general framework items relating to long-term planning and project selection, but there are also a number of specific things that relate directly to IT and I would like to highlight a few of them for you right now.

First of all, there is a strategic direction for information technology, a TB-Approved strategic framework, if you will. This is something that we have had in place for a couple of years and it has really provided value in orienting the government in its overall directions.

Second, I would like to describe our policy framework as it now exists and, perhaps, speculate with you on how it may evolve in the future. I want to talk about the very difficult area of what it means to specify architectures and how one migrates and makes progress.

I want to talk about people. We have always seemed to talk about people at the end, and yet, they are the ones that inherit the systems, that operate the systems we have now, that have to live with the how we build systems. I do not think they get enough consideration in the overall framework.

Let me talk initially about my role as Chief Informatics Officer. For the last couple of years, through the Council for Administration Renewal and the Deputy Minister's Treasury Board Information Management Sub-Committee, and supported by ACIM, we have been doing a good number of it initiatives across government. But the general observation was that there was no single focus for information technology. There was no single chair that you could go to and say, "I have a problem. What do I do about it?" I think it is safe to say, now, that in the area of IT, I am that chair, and I fulfil that role.

My responsibility is for enterprise-wide thinking. The issues that relate to the Government of Canada in a corporate sense are driven by the pressure to streamline our operations and to re-engineer our processes. This re-engineer is designed to provide better program service delivery at a lower overall cost. Finally, we recognize the need to provide an integrated approach to some of the major systems work that is going on.

I am responsible, therefore, for policy making in respect of information technology, exercising functional management over all departments, establishing a framework for the restructuring of programs that deliver services to Canadians, and thus directing the restructuring of administrative procedures. All of these factors are included in my mandate, and I feel that they allow me to perform my duties effectively.

Let me touch on our policy framework. We are the policy-centre for IT within the Treasury Board Secretariat, and like all policy centres, we develop and exercise policy. We use a "leadership" approach. This basically means amend the bad old days of developing policy in relative isolation and then rolling it out of the Treasury Board to descend on hapless departments. Today, we have to deal extensively with the departments, and they are consulted continuously. I firmly believe that if there is not general acceptance for a policy when it is issued, it will be a policy that will be very difficult to implement.

Our strategic directions document is worthy of note. It identified five strategic directions that the government should be moving towards. First, is renewing the delivery of services in the implementation of programs. This allows us to start our work in program restructuring areas, ... either a financial system, a personnel system or a social welfare system.

Second is investing strategically, which tells us that there is always a certain amount of money that the government is prepared to invest in new systems in order to improve the effectiveness of our services. The Treasury Board, even in these very tight times, has been quite good in saying, "We will have a hard look at any persuasive business case for the application of new technology." And, I think, over the last couple of years there has been a string of investments made in new systems designed to do just that.

The third is to create partnerships. One of the things about being virtually bankrupt is that you do not have a lot of money to invest. So, we are going out looking for partners. It fits nicely into the concept that one has a strategic confidence in an organization and, if it is not one of your core competencies, you look outside to find people who have the core competencies to make it happen. That is great, and I think that, increasingly, we will be doing that in the IT area. We love partners, especially those with deep pockets!

The fourth is to establish an open architecture and a base infrastructure that will allow us to do some of the sharing and co-ordinating of systems that has to be done.

The final objective is to empower the line managers and the front line employees with technology.

These are our strategic directions. To that, add the management of information technology in the long-term capital planning, and that really rounds out the broader framework that we have at present.

Let me talk for a moment about architecture, because it is a particularly interesting problem and it is one, quite frankly, that I have not really come to grips with. In principle, we want to be open. We have adopted an open system environment reference model, and that gives us the overall architecture definition for everything involved in the infrastructure: the interfaces, the services, the communication protocol, etc.

The whole point of doing this, of course, is to provide an environment in which we can have "interoperability" between systems. The systems can talk to one another. Anyone who attended our Technology in Government week and happened to wander down into the lower level at the conference centre would have seen a number of examples where we are starting to develop interfaces between materiel, personnel and financial systems to support this interoperability. Over the next few years, I foresee a dramatic reduction in the number and variety of administrative systems, and a common interface development that will allow those systems which remain to interoperate.

We have our TBITS standards, which the government is supposed to follow, and we require departments to develop a migration strategy. That's where the challenge comes. When you are sitting on a large legacy system and day-to-day operations occupy most of your budget, it is often difficult to migrate to these open platforms, particularly when the software is not necessarily there to support everything you want to do. I find this a little frustrating, and yet, I guess, that is the reality of change in the systems world.

What we have to do is to find a way to migrate to more open systems and at the same time recognize that is not going to happen overnight. In our view, major open systems with software portability will be among the key factors in our future planning. We would like to minimize our dependence on suppliers of computers and software. We would like to share systems, software, etc ... among all government departments. I believe it is essential that the federal government's systems be as open as possible.

Concerning TBITS guidelines, in addition to the ones that we already have, we are now working on application portability guidelines. An important item is case tools. It is now a virtual flower garden of case tools in departments. If we expect ever to have a repository of reusable code, we are going to have to get some standardization into this area.

And finally, the old Systems Development Life Cycle probably could use a good airing-out. That is, in part, why we are very interested in the outcome of the deliberations today.

I would like to spend just a moment, if I may, on people. I mentioned earlier that we always talk about people, but have you ever noticed they are always fourth on a list of four or five? I mean, they are never first! And yet, if the people do not really accept these systems that we have developed, then these systems are not going to work.

Mary Baetz, who will be on the panel this afternoon, came to one of our meetings a couple of years ago and told a fascinating story.

She described a survey that was done, a few years ago, to assess implementation of CAD-CAM (computer-assisted design, computer-assisted manufacturing) equipment in a sample of business enterprises in Ontario. They had criteria that I would commend to the Auditor General for stunning simplicity. Basically, the question posed was, "Is the equipment still in the building two years after the system was handed over?" More than 50 per cent of the firms failed the test. Why? Because they had failed to ensure that the end users, the people would access and use the new technology.

What that says to me, is that we must involve our people in the plan, right from the inception of the project. They are the ones that know the current systems and how they are used. These front line workers know what the system must do, particularly if we're talking about re-engineering. Re-engineering is driven by business process owners, not by technicians, not by people in central agencies, but by the owners of the systems! Now they may have to be assisted by people who know how to do process re-engineering, by people who know what the technology can give. But first and foremost it must be the process owners that do the reengineering. They know what is required and they are the ones that stay behind when the technological work is done and the system is handed over to them to operate.

So that suggests long-term planning and communication, and particularly, if we are talking about workforce readjustment, it means we have to open dialogues early with affected employees in unions so that we can plan the transition. I suppose it is unfortunate that it takes the government four or five years or more to put major systems into effect. On the other hand, in the human resources area, if you expect dramatic reductions in jobs (which tends to be what happens when we go after major paper-driven processes), having a long lead time means you can plan the workforce adjustment in a more systematic fashion.

I took this message to the public sector unions a couple of weeks ago, as part of Forum '93 and open discussion on some of the broader strategic issues. Basically, I said we would commit ourselves to open dialogue with the unions and the staff as soon as we knew what the implications were. Indeed, when one looks at the experience in a number of departments where they have major implications, very early on they identified how many jobs will be affected, and then have taken measures to begin retraining and other means to manage the effect on the workforce.

In the area of project management, one very important factor is the extent to which departments have the requisite skills and the necessary tools. As IT becomes more pervasive and more "owned" by the broader public service management community, project management skills and the ability to acquire people to advise on the technology will increasingly become an important part of every line manager's job.

If you say to the line manager, "You are the process owner and you are responsible for its re-engineering," and that manager does not have a strong technical background, yet is dealing with a consultant and a contracting firm, then that individual is going to need a lot of help. The line manager will need experience in project management, but may also require expert advice from a neutral third party.

This gives us real opportunities for partnerships with the private sector and for sharing among departments, particularly for shared systems development. We are being driven by the need to get more out of our shrinking resource base. One of the things about certain IT projects is that they offer the fascinating prospect of not only an increased level of service, but also a lower overall cost to the taxpayer and, perhaps, to the client we are serving. I think that is the thing that draws management, and it certainly draws me, into this whole area of information technology as applied to business operations.

I think I see some real change in roles in time of restraint. I am delighted to hear Denis Desautels say that he wants to get the external auditor involved up front in the projects. Frankly, I have always felt that, when the project is over and done, is not the time for the auditors to come in with their criteria and start picking over what you did or did not do well. I feel similarly about the internal auditors. All auditors have to be in there from the beginning.

We look to auditors to provide valuable input and advice on such things as the extent of management control of the project, the validity of our internal control processes, and whether what we're putting in place will, in fact, achieve what we want. That will be a variety of forms of advice, and it is not necessarily committing the external or the internal auditors or evaluators to not being able to change their minds down the road. We need the best advice we can get, from whatever source, as we implement major systems projects.

I have one other proposition for the internal auditor, or perhaps even the evaluator. I will raise it for your consideration during your deliberations today.

There are lots of business cases made to support IT projects. Some of these are submitted to the Treasury Board, the vast majority are dealt with in the departments. Many of those business cases are based on some questionable assumptions. However, who looks at those business cases when they are being developed? I wonder if there is not a role for internal audit to provide an attestation as to the quality of the information and the assumptions in the business case for a significant technology investment proposal before it is approved. I believe managers would derive great benefit from an objective review of the relevant business case.

One of the things that I have found is that the technology is so overwhelming that it tends to swamp senior management—the people who have to make the decisions—and they often defer to the people who obviously know more about technology than they do. Such people are the proponents of many of these projects, and as such are defacto not object. What we are dealing with here is not technology; we are dealing with business investment decisions, which have to do with the strategic business of the corporation. That is what we expect senior management to do; I believe they could help.

Another opportunity for internal auditors is the need for an independent tracker and scorekeeper for some of the benefits that are expected from information technology projects. World-wide, people are saying that they are not getting the benefits expected from their technology investments. It is very common in government, in financial institutions and in large corporations.

Part of the problem is that we tend not to define benefits appropriately. It is just not the traditional narrow cost/benefit analysis that I am talking about. It is much broader. In the private sector, if your product is not up-to-date, you will soon be out of the market—how do you cost that in a cost benefit analysis? It is an essential element in your corporate investment strategy. When it comes to IT benefits, I would look for assistance from internal auditors.

And finally, let me leave a message for the external auditors. I'm delighted to hear that you will be reporting in 1994 on how well we have done in our IT project implements. I am sure there will be a number of instances where our work was not up to scratch.

Let me remind you of a comment made by Ted Gabler. He commented on the importance of empowering people, he noted that they have to have the freedom to fail! Basically he was saying, if you want innovation and you want creativity, then you have to be allowed to fail. An absence of failure often is more indicative of a lack of creativity and initiative then of good management.

Another interesting comment Mr. Gabler made was that the people in government are often not the problem. They are not bad people. Rather, they are good people trapped in bad systems. I would suggest that my job, and the auditors' job, and all of our jobs, are to ensure that those bad systems no longer exist.