Public Accounts — Issue No. 21 — Monday, November 2, 2015 (HTML)

Present: Bruce Ralston, MLA (Chair); Sam Sullivan, MLA (Deputy Chair); David Eby, MLA; Simon Gibson, MLA; George Heyman, MLA; Marvin Hunt, MLA; Vicki Huntington, MLA; Lana Popham, MLA; Linda Reimer, MLA; Selina Robinson, MLA; Ralph Sultan, MLA; Laurie Throness, MLA

Others Present: Carol Bellringer, Auditor General; Stuart Newton, Comptroller General

2. The following witnesses appeared before the the Committee and answered questions on the Office of the Auditor General Report: Financial Statement Audit Coverage Plan for Fiscal Years 2016/2017 through 2018/2019

3. Resolved, that the Select Standing Committee on Public Accounts endorse the three recommendations listed on page 5 of the Financial Statement Audit Coverage Plan for Fiscal Years 2016/2017 through 2018/2019, as required by sections 10 and 14 of the Auditor General Act. (Sam Sullivan, MLA)

5. The following witnesses appeared before the Committee and answered questions regarding the Office of the Auditor General Report: An Audit of the Panorama Public Health System

The following electronic version is for informational purposes only.
The printed version remains the official version.

REPORT OF PROCEEDINGS
(Hansard)

SELECT STANDING COMMITTEE ON
PUBLIC ACCOUNTS


CONTENTS
Page
Auditor General Financial Statement Audit Coverage Plan	769
C. Bellringer
R. Jones
Auditor General Report: An Audit of the Panorama Public Health System	775
C. Bellringer
M. Sydor
S. Brown
P. Hamilton
P. Kendall
S. Newton

B. Ralston (Chair): Good morning, Members. We have an agenda before us that’s been agreed to. The first item on the agenda is the consideration of the Office of the Auditor General’s Financial Statement Audit Coverage Plan For Fiscal Years 2016-17 Through 2018-19. Here to present to the committee are the Auditor General and Deputy Auditor General — Carol Bellringer, Auditor General, and Russ Jones, Deputy Auditor General.

Each year we are required, under the Auditor General Act, to table a three-year plan for your approval that shows which government entity financial statement audits we plan to do and which the private sector firms will do. We prepared this plan in accordance with the recommendations in the Auditor General Act. The plan is also designed to meet Canadian generally accepted auditing standards. That allows me to form an overall view on government’s summary financial statements and then sign the audit opinion.

As you may recall from prior-year discussions, the coverage levels do not vary greatly from year to year, so the focus of this discussion is often just on the changes.

We expect that our office will carry out the plan within a budget envelope that’s very similar to last year’s recommendation by the Select Standing Committee on Finance and Government Services, and our overall budget will actually be discussed by that committee this coming Wednesday.

Russ Jones, Deputy Auditor General, leads our financial statement audit team. He has overall responsibility for the audit of the summary financial statements and the components of this plan, so I will now turn it over to Russ for a brief overview of the coverage plan.

R. Jones: Thank you, Carol, and good morning, Members. Hopefully, everybody’s got a copy of the plan in front of them. I’ll try and just highlight the most important parts of the plan for you, and then we’ll take questions at the end.

The annual audit of the summary financial statements is the largest audit performed in the province. It provides assurance on whether or not the financial statements present fairly the financial position and operating results of the province. The opinion on the summary financial statements is the Auditor General’s alone, but in British Columbia, the audit of the government reporting entity is accomplished through the combined work of the Office of the Auditor General — our office — and the private sector auditors.

Section 10(2) of our act requires the Auditor General to audit the ministries and other agencies that are part of the consolidated revenue fund. However, most of the organizations and trust funds that comprise the government reporting entity are audited by the private sector.

Today we’re looking for three approvals from members. The first is the proposed plan, which you will find in appendix A and which goes from pages 18 to 28. The second is for us to continue as the auditor for ten entities which exceed the five-year term, and that’s on page 15 — where that starts. Then the Auditor General is continuing as the direct auditor of one reporting entity, and we need approval for that as well.

This plan, as Carol mentioned, meets the professional requirements for audit coverage under generally accepted auditing standards and will allow the Auditor General to sign the opinion on government’s summary financial statements. Our selection process is risk-based and aligns with assurance standards specific to the audit of group financial statements. These standards require us to be involved in the audit of all significant components of the summary financial statements.

This plan also details the range of levels of involvement that we use to gain knowledge of the organizations and sectors during our audit of the summary financial statements. The extent of our involvement is on three different levels. It is there to recognize our judgment and assessment of risk at both the sector and government organization levels. Essentially, we define the three levels as….

Limited. We really don’t do very, very much at all on any of those organizations — the private sector audits, all of them. We have communication with the various private sector firms, and we take a look at the results of their audits to see if there is anything that might make us change our mind in the following year as to risks that we should be looking at.

One of the largest areas is the oversight area. That’s where private sector firms are appointed the auditor by the organization, and we conduct what we like to call extended procedures to better understand the business and risks of those organizations. That includes taking a look at their working papers after they’re done the audit.

It also includes us going to many of the audit committees of these organizations and also reviewing the plan and the year-end files of the various private sector companies. Included in that we also have monthly calls with many of the firms to discuss issues that they may be coming up with in their audits of the entities that they deal with. So we’re right in there with them, and it’s a very important part of what we do.
[ Page 770 ]

Then we have direct audits, and that’s where we actually do the audits ourselves. We also have a couple of contract ones. We contract with the private sector to do the audit on our behalf, but we sign the financial statement audit opinion. These types of audits provide us with the best in-depth knowledge of the entities.

In determining our involvement, we first look at each organization and do a risk assessment to determine if it’s a significant component. We look at whether it delivers a core service, it has a high level of public interest, it has a high inherent risk with respect to programs it delivers and its complexity and potential future issues or other factors that may come into play. Then we look at each sector to see if we’ve got sufficient coverage. You’ll see that when you take a look at our overall plan.

Finally, we look at all the new organizations that might be developed. Normally, we will do the audit of new organizations for the first three years. This year we don’t anticipate any in 2016-17, but it is possible that one called HSSBC, Health Shared Services B.C., might be set up as a separate Crown. If it is, then we would audit that organization.

The provision in our act is to request approval for extending direct involvement for five years or more to manage the inherent audit risk, where necessary. In this plan, we have to balance the benefits achieved through auditor rotation with the professional standards that require us to maintain appropriate knowledge and experience, as necessary, to fulfil our mandate.

For audits that exceed five years, including ministry audits, we employ senior staff rotation and other safeguards, as required by standards, to make sure that objectivity is maintained.

In the preparation of this plan, we reviewed each appointment exceeding five years and considered if rotation to a private sector firm would be appropriate. We don’t think any rotation of audits where we have been the auditor for more than five years is warranted. Rotation of organizations to private sector firms is either not feasible or will have limited benefits and add an increased cost. The rationale for our appointments you’ll find on pages 14 to 16 of the plan.

Now, turning to the detailed plan. This table is on page 11 of the plan, and it summarizes our planned audit coverage for the next three years for 145 entities covered by the plan for fiscal years 2017 through 2019. This is a rollup of appendix A. The first column shows the type of entity. The second column shows the expected number of entities in each type for the fiscal year ending 2016. The remainder of the table shows our planned coverage by entity, fiscal year and level of involvement.

For example, in 2016-17, we plan to have limited involvement in 16 of the 26 universities, colleges and institutes, have an oversight involvement in eight and directly audit two. That’s how you can read this chart. As you can see from the totals, our levels don’t change significantly over the three years. Also, our highest level of involvement is still in the Crown corporation group — nine oversight and 11 direct audits out of the 41 entities for 2016-17.

This chart is just to sort of show the magnitude of what we are covering off with our oversight and direct audits in terms of government expenditures. We’re covering off approximately 84 percent, if you look at the percentages up there. This is the same as in previous years.

Trends by organization types. School districts. There are no changes in the school districts we audit during the period covered by the plan. We also plan to have an oversight in three school districts each year of the plan, and we continue to rotate our involvement throughout the school districts.

In the post-secondary sector, there are no changes there either. We also plan to have oversight in seven or eight university colleges or institutes each year.

Then moving on to the health authorities and hospital societies. We continue to do a direct audit in one and do oversight of all the other health authorities and one hospital society, Providence Health Care.

Rotation of OAG Crown corps. Our plan is to cease auditing Destination B.C. We’ll have done that by 2018 and will take on the audit of the Royal B.C. Museum. We’ve never actually done that audit.

One change to our previous plan is that we were going to take on the Organized Crime Agency. Having done oversight for the last two years, we decided it wasn’t really something we needed to do.

Audits outside the government reporting entity. We’re finishing our audit of WorkSafe B.C. this year. It’s going to the private sector. We are also finishing the audit of the Langley School District Foundation. The only one that we continue to do is we do the audit of the provincial employees community services fund — a nice little audit for our staff to be involved in.

Consultation. We’re supposed to each year consult with the organizations impacted by the plan where we’ll be taking on oversight or direct audit coverage or ending it. This time we’ve met and done all the consultations that we need to and have been in touch with all of those organizations that will be affected.

Budget implications. As Carol mentioned, we have planned these changes so as not to impact our resource needs. However, when we do go to the Finance and Government Services Committee on Wednesday, we’re looking for a modest 1 percent increase in our budget. Most of that is to help cover off the increased occupancy costs that were related to our relocation.

As we’ve noted in the past, government organizations are required to follow Treasury Board regulations on deferred contributions. While we’re required to report on the summary financial statements in accordance with
[ Page 771 ]
GAAP, that is without regulations. This increases the audit work that we have to do each year in terms of coming up with what the difference is between what government’s recording and what we think should be recorded.

All of these types of things do impact the amount of work that we have to do. There are also a number of new standards coming out over the next two to three years that we will work through with the government. Each one of those, as Stuart knows, takes a little bit of extra work to make sure that the accounting standards are appropriately applied. Those are the types of things that could impact the work we do, but we don’t see it as majorly impacting the resources we’re going to need.

In conclusion, we’ve got the three approvals that we need from the committee today. We’d be happy to answer any questions that you might have on anything in the plan.

L. Throness: Thanks for this. I want to turn to the summary of proposed audit coverage on page 13 of the audit. I was looking at that. In my former life, I worked in Ottawa, and I worked in several federal departments. In the Department of Agriculture, there are 14,000 employees. In the Department of Justice, there are 4,000. Health — about 4,000. Transport — I think 6,000 or 8,000.

By contrast, the Fraser Health Authority alone has 27,000 employees. It is a massive, massive entity. Altogether the health authorities spend $18 billion to $19 billion a year, nearly 40 percent of the provincial budget. I note that you only have one direct audit of health authorities, although school districts, which spend about $2½ billion, have five — five times the number of direct audits.

I guess my question is: have you properly allocated the direct, most scrutinized audits to the places that are of highest spending and complexity?

R. Jones: A good point, Member. With the health authorities, we’ve deliberately made a decision to do one a year for a five-year period. As I said, in the oversight of all of the other health authorities, we attend all the audit committee meetings. We do look at the files of each one of the firms that are doing the audits and feel that we have a pretty good grasp, I think, of all of the issues in the health area.

R. Jones: It does. We’re doing the audit ourselves, so our staff are in there looking at all of the transactions and whatnot.

As I say, we have what I would call very good communication with the firms doing the other health authorities as well. For instance, we do the audit of Vancouver Coastal Health, and a lot of what goes on in the Lower Mainland is coordinated between Coastal and Fraser. We feel that just going to the audit committees and dealing with the firms, looking at their files and understanding all the issues, allows us good coverage.

C. Bellringer: I just wanted to add to the consideration that we also put together a three-year performance audit coverage plan, which we’re not required to do. We have circulated it publicly, so it’s available. But it’s not required to come to this committee. When we do that…. This is only for the financial statement audits that we’re doing, this plan that is before you today.

All of those organizations are given appropriate consideration when we put our performance audit coverage plan together. In there, you’ll actually note quite a lengthy list of organizations within the Ministry of Health that we have identified for broader audits, just to look at particular issues in depth.

L. Throness: I would simply note this as a matter of concern, Chair, and leave it on the table. My questions are clear, I think.

M. Hunt: I have a number of questions, but again, it’s most likely a part of me being new. This is my first time at this. As I read through this report, I got a….

B. Ralston (Chair): You’re only going to be able to work the new thing for so long, Marvin.

M. Hunt: At least for one year, one cycle, right? One year should be allowed, the cycle. It’s the first time I have seen this, so there are a bunch of questions that come up. But you’re absolutely right. I can only work it for so long.

Page 16 of your report. I’ll work through from there. Number 3 is the B.C. Transportation Finance Authority. Your second paragraph says: “This organization’s audit opinions since 2011 have been qualified, and our assessment of the financial reporting risk continues to be high.”

R. Jones: I’ll try and keep this as short as possible. It has to do with government transfers. It’s the accounting issue that is probably the one that we still have the most disagreement with government on at the current time. The B.C. Transportation Financing Authority did not adopt the regulation that government put in place. As a result, we feel that their financial statements are not in accordance with public sector accounting standards, so we qualify their statements, on a yearly basis, for that.
[ Page 772 ]

M. Hunt: You’re not challenging their accuracy. You’re challenging the reporting?

R. Jones: It’s the reporting of government transfers. As you can well imagine, it’s a large amount in Transportation.

M. Hunt: In my short history in the provincial government, I have heard of these disagreements between entities here, organizations.

Number 10 goes on, on page 17, with the Transportation Investment Corporation. The last sentence says: “As the risks in the construction phase and the operation phase are unique, our continued direct involvement is warranted.”

My question. The construction should be phased out now. Is this going to continue, or will that also phase out?

R. Jones: With the Transportation Investment Corporation, now that the construction phase is pretty much finished — there’s still some going on, but not much — it’s the operations phase, which is the tolling of a bridge. It is unique in the province, and we feel it is something that we should continue to be on top of.

R. Jones: It’s not a P3 as such. It is just owned and operated by government and has the tolls on it. Yeah, there is a separate tolling company that runs that part of the operation. It is unique, and given that there are some larger P3-type projects coming on board — maybe the Massey bridge, the possibility of its being tolled as well — we think we need to be on top of this.

M. Hunt: Well, that was going to be my next piece of the question. I realize I’m about to ask a hypothetical here. For example, the Massey Tunnel replacement, Site C. Do we do it on the basis of volume? Do you do it on the basis of complexity? How do we make those decisions?

R. Jones: Public-private partnerships are always complex. Every single one is different and new. We feel that we should be involved in any of those when they’re occurring.

C. Bellringer: Just, again, to refer back to that performance audit coverage plan, both of the examples that the member provided are included in that coverage plan as projects we want to look at in more detail.

M. Hunt: Over to page 19. You’re talking about the administration of the appointment process. Your last sentence in the first column says: “As part of our risk analysis, we also monitor the length of appointment, audit cost, frequency of competitive process across the government reporting entities.”

Looking at this from the higher level, when you talk about audit cost, do you actually sit there and say: “Hey, I think this bidding is a little on the high side here for the work that’s being done”? Is there that kind of a relationship that’s happening here? How does that work?

R. Jones: In most cases…. I’ll use Liquor Distribution Branch as an example. The RFP just went out for auditors for the next five years for the Liquor Distribution Branch. We will sit down with the entity and go over the RFPs that come back in and take a look at the costs, among other things, to assess whether or not who they are picking as the appropriate auditor is a good pick.

M. Hunt: So you would be a part of that process — as sort of the expert, I’d think.

M. Hunt: My last question is just a curious one with appendix A. I was trying to figure out…. You’re shifting from…. You’re low involvement to moderate involvement to high involvement. As we work through these, I’m just looking at boards of education where, for example, you’ll have oversight in the second and third years rather than in the first years for a couple of them. You shift that around, and it often looks like oversight is for two years.

Now, is that your…? I’m going to call it the arbitrariness of you sort of delving in and picking. No, I guess that’s my question: how do you do the pick, and how does that process work for you, doing what I’ll refer to as the arbitrariness of your audits or your oversight audits?

R. Jones: We take a look at a number of different factors when we’re looking at the school districts. As you can see, there are a lot of them across the province.

R. Jones: Yes. So we take a look at ensuring that we have audit coverage sort of demographically. As well, we look at all of the management letters that come in from the firms that are out there and whether or not they’re large firms that are auditing them or small firms — and just assess how the governance is at some of the boards of education as well.

We decided to go in and start auditing Vancouver school district this year for a five-year period. So we risk-rate them and try and just get good coverage for the five that we think we need to do to cover off the risks of this sector.

M. Hunt: Then my final question on that, Mr. Chair, is going to be this. As I look through, you’ve got some where you sort of do this two-year oversight. I’m going to
[ Page 773 ]
assume that your expectation is that this will be a more positive one. We have others where you have a year or two of oversight, and then you go into direct audit, which I would consider to be part of the process of understanding before you actually got into the direct.

Am I correct in that? That’s what you’re doing? And why do you extend it across five years rather than being, again, a shot in to do the audit and then back out?

R. Jones: In some cases, where we do have oversight and then show a direct audit, we may do the oversight for the two years and then decide we don’t need to do a direct audit. This is the way that we find…. It’s hard to describe it, but whenever you’re changing auditors, there’s always a learning curve and an extra cost to the organization. We feel that by doing oversight for a couple of years ahead of time, we get to understand the organization to make that transition much smoother for us coming in to do the audit.

In some cases — we’ve got down here, I think, board of education, North Van — for two years, we’re just doing oversight. That’s to see if there is a need for us to go in and do a direct audit later on.

M. Hunt: Then as the final, Mr. Chair…. I know I sort of said that for the last one. But a supplementary to that: then you really are viewing the three years as a living document in that it is subject to change as you go along, depending on what comes, what you discover.

R. Jones: Yes, Member, it is. It is a living document. As I said, one of the examples was the Organized Crime Agency. We did oversight of that for two years and then decided it didn’t need us to go in and do the direct audit.

L. Reimer: My questions are about pages 16 and 17, additional reporting requirements under the act. You’ve got ten organizations here where your appointment will be longer than five years. In some of these, you have stipulated dates, but for most of them you have not. Is this the total number of organizations, or does that change from year to year? What is the history with respect to the number of organizations that are greater than five years?

R. Jones: At this point in time, these are the ten. The Liquor Distribution Branch and a number of these are large, complex ones that we’ve done for a number of years. All of them are ones where we feel we need to have a presence. As you can imagine…. I’ll use B.C. Hydro as an example. It would be a wonderful one for us to go in and try to audit, but we just don’t have the resources to be able to do it as well as the private sector.

So in order for us to keep up our knowledge of the energy sector, we decided to continue with something like Columbia Power Corporation. It’s a more manageable size for us and has some complexities as well, but it keeps us involved in understanding the energy sector and that sort of thing.

Most of the other ones, once we get to the five-year point, like Destination B.C.…. We’ve decided that we don’t need to do that one any longer. It was a new organization. We did it five years. We’ve decided to allow the private sector to do the audit instead of us.

These ones, right now, I would say you can expect to see in this report for a number of years.

R. Jones: As you can see, there are also a number of transportation ones in here. A lot of that has to do with just the magnitude of transportation within the province, the P3s that are involved and everything like that.

Can the hierarchy, the three levels of invigilation, be adjusted or accelerated with new information that you receive from outside sources or from observers? You see an area that becomes problematic. Does this change the manner in which you do your audits, or is it…? Do you know what I mean?

In other words, some information comes available to you. Do you say: “Okay. We’re going to have to become a little bit more aggressive here”? How does that process work? How do you glean that information?

R. Jones: This is a living document — over the three-year period that’s here. This is our best assessment, at the time that we prepared it, in the current year.

As I mentioned, something like…. Health Shared Services B.C. is thinking of becoming a separate Crown corporation. We see that as an organization that we would, then, want to do. We wouldn’t have limited involvement in it.

Most of the limited ones are very small and really are not ones that we need to be doing. But yes, if information came to us, we would consider whether or not we should be doing a financial audit of them or maybe a performance audit.

C. Bellringer: During the year, we wouldn’t switch, though, from one level of involvement to another. Once the audit has been…. If it has been contracted out to a firm or we’re doing it, we would continue through the year on that basis. We might choose to do additional work in addition to that. We don’t move them around during the year once the contracts are in place.
[ Page 774 ]

I think I may have asked this before. I don’t think you do surprise audits. Do you? This is all documented. If I’m head of a particular Crown corporation, I know when you’re coming and what you’re going to do.

In the private sector, of course, as you know, there’s some element of surprise from time to time with audits. What is your comment on that?

C. Bellringer: For the purposes of conducting a financial statement audit, where we’re putting an opinion on a set of financial statements, there’s…. I mean, there may be an element of surprise throughout the audit, as sometimes there is. But no, they’re done on a very organized, scheduled basis.

That does not preclude us from choosing to go in on another matter, if and when something comes to our attention. But it’s not in the context of the financial statement coverage.

S. Gibson: My only other question is: with everything now being available on line — if you want to access a Crown corporation’s financial statements and they give you authorization to do that — is this potentially going to make your audits more economical, more efficient for the taxpayers? You can access a lot of information offsite without physically going to visit the particular organization.

I realize you still have to go, but now, increasingly, everything is available on line. You have the authority to do that. It seems to me that might be a way to make your audits a bit more efficient for taxpayers. That’s my question.

R. Jones: That’s very true. One of the things we continually look to improve is our audit methodology, as we’re undertaking audits. One of the — I shouldn’t say new ways of taking a look at audit evidence, but one of the things that’s becoming much more apparent is what they call data analytics and taking a look at trends and comparisons.

Instead of going out and doing hundreds of samples of invoices and whatnot, if the controls are appropriate and in place through the systems that are out there, you can sometimes use data analytics, which certainly helps reduce our amount of hours and on-site visits. But nothing can take the place, I don’t think, of getting there and talking to the organizations. I’m not a fan of audit by telephone or e-mail. I’m still old school in that regard.

C. Bellringer: I’m going to just echo what Russ said. The days of being able to look someone in the eye and ask the question…. They can’t be replaced by e-mail. But we already do a lot of our file work electronically. All of our working papers are retained electronically. In the old days, we used to use something they called 14 column. It’s till the joke around the office, that we don’t need any more 14 column. But when the lights go out, I always wonder if a paper and pen could still work.

Yes, we already do a lot of the things that I believe the member’s referring to. What it really changes is the effectiveness and the ability to actually look 100 percent at a particular file when we choose to do so.

V. Huntington: I don’t think it had clicked with me that the office doesn’t review its performance audits with us. For some reason, I hadn’t realized that after all these years.

Is that an issue of independence, in your mind? Why do we not see the table of performance audits?

C. Bellringer: It’s not an element that was designed in the Auditor General Act, so there’s no actual requirement. I’m sure it does have an element of independence to it, but we actually would welcome a discussion, at the committee or individual members, because we do make a lot of judgment calls when we put the plan together.

The plan we just issued has approximately 60 audits listed in it. There were 140 that we chose not to do because we just don’t have the resources to do them. Then there’s an infinite number beyond that.

Input from the members would be most welcome, even though it’s not a requirement.

V. Huntington: Would you be prepared to provide us with that table of audits? It would be interesting to see.

C. Bellringer: The report is public, and we’ll send the member a link to the website. It’s been circulated.

Okay. The next proposed item, then, is a motion. I think it’s been circulated to members, and I expect that the Deputy Chair is going to move it.

There are three separate motions there. They’re set out on page 5 of the report that you have before you.

S. Sullivan (Deputy Chair): I move that the Public Accounts Committee endorse the three recommendations listed on page 5 of the Financial Statement Audit Coverage Plan For The Fiscal Years 2016-2017 Through 2018-2019, as required by sections 10 and 14 of the Auditor General Act.

B. Ralston (Chair): We’ll take a brief recess to set up for the next paper.
[ Page 775 ]

B. Ralston (Chair): The next item on the agenda is we’re going to consider the report of the Auditor General entitled An Audit of the Panorama Public Health IT System. That’s a report that was issued in August of 2015.

I see representatives of the Office of the Auditor General there. I’ll ask Carol Bellringer, the Auditor General, to introduce the staff who conducted the audit on behalf of the office. Then we’ll hear from the Ministry of Health — Stephen Brown, the deputy minister, and Dr. Perry Kendall, who’s the provincial health officer.

C. Bellringer: Panorama had been on our office’s radar for a number of years. In early 2011, the office surveyed a number of B.C.’s e-health IT systems to understand project progress and issues. One of those systems was Panorama. At that time, the office briefed the Ministry of Health on concerns that we had about the project.

Because the implementation was just getting underway, we decided not to audit Panorama at that time. Instead, we gave the ministry another three years to work through the issues.

Large, complex IT projects like Panorama are often high risk and face challenges around system quality, budget and timelines. We focused our audit on those risks, and we did find problems in each area. Our report describes what we found in those areas, but we also included our assessment of why we believe this happened and what went wrong. We hope that this will assist the Legislature and, more specifically, the ministry in watching for and avoiding similar problems, going forward.

With me today, I’ll introduce Morris Sydor — who’s no stranger to this committee — assistant Auditor General, and Pam Hamilton, director of IT audit, who will provide you with a brief overview of the report. We have quite a few staff members in attendance.

M. Sydor: Thank you, Mr. Chair and committee members. As Carol indicated, we do have a number of staff members here. We’ve also got some guests that are with our office for some time. Two members of the audit team are also in the gallery. It’s Sarah Riddell, who was on the audit, as well as Adam Giles.

With us, as well, this year…. As you know, in the past, we’ve worked with the CCAF to have fellows from Third World countries join our office for about nine or ten months. They work on audit teams and carry out particular audits. As well, they develop a project to take back with them so that they can carry out an audit that they’ve helped develop here back in their home country.

We’ve agreed with CCAF this year to take two fellows. This year we have Mary Arthur, from Ghana, and Frank Mwalupale, from Tanzania. They’re both with us. They joined us in September, and they’ll be with us till about the end of April working on audits and working on their project, taking the opportunity to get some of the training that’s provided to our staff.

Just last week they, fortunately, had the opportunity to run up to Whistler to look at a water management project. I saw some photos of them this morning enjoying themselves quite immensely. I think their visit to British Columbia is getting off to a good start.

M. Sydor: What I’ll do this morning, committee members, is quickly give you a presentation, an overview, of our audit of the Panorama public health IT system. The story of Panorama began after the SARS epidemic hit Canada in early 2003. In May 2003, a report commissioned by the federal Minister of Health recommended that the country invest in a seamless public health system that will allow public health professionals to coordinate activities in the planned infrastructure.

In response, the federal government mandated Canada Health Infoway to work with provincial, territorial and federal governments to develop the system now known as Panorama. The Ministry of Health co-led and managed the project to build the national Panorama system on behalf of all provinces and territories. In B.C., Panorama supports public health functions such as family health, infant and child immunizations, vaccine inventory management, communicable disease management and outbreak management.

Panorama was a difficult and complex undertaking, involving numerous jurisdictions of varying sizes, with different public health processes and two languages. We expected Panorama to have the required functionality, stability and usability; to be built and implemented on budget; and to be built and implemented on time. The Panorama system did not meet our expectations.

It does not have the required functionality, level of stability and usability necessary to achieve all of the stated benefits of the system. It was not built or implemented on budget, and it was not built or implemented on time. Panorama was to be a national system, but at present, only five other provinces or territories are moving forward with partial implementation.

On functionality and usability, we found that key functional components, such as national outbreak capabil-
[ Page 776 ]
ities and national alerts, were de-scoped from the system. Others, such as the ability to identify patients who are who are due for immunizations….

B. Ralston (Chair): Mr. Sydor, I wonder if you could explain the term “de-scoped.” That is sort of a jargon term, and some people may not be familiar with it.

M. Sydor: That was something that was designed to be part of the original system and then, on reviewing it, either for cost or other reasons, was pulled out or contracted in terms of what the functionality was initially intended to undertake.

M. Sydor: Other functions, such as the ability to identify patients who are due for immunizations, do not work. Users reported that Panorama was neither intuitive nor user-friendly and used clinically incorrect or counterintuitive terms, and the system design has extended appointment times.

Panorama has been plagued with defects from the start. B.C. was responsible for accepting Panorama for IBM — that is, verifying that it met all contractual terms. At the time of acceptance, Panorama contained almost 1,200 defects. After its deployment in 2011, over 11,000 more defects were discovered. As of December 2014, approximately 2,500 defects continue to affect users.

B.C.’s version of Panorama is prone to errors, and slow performance and unexpected system outages mean the system cannot always provide clinicians with complete and accurate patient information when they need it.

The national project cost almost twice as much as expected: $66 million versus a budget of approximately $38 million. Most of the cost was borne by Canada Health Infoway, but individual provinces covered some of the cost overruns. B.C.’s implementation of Panorama was 420 percent over budget, five years late and did not meet all expectations. So far, Panorama has cost the province almost $115 million.

Now, Canada Health Infoway required Panorama to be built from existing commercial, off-the-shelf — or COTS — components, together with some custom development. IBM proposed knitting together a number of separate COTS products into a single public health IT system. In early 2007, IBM’s plan was determined to be unrealistic.

Now, at this point, we expected the ministry to evaluate options, including contract termination or alternative solutions. They did not and, instead, carried on with IBM, even though they identified a number of significant risks with IBM’s going-forward plan.

In continuing with IBM, the ministry agreed to a number of change orders that de-scoped significant functionality, lengthening the final delivery date by almost three years, downgraded the severity level for many defects and capped the number of defects IBM had to fix each year. These changes transferred the financial risk associated with delivering a quality program from IBM to the province.

Acceptance testing is critical to ensuring that a system is working as it should and meets user needs. The ministry relied heavily on IBM for the testing. IBM provided the test scenarios, and IBM carried out a substantial amount of the acceptance testing. Now, this is concerning, because IBM has an interest in having the system accepted quickly. After acceptance, the ministry and other participating jurisdictions become financially responsible for fixing deficiencies.

We found that Panorama was accepted prematurely. The ministry, IBM and other jurisdictions identified thousands of defects after the ministry accepted the system, and it was unusable in 2009 when IBM began training health authority staff to deploy it in B.C.

Finally, decision-making and leadership emerged as a concern during our audit work. There were reports that ministry staff and executives sanitized the information they provided to senior decision-makers. We share this concern, as we noted instances where decision-makers did not appear to have a good understanding of the Panorama project and system issues.

The ministry’s command-and-control leadership style was also somewhat troubling. Health authorities were instructed to implement the system despite concerns, and those who raised concerns were often dismissed as troublemakers or warned to stay quiet.

We had several recommendations for the Ministry of Health. First, given the ongoing challenges with the system, we recommended an independent review to identify the most cost-effective integrated approach to meet the current and future needs of public health in B.C.

Next, we had identified numerous deficiencies with the ministry’s management of the Panorama project and contract. We recommended that the ministry review its project management and contract management practices to ensure future IT projects are managed in accordance with good practice.

Finally, we recommended that the ministry review its current leadership practices and develop a collaborative leadership strategy for future IT projects.

B. Ralston (Chair): Thank you. We’ll reserve questions until we hear from the ministry.

I’d ask Mr. Brown, who’s the deputy minister, and Dr. Perry Kendall, the provincial health officer, to come forward. We’ll just take a moment while we switch over.

S. Brown: I’d just like to recognize Dr. Perry Kendall, the provincial health officer, who’s joined me here this
[ Page 777 ]
morning, and also to thank the Auditor for the report.

There’s quite a bit of duplication in this deck. We didn’t actually compare notes beforehand, so I will zip through some of the deck which is covering the same ground as Morris just walked through.

I think the key thing I wanted to start with was how this is one of the most complex audits, in my own experience, that I’ve had to deal with, given that it’s covering a ten-year period. We have got a series of decision points that were made over the ten years, with quite a significant change in personnel over that time, and it’s in the context of a national endeavour.

I think, both for the audit team and certainly speaking for myself, it’s been quite a complex audit to actually work through, to try to discern what may be historical and wouldn’t occur again and what was incidental and specific to what was a very complex attempt to do a pan-Canadian build of a system.

Also, then, and probably most importantly, in terms of what I’ll talk to this morning, is what we need to learn as a ministry — and perhaps broader than the ministry — into how you actually manage IMIT projects.

I’ll try to highlight some of those elements as I walk through this shot deck, and then, obviously, we can canvass the questions on the very specifics of what occurred.

I won’t cover this. This is the same ground that Morris has covered, and I think Morris did a good job of covering the history, which was the significant pressure that Canada felt following the SARS outbreak in terms of its ability to respond in a coordinated way. We saw the subsequent cost of that in terms of significant economic as well as human costs of the outbreak. This was set up as a pan-Canadian approach to actually build something that would go the breadth of Canada. Morris covered the ground in terms of some the history of that approach in terms of what was then added in.

I think the key here, as well, which perhaps Morris also didn’t quite highlight…. There was the response, but also there was a strong push for the need for a greater level of investment into the IMIT infrastructure for public health, which would then be of value to the multiple jurisdictions.

As Morris noted, B.C. became the co-lead on this. To my understanding, I think that was built around the experience that B.C. had had as being one of the first provinces to move ahead with a public health information system — that’s the iPHIS that’s in the Auditor’s report — which seemed to lead to B.C. being seen as in a reasonable position to co-lead.

I think the piece I’d like to emphasize is that, while co-leading, this was a very complex national committee that was made up of public health professionals as well as IT professionals from across the country.

The process, in terms of moving from what was an original ask in 2005, I think it was, based upon the then thinking of the Canada Health Infoway — that is, is the value, and there is real value, in trying to use COTS products — to actually experiencing, within a couple of years of trying to do that, that the adaptation of those COTS projects, or stitching them together into a system, created a whole range of challenges.

One of the most fundamental pieces of this was a decision that was made to actually move from the COTS to the custom-built. I think the custom-built approach has a whole series of consequences that then roll out. We can, if you want to, actually go into that detail, through the balance of the project.

The other piece I’ll highlight complements, really, what Morris was saying but was highlighted to me. Morris has set out the timelines, which are absolutely accurate. I think a piece that was highlighted to me, as I’ve been trying to understand what may have been some of the pressures, is the breakout of the H1N1, which was just prior to the Olympics. I think it was 2009.

It had a major impact of further complicating what was already a challenging process — which was the engagement of the very health professionals that you would want to be using for developing a system. They were also having to pay attention to what was a significant new threat in terms of the H1N1, which had further impacts in the development of the process. But the timelines, as Morris set out — I won’t cover those again.

There are a number of benefits, which we’ll obviously get into a discussion about, that have occurred through this process. We have been very clear in receiving the Auditor’s report and looking at the ten-year period. We absolutely acknowledge — I’m sure we’ll go through in some detail — the significant challenges that occurred with this project at multiple levels, both at the national build and then, subsequently, in terms of how we moved forward as a province.

There have also been significant gains made through this process. Some of those are listed here. Behind these has been a national dialogue on public health beginning to shape up standards in terms of how to actually label and determine and share information, which have been benefits that have come from this.

As Morris correctly says through the audit, not all of the aspirations that were set out early on in this project have been achieved. A fundamental issue we had to look at in receiving the report is: where is the viability of Panorama currently, and what would we do, going forward?

I will move to that in terms of the recommendations. The first recommendation the Auditor brought forward we only partially accepted, for this reason. Currently, we believe that in Panorama there is a solid platform. Whether it’s been painful to get to that platform or not, the platform is now in place for the public health system.

The key determination we have to make going forward, which we canvassed with the Auditor, was: was there an
[ Page 778 ]
alternate, and would an alternate be desirable at this point? Our own assessment of the environment, which was not part of the mandate of the audit, is that at this point there isn’t an alternate. In fact, we think in Panorama we’ve actually got a pretty good platform that has incrementally been implemented and is being implemented.

With it’s interoperability, as things have evolved over the last ten years in terms of the IMIT landscape, other elements can and should and will be added on to that platform, going forward. But the platform, in and of itself, is a solid platform at this point, notwithstanding some of the challenges that were found by the Auditor in looking at the evolution of the platform over the last several years.

Now, going forward — the plan forward. What I’ve asked for is that we develop a go-forward plan which will be three to five years in length. It will start with an assessment of the current modules, the six core and the seventh, set the platform in terms of where we’re at, in terms of the current deployment and what the next elements of deployment are — and that we have that specified and built out in a detailed plan going forward.

An independent consultant will be contracted to actually help develop that plan. That will actually give a very clear and public go-forward plan against which we internally can hold ourselves accountable while also a public statement of where we’re going. That will take some time to develop. We’ve set a time frame of June 2016 to have that plan fully completed. There will be elements of it already underway, but in terms of a fully developed plan, it will be June before that is completed.

The other piece, then, that we build out on in terms of one of the issues was the issue of the end user. When this project started, in terms of the actual technology and the way it was evolved, you had a certain technology. One of the biggest issues that has created concerns for staff as things have evolved is the whole scrolling piece.

There are a number of new approaches, and one of them is being applied as I speak, I think, to the immunization module, which will have a much more screen-based approach that is more contemporary and up to date. Those will be incrementally used. What we will be doing going forward is actually evaluating with end users in terms of the satisfaction. Direct feedback comes into the ministry in terms of what progress we are making in that area.

B. Ralston (Chair): Mr. Brown, I’m wondering if you could just explain the scrolling issue. Sometimes some of these IT terms are not apparent to some of the more casual readers that may be interested in this debate. So perhaps you could just explain what you mean by that and contrast it with the screen-based approach that you referred to.

S. Brown: The screen-based approach would be what I’m doing here. I press a switch, and it moves to another screen. The scrolling is literally using a mouse to scroll down what is a long list of elements going across multiple screens. So you don’t have a total picture in front of you. You’re having to move up and down along the screen in terms of understanding the elements.

That has been an issue of frustration for some users, although I understand that people have gotten used to it. They’ve gotten used to it, and they know where they are with the scrolling. But that’s something that would be modified going forward.

The other piece — which, then, does line up with what the Auditor recommended to us — is we will now conduct a thorough annual scan of other modules that are evolving or developed, available in this domain, that may be, through the interoperability capabilities, linked into the Panorama platform. That we will do annually and report out on and keep as a matter of public record that we’re actually doing that.

Now, because things have developed and this is looking back over a significant…. There were a number of actions already underway in the ministry that didn’t relate just to Panorama but were actual changes and evolutions that were taking place. This screen here represents some of the actions that were already underway in the ministry, and they link directly to the recommendations that the Auditor…. One of the things that Carol put at the very beginning of her report was just the sheer complexity of some of these projects and the need for a high level of skill set in terms of project management — so clearly a fundamental issue in terms of what we do in that area.

Steps were underway to actually strengthen the project management capability in terms of a refresh of the project management office. As well, there was a consolidation of the various pieces of work that were going on across the ministry into a single strategic business management unit. All of the skill sets are pulled together, and then we can pay attention to actually developing and strengthening the skill sets of that project management branch as well as the strategic management branch as they apply to the projects.

The team had already started a self-audit, which we have allowed to go ahead. I understand that that has just been completed.

That hasn’t yet come forward to me, but that has now been completed in terms of an internal audit. Also, there was an internal self-assessment with respect to the contract management. So those are pieces of work that were underway.

With the contract management piece, I think a fundamental issue that the Auditor is raising is not just about contract management in the sense of procurement and general contract management but, in fact, the capacity of the ministry to manage very complex contracts and interface effectively with the vendors.
[ Page 779 ]

Beyond what has already been done in the ministry, which is looking at just the straightforward contract management, I’ve also asked for a more fulsome look at how we manage contracts of this complexity and make sure we’re effective in dealing with the vendors, which will be completed over the coming months.

I think a piece that struck me from the audit was…. That’s our internal, but there was the point that the Auditor made in terms of expertise and assuring ourselves: do we have the kind of expertise that is required, and what may be some of the solutions?

In addition to this internal work, what I’ve asked for is that we engage with an external expertise to actually look at the work that the ministry’s been doing over the last year or so, in terms of beginning to change its practices in a number of areas, and bring to the senior team — and through me to the minister — a more detailed review of the actions that we’re undertaking, with some advice on how we can further strengthen project management, contract management as well as the governance functions of these kinds of projects.

That will be undertaken. I understand we’re just about to go out to RFP on that. While we’ve got September 2016, I envisage that that RFP will be in place in the next couple of months. Then we’ll begin to get some value and feedback as we actually take a second look at the action we’re taking to get that independent perspective.

On an immediate basis, another piece that struck us from the audit was the issue of project assurance.

Notwithstanding the sheer complexity of this…. We’re not engaged in a pan-Canadian project, at the moment, of this magnitude. But even our internal other projects that we have, just at a provincial basis…. You actually are subject to people leaving and changing.

We’re looking now at actually having, as a standard practice, that we have an independent project assurance function for any large, complex projects that we have underway that will actually then translate back to a senior level — which was one of the concerns that the Auditor raised, about whether we are comfortable with where the project is at — so we’re getting direct feedback in on projects. That has been implemented and will become an ongoing function as we move forward.

With respect to the issue about the style of leadership with respect to this project, I’ll first emphasize to you that we’re working very hard at collaborating. But I also want to balance it, to also emphasize the challenge in a complex sector like health, in terms of moving forward agendas.

There are multiple interests and perspectives. The complexity of health, in terms of what would be described as clinical autonomy, where individuals are actually doing their own practice, linked to a professional standard, and the diversity of opinions about how best to do work, leads to quite a diverse range of perspectives.

In this context, it was made even more complex because not only were you trying to do something provincially; you were actually trying to work collaboratively, at a pan-Canadian level, with other jurisdictions — all of which implied a certain range of compromise, negotiation, a give-and-take, in terms of how the project moved forward, while trying to hold together a pan-Canadian. As the Auditor points out, the pan-Canadian didn’t hold together in a number of aspects.

We also have the challenge, at a provincial level, of how much autonomy to give. This is where we should have a single system.

While it is big and complex, coming from Manchester, I do remind people regularly, who talk to me, that the 4.5 million people in B.C. would be about one-quarter of Manchester. Somehow Manchester, England manages to have a single system for the whole of Manchester.

We actually argue a lot about geography, complexity. The push to have single systems is hard. It’s challenging on a given day. But I will underscore that we have and continue to make changes in terms of the collaboration.

A key issue that the Auditor raised with us is: how do you allow active debate and disagreement and make sure that surfaces so that you understand where there’s disagreement between individuals? Certainly, the Auditor found, in some of the interviews they did, that some staff felt that that was not the case. Some of that disagreement was not getting up to the senior level. That’s something that we’ve looked at.

We’ve just done a refresh of what are called the standing committees of leadership council, which is made up of senior executives from the ministry as well as the CEOs, and we’re looking actively to make sure that those kinds of issues do surface. Notwithstanding, though, I will say that even when they surface, there is a need, sometimes, to push ahead, to get to solutions that are actually solutions that are systemwide solutions.

I think that pretty well covers the areas that I wanted to cover, and then we can open it up to questions. I think the point I would make, finishing, is that the assessment we made is that Panorama, notwithstanding the challenges — there were many, which we can now canvass — there is a platform in place. Much of that platform — even while the Auditor has been doing the audit, and subsequent to the audit — is incrementally improving and working well in many areas. There are still some challenges to be worked through, but we actually do have a good platform, and that platform can evolve and develop over the coming period of time.

B. Ralston (Chair): Thanks. I’ll take a list of people who may want to ask questions.

S. Robinson: Thank you for the presentation. I do have a long list of questions, but I’ll keep to just a couple. Then if we have a chance, Mr. Chair….
[ Page 780 ]

S. Robinson: I appreciate that this was quite the challenging project, given the scope and the undertaking. You kept saying, in your presentation: “Looking forward, looking forward.” But I appreciate, and I hope that you appreciate, that part of this is about looking backwards to identify where things went wrong.

I’d really appreciate hearing how things happened and how things evolved, because a lot of taxpayer dollars were spent on a project that didn’t deliver on time and didn’t deliver what it was supposed to deliver. We have an obligation, I believe, to learn from that. So we’ll certainly be asking questions about how things went sideways.

On page 26 of the Auditor’s report is “Key Findings.” I just want to ask some questions about the deficiencies. That’s where I’m going to start my questions. There were, from what I understand — and I’m not an IT expert, by any means — four different levels of deficiencies. The number of allowable defects for the original terms of the contract were listed in the first column. Then the terms were amended, and the number of deficiencies were increased somewhat. Then the number of defects at final acceptance were quite dramatic, compared to what the amended terms were.

I would really appreciate hearing how that became acceptable. Clearly, someone signed off. If you have some response for that, that would be really helpful.

S. Brown: A couple of things. I do need to be clear, from my understanding. Though we agree with the Auditor that, at the time of acceptance, the acceptance shouldn’t have, perhaps, gone ahead at that time…. In fact, there was some effort at the time — I understand, historically — to try to get an extension. There was some complexity in terms of the context, in terms of the funding from Canada Health Infoway, whether that was going to be extended, and so on.

There’s a number of complexities. I don’t know the details. But we are not in disagreement with the Auditor, looking back and saying that at the time of acceptance — whether the acceptance should have gone at that point, with all of the defects that were in the product — that was suboptimal.

I also would say that it has been explained to me…. Moving from a national COTS product, as it would have evolved, what you got from the national, once it evolved from there into customization, is you get a core, kind of, template approach, which actually then needs adapting and adopting to the individual. The actual testing-and-acceptance phase was then further complicated by the very fact that that was a shift in change — the whole acceptance period.

What I would also say, though, is that in terms of the information that I got…. You look at the defects compared to what was agreed, which is absolutely appropriate. That’s what the be Auditor chose to do. The information that I was given was that the numbers — in terms of whether they were fully agreed to — against what was negotiated originally…. The actual original negotiation for the number of defects that would be accepted going into production was exceptionally low, extraordinarily low, in terms of not really being practical — period.

Once you move away from a COTS project to a customization, the number of defects you would expect to see going into practice would be significantly more. While it wasn’t appropriate because it wasn’t part of the Auditor’s way they defined the project, the advice that I got was that for this kind of a project…. For every thousand lines of code, you would expect somewhere between 15 and 50 defects that you would then discover.

In this particular project, I understand that, in terms of total code, there’s something near 33 million lines of code in it. For a project of this size and magnitude, where it’s customized, you would actually expect to find somewhere in the area of 20-odd thousand plus — and it could be much higher than that — defects going into where you first bring it into practice, and then you quickly begin to move through those to remove those defects.

My understanding is that, in terms of the acceptance, yes, there could have been…. That’s a lesson to be learned, in terms of the acceptance. In terms of the amount of defects that were then subsequently found, that would not be out of line with normal expectations in terms of this kind of a project.

B. Ralston (Chair): I’m just wondering, before you go further, whether the Auditor General, the audit team, would like to respond to that point of view.

C. Bellringer: We acknowledge that moving from an off-the-shelf solution to a customization design would indeed shift what you would expect in terms of the number of defects. That was actually one of the considerations why we were surprised to see that that shift took place through change orders rather than a restart, given that it was a significant shift to move from off-the-shelf to customization. We did not see documentation of an analysis that took place as to why that choice was made.

Having said that, through the change orders, new arrangements were put in place that were agreed to, and there was a definition within that, of the now number of defects that were permitted. It was still significantly above that. What I’m hearing from the deputy minister is that that realignment was still insufficient, and that could very well be the case, but we did measure against that.

P. Hamilton: The way we analyzed it is: we looked at what was accepted in 2008 — that was after the user-acceptance testing — and at that point forward, the min-
[ Page 781 ]
istry is responsible for fixing the defects. In 2008, when it was accepted, the ministry records say that it was the acceptable amount of defects. Between 2009 and 2010, we analyzed the IBM defect log and saw that there were over 6,000 defects between that time. During that time, IBM fixed just over 4,800 defects at a cost of $21 million, and then in 2010 there were still 1,200 leftover defects.

B. Ralston (Chair): The cost was borne by the province and not by IBM. Is that correct?

Now, I’m not, like I said, an IT expert, and I’m also not a deck-building expert. I had to get a new deck this summer, and there were defects. I could see them. My husband, who has a little bit more expertise than I do, saw them. The people who redid our deck wanted to get paid, and I said: “No, not until every last little defect was fixed.” That was the contract.

So I’m puzzled by how it is that we signed off on a contract that people were done. They were the ones now…. Just like with building my deck, the owner said, “It’s all finished,” and so I did the inspection. I didn’t leave it up to them to do the inspection. I mean, they did their own inspection, but I certainly found five or six things that they needed to still fix, and I didn’t give them the last cheque until it was fixed as it was supposed to be.

I’m trying to wrap my head around how it is that IBM was seen as having lived up to their contract when, in fact, that doesn’t seem to have been the case. It actually cost taxpayers more money.

S. Brown: I think part of the process, as I understand it, is that with the shift from the COTS to the custom build, there was a negotiation which took place. Whether the negotiation…. It would be a matter of perspective as to whether that was effective, from different people’s perspectives. But there was a negotiation of the risk profile — what the risk would be to the provinces and territories and what the risk was to the builder. There was a series of adjustments made to the contract based on that new context. That actually was then used on the go-forward basis.

S. Robinson: I’ve two more quick questions. It sounds like we got taken by IBM. They got the better end of the deal. Was anyone fired as a result of either getting a bad deal at the beginning in terms of the contract, or at the end, signing off when it really didn’t meet the contract terms?

S. Brown: I would say that I haven’t had it expressed to me that IBM got the better part of the deal. In terms of the profile of puts-and-takes, that took place in terms of how the agreement was restructured. That would be a judgment call that you’d be making, but it would not be that it’s been presented to me as this was somehow giving away to IBM. It was seen as a negotiation that actually reprofiled the respective risks of the various parties, and that was agreed to by the individuals there. I don’t know the details of what went on in those negotiations, but it hasn’t been presented to me as you just framed it.

S. Robinson: Well, the way it reads at the end of the day is that we had to pay a lot more money for a product that was, in the end, deficient. In negotiating with my deck, we negotiated a price. It was fair. At the end of the day, I needed to get my deck done the way I expected it to get done, and that was a fair deal, because they said they could deliver that. If IBM said they could deliver this and then they didn’t, to me that would suggest that taxpayers got taken in.

I have one last question about where the extra money came from, in terms of what budget, in order to pay for all these added pieces. If you could perhaps tell us where that money came from, that would be really helpful.

S. Brown: The budget overall comes in part from the ministry budget in terms of the go-forward development of this, which would come from the division in terms of our IMIT division. And that would be replicated proportionally across the country for those who actually were involved in the development.

S. Robinson: Is that money that would have been spent on direct health care that has been moved now into an IT budget?

S. Brown: It would be part of…. I would have to look at the details historically of what were the adjustments made. It could well be that those were adjustments that were made against other elements of IMIT projects or other…. I would have to look at the detail of that. I haven’t gone back into the history to look at what adjustments were made at that time.

B. Ralston (Chair): Did the Auditor General have any comment on the question about, I suppose, the wisdom of the deal or the fairness of the deal in relation to contractual dealings with one of the globally leading IT firms?

C. Bellringer: We laid the facts out in the report. We didn’t make that definitive judgment call. I’d say that is one area where the ministry and our office are in disagreement.
[ Page 782 ]

D. Eby: In the report, it’s a project that’s 430 percent over budget. It takes a decade to deliver. Even then, the scheduling function is just one example of literally thousands of problems. The scheduling function still doesn’t work. They had to get the scheduling function from another provider.

Yet I read the Ministry of Health’s response to this — astounding: “Based on the amended contract, IBM delivered the national Panorama system on time.” That’s a quote from their response. Another one. The ministry has, quite frankly, the gall to say in response to the Auditor General’s recommendation that they have a review of this process that was such a disaster: “The ministry does not support this recommendation in its entirety.” They don’t feel that a review is needed.

I wonder whether Mr. Brown can explain, first of all, how the Ministry of Health could publish that this broken system was delivered “on time” when it was never actually delivered, and then secondly, how the ministry could say to the Auditor General: “We don’t think we need to do a review here. We don’t accept that recommendation.” It is astounding to me, in reading this report, that that is the reaction of the Ministry of Health.

S. Brown: In terms of the renegotiation that took place of going from a COTS to a custom build and the deliverables that were then negotiated, the scope of what would be in there, the judgment of the teams involved in this was that then IBM did meet those renegotiated deliverables. The issue of the profile, then, in terms of the defects and the subsequent work that has been done, that has been customized, is an issue of judgment that you could make.

I would say that in terms of looking at this myself, I’ve had to kind of…. I tried to disaggregate the number — because the number, as I saw the report as well, was quite an astounding number in terms of the size, the way it’s presented — to the best of my ability. I look at that number, and I have deconstructed it. There was the provincial capital build, which did go over budget from $27 million, I think, to $30-odd-million. Then the Auditor points out — correctly; we don’t disagree — that there’s another $86 million that needs to be accounted for, which is an astoundingly big number.

When I look at that number, I look at what is called the ongoing operating and maintenance agreement that is in place and will be in place, and I think the Auditor put that number at about $14 million. That is in place, where we make a contribution, and other jurisdictions make a contribution across the country. Our contribution is $14 million to that, which has got a number of elements. During the period that we’re actually reviewing there, which I think is five years, that would be a substantive part of that $86 million. That would be the ongoing maintenance and operating arrangements with the vendor.

Now, in addition then, you’ve still got a significant $25 million, $30 million of money that’s been spent. The explanation I have been given and looked through is that as the project moved forward with further customization — building interfaces to other parts of our system as well as dealing with some of the defects and errors — that that is the net build.

I look at that and say: “Is it significant?” Yes, it’s significant. I look at disaggregated and say: “It’s a real number.” But the components of it I look at, and in the context of a custom build and gradual improvement on a budget that is close to whatever now — $520 million — our investment into the IMIT portion of this in terms of continuing to work on it is around about 2 to 3 percent. So it looks, to me, reasonable that we would continue in that direction.

Now, the piece in terms of the review was, if I understood correctly, that it wasn’t a review writ large. It was a review about the other alternates that we should be considering for Panorama at this time.

In terms of our knowledge base of the environment, we’ve taken that on board, and we will do a review to see what other programs are actually available to the program that we can actually add to — including the example you give, which was the scheduling piece. It’s a subfunction of one of the modules, and there it was seen as more cost-effective to actually use and link in another provider, which is what we did.

I don’t see that as necessarily a failure but, actually, as a sensible, pragmatic approach as things have evolved and products become available. How we link them into the overall project, I think, is kind of a sensible way to go.

C. Bellringer: I just wanted to add that when I saw the action plan and the deputy minister’s presentation at the start, I was pleased to see the response on this particular recommendation. I actually felt that — without getting into the detail of it, which we haven’t had a look at — it was a response that would address the recommendation that we made. In fact, the introduction of using….

We thought that the independent look at this was important. There was definitely, when we spoke to people within the ministry, an indication that this was the answer and the only answer. We did feel that that needed to be broadened and that there needed to be consideration of some of the other views that was greater than what we were hearing.

I did hear that in the action plan. We didn’t hear it at the time that we were finalizing this report and in the response that was included in the report, so that has been a shift in a positive way.

S. Brown: Could I just add to that comment? Sorry, I think it’s an important comment.

Carol and I spent quite a bit of time discussing this audit. As you were raising, Member, this is very complex, and there’s a range of serious issues that were raised by
[ Page 783 ]
this experience of ten years. We are really trying to do due diligence of working our way through, understanding and going down into more detail, because of the history of this, to see: what do we need to pull from it that we can actually learn from?

I would also observe that I think there was a massive learning at a pan-Canadian level of just how difficult it is given different political cycles, different structures, different practices across the jurisdiction. Perry and I were discussing the U.K. compared to Canada. While I personally believe that the actual aspiration at the beginning of this project, as I read it, was the correct aspiration — which is to build a solid national program — the actual challenges in the context of Canada are not insignificant. I think that’s been a big learning, and I can only imagine that that would not be repeated.

I think where we’ve got to on a number of areas…. One area would be, say, outbreak management, which is a significant piece that the Auditor identified. We haven’t got a national outbreak. What we have got are provincial and territorial outbreak systems, which allow exchange of information.

Is that significantly better than it was before? The answer is yes. Is that what was aspired to at the beginning? The answer is no.

But the complexity of trying to make that work across multiple jurisdictions, I think, has been just very complex and a very difficult lesson to actually learn about how far you can go in the context of Canada with that kind of an approach.

D. Eby: I’m certainly relieved to hear that the ministry will be doing that review. I can’t help but note…. I mean, it took a decade of failure to start looking at other alternatives. I don’t understand that, but I’m very glad that it’s happening, despite the response to the audit initially.

This question is more for the Auditor General, but it does feed into my first question. In my first question, I was saying…. It seems like they’ve never actually delivered this thing, and yet the Ministry of Health is saying that not only was it delivered, but it was delivered on time, which was quite astounding.

In exhibit 1, the Auditor General has set out two different types of functionality findings. One is “Not delivered, removed from scope,” and one is “Delivered but unusable.” I’m wondering what the distinction is, because to me, it seems a distinction without a difference — “Not delivered, removed from scope” and “Delivered but unusable.” What is something that is delivered but unusable? I’m picturing a car without a steering wheel. What is “delivered but unusable”? Isn’t that, functionally, just not delivered?

One thing I just want to point out about this exhibit…. It was a conversation we did have when we were finalizing the report. This is only a list of the problems. We did not try to summarize all of the positives. My first reaction to looking at the chart was: “You mean nothing got delivered at all?” Because I misread that, in fact, in the draft. That’s not the case. You do need to just appreciate that when you’re looking at this chart.

Having said that, there were a number of things that were, at the end of the day, not included in the final project.

Is there a technical distinction that’s useful, or is it just fair to say that’s exactly the point — that they’re not there?

P. Hamilton: The de-scoped items in the beginning were contractually removed through change orders. The other ones, delivered but not usable, were delivered, but they don’t function in a way that the health authorities can actually use them.

B. Ralston (Chair): Was there a financial consequence to IBM as a result of those actions?

B. Ralston (Chair): Did they lose money, or was there any penalty for failing to follow through and meet the contract that they’d bargained?

G. Heyman: I think it’s important to look at the context of this report and the patterns within the ministry. The context is: by far the biggest budget in government. That means that other services required by the public are sacrificed in order to provide an adequate budget for health. In fact, other arguably important health services are sacrificed in order to balance the budget.

Given the size of the ministry, the size of the ministry’s budget and the overall impact on British Columbia, one would expect that there would be a pretty rigorous system of checks, balances, accountability and reviews in place. Yet what I see with this report and a previous report from the Auditor General with respect to IT systems, specifically the Maximus contract, is what looks to me like a pattern.

I want to look at a couple of comments in the report. In “Why Did Things Go Wrong?” there’s a notation:

“Change is a normal part of large IT projects. However, when change occurs, it’s important for organizations to update their project goals or make sure that the changes still enable them to achieve the objectives set out at the start of the project. Neither of these actions were taken in response to IBM’s revised plan.

“We found no evidence that the ministry evaluated its options, including contract termination and alternative solutions. Instead, the ministry continued forward with IBM.”
[ Page 784 ]

If I look back in my memory to the Maximus report, the Auditor General pointed out that, despite assurances to the public of British Columbia that a number of measures would be taken to guarantee the security of personal, private medical information, this whole project was necessary in order to save the government money in terms of changing legacy systems.

The Auditor General found a couple of things. Most of the promised changes in the contract were delayed. I think, in terms of changing one significant part of the legacy system, that never happened at all.

In terms of assurances of protecting the privacy of personal medical information — I’ll come to this point, in relation to this report, again in a minute — it turns out that the ministry relied on the contractor itself to review the actions of its subcontractor, basically self-reporting and self-regulation, with no ability of the ministry to actually determine if the deliverables contained in the contract were being met or delivered. In fact, in a number of cases, the ministry was aware that deliverables that were part of the contract were not met and not delivered.

We look at this report. It talks about change orders being made by the ministry, including capping the number of defects that IBM was required to resolve per year at 1,450, which previously had been unlimited — which transferred risk from IBM to the ministry and, ultimately, to taxpayers. Of course, the whole point of these contracts is supposed to be to transfer risk to the private sector and protect taxpayers and the public interest. The ministry did not receive any tangible compensation in return for these significant concessions.

There’s no end of points here, but I’m just going to check on a couple more that I think are significant.

The ministry contracted IBM to carry out a substantial amount of national acceptance testing, which is unusual. As the service provider, IBM has an interest in having the system accepted quickly. Again, there’s a parallel to what happened with the Maximus contract, essentially relying on the contractor to say, “Don’t worry. Everything is fine. We’re delivering,” when in fact they’re not.

My question, first of all to you, Mr. Brown, is: what possible rationale does the ministry have for taking large, wealthy, multinational corporations like IBM — or Maximus, for that matter — that sign contracts with firm deliverables with the government of B.C. — i.e., with the taxpayers — and simply excusing them from their obligations? What is the thought process that goes into making that kind of decision?

S. Brown: The context of moving from a COTS product to a custom build adds significant impact. The Auditor made the point…. We have accepted that point and, actually, just acted on that point in a recent scenario, which is that that should and could have been a point of pause to say: “Let’s look at the contract. Do we want to go all out with this contract? Should we go with an off-ramp from the contract?”

At the time, those involved — there were senior people, from across multiple jurisdictions, involved and engaged — did not say that was an issue for the vendor. It was an issue of a decision that had been made that it was possible to stitch together the COTS projects into a single national product.

In that context, if I understand correctly, they determined that that wasn’t an issue with the vendor. It wasn’t about the vendor not delivering. It was that the actual COTS couldn’t work in the context of Canada and that there were a number of issues with different jurisdictions around how public health is practised in Canada compared to some of the products from the States — and certain processes. That became a factor of the complexity of trying to get to a national build in Canada across multiple jurisdictions. That led to some adjustments in the risk profile between the vendor and the receiver of the service.

There were a number of adjustments, which you may or may not feel, with the right adjustments…. I only have some of them here. But I was informed that there was a change in terms of a $7.3 million fee reduction for the master licence agreement. There was a $2 million reduction in jurisdictional licence fees that was part of this negotiation. There was a $1 million letter of credits that was made available.

There was a change in the liability that IBM could be subject to going forward, which went from $14 million to $23 million. There was a range of additional reports that now were required because it was now going to be a customized approach. There was a range of things that took place.

Now, whether that was a good negotiation or a bad negotiation, I really, honestly cannot comment — other than there was a negotiation about changing the risk profile. That wasn’t about giving away taxpayers’ money. It was a business negotiation that took place between the vendor and the jurisdictions involved in this.

The characterization…. I’m not sure that that’s an accurate characterization of what occurred. Having said that, the point you’re making about the acceptance process, the actual delegation of that acceptance process and relying on the vendor, which was reported to us by the Auditor…. As we looked back, looked at that context, there were some externalities, pressure that seemed to be on the various parties in terms of moving ahead because of various funding streams coming from federal levels. That led to a judgment call. That was not optimal in terms of that reliance, and that was acknowledged to the Auditor.

G. Heyman: With respect, I think the characterization of it as not optimal is a gross understatement of the actual result. With respect to whether it was a good negotiation
[ Page 785 ]
to end up with taxpayers having a significant obligation that should have belonged to the vendor…. I think the test of a good negotiation is the result. If the result is bad, it’s a bad negotiation. It’s pretty straightforward.

The only thing I take from your comments is people within the ministry who were faced with things not going the way they should have responded to pressure from the vendor to make concessions for which no tangible benefit was received.

I think it’s significant, again, that there’s a pattern here. This isn’t the only contract in which we’ve heard that contractors have been allowed to get away with not delivering on what they contracted to provide, with no penalty. In fact, in the case of Maximus, even when this information had come to light, their contract was renewed — renewed with a substantial increase in payment. So it doesn’t appear to me that the ministry is either learning from the experience or dealing with contracts that are supposed to relieve the taxpayers of B.C. from risk in a particularly constructive way that actually meets that result.

I would appreciate it…. I asked some questions about the thought process that went into allowing corporations like IBM to simply not deliver or be relieved of obligations, which transfers risk to the taxpayer. Perhaps I could ask, through the Chair of the committee, that people from the ministry who can actually describe in detail the discussions that took place and the thought processes that led to that rather perverse result could come to the committee so we can question them.

Finally, I have a question for the Auditor General. Given that we now have two examples of significant expenditures of public money through contracts in which the ministry, on behalf of the taxpayers, apparently thought it was quite all right to relieve the contractors of their contractual obligations or to self-audit, when in fact the ministry should have done it themselves or had an independent auditor, have you considered doing a further review of other major contracts and suggesting a set of guidelines that could be used throughout government to actually assure the taxpayers of British Columbia that it isn’t simply routine business to sign a contract and then give large multinational contractors whatever they ask for?

Actually, on the performance audit coverage plan, we’ve identified…. We are looking at Cerner, which is an IT project very much larger than this in the Ministry of Health.

We’re also, at the moment, finalizing a review of the major challenges facing large IT projects within the province, which is broader than the Ministry of Health, because of a number of problems that we’ve identified in other IT reviews, including ICM. That’s coming up in the spring.

The element of this around oversight is more fundamental than it may appear on the surface. I mean, it’s more than just, in this instance, shifting the acceptance testing to the supplier. It’s a whole cultural and organizational, structural issue that needs to be integrated within the ministry and right across the system. We haven’t specifically looked at that across all of government, but it is a fundamental shift in thinking and how you go about doing something.

It’s actually common in completely unrelated areas where suppliers are contracted to do something and consistently…. In my experience, it’s not unique in British Columbia. In governments where the majority of services are being delivered outside of core government, there is always a risk that the oversight is forgotten. While you can delegate the delivery, you can’t delegate the responsibility for oversight, in my view.

In this case, now bringing that down to the very specific, in the IT world, it’s a very dangerous thing when a supplier is hired. You have to be very careful to make sure that all of the various levels of oversight are conducted by the ministry and not by the supplier. That would be a very unusual practice in any…. To do it the way it was done is just not done. You do your testing yourself.

Getting to a point where the capacity does exist within the ministry is a very important one for the ministry — to get to that point.

S. Brown: Just to, really, dialogue with you a bit more on it, my understanding of the process…. If we could, perhaps…. I don’t know whether it’s appropriate to use the deck example. There was an understanding between the vendor and the jurisdictions about buying something, and that was the base of the original deal. With no blame on anybody, what was perceived as being a good purchase, which is this range of COTS products, was then not seen as actually being what was required. So there was a course correction.

I think that as the Auditor points out correctly, there could have been a choice point there to actually go back to the marketplace and say: “These are the new requirements. These are things that we no longer believe are the priorities within scope. This is now what we want to have in scope. This is where we require more customization across the different jurisdictions.”

That is a choice point. We have taken that point from the Auditor of making sure that gets more hard-wired in — that there is a choice point. However, I don’t believe — the way it has been explained to me and what I’ve looked at — that it was simply: “Now let me hand over to you all of the resources, and I will assume the risk.”
[ Page 786 ]

I believe there was a genuine discussion and negotiation around what the right risk profile should be, given the variety of needs that the different jurisdictions were then asking the vendor to supply. Without that being a blame on the vendor that they didn’t provide the original, I think it was that the original proposition of going to COTS was not going to work in the context of the different jurisdictions.

I take your point, but I think it’s not that it was without thinking or that it was without negotiation or that it was simply a handoff to a large vendor. There was an active negotiation and debate that took place to actually profile that.

G. Heyman: Very brief. I’ve had experience chairing an audit committee of a large organization, a large Crown corporation, as well as a reasonably large organization.

It hasn’t been uncommon, for well over a decade, for there to be policies in place, to have clear procedures when you hit a point of failure to deliver or going over budget, to make decisions and do it in a unified way, not break down projects into discrete samples.

I’m not disputing there might have some negotiation. What I’m questioning is the thinking that went into it, the appropriateness of that thinking. That’s why I asked for people who were directly involved in that decision-making to come before the committee so we can determine what the thought process actually was so that we can have some certainty that if it’s a bad thought process, it won’t be baked into any new process going forward.

B. Ralston (Chair): I would say that the deputy minister is considered to be the person accountable and responsible to provide to the committee answers, even though they may be historical. It’s not a question of fobbing it off to previous deputy ministers or previous officials in the past. The deputy minister here before us is the accountable officer and should be prepared to answer any questions that the committee puts forward that are part of the report.

The suggestion that…. Mr. Brown, I don’t agree with your point of view that somehow this is the responsibility of other officials and you are not informed of that or are not fully informed of that. In my view, you should be prepared to answer those questions. I’m not sure we’ll conclude. We may resume this at a later date. We have another date on the calendar, so that, perhaps, will give an opportunity for you to provide those answers to the committee directly, yourself.

R. Sultan: Deputy Minister, I do appreciate that in the main, this project was launched and guided by others, not on your watch. But the Chair’s point is well taken.

I also appreciate that this is only one in a whole series of very large-scale IT projects that, slowly, politicians try to understand from time to time. If they have any common characteristic, they are always overly ambitious, perhaps under-resourced, based on dreams as much as reality, and there are lots of tears at the end and lots of finger-pointing — particularly on either side of the aisle.

I don’t think this is endemic in any particular political party. I think this is something you will see right across Canada and the world. Perhaps it’s part of the process of growing up, as we come to grips with what, in fact, computers and large-scale IT systems can actually do, realistically. It’s easy to get carried away, and I think this is one example of that.

In all the years I’ve served on this committee off and on, I’ve never seen a report so clearly damning as the one the Auditor General has brought forward. It’s quite astonishing, really, in terms of her listing of the missteps along the way, over a 12-year saga. But what’s done is done. The money was spent, and here we have kind of something that, I guess to some degree, is useful to some people, so we shouldn’t say that it has been a total waste.

The committee, I sense, from the remarks around the room, is looking for somebody’s head here. I think that somebody’s head, in this case, is the IBM Corp., which has apparently not performed up to contract and has been given lots of leeway and, indeed, has had positions of advantage to sort of verify and qualify its own work, which the Auditor General finds rather surprising.

My question is: given all of this history, why on earth would you carry on with this organization which has let you down so badly? Isn’t it time for a new start, with somebody else?

S. Brown: I’d go back to what the Auditor said. There is a list of items that were identified in exhibit 1 or 2, I think it was, some of which have now been resolved, some of which are in the process of being resolved. As the Auditor correctly pointed out, there is a level of functionality across the modules that were delivered by IBM with the jurisdictions to deliver the platform that we have.

I’d be happy to read into the record the functionality that is available and also what functionality is coming on line. I don’t think it’s a fair characterization to say that there was not a significant deliverable here. There is a significant deliverable. I’d be happy to actually walk through the list of the modules and talk about the functionality, about what is being delivered, what is in operation, what has been deployed. There are multiple projects coming on line over the fall and winter.

P. Kendall: I think the history of this has obviously been challenging and troubled, and we didn’t get everything we wanted. But I think what we have now is a really
[ Page 787 ]
solid foundation going forward.

We have the case, which is individual communicable diseases that can be managed. We have data systems which are consistent, certainly within B.C., and will become consistent in those provinces that are using it. We have an immunizations registry, which is linked with the family health, which for the first time gives us an almost total, as far as public health nursing goes, family health record for prenatal, postpartum infant immunizations. It will be linked over time with the doctors’ health records and with pharmacists who are delivering immunizations. That’s a big piece.

We have inventory which allows us to better manage immunizations. It isn’t quite there yet in terms of returns or looking at call chain breakdowns in doctors’ offices, but that can also be fixed. We will have STI — that’s sexually transmitted infections — and TB outbreak or TB case management systems, which will be replacing legacy systems that are breaking down.

We’ve taken about 80 individual databases from B.C. Centre for Disease Control and moved them into one database, which is now accessible to the regional health authorities, which it wasn’t before. In Fraser Health, which is the largest maternal health youth population, we’ve now moved to comprehensive electronic recording.

My colleagues who are using this across the country and in B.C. with the First Nations Health Authority really would not recommend stopping this and trying to start something else, because we’d be in a worse-off position than we currently have painfully got to, where we are now, with respect.

When I went out in 2013 to see how this first instance was working for the family health, it was clunky. It was slow. It was not something you would want to have spent money investing in. When I went out this summer, in 2015, and observed it in a front-line clinic, it was smooth. It was functional. It was unrecognizable from the one that I saw in 2013. I asked the nurses how they liked it, and they said: “We love it.” There are nurses who don’t, but that was my experience on the front line.

I think we have a foundational piece. It’ll need more investment. It’ll need more work. It took us a long time to get here, as you’ve heard — the issues — but I think what we have now is not something that we should throw away.

R. Sultan: Dr. Kendall, I find your assessment reassuring. I’m sure it’s very reliable testimony. Thank you. But that wasn’t really the point I was asking. Despite the positive accomplishments that you’ve described, it hasn’t been achieved without a certain amount of agony, delay, cost overrun and perhaps a little bit of unreality.

I’m just suggesting taking this foundation which has been created, which you’ve very eloquently described…. Isn’t it now time to turn the page and have somebody else come in and do their best? The track record here has not been totally unblemished.

S. Brown: I’d have to take that under advisement, to be honest. I think the judgment that this is a failure of IBM, if that is the point, or there is time for a transition — that’s a business analysis that has not been presented to me.

Nor has a case been made to me as to why IBM is not actually meeting its obligations in terms of the maintenance and operation agreement.

In fact, as Dr. Kendall said, in the last couple of years, the moving forward in terms of addressing issues, getting the system functional…. The maintenance of the system has improved significantly in a whole range of areas.

That doesn’t mean there are not challenges, but no one has presented to me an analysis saying that we should actually try to change vendor. I’m sure I could come to this internally, but that’s not been presented to me as something that would warrant a decision or a comment from me.

R. Sultan: Well, Deputy Minister, I would find your remarks reassuring, but I would also suggest they are inconsistent with what I read in the Auditor General’s report.

L. Throness: Two quick questions — the first to the deputy minister. When it was first decided to run a COTS project or product, was there not expert advice? Did people not know the capacities and possibilities of COTS products?

S. Brown: I think the answer is: yes, they did. I’d say that for Canada Health Infoway at the time there was a swinging away from customization, because of some of the costs and implications and complexity of doing customization, to try and get off-the-shelf products. So there was a strong push in that direction.

Subsequently, as things have evolved, there has been a swing away from that, to some degree, in terms of how the technology has evolved and interoperability has developed. That has shifted again, in terms of where the emphasis would be, by the time there was a strong push towards the COTS.

In practical terms — and I think this wasn’t known at the time — you look at public health and think it’s being practised the same way across all jurisdictions. That was not the case. There are multiple different ways. There were multiple different schedules, I understand, for how people think about vaccines, immunizations, the protocols. The complexity of the Canadian context was not built on a standardized approach across North America. That in itself led to a move away COTS to actually saying there’s a need for some customization.
[ Page 788 ]

L. Throness: The initial decision seems to me like sort of a back-of-the-envelope approach — “Well, let’s try COTS” and spend dozens of millions.

My second question is to the Auditor General. This situation is not a unique situation. I’ve seen it on the federal side — you know, huge cost overruns and all types of problems with IT systems. When you get wind of a project like this, of this magnitude, do your Spidey senses start to tingle? Should you establish a separate and formal category with automatic, direct, ongoing attention from the Auditor General on these types of projects across government?

C. Bellringer: I’d say we don’t do it in as formal a way as the member suggested, but it’s certainly something that is evolving and we’re thinking about more. I’ll take that into consideration for sure.

We definitely, though, when we put our coverage plan together, did do an assessment at that point in time of all of the large projects that we thought could have some risk attached to them and then made a selection from there.

We will update that plan every year, so I think what we’ll do is incorporate that suggestion into our next reassessment.

S. Brown: I think as part of our own learning, as Carol referred to earlier, on the CST project, which is of a significant size and scope, we have engaged with the Auditor straightaway and are keeping the Auditor linked in at key decision points, in terms of sharing information with the Auditor to actually assist and help in terms of some additional oversight.

B. Ralston (Chair): Just further to the member’s question, there was an opportunity in 2011 to intervene and audit. That was, I think, forgone, and there was not an audit undertaken for another three years. In retrospect, was that the wrong decision?

C. Bellringer: I would say that even in hindsight, no. But what we had expected from that was we didn’t see the outcome of that, which was: “We’ll give you the chance to do it, and then we’ll come in later.” We actually expected to see far more progress than we did when we did end up going back and doing the audit. That’s counterintuitive.

V. Huntington: I just want to follow up a little bit on Mr. Sultan’s suggestion that it is perhaps time to leave IBM in the dust on this. I’m very troubled by your response, Mr. Brown, by “that business analysis was not presented to me,” which is precisely the point of the Auditor General’s primary recommendation: that there is a need for an independent review to do just that, to analyze whether or not the process that was engaged in with IBM should continue or should look at whether there are alternatives.

To that, clearly the ministry has said: “No, we’re not interested in that.” And yet your comment shows to me, at least, that it is essential that that happens in this instance.

I believe the ministry is too close to this issue, to this contract and to the entire ten years of effort. I believe that you should be looking very closely at whether or not there’s a conflicted sympathy here. I think you should regroup and take a very close look at the Auditor General’s primary recommendation. I think your quote would show that that recommendation is a sound one that ought to be considered very seriously.

As I read this report, it reads like a movie script: intimidation, sanitization, 420 percent implementation budget overruns, de-scoping, defective products, poor leadership, poor contract follow-up, poor practice. We see this fairly often — maybe not to the extent of this, but I’m sure it exists with other huge contracts.

I guess I would like to ask, first, the comptroller general, and perhaps the Auditor General could comment. What is the role of the comptroller general in reviewing how ministries undertake these types of large corporate contracts — how they administer them, what the financial obligations are, what the risk assumptions are or the transfer of risks? Where is the comptroller general in all of this? When do you, or are you entitled to, step in and say: “We have to take a good, hard look at this, and practices are going to improve, or I’m going to wave my little wand”?

S. Newton: We set, through the government’s Core Policy And Procedures Manual, the requirements for contracting across government. There are very specific rules and requirements. We also test, periodically, whether or not those rules are being followed. We also provide the information back to ministries so they’re aware of whether or not they’re meeting the contract requirements.

In significantly large deals, our office also is involved in the aspects of, on the financial side, whether something is capital or operating and how that would actually impact government’s financial statements.

As far as the governance on the contract itself, the expectation is that the ministry is putting together the appropriate business case in order to enter into the contract and is monitoring the contract and the deliverables as they go through the process. Decisions that are made throughout that process that are taken at a significantly high level — some of them are a strict negotiated funding choice. “We are going to continue based on the investment to date” versus “We’re going to wind this thing up.” And there are a lot of factors involved. In something like this, there’s a bit of a jurisdictional issue, because it’s a national program with all the various jurisdictions involved.

From that perspective, it would be an interesting question that I haven’t really put my mind to as far as whose
[ Page 789 ]
jurisdiction was the decision to contract through Infoway in order to move forward, which all the provinces paid into — in which case, it might not be a province of B.C. contract.

The other piece is that for large, significant contracts, as we hear of issues and concerns, we do follow up with ministries in order to be able to get some clarity. I have regular conversations with the Deputy Auditor General on a variety of financial issues related to year-end, so we are also made aware of things as they come up, if Russ and I feel they’re worth discussing in order to be able to follow up with ministries to see where they’re at.

So there’s a very specific requirement. There’s also ongoing dialogue. As far as the business decisions that ministries make in order to move forward, those are decisions that they make at their senior executive level with consultations with central agencies like Treasury Board staff.

It’s not my place to second-guess a Treasury Board decision or a government policy decision, but certainly, whether or not the contracting policies and procedures are being followed, I would be concerned.

B. Ralston (Chair): Just before you begin, Vicki, we’re clearly not going to finish today. The management subcommittee had agreed that we would continue this, if necessary, on November 16 at 10:05 for the same: 10:05 to 12:30.

I think we have time for one more question and one more response, and then we’ll adjourn.

V. Huntington: Well, just to follow up a little bit. If the comptroller general’s office is responsible for good practice for contract-letting policies for the management of contracts and the guidelines, then what responsibility do you have, or do you feel you have, to ensure or to review a situation when you see a report like this coming from the Auditor General?

Having the ministry examine its own policies in a situation this grave is like having IBM self-report, and I think there has to be some oversight in how the corrections are made to process and practice within a ministry, and I’m asking: who does that? Who’s responsible for that?

S. Newton: For the rules that are followed for ensuring that that framework is well described and everybody is cognisant of what they are required to know, my office would have that responsibility. Whether or not the specific requirements in contracting are followed is the responsibility of the ministry in order to do that.

Where we see issues or areas of concern, we do follow up to understand what occurred. In a lot of cases, where the Auditor General has already provided information as to what has occurred, then that clearly gives us the answer as far as what may be the issue.

As far as practices going forward, there is, within the Ministry of Finance, and also government’s internal audit and advisory services as well, that is doing the risk assessments as far as what pieces of work they need to do on an ongoing basis…. They also would have the ability to determine that contracting would be something that would specifically be looked at.

In relation to this issue, we would look at the testing that we do of contracts and payments within government regularly to determine if there are changes that we need to do in our risk assessment profile to determine how best to find these things in the future.

Certainly, in relation to this, I will be keenly watching as the ministry addresses the recommendations. Certainly, anything where the ministry is going to seek an external party to do a review is definitely appropriate, and it doesn’t necessarily need to be my office that does the review or even that would contract that. That would be if the ministry is doing something externally to demonstrate accountability for a recommendation within the Auditor General’s report. I think that’s appropriate.

B. Ralston (Chair): Marvin Hunt had a question which relates to preparation for the next session, so I’ll let him go and ask the question.

M. Hunt: It’s just that I’m getting confused in this federal project and the provincial expenditures, and I’m getting lost in it all.

So my questions are going to be these. How much of these expenditures were the COTS project that failed? How much is the custom project that is ongoing? And how much of the numbers that we’re talking about here — I believe it’s the $115 million number — is the ministry going to describe as yearly implementation costs that are rolled into this $115 million? I want to sort of get a feeling of what are federal costs, what are provincial costs and what’s going on here. It seems to me we’ve patched a bunch together, and I just want to separate it out so I can understand.

With that, we’ll adjourn to Monday, November 16, at 10:05. Is there a motion to that effect?

Hansard Services publishes transcripts both in print and on the Internet.
Chamber debates are broadcast on television and webcast on the Internet.
Question Period podcasts are available on the Internet.

Chair:	Bruce Ralston (Surrey-Whalley NDP)
Deputy Chair:	Sam Sullivan (Vancouver—False Creek BC Liberal)
Members:	Kathy Corrigan (Burnaby—Deer Lake NDP)
	David Eby (Vancouver—Point Grey NDP)
	Simon Gibson (Abbotsford-Mission BC Liberal)
	George Heyman (Vancouver-Fairview NDP)
	Marvin Hunt (Surrey-Panorama BC Liberal)
	Vicki Huntington (Delta South Ind.)
	Greg Kyllo (Shuswap BC Liberal)
	Mike Morris (Prince George—Mackenzie BC Liberal)
	Lana Popham (Saanich South NDP)
	Linda Reimer (Port Moody—Coquitlam BC Liberal)
	Selina Robinson (Coquitlam-Maillardville NDP)
	Ralph Sultan (West Vancouver–Capilano BC Liberal)
	Laurie Throness (Chilliwack-Hope BC Liberal)
Clerk:	Kate Ryan-Lloyd

REPORT OF PROCEEDINGS(Hansard)

SELECT STANDING COMMITTEE ON PUBLIC ACCOUNTS

REPORT OF PROCEEDINGS
(Hansard)

SELECT STANDING COMMITTEE ON
PUBLIC ACCOUNTS