Final Report

August 2020

Table of Contents

• Executive Summary
• Background
• Audit Approach
• Key Findings
  • Project Review and Approval Process and Governance
    • Finding #1: The role that the OGD Committee undertook as part of the project review process diverged from its intent
    • Finding #2: Lack of evidence that established procedures are being followed
    • Finding #3: Lack of process and guidance related to monitoring of climate lens deferrals and exemption
  • Risk Management
    • Finding #4: The program, recipient and project risks were consistently assessed and mitigated according to their risk levels, but the approvals were not well documented
  • Outcomes-Based Approach
    • Finding #5: Some gaps need to be addressed to have a fully comprehensive Performance Information Profile
    • Finding #6: Performance measurement governance controls have been established
• Audit Conclusion
• Statement of Conformance
• Management Action Plan (MAP)

Executive Summary

Audit Objectives and Scope

The objective was to provide reasonable assurance on the efficiency and effectiveness of the Investing in Canada Infrastructure Program's (ICIP) process for assessing and recommending infrastructure projects for funding approval that support the achievement of ICIP objectives.

More specifically, the audit assessed whether Infrastructure Canada (INFC) had in place effective internal control processes for:

• Reviewing and recommending for approval projects submitted to INFC under ICIP; and
• Managing program and project/recipient risks.

The audit also aimed to provide reasonable assurance that ICIP's newly introduced outcomes-based approach to determine eligibility at the project submission stage was designed appropriately. This approach should also provide INFC with sufficient and accurate performance data on expected results and be aligned with Treasury Board (TB) requirements regarding performance measurement.

The just-in-time auditing approach was used to provide timely assurance to INFC senior management on the effectiveness of core controls which will allow management to strengthen early monitoring of a program's lifecycle, instead of relying on the traditional audit approach, which generally does not start until the end of a program.

The audit scope covered the period between April 1, 2018 and June 30, 2019.

Working as Expected

Various elements related to the project review and approval process and governance were in place and working as expected, including:

• Performance measurement governance controls had been established;
• The Other Government Department (OGD) Committee and the External Review Committee (ERC), which provide project-specific considerations as part of project review, had documented Terms of Reference;
• The ERC had established assessment criteria for selecting members;
• The ERC members collectively possessed sufficient knowledge and experience to effectively exercise their function;
• The Program Secretariat played a vital role in the project review process; and
• The committees had sufficient time to exercise their function effectively, met regularly, had quorum for all meetings, and documented all records of decision.

The ICIP's risk management approach was working well for program, recipient and project risks, including the use of formal risk assessments, identification of risk mitigation strategies and documented risk responses/mitigation measures. The approach was proactively modified over time as the program matured.

The ICIP's outcomes-based approach was designed appropriately which provided INFC with sufficient and accurate performance data on expected immediate outcome results.

Areas for improvement

The OGD Committee is composed of other federal departments whose areas of expertise align with the types of infrastructure projects that are anticipated to be submitted for approval. The Committee demonstrated strong horizontality and collaboration across the federal government. While efforts were made to better focus the role of the Committee with respect to the advice provided for project approval, its expected role as a horizontal strategic advisory panel partially differed from the role that the Committee took concerning the review process. Given the significant resources and effort required from INFC and OGDs, the difference in role impacted the efficiency of the project review process.

While the risk management approach was working well, evidence of management approval was inconsistent for the program, recipient and project risk assessments.

Similarly, many procedures and guidance materials had been established for reviewing projects; however, the audit identified that documentation was not always available to demonstrate that procedures were consistently being followed. Evidence that procedures are followed decreases risk exposure and demonstrates sound stewardship and the highest level of integrity, transparency, and accountability.

Climate lens assessments are a federal requirement for many projects under the ICIP. These assessments are intended to encourage behavioral change and incorporate consideration of climate impacts into the planning of infrastructure projects. Early in the Program, prior to the completion of the federal guidance on this requirement, climate lens deferrals were required in order for projects to move forward. Climate lens deferrals continue to be applied in certain circumstances.

The audit identified a gap in process and guidance related to monitoring deferrals. Without a clear process to follow up on projects with deferred assessments, there is an increased likelihood of three risks. First, that payments to provinces and territories (PTs) could be made without having satisfied the payment conditions; second, that forecasting of departmental cash flow needs is further complicated with outstanding climate lens deferrals. Finally, that the national level Green House Gas (GHG) reduction target of 10 mega-tonnes may not be achieved.

The Performance Information Profile (PIP) meets most requirements of TB's Policy on Results, but some gaps need to be addressed for the PIP to be fully comprehensive, such as a data strategy to ensure the availability of intermediate and long-term performance and outcome information. These gaps could impact INFC's ability to report ICIP results and benefits back to Parliament and Canadians.

Conclusion

For the primary objective, the audit concluded that INFC has generally designed an effective review process to assess and recommend infrastructure projects for funding approval. There are concerns that some controls are not working as intended.

In most cases, the risk exposure of the control failures are partially reduced by other controls. In some cases, these also resulted in duplicative processes. That said, we found that projects approved by INFC had documentation to support eligibility according to the Program's Terms & Conditions.

For the secondary objective, the audit concluded that ICIP's outcomes-based approach was appropriately designed to provide sufficient and accurate performance data. However, some work remains to better define roles and responsibilities within INFC in relation to performance measurement and the collection of data for intermediate and long-term outcomes/indicators. At the same time, PIPs are evergreen documents, and the performance measurement approach under the Program represents an improvement compared to past programs.

Recommendations

As this just-in-time audit was completed early in the program's lifecycle, the following recommendations are aimed at improving the program's efficiency and reducing risks as implementation continues.

• It is recommended that the Assistant Deputy Minister (ADM), Program Operations Branch (POB), reconsider the mandate of the OGD Committee in order to make better use of INFC's and OGD's resources.
• It is recommended that the ADM, POB, review the project review checklists process, for documenting the project assessment and due diligence, to improve efficiency and increase effectiveness of this control. Consideration should be given to separating all of the steps completed by the regional analysts as part of their due diligence, from the key controls that require management sign-off.
• It is recommended that the ADM, POB, clarify the climate lens process to incentivize consideration of GHG emissions and/or climate resilience in infrastructure projects and update program guidance for its management, including deferrals. In addition, a process should be established for the public notification of approved exemptions.
• It is recommended that the ADM, POB, ensure that supporting evidence exists and is available to confirm that program, recipient and project risks are approved according to the appropriate level of authority identified in the management control framework to ensure proper program governance.
• It is recommended that the ADM, POB, with support from ADM, Policy and Results Branch, ensure that the PIP is updated by addressing TB's Directive on Results' requirements that were not explicitly identified or included in the PIP in order to have a fully comprehensive PIP.

Background

Under the Investing in Canada Plan (IICP), the Government of Canada is investing more than $180 billion over 12 years in public transit, green infrastructure, social infrastructure, trade and transportation, and rural and northern communities.

The $33.5-billion Investing in Canada Infrastructure Program (ICIP) is the centrepiece of Infrastructure Canada's (INFC) new funding initiatives under the IICP. This new allocation-based Program is being delivered through Integrated Bilateral Agreements (IBAs) with provinces and territories (PTs) over ten years and includes four funding streams:

• Public Transit Infrastructure Stream (PTIS);
• Green Infrastructure Stream (GIS);
• Community, Culture and Recreation Infrastructure Stream (CCRIS); and
• Rural and Northern Communities Infrastructure Stream (RNIS).

The remainder of the funding under each stream is allocated per capita and, using 2016 Statistics Canada Census data, is based on populations of communities under 30,000 for RNIS.

As part of the ICIP governance process, INFC established two new external committees. The first is the Other Government Departments (OGD) Committee, comprised of Director General-level members from other federal departments. The second includes members external to the federal government, referred to as the External Review Committee (ERC).

These two committees review projects and provide strategic advice to the INFC minister and Rural Economic Development (RED) minister to consider during approvals. While committee functions enhance the oversight of the program, they do not replace departmental authorities or responsibilities. Once the committees complete their review, the Deputy Minister (DM) makes a recommendation to the Minister of Infrastructure and Communities or to the Minister of Rural Economic Development for project approval or for the appropriate Minister to seek Treasury Board (TB) approval, as required. More details on both committees is provided in the next sections.

Other Government Department Committee

The mandate of the OGD Committee is twofold: to provide strategic considerations on projects to support the Minister of Infrastructure and Communities in his/her approval of projects, and to provide a horizontal table for departments to share and exchange information on issues related to the delivery of the IICP, including longer-term considerations and opportunities for the IICP.

The OGD Committee is composed of other federal departments whose areas of expertise align with the types of infrastructure projects that are anticipated to be submitted for approval. Members include directors general and directors from:

• Environment and Climate Change Canada;
• Natural Resources Canada;
• Indigenous Services Canada;
• Crown-Indigenous Relations and Northern Affairs Canada;
• Public Safety Canada;
• Canadian Heritage;
• Transport Canada;
• Innovation, Science and Economic Development; and
• The Treasury Board Secretariat in an observer role.

External Review Committee

The External Review Committee (ERC) provides an external perspective on projects and additional strategic considerations related to project alignment to program outcomes and risks, as well as challenges and opportunities. Committee members are selected from outside the federal Public Service and have backgrounds representing various infrastructure sectors. The committee's strategic considerations are provided to the INFC minister and RED minister to contemplate during approvals.

It should be noted that, as per the program's terms and conditions, prior to the establishment of the ERC, projects only went to OGD. As such, 13 projects were reviewed by OGD prior to the establishment of ERC in February 2019.

Risk Management Approach

INFC has refined its risk management approach under ICIP through the introduction of a split risk assessment process, with separate recipient and project assessments. Under previous programs these were typically combined in a single assessment.

In addition to this, the new risk assessment process is used to assess recipient and project risks throughout the program's lifecycle in order to inform project approvals, set baselines for monitoring, provide standardized risk responses and support effective risk management for the Department.

Two key tools were developed to support this process:

• The Recipient Risk Assessment Tool; and
• The Project Risk Assessment Tool.

Outcomes-Based Approach

ICIP is an outcomes-based program where eligibility, funding levels and other program requirements are linked to the outcomes a given project is expected to achieve. Under this approach, a project's alignment to expected outcomes is a key factor in the application review process. Under previous programs, more emphasis was placed on the asset type. Focusing on expected immediate outcomes rather than asset types allows for a greater variety of project types and approaches and this, in turn, provides greater flexibility for provinces and territories to select projects that did not previously fit into eligible asset categories.

Project eligibility is also dependent on stream-specific requirements and exclusions as well as horizontal federal requirements (e.g. Climate Lens, Community Employment Benefits).

Operating Context

Infrastructure Recipient Information System (IRIS)

IRIS is a web-based application that enables direct inputting/uploading of data by PTs to improve efficiency and to reduce duplication of effort―a new approach under ICIP. Expected immediate outcomes, including a list of specific indicators, are submitted by PTs as part of the project submission and updated through progress reporting through IRIS. This new process allows INFC to report on estimated progress being made towards achieving results and targets established in the IBAs before projects are completed, and will allow for comparison to actual outcomes once projects are completed and in use.

Resources

Within the Program Operations Branch (POB), there are three directorates responsible for the ongoing implementation of programs and the corresponding oversight functions. Two are organized by regions (i.e. North-Atlantic-Ontario and West-Quebec) and are the key Federal-Provincial-Territorial interlocutors and have the lead responsibility at the project level. The third directorate, Program Integration, manages across programs―providing a support function and is the lead at the program level.

Regional analysts do not work only on ICIP projects, but rather work on and oversee projects associated with multiple programs within their assigned region. As with other programs, the receipt of project submissions from PTs has sometimes been extremely heavy. This creates additional pressure on staff for the specific region in POB to review the project submissions in a timely manner and has been cited as a reason why some procedures or controls have not been followed appropriately in some instances.

Additionally, POB is coming out of a high turnover period, as shown below, which should be taken into consideration when assessing the results of this audit. This has led to a significant loss of corporate memory within program operations staff over the last three years and a need to constantly train new employees.

Figure 1 - POB Turnover Rates*

	2016/2017	2017/2018	2018/2019	2019/2020**
Indeterminate turnover	28	39	29	10
Indeterminate Population	129	141	152	175
Turnover rate	21.7%	27.7%	19.1%	5.7%

* Numbers include Major Bridges division

** Fiscal Year still in progress (data up to October 31, 2019)

Audit Approach

Audit objectives and scope

The audit's main objective was to provide reasonable assurance to INFC senior management and the Deputy Minister on the efficiency and effectiveness of the new review process introduced under ICIP for assessing and recommending infrastructure projects for funding approval to support the achievement of ICIP objectives. More specifically, the audit assessed whether the Department had in place effective internal control processes for reviewing and recommending projects for approval and for managing program and project/recipient risks related to the approval process.

The audit also aimed to provide reasonable assurance that ICIP's newly introduced outcomes-based approach to determine eligibility at the project submission stage was designed appropriately. This approach should also provide INFC with sufficient and accurate performance data on expected results and be aligned with Treasury Board requirements regarding performance measurement.

The audit scope was divided into two phases:

• Phase 1: Project review and recommendation for approval process and governance covering the period of April 1, 2018 to May 31, 2019.
• Phase 2: Risk management and outcomes-based approach covering the period of April 1, 2018 to June 30, 2019.

Approach and methodology

The just-in-time auditing approach was used to provide timely assurance to INFC senior management on the effectiveness of core controls which will allow management to strengthen monitoring, instead of relying on the traditional audit approach, which generally does not start until the end of a program. This audit engagement was performed early in ICIP's implementation, at a time when some program guidance was still being developed. Based on the integrated audit and evaluation plan for 2019-20, there are two additional just-in-time audits planned over the program lifecycle. They will likely focus on program and project governance, including the management of oversight committees, as well as agreement monitoring and oversight, including payments and reporting.

This audit was conducted in accordance with the TB Policy on Internal Audit and the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing.

The audit engagement included various tests, as considered necessary, to provide reasonable assurance on the management of the ICIP. These tests included, but were not limited to, interviews, observations, audit procedures, review of supporting documentation, attendance at key meetings and an analytical review.

As part of the work performed for the outcomes-based approach, the audit assessed the extent to which the Performance Information Profile (PIP) contained all elements required under TB's Policy on Results, as well as whether the indicators within the PIP were aligned to those in the Departmental Results Framework. An analysis of the quality of the PIP's content was not conducted as it was outside of the scope of this review.

Field work was substantially completed on September 20, 2019, with some additional testing completed in January 2020. Audit findings were communicated to the auditee to validate facts and to confirm the clarity, accuracy, and completeness of the information reported.

A risk-based approach was used to establish the objectives, scope, and approach for this audit engagement and included key interviews and the review of important documents.

Taking into account these risks, detailed audit criteria were developed which guided the audit field work and formed the basis for the overall engagement conclusion.

Key Findings

Project Review and Approval Process and Governance

It was expected that there would be an efficient and effective review process in place to assess and recommend infrastructure projects for approval, and to ensure that approved projects aligned with the terms, conditions and expected outcomes of the ICIP.

Following ICIP's launch in February 2018, INFC intended for each project submitted for funding consideration to be reviewed by both OGD and ERC committees, regardless of the risk profile and materiality of projects. Given a larger than expected volume of projects and larger than anticipated low dollar value projects, in February 2019, INFC implemented a risk-based approach to the external review process in order to focus attention on higher dollar and higher risk projects.

Under the revised approach:

• Projects under $10M that are low-risk, do not require a climate lens assessment and do not require approval from TB must undergo a streamlined review process without requiring OGD/ERC review (Path 1);
• Projects under $10M that are low-risk and require a climate lens assessment must be reviewed by the OGD Committee (Path 2); and/or
• Projects over $10M, all medium/high risk projects, and/or all projects that require TB approval must be presented at both OGD/ERC committees for review (Path 3).  

Figure 2: ICIP's Risk-Based Streamlined Approach for Project Review and Approval

Text description for Figure 2

Figure 2 is entitled ‘ICIP’s Risk-Based Streamlined Approach for Project Review and Approval’. It is split into two stages. Each stage is divided into four different rows, with each row following a different path for project review and approval.

The first stage is entitled ‘Stage 1: Project Review'. The Stage 1 timing depends on the quality of submission and volume of projects. The second stage is entitled ‘Stage 2: Project Approvals’ and the timing depends on the project path required.

Projects that fall within Path 1, which are projects under $10 million that are low-risk, do not require a climate lens assessment and do not require approval from Treasury Board (TB), must follow the following steps:

Strage 1:

1. Project submission from the province/territory;
2. The eligibility review completed by Infrastructure Canada (INFC);

Stage 2:

3. The recommendation made by INFC staff;
4. The Minister’s approval; and
5. The approval letter.

The approximate length of time to complete Stage 2 for Path 1 projects is around four weeks.

Projects that fall within Path 2, which are projects under $10 million that are low-risk, require a climate lens assessment but do not require approval from Treasury Board (TB), must follow the following steps:

Strage 1:

1. Project submission from the province/territory;
2. The eligibility review completed by Infrastructure Canada (INFC);

Stage 2:

3. The Other Government Departments (OGDs) Committee’s review;
4. The recommendation made by INFC staff;
5. The Minister’s approval; and
6. The approval letter.

The approximate length of time to complete Stage 2 for Path 2 projects is around six weeks.

Projects that fall within Path 3 (without TB’s approval required), which are projects over $10 million that are medium and high risk but do not require TB approval, must follow the following steps:

Strage 1:

1. Project submission from the province/territory;
2. The eligibility review completed by Infrastructure Canada (INFC);

Stage 2:

3. The Other Government Departments (OGDs) Committee’s review;
4. The External Review Committee’s (ERC) review;
5. The recommendation made by INFC staff;
6. The Minister’s approval; and
7. The approval letter.

The approximate length of time to complete Stage 2 for Path 3 projects without TB’s approval required is around eight weeks.

Projects which fall within Path 3 (with TB’s approval required), which are projects over $10 million that are medium and high risk but require TB’s approval, must follow the following steps:

Strage 1:

1. Project submission from the province/territory;
2. The eligibility review completed by Infrastructure Canada (INFC);

Stage 2:

3. The Other Government Departments (OGDs) Committee’s review;
4. The External Review Committee’s (ERC) review;
5. The recommendation made by INFC staff;
6. The Treasury Board’s approval; and
7. The approval letter.

The approximate length of time to complete Stage 2 for Path 3 projects requiring TB’s approval is around twelve to eighteen weeks.

The project submission, review and approval process, in effect at the time of the audit, is outlined below:

1. PT submits a completed project application for funding consideration to INFC, which includes a signed attestation by designated PT official.
2. 1. INFC's program analyst reviews the project application for eligibility by conducting a multi-step due diligence assessment against the program's terms and conditions and other federal requirements. The detailed project assessment is summarized in the project fiche. The project review checklist is also completed to confirm that the various due diligence steps have been taken.
   2. Following review of the detailed project assessment (e.g., the project review checklist and supporting documentation) and project fiche, the Regional Director and Director General approve the recommendation regarding project eligibility. This approval is captured by signing the project review checklist.

   3. The project fiche is reviewed by INFC's Program Secretariat for consistency of analysis and quality assurance purposes.
3. The project fiche is shared with the OGD and ERC committee members (as applicable) to get their input.
4. The project fiche is submitted to the Deputy Minister for final recommendation to the Minister.
5. The project fiche is submitted to the Minister for approval (or Treasury Board if outside of Minister's delegated authority).
6. Approval letters are sent by INFC's Minister to the PT to inform representatives of project approval, and any applicable conditions.

INFC documents program controls related to the project review and approval process through ICIP's Management Control Framework^{Footnote 1} (MCF). The ICIP was announced in spring 2017 and officially launched in February 2018. While elements of the MCF were in place within the first year of the program launch, the MCF was fully developed and approved in June 2019.

The following elements related to the project review and approval process and governance are in place and working as expected:

• Terms of Reference (ToRs)―including the mandate, roles, responsibilities and accountabilities―for the OGD and ERC committees were documented and communicated to members;
• Assessment criteria were established and documented for selecting ERC members, and ERC members collectively possess sufficient knowledge and experience to perform their function effectively;
• Program Secretariat, the group within INFC responsible for coordinating the OGD and ERC committees, plays a vital role in the project review process and ensures a high level of quality for the project fiches before they are shared with OGD and ERC members;
• OGD and ERC committees had sufficient time to perform their function effectively; and
• Committee meetings occurred regularly, had quorum and resulted in documented records of decision.

Figure 3: OGD Committee/ERC's Metrics

	OGD Committee	ERC
Meetings	Start date: April 2018 Monthly (12 regular, 4 ad-hoc)	Start date: February 2019 Monthly (3 regular, 1 ad-hoc)
Quorum	Satisfied (3 members including chair) Meeting Membership: 10	Satisfied (3 members including chair) Meeting Membership: typically 4
Participation	Average: 74% DG-level: 24%	Average: 106% (Based on 4 members per meeting)
Project Distribution	Standard # of business days: 8 7/12 – 8 days or more 4/12 – 6-7 days 1/12 – data not available	Standard # of business days: 4 Data not available

Finding #1: The role that the OGD Committee undertook as part of the project review process diverged from its intent

It was expected that governance committees would operate in accordance with their terms of reference.

OGD Committee

The original OGD Committee's ToR (April 2018) defined the role of members as providing ‘technical expertise' advice with respect to projects and providing a forum for information sharing between the federal organizations.

As INFC is not involved in the design and construction of the projects, technical design advice generally does not enhance INFC's ability to review project proposals.

The OGD Committee's ToR were updated in June 2018 to reflect, among other changes, that the OGD member's role was to provide ‘strategic considerations' with respect to project review. INFC also met with the OGD members through a ‘take stock' exercise in the fall 2018 to discuss and clarify their expected role.

The audit found that in one-third of projects, the Committee provided INFC with strategic considerations that were ultimately included in the final project fiche. Examples of strategic considerations include site-specific knowledge, recipient history or future plans, or legal concerns that may not be obvious or known to INFC.

While early actions were taken to address challenges, the OGD Committee continued to focus on technical design concerns, and provided relatively limited strategic considerations. In interviews held in June and July 2019, OGD Committee participants noted a desire for more information to complete their technical analysis, which suggests a continued misalignment between the expected strategic role, and the role undertaken by the participants. While the OGD Committee membership is identified at the Director General level, participation is largely delegated to Directors, Managers or Senior Analysts, with a participation rate of only 24% at the Director General level.

External Review Committee

The ERC provides an external perspective on projects and additional strategic considerations related to project alignment to program outcomes and risks, as well as challenges and opportunities.

Similar to the OGD committee, the audit found that in one-third of projects, the ERC provided INFC with strategic considerations that were ultimately included in the final project fiche. The ERC operated in accordance with the Committee's ToR.

Why is this important?

During the audit period, the OGD Committee was not efficiently fulfilling its role with respect to project approval as it required a significant level of resources and effort from INFC and OGDs.

INFC will need to decide if it wants to continue to rely on the OGD Committee for strategic considerations. The second purpose of the OGD Committee is to be a forum for sharing information with other government departments related to the delivery of the IICP. OGDs could continue to add value by providing technical and other insight that would inform and support INFC's discussions with PTs for the ICIP, including on provincial or territorial infrastructure plans.

Finding #2: Lack of evidence that established procedures were consistently being followed

It was expected that program procedures for reviewing eligibility of infrastructure projects would be established and followed.

The audit found that, although many procedures had been established for reviewing projects, evidence was not always available to demonstrate that the procedures were consistently being followed.

The project review checklist, which confirms the due diligence steps taken by the program analyst, was designed to be approved by the manager, director, and director general, and was identified by the Program as a key control. The audit sample, including 32 projects from April 2018 to May 2019, found five checklists were missing. Of the 27 available checklists, 12 did not have important steps checked off, and 17 were missing approval signatures.

An example of an important step not completed was the assessment of the reasonableness of project cash flows and confirmation by the Finance Unit that the required funds were available. In three instances out of 27 available checklists (11 percent), there was no evidence that cash flows were provided to Finance. More concerning, there were two instances where approved checklists indicated that Finance had verified the cash flow, when in fact that step had not occurred before the checklist was approved. In another instance, the step to indicate that the cash flow had been verified by Finance was removed from the checklist, and the audit confirmed that the verification by Finance had not occurred before the checklist was approved.

The project review checklist was designed to validate key due diligence steps performed by the program analyst, such as confirming project eligibility, ensuring that requirements for environmental assessment and duty to consult with Indigenous groups had been met, and confirming the project risk assessment results. This control was designed to confirm projects were assessed appropriately against the program terms and conditions, and to ensure compliance with some requirements of federal legislation (e.g. environmental laws or the Financial Administration Act). The number of control failures in a relatively small sample is sufficient to demonstrate the control, as designed, is ineffective.

This control failure is somewhat mitigated by the challenge function played by Program Secretariat in reviewing the quality of the content included in the project fiches. That challenge does not involve an in-depth review of the analysis performed by regional analysts (i.e. the quality of documentation or analysis to support the due diligence that is included the project fiche); therefore, the review of the fiche is not sufficient to replace the control.

A total of 35 project files were sampled to confirm the project submissions included all the required supporting documentation, as outlined in the Project Submission Guide. In addition, select eligibility assessments were reperformed, based on what was included in the project file, to validate the results of the program analyst. We found that projects approved by INFC had documentation to support eligibility according to the program's terms and conditions.

For seven projects, the requirement to complete a climate lens assessment was deferred. This issue is explained in more detail in the next finding.

Program management has indicated in their Management Action Plan in section 7 that since the audit was performed, changes have been made to their processes. We will assess their effectiveness in future work.

Why is this important?

Being able to demonstrate that procedures were established and followed is a key component of ensuring INFC respects the objective of TB's Policy on Transfer Payments, particularly to ensure programs are managed with sound stewardship and the highest level of integrity, transparency, and accountability.

Finding #3: Lack of process and guidance related to monitoring of climate lens deferrals and exemptions

An important element of ICIP's program design is the Climate Lens requirement, which is intended to promote better incorporation of climate change considerations in the development of infrastructure assets.

According to the Climate Lens guidance:

“The assessments are intended to incent behavioral change and consideration of climate impacts into the planning of infrastructure projects with a view to implementing Canada's mid-century goals of a clean growth low-carbon economy. The prescribed assessments are intended to encourage project proponents to incorporate climate change considerations into their project development process for the first time. By systematically evaluating each project's GHG emissions and/or resilience to the impacts of climate change, project planners will become increasingly familiar with key considerations, risks, and mitigation strategies, which will facilitate better decision making in both current and future infrastructure projects.”

The Climate Lens is a horizontal requirement applicable to Infrastructure Canada's ICIP, Disaster Mitigation and Adaptation Fund, and the Smart Cities Challenge.

The Climate Lens has two potential components:

• The greenhouse gas (GHG) mitigation assessment designed to measure the anticipated GHG emissions impact of an infrastructure project, and
• The climate change resilience assessment, which employs a risk management approach to anticipate, prevent, withstand, respond to, and recover from a climate change related disruption or impact.

Individual proponents are asked to undertake one or both types of assessment, depending on the program, funding stream, and the estimated total eligible cost of the project.

Under ICIP, climate lens assessments are required for all projects when total eligible costs are $10M or greater, regardless of the funding stream. Additionally, the Climate Lens assessment is mandatory for project eligibility under the Green Stream as follows:

• A GHG Mitigation assessment is mandatory for projects under the Green Climate Change Mitigation sub-stream (resilience assessment not mandatory);
• As part of the GHG Mitigation assessment, projects will be required to provide a cost-per-tonne calculation indicating the amount of investment required to achieve one tonne in GHG reductions.
• A Climate Resilience assessment is mandatory for projects under the Green Adaptation Resilience & Disaster Mitigation sub-stream (GHG assessment not mandatory).

There are no minimum requirements that climate lens assessments have to meet as a condition of funding.

The Minister has the authority to exempt climate lens assessments (except for the two previously mentioned green sub-streams), but must make such exemptions public. Program management also allowed for deferrals (i.e. an assessment was not mandatory at the project approval stage) in some cases.

As noted in a policy decision document, “Deferral of one or both climate lens assessments should be considered primarily for shovel-ready projects, where delays in approval would prevent time-sensitive procurement processes from moving forward, or where the Climate Lens would be completed during detailed design work, which will occur some time after project approval.” In addition, it should be noted that at the launch of the ICIP, the guidance to support the climate lens assessments was not yet available.

For deferred assessments, the completion of the climate lens is being treated as a condition prior to INFC paying the federal contribution, similar to Aboriginal consultation or environmental assessment requirements. This condition is explicitly included in the approval letter to PTs for deferred projects, but is not included in the integrated bilateral agreement.

In the sample, for the fifteen projects requiring a climate lens assessment, eight were completed at project approval, while the remaining seven projects were advanced for approval seeking a deferral or exemption. All projects with climate lens assessment deferrals (or exemption in one case) were granted by the Minister, and the related condition was included in the PTapproval letters.

The audit found that a process and guidance existed to obtain ministerial deferrals, but there was a lack of a process and guidance of post-deferral monitoring to ensure the outstanding requirements were met prior to issuing payments. Similarly, there is no documented process regarding when and how exemptions to the Climate Lens assessment will be made public.

While the Claims Checklist requires verification that all payment conditions are satisfied before issuing a payment, previous audit results have shown that this control may not be sufficient unless there is a robust quality assurance function in place. As the audit was being substantially completed, a new process was being implemented within the Program Information Management System (PIMS) to add a control to prevent payments from being issued before all applicable approval conditions were met, such as completing a climate lens assessment when required. This control was not tested as part of the audit, and is expected to be tested as part of a future audit of the ICIP.

As well, there are many elements that affect project cash flows over time for which the department has no control, such as the impacts of inclement weather. Deferred climate lens assessments introduce an additional element of risk to financial forecasting given that if they are not approved by the time the first payment is submitted, they would delay INFC's ability to make payment. Proper monitoring is an essential tool within the Department's control to mitigate this risk.

This audit did not validate the quality of the Climate Lens information or assessments as part of the testing procedures. Future work by the Audit and Evaluation Branch may examine the quality of the Climate Lens assessments, and whether it is having its intended effect.

Why is this important?

In some cases, by deferring the completion of the Climate Lens assessment, the intended purpose may not be fully achieved as the project may advance too far to incorporate any identified considerations from the Climate Lens assessment into the project. For example, a project where the Climate Lens assessment is a project eligibility requirement was submitted in July 2018 and approved in September 2018 with a condition to complete the Climate Lens assessment before payments would be made. As of February 2020, while no payment had been issued, the Climate Lens assessment was still outstanding.

Additionally, if the Climate Lens requirement is not well managed, it increases the likelihood of three risks occurring:

1. That payments could be made to PTs without having satisfied the payment conditions;

2. That INFC's ability to make payments is delayed while the Climate Lens assessment remains outstanding, thereby impacting the accuracy of forecasted cash flows; and,

3. That the national level GHG reduction of 10 mega-tonnes target may not be achieved.

Recommendation #3:

It is recommended that the ADM, POB, clarify the Climate Lens process to incentivize consideration of GHG emissions and/or climate resilience in infrastructure projects and update program guidance for its management, including deferrals. In addition, a process should be established for the public notification of approved exemptions.

Risk Management

There are several indications that ICIP's risk management approach is working well. Formal risk assessments at the program level, as well as for all PTs and projects, were completed. As well, risk mitigation strategies were identified, program and project risks had documented risk responses/mitigation measures and independent oversight controls for monitoring risks at the program level and for riskier projects have been established.

We cannot conclude on the effectiveness of the risk monitoring processes as it is still too early in the program implementation. This may be included in a future audit.

Finding #4: The program, recipient and project risks were consistently assessed and mitigated according to their risk levels, but the approvals were not well documented

It was expected that program, project, and recipient risk assessments were completed, approved, and updated as necessary.

Program Risk Assessment

The ICIP Program Risk Profile, developed with the input of a risk management expert, was approved in February 2018.

There is evidence that the ICIP program risk profile was being updated during the winter of 2018-19; however, there was no indication that those changes were finalized and approved. The MCF requires an annual update of the program risk profile.

Recipient Risk Assessment

The initial recipient risk assessments were completed, but there was no evidence of management review and approval. To rectify this issue, a signature block was added to the revised recipient risk assessment template. As outlined in the Management Action Plan (MAP) in section 7, a new template will be used for the review of recipient risk assessments, starting in the fall of 2019. The new template was not yet used by the time the audit work was substantially completed.

Project Risk Assessment

There is evidence that a risk assessment was performed for all projects under review; however, the manner in which it was captured was not consistent. Some projects included a completed Risk Assessment tool (as was expected based on the process in place at the time), and in other cases, only some risks were included as part of the project fiche. The project review checklist was the expected approval mechanism for project risk assessments, which, as noted in Finding #2, was not consistently approved. As such, 22 out of 35 projects did not have evidence of management review or approval of the project risk assessment.

Program Secretariat did monitor the quality and consistency of project risk assessments and related mitigation strategies when reviewing project fiches, and challenged the risk information section with the regional staff when appropriate. The audit found that this challenge function was being done regularly; however, Program Secretariat does not review the risk assessment tools, which would include the calculation and justification for each risk factor, and does not assess the lines of evidence used by regional analysts in the ultimate risk determination. Therefore, Program Secretariat's review is not considered sufficient to fully compensate for the risk assessment not being reviewed and approved by regional managers and directors.

Additional testing was done to determine whether project risks were assessed consistently across regions and that risk mitigation strategies were documented.

The audit sampled 18 projects to confirm that the project risks were assessed and mitigated consistently across POB regions, and for similar types of projects and risk levels. The audit confirmed the risk assessment results were consistent between regions and between similar types of projects. In general, the level of detail increased based on the risk level of the project; in other words, the risk assessment results were more detailed in medium and high risk projects, compared to low risk projects.

The risk mitigation measures were also consistently applied for projects in similar funding streams and risk levels. This was likely due to the fact that a list of possible risk responses was established in the risk assessment tool as guidance, depending on the project complexity and readiness, public sensitivity and ultimate recipient capacity.

The introduction of a peer review process for conducting risk assessments or conducting risk assessments as a team, which was a noted practice in one POB region, is a best practice that could be adopted across regions.

Why is this important?

If the risk assessment approval process is not followed, assessments may not appropriately identify and mitigate risks, which may lead to an improper level of monitoring. A risk assessment approval process also forms part of ensuring sound stewardship and the highest level of integrity, transparency, and accountability.

Recommendation #4:

It is recommended that the ADM, POB, ensure that supporting evidence exists and is available to confirm that program, recipient and project risks are approved according to the appropriate level of authority identified in the MCF to ensure proper program governance.

Outcomes-Based Approach

The Treasury Board (TB) Directive on Results (2016) requires that federal departments establish, implement and maintain a Performance Information Profile (PIP) for each designated program. The purpose is to:

• Ensure that valid, reliable, useful performance data is collected and available for managing programs;
• Assess the effectiveness and efficiency of programs; and
• Meet the performance information requirements of Treasury Board of Canada submissions, evaluations, and central agencies.

ICIP's outcomes-based approach also relies heavily on the PIP, as the expected immediate outcomes of the PIP are a key factor in project approval. Specific indicators for each outcome are provided by PTs at project submission and validated through progress reporting and completion stages. There is a requirement to submit project-specific data that demonstrates a contribution towards results. Certain projects will be required to have targets in addition to their indicators. Progress towards targets by each jurisdiction will be publicly reported on INFC's website.

Underlying each immediate, intermediate and long-term results statement in the PIP is a series of indicators meant to provide a clear mechanism to identify and report on progress towards key infrastructure results. Integrated into the ICIP indicators are a subset of eight indicators to which targets have been assigned. INFC will synthesize and aggregate project-level data to report nationally on tangible and practical results that are meaningful to Canadians.

Consultations with PTs on performance measures and indicators and related roles/responsibilities were conducted by INFC during the program design phase in Fall 2017 and Winter 2018.

The PIP has been developed considering ICIP's scope and complexity as well as its risks. The PIP is expected to be evergreen and will be reviewed and updated on an annual basis, at minimum, or as needed in real time to ensure indicators, data methodologies and data collection efforts benefit from continuous improvement opportunities.

Finding #5: Some gaps need to be addressed to have a fully comprehensive Performance Information Profile

It was expected that the PIP would include all core elements as required by TB's Directive on Results and that it would be consistently applied. While the PIP complies with six of ten TB directive requirements reviewed during this audit, four requirements were not met.

In particular, the PIP does not:

• clearly describe the target population:
• A more detailed description of the characteristics of the targeted population, including any relevant environmental and socio-economic characteristics, would contribute to measuring the effectiveness of the Program.
• Impact: It could be difficult to monitor progress and report tangible results of INFC's funding impacts on the various socio-economic groups if they are not identified in the PIP. It could also be harder to identify strong indicators and targets.
• describe a data strategy to ensure the availability of intermediate and long-term performance and outcome information:
• Assessing program performance and reporting to Canadians may be difficult if performance information cannot be consolidated at the national level with a reasonable degree of certainty.
• While intermediate and long-term data owners have been designated, the process for data collection should also be set up early in program implementation and tracked over time. Baseline data should be collected, and consistent data collection procedures should be used to enable comparisons over time.^{Footnote 2}
• Impact: INFC may not have access to all necessary performance data for assessing the effectiveness and efficiency of the ICIP and for meeting the performance information requirements of TB submissions, evaluations, and central agencies.

The audit also assessed whether the program outcomes established in the PIP were aligned with those established under INFC's Departmental Results Framework (DRF). While 69 of 72 ICIP indicators were aligned with the DRF indicators, three indicators for two ICIP outcomes (IM3- Improved access to public transit systems and LT4- Investments make Canadian communities more inclusive and accessible) were not aligned.

For example, the DRF includes as an indicator the percentage of Canadians living within 400 metres of a transit station or stop, while ICIP's indicator is limited to social and affordable housing within 1000 metres. This could lead to difficulties in reporting on performance at the departmental level.

POB is currently updating the PIP for ICIP to ensure that reliable data sources for all intermediate and long-term outcomes/indicators have been identified. With implementation of the updated PIP, there is an opportunity to better define the roles and responsibilities of different groups in INFC.

Recommendation #5:

It is recommended that the ADM, POB, with the support from ADM Policy and Results Branch (PRB), ensure that the PIP for ICIP is updated by addressing TB’s Directive on Results’ requirements that were not explicitly identified or included in the PIP in order to have a fully comprehensive PIP.

Finding #6: Performance measurement governance controls have been established

The TB's Policy on Transfer Payments (2008) states that Deputy Heads are responsible for ensuring that cost-effective oversight, internal controls, performance measurement and reporting systems are in place to support the management of transfer payments.

The audit found that governance controls were established to ensure that required performance information was collected at the immediate outcome level. At project approval, our testing revealed that for 56 of 65 projects (86%), the required data for all relevant immediate outcomes and indicators was provided by PTs and entered in IRIS.

Two of those governance controls include the establishment of oversight committees and submission of three-year infrastructure plans by PTs. The oversight committees are the formal venue for INFC and each PTto discuss progress towards achieving results, including targets. To monitor progress, INFC will consider the approach to targets proposed in the PTinfrastructure plans, the projects that contribute to targets in the infrastructure plan, and data on proposed and approved projects that are contributing to targets.

All 13 oversight committees established under ICIP held initial meetings to establish the mandate and discuss roles and responsibilities between the co-chairs. At the time of the audit, only five additional meetings had occurred where monitoring of projects would have taken place. As well, testing revealed that performance measures were discussed at two of those five meetings. It would be important, moving forward, to ensure that performance measurements are discussed regularly during oversight committee meetings to continuously monitor progress towards achieving program requirements and performance targets.

As noted under section 4.2 above, it is too early in the program implementation to determine the effectiveness of the governance controls for all levels of outcomes. We do note that the Department continues to improve the performance measurement process and that the performance measurement approach under the program represents an improvement compared to past programs.

Audit Conclusion

For the primary objective, the audit concluded that INFC has generally designed an effective review process to assess and recommend infrastructure projects for funding approval. However, there are concerns that some controls are not working as intended.

The role the OGD Committee undertook diverged from its original strategic intent, and the tools and procedures put in place to operationalize the management control framework (e.g., the risk assessment tool, and the project review checklist) were not always available or consistently followed. In most cases, the risk exposure of the control failures are partially reduced by other controls. In some cases, these also resulted in duplicative processes.

For the secondary objective, the audit concluded that ICIP's outcomes-based approach was appropriately designed to provide sufficient and accurate performance data. However, some work remains to be accomplished to better define roles and responsibilities within INFC in relation to performance measurement and the collection of data for intermediate and long-term outcomes/indicators. At the same time, PIPs are evergreen documents, and the performance measurement approach under the Program represents an improvement compared to past programs.

Recommendations can be found in Section 7: Management Response and Action Plan.

Statement of Conformance

In my professional judgement as Chief Audit and Evaluation Executive, the audit conforms to the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing and the Government of Canada's Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.

____________________________

Isabelle Trépanier

Chief Audit and Evaluation Executive

Internal Audit and Evaluation Branch, Infrastructure Canada

Management Action Plan (MAP)

As noted in the individual responses to the recommendations below, INFC's program management team began addressing elements of the recommendations when they were presented with the emerging audit findings in late 2019. Their effectiveness will be assessed by INFC's Internal Audit team as part of the monitoring of management action plans.

#	Recommendations	Management Action Plan	OPI and Due Date
1	It is recommended that the ADM, POB, reconsider the mandate of the OGD Committee in order to make better use of INFC's and OGD's resources.	Work is currently underway to reconsider the mandate of the OGD Committee POB will consult internally on a way forward. Options and a recommendation will be developed for approval. POB will implement the approved way forward. Depending on the complexity of the desired changes, the timeline for completion remains unknown. This could involve amendments to the ICIP terms and conditions on governance.	POB July 2020 TBD
2	It is recommended that the ADM, POB, review the project review checklists process for documenting the project assessment and due diligence to improve efficiency and increase effectiveness of this control. Consideration should be given to separating all of the steps completed by the regional analysts as part of their due diligence from the key controls that require management sign-off.	POB has updated the project review checklist based on the findings of this audit, taking into consideration the separation of the various steps and requirements for the analysts, managers and directors. The manager sign-off on the project review checklist includes an attestation that the manager has reviewed outcomes and indicators, risk assessment and cash flow. The director sign-off includes an attestation that the director has reviewed and approved the project fiche. Both signature boxes also now contain a comment box to note any specific considerations. The updated project review checklist was approved by the director of Program Stewardship, PI, on October 23, 2019.	POB Complete
3	It is recommended that the ADM, POB, clarify the climate lens process to incentivize consideration of GHG emissions and/or climate resilience in infrastructure projects and update program guidance for its management, including deferrals. In addition, a process should be established for the public notification of approved exemptions.	INFC is currently examining an enhanced and risk-based approach to optimize the Climate Lens in recognition of these findings and in response to the current world pandemic. The Department has already updated the Climate Lens guidance to clarify requirements and to ensure results can be incorporated into program planning. POB developed the process for review and approval of deferred Climate Lens assessments, incorporated it into the ongoing POB training and communicated it to analysts. Guidelines and processes around public notification are being developed, all while aligning with current federal priorities and realities related to the global pandemic.	POB November 2020
4	It is recommended that the ADM, POB, ensure that supporting evidence exists and is available to confirm that program, recipient and project risks are approved according to the appropriate level of authority identified in the MCF to ensure proper program governance.	Program Risk Profile POB is currently updating the ICIP program risk profile. Presentations will be made to Deputy Minister's Executive Committee and Departmental Audit Committee and approval will be sought by the ADM as outlined in the MCF. Recipient Risk Assessment As recipients' risks are reassessed in accordance with the program management control framework, a revised ICIP recipient risk assessment template will be used, which includes signature blocks to formally capture the approval of Regional Directors. Project Risk Assessment The project risk assessment process has been revised to clarify how project risk assessments are to be captured and approved. The Risk Assessment Tool will now serve as guidance for regional analysts and the assessment is expected to be captured on the project fiche, which is reviewed by Program Secretariat and approved by Regional Directors. These changes have been communicated as part of the ICIP analysts' training to ensure consistent application across all POB regions.	December 2020 Complete Complete
5	It is recommended that the ADM, POB, with support from ADM, PRB, ensure that the PIP for ICIP is updated by addressing the TB Directive on Results' requirements that were not explicitly identified or included in the PIP in order to have a fully comprehensive PIP.	POB-PI is currently working on the review of the ICIP PIP, in collaboration with ADM, PRB, and in consultation with the Head of Performance and Head of Evaluation. The results of the review would be presented at PMEC in March 2020. The PIP will be updated as per the feedback from PMEC.	April 2021

Footnotes

Footnote 1

The management control framework defines the key control points of the program and the accountability for each control.

Return to Footnote 1

Footnote 2

Impact evaluation in practice, The World Bank, 2011

Return to Footnote 2