Table of Contents

• 1. List of Acronyms
• 2. Program Overview
• 3. Summary of Key Findings and Conclusions
• 4. Recommendations and Considerations
• Annex A: Detailed Objectives, Scope, and Findings from Internal Audit
• Annex B: Detailed Objective, Scope, and Findings from Evaluation
• Annex C: Management Action Plan
• Annex D: Internal Audit and Evaluation Methodology
• Annex E: Estimated Value for Money of P3CF Projects

1. List of Acronyms

ADM

Assistant Deputy Minister

BN

Briefing Notes

CANCEA

Canadian Centre for Economic Analysis 

COSO

The Committee of Sponsoring Organizations of the Treadway Commission

DAC

Departmental Audit Committee

DBFM

Design, Build, Finance, Maintain

DBFOM

Design, Build, Finance, Operate and Maintain

DCC

Direct Construction Costs

FA

Financial Agreements

FTE

Full-Time Employee

GBA+

Gender-Based Analysis Plus 

IAD

Internal Audit Directorate

INFC

Infrastructure Canada

IPI

Investment, Partnerships and Innovation Branch

IT

Information Technology 

MC

Management Committee

MCF

Management Control Framework

O&M

Operations and Maintenance 

OPI

Office of Primary Interest

P3

Public-Private Partnership

P3CF

P3 Canada Fund

PIMS

Program Information Management System

PO

Program Overview

RA

Recipient Audit

RASCI

Responsible, Accountable, Supporting, Consulted and Informed

SCC

Substantial Completion Certificates

SOP

Standard Operating Procedure

TB

Treasury Board

Ts&Cs

Terms and Conditions

VfM

Value for Money  

2. Program Overview

P3CF Description

The P3 Canada Fund (P3CF) funded infrastructure projects are undertaken through P3 arrangements. The P3CF was created through the Government of Canada's 2007 Budget Plan.

The P3CF is focused on supporting and promoting P3 projects across Canada. Funding is available to First Nations communities, provincial governments, municipal governments, and their private sector partners. Eligible projects include water and wastewater systems, public transit expansions, enhancements to the core national highway system, brownfield redevelopment infrastructure, and better local roads.

From 2008-2009 to 2017-2018, P3CF was managed by PPP Canada Inc. which was a Parent Crown Corporation^{Footnote 1} created in 2008 to spearhead the Government of Canada's efforts to promote the use of P3s within the federal government, including the management of the P3CF . Its mandate was to improve the delivery of public infrastructure by achieving better value, timeliness and accountability to taxpayers through P3s.

In 2017-18, the federal government determined that PPP Canada Inc. had fulfilled its mandate and ceased its activities as of March 31, 2018, transferring responsibility of the P3CF project Financial Agreements (FAs) to the IPI Branch within INFC.

IPI is a centre of expertise on innovative approaches to infrastructure delivery, including alternative approaches to funding, financing and procurement, as well as opportunities for partnerships with the private sector to deliver public infrastructure.

P3CF Objectives

The P3CF was designed to support the following objectives:

• Leverage private sector expertise and capital in support of efficient, innovative, and timely infrastructure investments;
• Help in managing public sector exposure to project risks, including cost overruns;
• Increase private sector capital financing for infrastructure renewal;
• Act as a catalyst for the development of the P3 market in Canada;
• Foster economic growth;
• Support a cleaner environment; and
• Promote stronger communities.

• Figure 1 - Text version

  P3CF has 25 total projects spread over 8 infrastructure categories

  	Number of Projects
  Public Transit	9
  Local Road	4
  Core National Highway System	1
  Water/Wastewater	4
  Solid Waste	4
  Regional & Local Airport	1
  Brownfield Redevelopment	1
  Green Energy	1

Source: Program document as of March 31,2021

Program Funding

P3CF funding was allocated to projects that have private sector involvement in at least two of the following five structural elements: design, build, operate, maintain, or finance. One of those elements must include operate, maintain or finance.

The maximum federal share of the total eligible cost of each project funded through the P3CF was 25 percent. The exception would be projects in jurisdictions that lack the internal capacity to manage a P3 procurement project.  In this case, P3CF funding could be up to 50 percent.

The P3CF committed over $1.3 billion to 25 projects across seven provinces and two territories. As of March 31, 2021,  the P3CF has allocated $1.2 billion under asset categories such as solid waste, public transit, water/wastewater, brownfield redevelopment, core national highway system, regional and local airports, green energy, and local roads. Among the 24 projects^{Footnote 2} transferred to INFC, one was a low-interest loan agreement, the first of its kind in Canada.

• Figure 2 - Text version

  Since March 31, 2018, the majority of funds committed under financial agreements were disbursed (Amount in $ million)

  	Value (in $ millions) of undisbursed FAs
  Program Inception to March 31, 2018	1,289
  As of April 1, 2018	841
  As of March 31, 2021	448

Source: Program document as of March 31, 2021

• Figure 3 - Text version

  Out of $1.2 billion allocated to provinces and territories, $841 million was paid as of March 31,2021

  Province/Territory	P3CF Contribution (Allocated Amount in $ million)	P3CF Claims Paid ($ million)
  Alberta	343	103
  Saskatchewan	294	292
  Ontario	126	125
  Manitoba	117	117
  British Columbia	98**	50
  Nunavut	77	77
  Quebec	68	25
  Northwest Territories	53*	N/A
  New Brunswick	52	52

Source: Program document as of March 31,2021

*The project in the Northwest Territories is currently under construction.

**  Includes a loan of $7.2 millions.  

P3CF Transfer to INFC

Prior the transfer to INFC, PPP Canada Inc. was responsible for overseeing the P3CF , which included ensuring compliance against the project's Ts&Cs. Key recipient requirements included providing advisory services to support project-level management and monitoring of activities such as mandatory compliance audits.  Recipients were also required to establish Management Committees (MCs), which are co-chaired by two members, one appointed by INFC and one by the recipient. These meet regularly and play an important role in project oversight. At PPP Canada Inc., 53 employees were assigned to work on the P3CF .

At the transfer (April 2018), INFC was mandated to ensure that all financial commitments and contractual obligations under the FAs were honored. However, no resources were allocated to INFC for internal management of the program. When the program was assigned to IPI, the Branch was able to staff three former PPP Canada Inc. employees to be responsible for the administration of the FAs.

At that time, there were no additional project intakes, and the management role of P3CF shifted to program administration.

The Ts&Cs requirements expired upon the program transfer from PPP Canada Inc. to INFC, leaving only the requirements set out in the projects' FAs. Although the program was not subject to the TB Policy on Transfer Payments, INFC is required to provide due diligence on the FAs, payments and project monitoring and evaluation.

As a result of the transfer of the P3CF from a Crown Corporation to INFC, the inherited documents and management practices did not always align with the usual INFC programmatic approaches. This includes the absence of a program performance measurement strategy, which would outline the program's intended outcomes and approach for demonstrating results. Since such a strategy was not established when the P3CF was developed, there was a subsequent lack of information on P3CF projects' performance results.

• Figure 4 - Text version

  After the novation on April 1, P3CF focused on completing projects

  	Substantially Completed	in Construction
  As of Apr 1 '18	13	11
  As of Mar 31 '21	21	3

Source: Program document

*Substantial Completion refers to the completion of the construction phase of the projects, at which point the terms of the FAs are not yet completed.

Project Governance

As per the FAs, MCs will exist up to three years after the project's substantial completion. The mandate of the MC is to:

• Monitor the recipient's performance and the progress of the project in accordance with the FA;
• Review the compliance audit plan;
• Facilitate the conduct of the audit and review audit recommendations and any impact on the FA; and
• Prepare and submit the annual progress reports, establish subcommittees, as needed.

All decisions and recommendations of the MC must be unanimous and recorded in writing through a record of decision.

• Figure 5 - Text version

  There is a wide range of variation in the number of years between when a project is substantially completed, and its management committees expire.

  Categorie	Maximum number of years before management committee expiration
  Water/Wastewater	3
  Solid Waste	3
  Public Transit	3
  Local Roads	3
  Regional & Local Airports	2
  Core National Highway System	2
  Brownfield Redevelopment	1

Risk Assessments

A Program Overview (PO) document and supporting tools are in place and assist in the identification, prioritization and communication of risks. They also support the tracking of payment timing to ensure payments are made upon substantial completion of projects or other key milestones, and that lessons learned are documented to support the continuous improvement of the process. In addition, through communications with recipients, the P3CF team informally tracks mitigating risks of novated projects.

Although advisory services are no longer a program requirement, the P3CF team provides ad hoc support to recipients by referring to past guides and policy decisions. It also connects recipients with each other to share solutions and best practices to mitigate risks.

Project Monitoring

The current process for project monitoring includes site visits (performed virtually during the pandemic) and the review of the substantial completion certificates signed by independent engineers to confirm project progress updates. As per the audit clause of the FA, "INFC may, at its expense, audit all accounts, records and claims for reimbursement relating to the Project and may undertake reviews."Moreover, "INFC may at any time, at its own expense, conduct an audit of actual costs claimed by the recipient as eligible costs under the Agreement. If such audit reveals an underpayment by INFC, it will promptly remit the amount of such underpayment to the recipient and if such audit reveals an overpayment by INFC to the recipient, the recipient will repay the amount of such overpayment to INFC."

P3CF Reporting

As per the FAs, recipients are responsible for providing an annual progress report to INFC no later than June 30th in each year during the term of the agreements, up to and including the fiscal year following the completion date. Each annual report shall include, among other requirements, a general description of the project focusing on major achievements, detailed summary information on the project's progress, and an update of project economic benefits achieved during the applicable fiscal year. All 24 P3CF legacy projects are scheduled to be substantially completed by 2023. However, their reporting requirements will continue until 2049 with an average of 6 reports required per year, starting with 19 reports due in 2023.

• Figure 6 - Text version

  All 24 P3CF legacy projects, inherited by INFC in 2018, are scheduled to be substantially completed by 2023. However, INFC must process and analyze recipients' annual reports until 2049.

  Year	# of Projects Submitting Reports
  2018	21
  2019	21
  2020	21
  2021	21
  2022	21
  2023	19
  2024	18
  2025	18
  2026	17
  2027	17
  2028	17
  2029	17
  2030	17
  2031	17
  2032	16
  2033	16
  2034	16
  2035	15
  2036	15
  2037	14
  2038	14
  2039	14
  2040	13
  2041	12
  2042	11
  2043	11
  2044	10
  2045	7
  2046	4
  2047	3

At the project intake phase, provinces, territories and municipalities were required to assess a range of procurement options, including an assessment of a quantitative and financial analysis to determine whether a P3 approach would provide better value for money^{Footnote 3} than a traditional procurement approach. Recipients were required to public the value for money reports.

3. Summary of Key Findings and Conclusions

P3CF has elements of an effective long-term project management framework^{Footnote 4} in place to support the successful delivery of the 24 legacy projects inherited by INFC following the dissolution of PPP Canada Inc. 

Elements of a sound program management structure are in place to effectively achieve P3CF 's objectives and outcomes, but some improvements are required to strengthen the long-term project management framework. (Key Finding #1)

The risk management processes currently in place for the identification, assessment and mitigation of project and program risks are not always well documented and should be followed consistently for all P3 projects so that a risk-based monitoring approach can be implemented. (Key Finding #2)

P3CF has put in place some compensating controls to adequately monitor the successful delivery of the P3CF projects. Further improvements could benefit P3CF and improve the monitoring of ongoing projects and future programs. (Key Finding #3)

P3CF is aligned with the Government of Canada's priorities, supported municipalities, provinces and territories in meeting their needs, and has made progress towards achieving its objectives.

P3CF was aligned with federal government priorities and helped provinces and territories invest in green infrastructure, public transit, highways and roads, and brownfield redevelopment infrastructure. (Key Finding #1)

P3CF has contributed to leveraging private sector financing for building infrastructure projects. P3CF projects were facilitated by public-private partnerships, where recipients benefited from the financial resources of, and transfer of risk to, the private sector. P3CF helped in managing public sector exposure to project risks, including cost overruns. P3CF has contributed to increased private sector capital financing for infrastructure renewal (Key Finding #2)

P3CF supported P3 market development, particularly in inexperienced jurisdictions by providing advice, tools and guidance that helped municipalities navigate the P3 model and develop capacity to pursue P3 projects in the future. (Key Finding #3)

Identifying communication processes and practices could help adequately support recipients' project management needs and could have supported the project development, particularly in Northern communities. Supplemental support following the transfer to INFC could have contributed to improved communication between recipients and INFC. (Key Finding #4)

While P3CF has leveraged private sector capital in support of efficient infrastructure investments, a lack of data following substantial completion prevents a full assessment of their performance. (Key Finding #5)

Overall, P3CF project delivery time was shorter than the Canadian average for P3 projects. Although some experienced delays, these were comparable to traditionally procured projects. (Key Finding #6)

4. Recommendations and Considerations

Recommendations

Given that the P3 Canada Fund will soon be fully entering the Operations and Maintenance phase of the program life-cycle for the next 20-30 years, with the last three outstanding legacy projects nearing substantial completion over the next year, we recommend that:

• IPI document a complete risk-based forward-looking management control framework, incorporating existing documented processes (e.g., Program Overview and Standard Operating Procedures),and in collaboration with the Grants& Contributions Centre of Expertise, Corporate Services Branch, re-evaluate the program risks and include the following key elements^{Footnote 5}:
  1. Established and understood effective oversight procedures;
  2. Clearly defined and communicated roles and responsibilities;
  3. Risk management that includes senior management's risk tolerance and mitigation measures; and
  4. Updated relevant monitoring controls, such as a centralized repository of project information, including the conduct of recipient audits (where applicable) and lessons learned. This would be conducted with the objective of ensuring operational efficiencies and that cost-benefits are achieved in terms of establishing appropriate monitoring controls over the long-term management for this legacy program, which should be aligned to the program's and recipients' risk profiles as a best practice.
• IPI document and implement a performance measurement strategy that aligns to the existing program structure and available financial and performance information, given the requirements of the FA. This will enable effective long-term results tracking and reporting on P3CF 's projects over the full monitoring period.

Considerations for future programming

Going forward, INFC should ensure adequate resources are assigned within the Department and a formal integration process (e.g., program management guidance, training and tools) is provided to program delivery staff when existing programs are transferred from Other Government Departments or Crown Corporations, including:

• Design and delivery: defining/identifying communication processes and best practices that could adequately support the recipients' project development and management needs, particularly in Northern communities; and
• Accountability: developing management control and performance measurement strategies upon program transfer to help effectively monitor and assess the achievements of program objectives and outcomes.

Annex A: Detailed Objectives, Scope, and Findings from Internal Audit

Internal Audit Objectives and Scope

Objective

The objective of the internal audit component of this engagement was to assess the effectiveness of the long-term project management framework, specifically the design and implementation of monitoring and oversight controls.

Sub-Objective #1 – Governance 

To determine whether there is an appropriate program management structure in place to effectively achieve P3CF 's objective and outcomes.

Sub-Objective #2 – Risk Management

To determine whether risk management processes are in place to identify, assess and mitigate program and project risks, are effective, and are sufficiently documented.

Sub-Objective #3 – Monitoring and Resourcing 

To determine whether INFC has in place an adequate ongoing monitoring plan and adequate support to achieve the successful delivery of the legacy P3 projects inherited.

Scope

Based on preliminary research and the risk assessment exercise, conducted by the IAD team as part of planning for this engagement, the scope of the P3CF Internal Audit included the 24^{Footnote 6} projects transferred to INFC at the time of its dissolution (March 2018) and the period covered was between April 1, 2018, and March 31, 2021.

Statement of Conformance

The internal audit portion of this combined engagement conforms to the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing, as supported by the last results of the Quality Assurance and Improvement Program.

A detailed approach can be found in Annex D.

Key finding 1: Elements of a sound program management structure are in place to effectively achieve P3CF 's objective and outcomes, but some improvements are required to strengthen the long-term project management framework.

Sub-Objective #1: Governance

To determine whether there is an appropriate program management structure in place to effectively achieve P3CF 's objective and outcomes.

Internal Audit Criterion: 1.1 Program oversight processes are in place and if applicable, aligned with the Program Guide and FA.

What is in place and working well?

A Program Overview (PO) document as well as Standard Operating Procedures (SOPs) were developed by the P3CF team to facilitate transition following the novation of P3CF Funding Agreements in March 2018. They provided an overview of the P3CF transfer; the remaining obligations of the P3CF to be governed; outlined the due diligence on the FAs required including MCs, Value-for-Money reports, site visits, scope-change requests, payments, project monitoring and evaluation, and compliance audits; and were updated to include progress-to-date and next steps. An Action Plan was also developed by IPI to list key processes and templates to be developed (including working with IT to assess whether adding P3CF in PIMS for project tracking was feasible at the time of novation) and some of those were completed (e.g., Compliance Audit requirement BN to ADM, IPI, and claims process and checklists). Some templates were completed at the time of the review: MC templates, claims process checklists and Compliance Audit requirement briefing note template for INFC's senior management approval.

What is missing and why it is important?

While the PO listed the key requirement processes for managing the P3CF , it did not outline the roles and responsibilities for each process, nor risks and control measures that would normally be outlined in a Management Control Framework (MCF) found in existing INFC programs nor was it officially approved by the Assistant Deputy Minister, IPI Branch. Some key components of the Action Plan also were not completed e.g., project tracking requirements and formal standard operating procedures with roles and responsibilities. The purpose of having these key elements of an MCF^{Footnote 7} in the PO is to ensure that P3CF has the necessary practices and procedures in place to support the achievement of program objectives, to mitigate known risks to the department and allow for transfer of corporate knowledge.

What should be in place?

The PO in place should further cover all key roles and responsibilities for the delivery of P3CF and monitoring/oversight processes, including formal approvals, documented decision-making, regular updates and sharing of all foundational documents (e.g., SOP s) with key stakeholders. The PO should also outline the program controls that are in place and describe their function to illustrate how risks related to P3CF are identified and mitigated by the department. The MCF should answer several key questions: How are major risks to the program being managed? What controls are in place to mitigate these risks? What outputs are generated by the application of these controls? Who is accountable for these controls? How do the major program risks link to the risk categories in INFC's Corporate Risk Profile?

Internal Audit Criterion: 1.2 Oversight bodies are in place to govern portfolio management of projects, specifically MCs, and are operating effectively.

What is in place and working well?

At the time of the review, the P3 team included three former PPP Canada Inc. employees that had the level of expertise required to manage more complex projects, along with the knowledge and history of the P3 projects inherited by INFC. Although no longer required to provide advisory services, the IPI team demonstrated they were available to meet with recipients as needed to assist them in addressing emerging issues. This included referring recipients to relevant policy papers, connecting them to other recipients with similar situations and strategically meeting with recipients for projects nearing substantial completion to ensure the timeliness of payments.

What is missing and why it is important?

While ad hoc oversight meetings did occur, they were not found to be aligned with all the FA's requirements. The MC clause to establish the committees, nominate co-chairs, hold regular biannual meetings, follow official agendas and maintain records of discussions were not in place for any of the novated projects. The intent after novation was for INFC to carry out oversight meetings as outlined in the MC clause. However, due to limited resources, the P3CF team used targeted meetings instead of regular biannual meetings to monitor the progress of active projects and to deal with any emerging issues. The absence of a formal MC process does not comply with the current FA requirements and increases the possibility of risks related to P3 projects going unnoticed.

What should be in place?

Regular (i.e., biannual), documented and implemented processes related to MCs (e.g., Establishment of MCs, Co-Chairs nominations, Agendas and RoDs which are well documented and approved appropriately) to comply with the FA requirements. However, IPI senior management, based on available resources and the legacy nature of the program, could consider the required frequency of MCs from  biannual to targeted, using a risk-based approach, for the remaining P3 projects that would still require MCs.

Key finding 2: The risk management processes currently in place for the identification, assessment and mitigation of project and program risks are not always well documented and should be followed consistently for all P3 projects to facilitate the implementation of a risk-based monitoring approach.

Sub-Objective #2: Risk Management

To determine whether risk management processes are in place to identify, assess and mitigate program and project risks, and whether they are effective and sufficiently documented.

Internal Audit Criterion: 2.1. Processes are in place to identify, assess, mitigate and monitor project and program management risks.

What is in place and working well?

The P3CF team maintained positive relationships with key stakeholders, and when projects faced emerging risks (e.g., potential construction delays), the P3CF team assisted the recipients with mitigation strategies. Measures included reaching out to P3 partners and Legal Services and following up to ensure the issues had been resolved. Furthermore, at the program onset the team at PPP Canada Inc. performed and documented risk analyses of all significant projects to ensure suitable P3 projects were selected for funding. The P3CF team used the results of these conducted-risk workshops to support the monitoring of project-level risks and projects that were reaching successful completion, on time and on budget.

What is missing and why it is important?

The audit team found there was an absence of documented ongoing project risk management in the project files. While emerging risks have been managed effectively by the P3CF team, the documentation for the communications with recipients is inconsistently retained and therefore poses a risk that historical records are not available for audit purposes, if required. Although a program-level-risk assessment was completed at the onset of the program (by PPP Canada Inc.) the P3CF team has not since updated it to reflect changes in the delivery model of the program now at INFC. The documentation and ongoing monitoring of program and project-level risk assessments prepares the management team for unexpected risks, thereby minimizing risks associated and extra costs before they materialize. Revisiting the program risk.

What should be in place?

Formalized and consistent documentation of program and ongoing monitoring of project-level risks, including the collection of recipient data on their risk assessments, and any results of risk management exercises or decisions including risk tolerance and risk mitigation measures. Given the legacy nature of the program, the risk assessment exercise should include consideration for operational efficiencies and the cost benefits of monitoring activities based on program and project-level risks.

Key finding 3 : INFC has put in place some compensating controls to adequately monitor the successful delivery of the P3CF projects. Further improvements could benefit P3CF and improve the monitoring of ongoing projects and future programs. 

Sub-Objective #3: Monitoring and Resourcing

To determine whether INFC has in place an adequate ongoing monitoring plan and adequate support to achieve the successful delivery of the inherited legacy P3 projects.

Internal Audit Criterion: 3.1 P3CF has in place adequate monitoring activities, such as recipient audits, that are aligned with relevant legislation outlining program monitoring requirements.

What is in place and working well?

The Program Overview document combined with the knowledge of the P3CF team supported effective project-level monitor activities; Furthermore, the claim payment process in place ensured oversight of project activities; specifically, all expenses were aligned with the agreements and all covenants and other obligations had been met before issuing payments. In response to the COVID-19 pandemic, the P3CF team contacted recipients and worked collaboratively to determine the impacts of COVID-19 on P3CF  projects. The team was also very innovative in its monitoring activities using media clippings and virtual site visits. 

What is missing and why it is important?

At the time of the review, the construction phases of most projects were complete and the operations phase underway. At the operations phase, INFC entered long-term project monitoring, which continues for 30 years following the construction phase for most projects. No evidence of a formal tracking system that documented monitoring activities at the project level was found; there was no evidence that the MC / Targeted meetings are proactively monitoring P3CF key risks to improve opportunities for risk mitigation; and there was no documented program-level monitoring plan that would ensure that sufficient resources are allocated to P3CF to adequately monitor the P3CF projects at INFC over the long-term. Additionally, conducting recipient audits was previously a mandatory requirement under PC3F's Ts&Cs, but became optional under the novated FA's due to insufficient resources and lack of budget allocation to the P3CF team to conduct them. Although the P3CF  Ts &Cs expired at novation, the P3CF team acknowledged the value of completing recipient audits and that the impact of not conducting project-level monitoring activities, such as recipient audits, results in the lack of assurance regarding a recipient's compliance with the FAs and whether P3CF funds were used as intended.

What should be in place?

A project-level tracking system that documents and plans for long-term project monitoring activities, including recipient audits, as part of the INFC-wide integrated risk-based recipient audit framework. This would include allocating sufficient resources and budget to the program for conducting monitoring activities including recipient audits to reflect key practices set out in the Treasury Board Guideline on the Directive on Transfer Payments^{Footnote 8} on recent audits issued under the Policy and the Directive on Transfer Payments.

Internal Audit Criterion: 3.2 Oversight processes in place are communicated and working as intended.

What is in place and working well?

The P3CF team was aware of the monitoring required to manage the P3CF . It has been managing project oversight requirements to date, including ongoing discussions with key stakeholders on emerging project-level risks and maintaining communications with the appropriate levels of management to provide project-level updates.

What is missing and why it is important?

At INFC, the ADM of IPI is accountable for approving reports received; however, the audit team found that reports did not contain a section for approvals or documentation that approvals were made. Approvals for key program and project decisions were also missing from project files and from the current oversight processes. The audit team also noted through interviews and reviewed documents that, upon novation, an insufficient level of resources were allocated to PC3F to carry out the long-term management and oversight of the P3CF . Sufficient resourcing and good communication practices ensure that key stakeholder roles and responsibilities are aligned to program objectives.  

What should be in place?

Oversight processes and practices should be in place, documented and communicated, as part of the overall program and project-level monitoring practices and processes, to effectively support the monitoring of P3 legacy projects. However, IPI senior management should consider adjusting the approval levels required for documenting key program and project decisions and for oversight processes based on the results of a program and project-level risk assessment.

Internal Audit Criterion: 3.3 An adequate process is in place to support the FA's annual reporting requirements.

What is in place and working well?

Although a formal process was not in place to support the FAs' annual reporting requirements, the P3CF team does maintain an evergreen checklist to document the receipt of recipient annual reports and their contents. The completion of the reporting checklist is done throughout the year as pieces of the annual reporting contents are received by project. Each project varies in the FA reporting requirements for example one project could have to submit 9 items and another project 10 items as part of their annual reporting requirement, the items vary by project types. Although for most projects the annual reports were not submitted on time, this was especially during the pandemic, the P3CF team did reach out to affected recipients at the onset of the pandemic and were provided with project updates that included pandemic-related risks and mitigation strategies. 

What is missing and why it is important?

There were inconsistencies in the collection of recipients' annual reports. Most project files were missing annual reports as required by their FA terms. Additionally, there was a lack of a documented process for analyzing and reviewing the projects information in the annual reports. And although there was evidence of a tracking tool for the current year, there was no evidence that a tracking tool was completed for each reporting year to record the collection of all projects and all required reports, including when they were received, completeness of reports received, and any related communications between the IPI team and recipients requesting reports or providing extensions. Not having an adequate process in place to analyze the content of annual reports, and to track whether all FA reporting requirements were met has resulted in some recipients not following their FA reporting requirements and INFC not having sufficient data to report results achieved at the program level.

What should be in place?

A formal centralized process for the monitoring of all required recipients' reports. This should include a tracking tool highlighting which reports are outstanding, completeness of the reports received, and any related communications between the IPI team and recipients requesting reports. The process should also include standardized reporting templates and guidelines for the storage and review of the annual reports. These guidelines should include a mention of when an exception or exemption is provided to the recipient, in cases where they are unable to submit required reports or have provided another form of project update. IPI management should also assess the reporting requirements outlined in the FAs and the information they require to report on program-level results and consider the number of reports required. Does INFC require annual reports over the next 20-30 years to effectively report on the program? This finding aligns to the evaluation limitation related to data availability to assess long-term results (slide #21) and the corresponding recommendation #2 to develop a program performance measurement strategy. 

Internal Audit Criterion: 3.4 P3CF has processes in place to ensure claims for payments paid by INFC follow relevant legislation and the requirements stated in the FAs.

What is in place and working well?

P3CF had processes in place to ensure claims for payments paid by INFC were made on time and followed relevant legislation. This includes written certification with respect to the substantial completion of projects and the eligibility of costs for direct construction and project implementation costs. All claim payments tested were found to be aligned with relevant legislation (e.g., Financial Administration Act and TB's Policy on Transfer Payments), as well as the payment process as outlined in the P3CF  program FAs and INFC's internal claim processes.

What is missing and why it is important?

The claims process used prior to the pandemic was a circulated claim binder that contained all required documentation and approvals necessary to release payments. Not all documents were saved in the project's electronic files, making it difficult to locate all the evidence during the review. Although all claims processed had a form of evidence that the required due diligence was in place prior to payment, the audit team found that all claim documentation submitted by recipients and final signed INFC claim documents were not always electronically scanned and saved in the individual claim file folders. Given the TBS Directive on Service and Digital^{Footnote 9}, having consistent processes such as a centralized electronic repository will ensure there is supporting evidence that all claims paid by INFC follow relevant legislation and the requirements stated in the FAs.

What should be in place?

Considering all claim payments were supported by evidence and only 3 payments remain outstanding, as best practice for future programs, a standardized information management practice should be in place, in coordination with Finance. This would ensure all required claims documentation submitted by recipients, evidence of the INFC analyst due diligence review and final signed section 34 claim documents are electronically saved in the individual claim file folders. This coordinated process could include an internal checklist to ensure claims for payments paid by INFC follow relevant legislation and that all evidence is on file before releasing the payment. 

Annex B: Detailed Objective, Scope, and Findings from Evaluation

Evaluation Objective, Scope, and Methodology

Objective

In alignment with the 2016 Treasury Board Policy on Results, the objective of the evaluation component of this engagement is to assess the relevance, progress towards outcomes, and design and delivery of the P3CF . The evaluation focused on the P3CF objectives related to supporting and promoting P3 procurement models for infrastructure renewal and took into consideration Gender-Based Analysis Plus (GBA+). Longer-term objectives such as fostering economic growth, supporting a cleaner environment, and promoting stronger communities could not be evaluated, due to a lack of performance indicators and related data.

Scope

The evaluation covers the timeframe from the inception of the P3CF on April 1, 2008, to March 31, 2021. Given the timeframe considered, all 25 P3CF projects are included in the evaluation scope, including the one project (the AMT Lachine Transit Maintenance Facility project) that was not transferred to INFC in 2018 because it had been completed. 

Methodology

The evaluation used multiple lines of evidence to gather information in support of the findings. This included a review of program documents,  program data, and external data, a literature review , and key informant interviews. A detailed methodology can be found in Annex D.

Evaluation Questions

The evaluation questions were developed based on a review of program documentation and objectives.

1. To what extent was the P3CF relevant for Canadians?
   
2. To what extent does the P3CF support the achievement of its expected outcomes?
   
3. To what extent is the design and delivery of P3 Model effective and efficient?
   
4. To what extent are  inclusiveness and accessibility part of the P3CF design and delivery?

Evaluation Limitations and Mitigation Strategy

Limitations encountered during the evaluation include:

Absence of performance measurement and information gathering mechanism: P3CF  was created under PPP Canada Inc., a Crown Corporation. A performance measurement strategy^{Footnote 10}, including performance indicators and targets to assess project results, was not in place for the program at that time. Data was not collected against some project data elements, such as project final cost. As a result, the evaluation could not assess P3CF 's on-budget performance, or the extent to which the P3CF objectives were achieved. This includes the longer-term outcomes of fostering economic growth, supporting a cleaner environment, and promoting stronger communities. This limitation is the subject of the second recommendation of this report. The evaluation took this limitation into account, focusing on assessing the P3CF objectives related to supporting and promoting P3 procurement models for infrastructure renewal.

Program data and information consistency and accessibility: While files created with information on each P3CF project provided information to conduct the evaluation, there was no centralized and easily accessible repository including updated financial and performance information on the projects funded, which would allowed the department to track achievement of program objectives and outcomes. For each project, the same type of information was found in different document types, saved in different places, or reported differently. The evaluation worked with the program team to validate the available data and information to ensure its reliability and consistency. 

Lack of data on Gender Based Analysis Plus: The P3CF did not require inclusiveness and accessibility considerations in the design and delivery of its projects, and GBA+ was not a requirement in the development of the P3CF . Therefore, no related data was collected. Based on this lack of data, an assessment that takes GBA+ into account was not possible. 

Key Findings

Key finding 1: P3CF has supported municipalities, provinces and territories to fund infrastructure that meets their needs.

Several factors facilitated recipients' choice to use the P3 model: 

• Recipients were generally supportive of the P3 approach and indicated that federal support was needed. Some of them mentioned pursuing P3 projects beyond those funded by P3CF .
• While the P3 procurement process was new to some recipients, P3CF was aligned enough with recipients' own procurement processes to address their needs.
• Market appetite, strong examples of P3s in some provinces, recipients' internal P3 policies, and limited ability to access and qualify for alternative funds were determinants in recipients' choice to use the P3 model.

P3CF is aligned with GoC priorities:

• The objectives of P3CF of  to achieve achieving more efficient project delivery (on time and on budget) aligned with GoC priorities related to public investment in infrastructure at the time the program was developed. 
• After the 2008-09 recession, public savings and private-sector spending were prioritized, as well as quick infrastructure investments as for economic stimulus. 
• The implementation of various types of public infrastructure projects funded under the P3CF aligns with the GoC's more recent priorities of supporting public transit, green infrastructure, and social infrastructure.

• Figure 7 - Text version

  P3CF has allocated $1.229 billion to provinces, municipalities and territories over 8 infrastructure categories

  	P3CF Allocated Amount ($ million)
  Public Transit	602
  Core National Highway System	155
  Local Road	169
  Water/Wastewater	109
  Solid Waste	79
  Regional & Local Airport	77
  Brownfield Redevelopment	29
  Green Energy	7

Source: Program document as of March 31, 2022

Key finding 2: Projects were facilitated by public-private partnerships, where recipients benefit from the financial resources of, and transfer of risk to, the private sector.

P3CF helped in managing public sector exposure to project risks, including cost overruns.

• The most common procurement model is Design-Bid-Build, where  governments are directly involved in the design phase; tenders the construction; is responsible for any design flaws or cost overruns and has little control over timelines and completion date. During operations, the performance of the asset is typically the responsibility of governments.
• In the standard P3 procurement model, DBFOM, the private partner is responsible for all or most phases of the lifecycle; assumes most risk between the phases; is fully accountable for delivery and is incentivized to produce the most effective result over the lifespan of the asset. Issues such as poor design, substandard construction or inadequate or deferred maintenance become the responsibility of the private sector.
• P3s enabled improved risk management for all parties involved by transferring the risks to the parties best suited to manage them.  Across the P3CF projects, risks transferred to the private sector were typically:
  • building and construction
  • design
  • financial  (e.g., related to cost overruns or delays)
  • maintenance
  • operating
    
• Risks that were shared between the public and private sectors varied based on the project. Some commonly shared risks include utilities and environmental (site) conditions and considerations.
• Literature suggests that while the public sector may be paying more than the cost of the project phases under the P3 model, not only is it not taking issue-related additional costs, it also has access to the financial resources of the private sector not available to municipalities.

P3CF has contributed to increased private sector capital financing for infrastructure renewal^{Footnote 11}

P3CF was a part of an increasing trend in private spending on infrastructure since 2009, although a slight decline was noted in 2016.

• Figure 8 - Text version

  Private Spending in Infrastructure 2009-2017

  Year	Private Investment($ Billions)
  2009	14
  2010	17
  2011	21
  2012	21
  2013	22
  2014	27
  2015	28
  2016	26
  2017	27
  2018	21
  2019	26

Source: Statistics Canada, Infrastructure Economic Accounts Accounts

Key finding 3: P3CF supported P3 market development, particularly in inexperienced jurisdictions.

Recipients gained capacity in P3 projects

• Inexperienced participants refer either to jurisdictions without prior experience in the P3 market, or projects in categories new to the P3 model, such as wastewater and solid waste. 
• Of the 25 projects funded by P3CF , 17 were considered inexperienced participants. 
• Recipients built capacity and gained P3 experience through participation in the P3CF process.
• Eight (8) out of the 25 P3CF projects were recognized by the Canadian Council for Public-Private Partnerships for Innovation and Excellence in P3 projects.

Interview respondents reported gaining P3 experience through their P3CF projects

• Respondents in inexperienced jurisdictions reported that the municipality guide, business case template, other program tools and guides, documentation of lessons learned, and other program resources were helpful. These resources allowed jurisdictions like municipalities to:
  • Minimize cost and report to provinces and territories with greater ease; 
  • Navigate the procurement process; Adopt some of the federal guidelines at the local level; and,
  • Receive support to navigate processes across the stages of a P3 project.
• In jurisdictions where the P3 process was well developed, respondents found existing supports outside of P3CF to also be helpful.  This included using provincial P3 structures and guidelines, sharing tools and knowledge through existing connections with municipalities, and sharing advice with industry organizations, such as transit authorities.  Further, some jurisdictions reported pursuing P3 projects beyond those funded by P3CF .
• Other factors that respondents mentioned as facilitating a successful project included hiring external experts and keeping their own advisors on the projects. Program documents indicate that technical, legal and financial experts were on the P3CF project procurement teams, providing expert advice through the P3 process.

Supplemental support was required following the transfer to INFC

• Respondents reported that the transition of projects to INFC was not always seamless: communication, roles and responsibilities were not clear.
• Respondents noted that they needed to access advice through other connections. For some, those connections were facilitated by INFC, while for others they used connections that existed before the involvement of INFC.

Key finding 4: Federal support of P3s instilled confidence in the process for recipients despite P3CF design and delivery processes were not always aligned with recipient processes

Federal support of P3s instilled confidence in the process for recipients and private sector

Document review, interviews and literature review revealed that:

• Private sector stakeholders view federal government support as important to reducing perceived risks associated with the P3 model, as federal involvement provides an increased sense of comfort when bidding on projects. 
• With federal investment, private sector stakeholders are more confident that the organizations they are working with will have sufficient resources and support to procure projects properly.
• The regular meetings and support provided at the federal level under the P3CF FAs helped address potential challenges that may have arisen during the project construction phase. 

More alignment between the program and particular project challenges during the project development process would have benefitted some recipients

• Interviewees highlighted a lack of understanding within PPP Canada Inc. of the unique needs faced by communities, specifically in the North, such as their climate, small community populations, and remoteness, which can lead to delivery issues.
• Interviewees explained that the applications and documentation requirements provided in the P3CF templates were not flexible nor sensitive  with Northern or Western Canadian individual needs.  
• Some interviewees also indicated the templates were too general in nature for use for all types of infrastructure projects, including transit projects. 

Key finding 5: While P3CF has leveraged private sector capital in support of efficient infrastructure investments, a lack of data following substantial completion prevents a full assessment of their performance.

P3CF projects reported estimated value for money comparable to those achieved in other P3 projects.

• Determining the estimated VFM involved holding risk "workshops"  where consultants and experts on behalf of provinces, territories, municipalities and transit agencies calculate the risk-adjusted costs of delivering a project. These costs become the cost of the P3 delivery in the calculation of the project VFM.
• The 25 P3CF projects were estimated to provide an average of 15.5% VFM.
• A comparative VFM analysis from 79 Infrastructure Ontario projects (from 2007 to 2020) found a value for money average of 13%. 
• According to the Canadian Centre for Economic Analysis (CANCEA), P3 projects were estimated to provide an average of 24% VFM across Canada.
• Value savings from P3CF projects range from $2.4 million to $574 million.

• Figure 9 - Text version

  P3CF projects demonstrated estimated value for money across project categories

  P3 project asset category	Capital Cost ($ millions)	Value for Money ($ millions)
  Public Transit	2,940	1,357
  Local Road	696	525
  Core National Highway System	1,219	380
  Water/Wastewater	487	124
  Regional & Local Airport	192	100
  Solid Waste	365	36
  Brownfield Redevelopment	122	5

Key finding 6: P3CF has leveraged private sector capital in support of timely infrastructure investments. Although some projects experienced delays, these were comparable to traditionally procured projects.

• P3CF projects demonstrated shorter delivery time, an average than three years, than the Canadian average for P3 projects. CANCEA reported th at Canadian P3 construction projects take on average six years to deliver, or eight years when including the procurement process.
• INFC's conventional projects of a similar scope to the PC3F projects also had an average delivery time of about three years:
  • 60% (21 out 35) of these conventional projects were completed behind schedule, compared to 55% (12 out of 22) of substantially completed P3CF projects. For those 12 P3CF projects, delays were less than one year.
  • Solid waste and public transit projects tended to experience more delays due to unforeseen events.

• Figure 10 - Text version

  The timeliness of P3CF projects was comparable to INFC's conventional projects

  	As planned	Delayed
  INFC Project	40%	60%
  Local Road	45%	55%

Source: Program document and Infrastructure Financial Report

• Figure 11 - Text version

  Distribution of projects delayed by Category

  Category	%
  Public Transit	42
  Solid Waste	25
  Water/Wastewater	25
  Brownfield Redevelopment	8

Source: Program document as of March 2021

Primary Causes of Delay

Document review and interviews indicated the common reasons for P3CF project delays included:

• Poor weather including rain and snowfall;
• Unforeseen events such as contaminated soil;
• Delay from suppliers or change in partners;
• Lack of expertise or P3 experience from contractors; and, 
• COVID-19 pandemic related delays.

Projects well-suited for a P3 have a well-defined scope and are less likely to be impacted by unexpected changes.

Key finding 7: The P3CF did not require inclusiveness and accessibility considerations in the design and delivery of its projects. Because of this and a subsequent lack of related data, an assessment of gender-based analysis plus was not possible.

The 2016 Directive on Results requires evaluators to consider government-wide policy commitments, including Gender-based Analysis Plus (GBA+). GBA+ factors were not required considerations in the development and delivery of P3CF projects.  Because of this and a subsequent lack of related data, an assessment considering gender-based analysis plus was not possible. However, evaluators examined whether the program achieved unexpected GBA+ related results.

It was found that:

• Indigenous and First Nations as well as certain vulnerable communities were consulted during the design and construction of five P3CF projects in the categories of public transit, local roads, regional airport, brownfield redevelopment and green energy. 
  • For example, directly impacted communities were consulted during the design phase of the projects to ensure alignment with cultural considerations and sensitivities. 
• Indigenous and Northern/remote communities benefited from P3CF projects.
  • The First Nations communities were direct project participants and partners in two projects.
  • Projects specifically included Indigenous and First Nations employment and training and/or hired Indigenous owned business.
  • Green energy brought consistent and reliable electricity to a First Nation community in British Columbia.
  • Projects facilitated access to northern and remote communities through spending on regional and local airports, core national highway systems and local roads.

Annex C: Management Action Plan

Management Action Plan Number 1

Recommendation

It is recommended that IPI document a complete risk-based forward-looking management control framework, incorporating existing documented processes (e.g., Program Overview and Standard Operating Procedures), and in collaboration with the Grants & Contributions Centre of Expertise, Corporate Services Branch, re-evaluate the program risks, and include the following key elements^{Footnote 12} in line with the COSO Framework:

• Established and understood effective oversight procedures;
• Clearly defined and communicated roles and responsibilities;
• Risk management that includes senior management's risk tolerance and mitigation measures; and
• Updated relevant monitoring controls^{Footnote 13}, such as a centralized repository of project information, including the conduct of recipient audits (where applicable) and lessons learned. This would be conducted with the objective of ensuring operational efficiencies and that cost-benefits are achieved in terms of establishing appropriate monitoring controls over the long-term management for this legacy program, which should be aligned to the program's and recipients' risk profiles as a best practice.

Management Action Plan

As a first step, IPI is implementing a risk-based monitoring practices: including regular engagement with project proponents at least once a year; collection of project annual reports as required in FA and lessons learned on a voluntary basis; tracking of project hand-back requirements and planning; tracking of staff transition for both public and private sector partners.
Secondly, IPI will then be updating the P3CF Program Management Guide to document the risk-based practice above which will act as the new Program Overview and Standard Operation Procedures (POSOP). The renewed document will incorporate the findings of this audit and include oversight procedures, clearly defined roles and responsibilities, mitigating measures and requirements for monitoring project information.

A Claim Process Map has been completed, which provides a step-by-step process for issuing P3CF recipient payments and will be included as part as the oversight procedures in new POSOP.

Deliverables

Program Overview and Standard Operation Procedures

OPI and Due Date

Glenn Campbell, ADM, IPI

April 2023

Management Action Plan Number 2

Recommendation

It is recommended that IPI document and implement a performance measurement strategy that aligns to the existing program structure and available financial and performance information given the requirements of the FAs. This would enable effective long-term results tracking and reporting on P3CF 's projects over the full monitoring period.

Management Action Plan

IPI is developing a performance measurement strategy that outlines the program's expected outcomes, including associated indicators that align to available data, including through the Financial Agreement requirements. The purpose for the exercise is to build capacity and database for future programs and projects.

IPI will also ensure it tracks the results/outcomes progress of projects throughout the period of the agreement.

Deliverables

Performance measurement template (including tracking tool such as project database)

OPI and Due Date

Glenn Campbell, ADM, IPI

April 2023

Annex D: Internal Audit and Evaluation Methodology

Internal Audit Methodology

In accordance with INFC's 2021-22 to 2025-26 Integrated Audit and Evaluation Plan (IAEP), the Internal Audit Directorate undertook the Review of P3 Canada Fund as part of the approved Audit and Evaluation Branch Combined Internal Audit and Evaluation engagement of the Legacy projects from P3 Canada. 

A risk-based approach was used to establish the objectives, scope, and approach for this internal audit engagement. 

Considering these risks, detailed audit criteria were developed to guide the audit field work and formed the basis for the overall engagement conclusion. 

The internal audit engagement included various tests, as considered necessary, to provide reasonable assurance on the overall engagement conclusion. These tests included, but were not limited to, interviews, walkthroughs, audit review of supporting documentation, and other audit procedures. All project files were reviewed as part of the testing procedures.

The field work was substantially completed on December 2, 2021. 

Internal audit findings were communicated to the auditee to validate facts and to confirm the clarity, accuracy, and completeness of the information reported.

Additional Reference Material

The audit criteria were also derived from the COSO Enterprise Risk Management Framework. The Committee of Sponsoring Organizations of the Treadway Commission (COSO^{Footnote 14}) is an international organization dedicated to improve management performance by developing and sharing best practices in internal control, risk management, governance and fraud deterrence.

Lines of Evidence

Multiple lines of evidence have drawn on both quantitative (e.g., administrative and financial data) and qualitative data (e.g., key informant interviews). The analytical methods used for this evaluation were tailored to the nature of the data available. The evaluation design and level of effort were calibrated with available INFC resources. The following lines of evidence were used for data collection:

• Document Review
• Literature Review
• Interviews
• Program and External Data Review

Document Review

The document review addressed the program context evaluation question of relevance and effectiveness. Program foundational documents were examined to address the relevance for the program and reports on plans and priorities, departmental plans, Budgets and Speeches from the Throne examined the question of alignment with government priorities. The annual progress reports, Value for Money reports and documents related to the financial agreements, and management committee were reviewed to assess the effectiveness and the efficiency of P3 Canada Fund (P3CF).

Literature Review

The literature review examined academic literature to get a better understanding of Public-Private partnerships, identify links between P3CF projects, P3CF objectives and outcomes, and efficient program delivery mechanisms.  It was also used to get comparison and assess the magnitude of program results.

Interviews

The key informant interviews were conducted to assess the relevance, the design and delivery, the achievement of expected outcomes and areas of improvements of P3CF and included key stakeholders such as three INFC program managers and analysts, eight representatives from the Provinces and Territories, and two other recipients such as transit agencies.

Program and External Data Review

The program and external data review consisted of reviewing administrative and financial data from P3CF project files and INFC's economic accounts.  The evaluation employed  analytical methods used by organizations such as the CANCEA and Infrastructure Ontario, to measure P3CF 's on-time performance. Data from projects' VFM reports were used for assessing whether P3CF projects demonstrated value savings.

Annex E: Estimated Value for Money of P3CF Projects

• Figure 12 - Text version

  P3 Projects are projected to save $2,527 million according to the value for money reports.

  Project	VfM ($ million)
  Evan Thomas Water/Wastewater plant	2
  Lac La Biche Wastewater Facility	3
  BC Eastside Housing Social Housing	5
  Sudbury Biosolids Waste Management Facility	11
  Surrey Biofuels Waste Facility	11
  Hamilton Biosolids Facility	13
  AMT Lachine Transit Maintenance Facility	18
  Saint John Water Treatment Facility	24
  Chief Peguis Trail Extension (Road)	32
  Pt. Sainte Charles Maintenance Facility in Montreal	50
  Tlicho All Season Road Project	53
  North Commuter Parkway (road) and Traffic Bridge	69
  Barrie Transit Maintenance Facility	89
  Saskatoon Civic Operations Centre	92
  SW Transit Way and Pembina Highway Underpass	92
  Regina Wastewater Facility	95
  Iqaluit Airport	100
  Lincoln Station – Coquitlam Transit station on Evergreen Line	134
  Go Transit East Rail Maintenance Facility	144
  Stoney Natural Gas Bus Maintenance Facility	163
  North Saskatchewan River Bridge NEAHD Road	371
  Regina Bypass Road	380
  Edmonton Light Rail Transit System	574