Internal Audit Report - Audit of the Project Review Panel

October 2014

Table of Contents

• 1. Executive Summary
• 2. Background
• 3. Audit Objective and Scope
• 4. Audit Approach
• 5. Audit Findings
  • 5.1 Governance
  • 5.2 Information for Decision-Making
  • 5.3 Project decision process due diligence
• 6. Conclusion
• 7. Statement of Conformance
• 8. Management Response and Action Plan

1 EXECUTIVE SUMMARY

Introduction in Executive Summary

The Project Review Panel (PRP) provides assurance to the Associate Deputy Minister, Deputy Minister and the Minister responsible for Infrastructure, Communities and Intergovernmental Affairs that for each project within the Minister's delegated authorities: (1) the authority to proceed is documented and justified; (2) the conditional acceptance of a project is supported by evidence and risks have been identified; and, (3) the risks identified are appropriately mitigated in the Contribution Agreement.

The PRP reviewed project information packages that were prepared and presented by Policy and Communication Branch policy analysts and Program and Operations Branch analysts related to three programs during the time period under audit: the Building Canada Fund Major Infrastructure Component (BCF–MIC); the Green Infrastructure Fund (GIF); and, the Infrastructure Stimulus Fund (ISF).

As of August 2014, the PRP is chaired by the Assistant Deputy Minister (ADM) Corporate Services and Chief Financial Officer and standing members of the Panel include the ADM Policy and Communications and the ADM Program Operations Branch. There is also an external member (non-public sector), a non-voting observer from the Audit and Evaluation Branch, and an official from the Treasury Board of Canada Secretariat who participates as a non-voting observer for projects that exceed the Minister's delegated authorities and will therefore be sent for Treasury Board approval. The Panel is supported by a Secretariat, which is comprised of personnel from the Corporate Services Branch.

The Panel will continue to operate and support the Minister under the 10-year $14 billion New Building Canada Fund. The Panel's terms of reference were updated in July 2014 to reflect new program requirements.

Audit Objective and Scope in Executive Summary

The audit objective was to assess whether the Project Review Panel functioned effectively.

The audit scope included BCF–MIC, GIF and ISF projects that were presented to the PRP between June 2009 and February 2014. Only projects that had undergone both stage one (agreement in principle) and stage two (contribution agreement) assessments were selected as part of the sample of files reviewed as part of the audit.

The audit work included conducting interviews, meeting with representatives from other government departments with similar governance bodies, examining documentation such as policies and procedures and committee terms of references, as well as, reviewing a sample of files that had been presented to the Panel during the period under audit.

Preliminary audit findings were validated by the ADM Corporate Services and Chief Financial Officer to confirm the clarity, accuracy, and completeness of the information reported.

Conclusion in Executive Summary

Overall, during the time period under audit, the Project Review Panel operated effectively. Governance mechanisms supported the functioning of the Panel; the PRP Secretariat had an established process in place to obtain information from program and policy analysts and provide complete information packages to members in advance of meetings; and, due diligence reviews supported decisions for all projects in the audit sample.

The areas identified for improvement will further strengthen the functioning of the Panel as it transitions into the implementation of the New Building Canada Fund. The audit recommendations are as follows:

• The PRP Secretariat should put in place formal training and guidance for PRP members in order to clarify expectations, processes and procedures related to the functioning of the Panel;
• The Assistant Deputy Minister Policy and Communications Branch should develop a guidance document for analysts that provide and present information to the PRP. The PRP Secretariat should review the guidance documents produced by both Branches; and,
• The Assistant Deputy Minister Program Operations Branch and the Assistant Deputy Minister Policy and Communications Branch in collaboration with the PRP Secretariat should agree on a consistent approach to the presentation of project risks to the PRP.

Management is in agreement with the audit findings and recommendations. Management has developed action plans to address the recommendations which have been included in the report.

2 BACKGROUND

The Project Review Panel (PRP) provides assurance to the Associate Deputy Minister, Deputy Minister and the Minister responsible for Infrastructure, Communities and Intergovernmental Affairs that for each project within the Minister's delegated authorities: (1) the authority to proceed is documented and justified; (2) the conditional acceptance of a project is supported by evidence and risks have been identified; and, (3) the risks identified are appropriately mitigated in the Contribution Agreement. The original PRP terms of reference were approved in June 2008 and have been revised to take new factors and lessons learned into consideration.

The PRP reviewed project information packages that were prepared and presented by program and policy analysts related to three programs during the time period under audit. These included: The Building Canada Fund Major Infrastructure Component (BCF–MIC); the Green Infrastructure Fund (GIF); and, the Infrastructure Stimulus Fund (ISF). A brief overview of the number of projects is provided in table 1.

The Panel's roles and responsibilities differed by program. For BCF–MIC and GIF projects, the PRP advised the Minister at two stages: at stage one prior to the issuance of a letter from the Minister to a proponent indicating conditional acceptance of a project (agreement in principle); and, at stage two prior to the Minister signing the Contribution Agreement. For ISF projects, the PRP provided advice to the Minister prior to the signing of a project into a schedule of the Contribution Agreement.

Table 1 - Population of projects reviewed by the PRP from June 2009 to February 2014

Program	Total Number of Projects	Total Value
BCF–MIC	123	$2.5 billion
GIF	16	$456 million
ISF	5,710 (in 282 batches)	$4.676 billion

The Policy and Communications Branch prepared information packages related to BCF–MIC and GIF projects that were presented for PRP stage one review. The Program Operations Branch prepared information packages related to BCF–MIC and GIF projects that were presented for stage two reviews, as well as for all ISF project information packages.

PRP membership has changed over time. As of August 2014, the PRP is chaired by the Assistant Deputy Minister (ADM) Corporate Services and Chief Financial Officer and includes the ADM Policy and Communications and the ADM Program Operations Branch as members. There is also an external member (non-public sector), a non-voting observer from the Audit and Evaluation Branch, and an official from the Treasury Board of Canada Secretariat who participates as a non-voting observer for projects that exceed the Minister's delegated authorities and will therefore be sent for Treasury Board approval.

The Panel is supported by a Secretariat, which is comprised of personnel from the Corporate Services Branch. The Secretariat receives and analyzes project proposals and prepares PRP Assessment Notes for approval by the PRP and the Project Assessment Form to record the assessment results. The Secretariat's briefing note to the PRP recommends that the Panel support the recommendations being made to the Minister with respect to the project being presented. As such, the Secretariat's role at the meeting relates to answering questions about its recommendation, recording decisions, and implementing the required file management structure to maintain documents and records related to the deliberations.

The Panel will continue to operate and support the Minister under the 10-year $14 billion New Building Canada Fund (NBCF) that was announced as part of Canada's Economic Action Plan. The Panel's terms of reference was updated in July 2014 to reflect new program requirements.

3 AUDIT OBJECTIVE AND SCOPE

The audit objective was to assess whether the Project Review Panel functioned effectively.

The audit scope included BCF–MIC, GIF and ISF projects that were presented to the PRP between June 2009 and February 2014. Only projects that had undergone both stage one (agreement in principle) and stage two (contribution agreement) assessments were selected as part of the file sample reviewed.

4 AUDIT APPROACH

The audit subject was approved as part of the departmental Risk-Based Audit Plan 2014-17. The approach and methodology conformed to generally accepted practices, processes, procedures and standards of internal audit in the Government of Canada, and conformed to the Treasury Board Secretariat Policy on Internal Audit and the Institute of Internal Auditors Standards for Internal Auditing. Audit criteria were sourced from the Treasury Board of Canada Secretariat, the Office of the Comptroller General of Canada, and other best practices.

The audit work was conducted in three phases - planning, examination, and reporting - with deliverables prepared at key points. The planning phase included interviews, a risk assessment and general documentation reviews (i.e. Panel terms of reference, project assessment documentation) to gain an understanding of the audit entity and its operating environment. This knowledge formed the basis for determining the areas for detailed review in the examination phase.

The examination phase included interviews with officials from INFC and other government departments, as well as file review in order to complete audit testing, and to identify and assess the significance of issues and findings. Preliminary audit findings were validated with the ADM Corporate Services and Chief Financial Officer to confirm the clarity, accuracy, and completeness of the information reported.

In order to assess the effectiveness of the project decision process, a directed sample of files that had been presented to the Panel was reviewed. Since the PRP was used for three unique programs, samples were selected from three distinct population groups. The audit sample included the following elements:

• For BCF–MIC projects – A sample of 15 out of 104 projects was selected. This included 10 INFC projects and five Transport Canada projects.
• For GIF projects – A sample of four out of 12 projects was selected based on project category and geographic location.
• For ISF projects – A sample of four (which included 22 projects) out of 282 batches (which included 5,710 projects) was selected based on the type of recipient and geographic location.

This following section consolidates the findings and presents management's responses and action plans to address recommendations.

5 AUDIT FINDINGS

5.1 Governance

Audit criterion for section 5.1

It was expected that governance mechanisms would support the functioning of the PRP. This would have included clear roles, responsibilities and accountabilities; guidance, training and tools provided to members; and, potential conflicts of interest being addressed.

Roles and responsibilities for section 5.1

The Panel's mandate was clearly outlined in successive iterations of the terms of reference and has not changed significantly since it was first put in place in 2008. Modifications to the terms of reference were made to ensure that the Panel's guiding principles are in line with policy and program requirements as well as delegated authorities.

The roles and responsibilities of PRP members and the Secretariat were clearly outlined in each version. The Secretariat is the conduit between the PRP and the departmental Branches that produce the analyses and assessments. The Secretariat is responsible for collecting information and providing it to members for decision-making purposes. The Secretariat has been effective in performing its role in supporting the Panel.

Training and guidance for section 5.1

As the Panel has progressed since 2008 and membership has evolved due to changes in the senior ranks of the Department, the terms of reference has been the only formal guidance provided to members. Individual Panel members have informally discussed their roles and responsibilities with colleagues prior to commencing their work on the Panel.

During discussions with Panel members, there was a general consensus that a more formal approach to training would be beneficial. This was deemed to be particularly relevant given the introduction of a new external Panel member and other observers as part of the revised approach being introduced for New Building Canada Fund projects. Furthermore, this approach is in line with best practices from other government departments that have similar governance bodies.

Records of decision and discussion for section 5.1

Prior to April 2014, the only records of meetings were the individual project decision notes signed by all members of the Panel. In April 2014, the Secretariat began documenting records of decision; however, these were not circulated to Panel members for approval. As of August 2014, PRP members are approving records of decision at subsequent meetings. This modification was made to capture information related to the decision-making process and to maintain a record of important discussions related to program and policy interpretations and potential areas for improvement. This change also provides documented support to panel members as part of their consideration of past decisions in making project recommendations, which will promote consistency in decision-making.

Addressing potential conflicts of interest for section 5.1

The Panel's terms of reference identifies that in cases where a PRP member is in a conflict of interest situation or perceived conflict of interest situation with a proponent, he or she will excuse himself or herself from the discussion of a project submission. This approach ensures that decisions made by the PRP remain objective and free from bias. As all Panel members were departmental officials, they would have signed a Conflict of Interest form as a condition of employment. To meet the PRP conflict of interest requirement, the practice was for members to proactively self-identify during meetings in cases where they perceived that they could potentially be in a conflict of interest situation.

During the course of the audit, the audit team was informed that there had been at least one instance where a Panel member excused themselves from deliberations due to a potential conflict of interest situation. This member did not sign the PRP assessment note. However, the potential conflict had not been formally documented in any of the files related to the deliberations.

In August 2014, the Chair instituted a new practice whereby the issue of potential conflicts of interest is now raised as a standing agenda item at the start of each meeting and documented in the records of decision. This practice has been instituted to take into consideration the changing nature of the Panel with the inclusion of external members and observers.

Recommendation 1: The PRP Secretariat should put in place formal training and guidance for PRP members in order to clarify expectations, processes and procedures related to the functioning of the Panel.

Management Response: INFC Senior Management agrees with this recommendation and an action plan has been developed in response to the recommendation.

5.2 Information for Decision-Making

Audit criterion for section 5.2

It was expected that information for decision-making would have been provided to Panel members in a timely manner in order to support due diligence and meaningful deliberations. It was also expected that the PRP Secretariat would have established procedures for receiving information to be provided to members.

The provision of information to the Panel for section 5.2

The PRP Secretariat aims to obtain documentation from presenters in a timely manner so that information packages may be provided to Panel members for review prior to meetings. The Secretariat requests that draft information from the analysts is provided at least eight days prior to meetings so that final drafts may be provided to Panel members three days prior to the meeting. Panel members indicated that documentation was generally provided in a timely manner; however, there were instances when this had not been the case. These situations were beyond the control of the PRP Secretariat.

The PRP Secretariat did not keep performance data on timeliness of information received prior to meeting dates. In August 2014, the PRP Chair, in consultation with the PRP Secretariat, instituted the practice of tracking performance in this area against the target set in the PRP terms of reference. The most recent version of the terms of reference indicates that Branches must provide final versions of documents to the Secretariat three days prior to the meeting.

Standardized process in place for section 5.2

The documents provided to Panel members are based on templates related to assessment notes and forms, briefing notes, letters and reports. The use of these templates ensures that information presented to the Panel comes in the same format for each individual project. This assists members by providing information for decision-making in a consistent manner.

While the templates ensure that documents are provided in a consistent format, the content of information for similar projects varies. This is a function of which Branch or Directorate is responsible for the information presented to the Panel. Analysts had varying degrees of information related to what was expected of them when providing and presenting information to the PRP. One factor that analysts considered important was guidance provided to them in advance of presenting information to the Panel. For example, while the Program Operations Branch has a guidance document for analysts on how information should be presented to the Panel, the Policy and Communications Branch do not. It would be beneficial for Policy and Communications analysts to have more formal guidance related to presenting to the Panel. This guidance would assist in outlining expectations, which in turn would increase awareness concerning expectations and provide more consistent information for PRP member deliberation and decision-making.

Recommendation 2: The Assistant Deputy Minister Policy and Communications Branch should develop a guidance document for analysts that provide and present information to the PRP. The PRP Secretariat should review the guidance documents produced by both Branches.

Management Response: INFC Senior Management agrees with this recommendation and an action plan has been developed in response to the recommendation.

5.3 Project decision process due diligence

Audit criterion for section 5.3

It was expected that due diligence would be performed with respect to project decisions. Specifically, it was expected that stage one project recommendations would be consistent with program terms and conditions and that stage two contribution agreements would be consistent with approved terms and conditions. It was also expected that the Panel would evaluate project review packages to determine whether key project risks were identified and mitigated.

Meetings and agenda items for section 5.3

The Panel held 92 meetings between June 2009 and February 2014. During this period the Panel assessed 172 BCF–MIC and GIF projects (84 at stage one and 88 at stage two). The average number of projects that were presented at each meeting was two. The average number of days between meetings was approximately 16. In the past, meetings were called as required; however, since July 2014, a set schedule has been established for meetings to occur every two weeks and more frequently if required.

Project approval stages – BCF–MIC and GIF for section 5.3

Most BCF–MIC and GIF projects were presented to the PRP two times — once to obtain agreement in principle status, and secondly to obtain approval of the contribution agreement. At stage one, the PRP reviewed project documentation to ensure that:

• The authority for the Minister to proceed without Treasury Board approval was documented and justified;
• The project conformed with government policy on the parameters of the program;
• The project conformed with Treasury Board policy;
• The project conformed with the program's terms and conditions;
• The risks and mitigation measures were appropriately identified; and,
• The project had a sound and appropriate governance structure.

At stage two, the PRP made a determination as to whether contribution agreements conformed to program terms and conditions and that key risks were identified and mitigated. The average time between stage one and stage two reviews were approximately 10 months, with a median of approximately 9 months.

Stage one projects reviewed

All files related to the 19 projects (15 BCF–MIC and four GIF) in the audit sample that went for stage one review were complete and the information made available to Panel members was appropriate to support due diligence reviews. Additionally, all projects in the audit sample were consistent with program terms and conditions.

Stage two projects reviewed

All files related to the 19 projects (15 BCF–MIC and four GIF) in the audit sample that went for stage two review were complete and the information made available to Panel members was appropriate to support due diligence reviews. Furthermore, contribution agreements were developed based on departmental templates and consistent with the program's approved terms and conditions. It was also noted that in cases where a contribution agreement was not consistent with the approved template, the memo from the responsible ADM to the PRP outlined the deviations from the template — specifically related to provisions added, changed or removed.

Project approval – ISF for section 5.3

For the ISF program, the Minister signed one umbrella contribution agreement with each province or territory that listed multiple approved projects in a schedule. For ISF projects, the PRP performed one stage of review prior to the inclusion of projects into the schedule. The schedule lists the names of projects that had received approval and basic information such as project location, funding amounts, as well as start and end dates. This streamlined PRP process ensured that:

• The authority for the Minister to proceed without Treasury Board approval was documented and justified;
• The projects conformed with government policy on the parameters of the program;
• The projects conformed with Treasury Board policy;
• The projects conformed with the terms and conditions of the program; and,
• The risks and mitigation measures were appropriately identified.

ISF projects recommended for approval

All files related to the four ISF batches (22 projects) in the audit sample that went to PRP for review were complete and the information made available to Panel members was appropriate to support due diligence reviews. Additionally, all projects in the audit sample were consistent with program terms and conditions.

Risk management information provided to the Panel for section 5.3

Risk and risk mitigation material was included in the information packages provided to Panel members. Policy analysts used the Priority Initiatives Risk Tool (PIRT) and program analysts used the Program Operations Risk Tool (PORT) as the basis for deriving risk rankings for proposed projects.

For projects presented for stage one review, risk information was identified in the documentation packages provided to Panel members – assessment notes and forms, briefing notes, letters and reports. There were some inconsistencies in the presentation of project risk information throughout the documents. In addition, risk information presented for similar types of projects had the same risks presented to the Panel with few deviations.

In the audit sample of projects that were presented for stage two reviews, it was noted that each of the contribution agreements indicated the risks associated with the project and potential mitigation strategies.

The PRP considered risks and mitigation strategies during deliberations for projects presented at stage one and stage two. This was evident in the documentation provided to Panel members. The area for improvement relates to ensuring project risk information provided to members is consistent throughout the information package presented for decision making purposes.

Recommendation 3: The Assistant Deputy Minister Program Operations Branch and the Assistant Deputy Minister Policy and Communications Branch in collaboration with the PRP Secretariat should agree on a consistent approach to the presentation of project risks to the PRP.

Management Response: INFC Senior Management agrees with this recommendation and an action plan has been developed in response to the recommendation.

6 CONCLUSION

Overall, during the time period under audit, the Project Review Panel operated effectively. Governance mechanisms supported the functioning of the Panel; the PRP Secretariat had an established process in place to obtain information from program and policy analysts and provide complete information packages to members in advance of meetings; and, due diligence reviews supported decisions for all projects in the audit sample.

The areas identified for improvement will further strengthen the functioning of the Panel as it transitions into the implementation of the New Building Canada Fund. The audit recommendations are as follows:

• The PRP Secretariat should put in place formal training and guidance for PRP members in order to clarify expectations, processes and procedures related to the functioning of the Panel;
• The Assistant Deputy Minister Policy and Communications Branch should develop a guidance document for analysts that provide and present information to the PRP. The PRP Secretariat should review the guidance documents produced by both Branches; and,
• The Assistant Deputy Minister Program Operations Branch and the Assistant Deputy Minister Policy and Communications Branch in collaboration with the PRP Secretariat should agree on a consistent approach to the presentation of project risks to the PRP.

Management is in agreement with the audit findings and recommendations. Management has developed action plans to address the recommendations which have been included in the report.

7 STATEMENT OF CONFORMANCE

The audit conforms to the International Standards for the Professional Practice of Internal Auditing and the Internal Auditing Standards for the Government of Canada as supported by the results of the quality assurance and improvement program.

8 MANAGEMENT RESPONSE AND ACTION PLAN

#	Recommendation	Management Response and Action Plan	OPI and Due Date
1	The PRP Secretariat should put in place formal training and guidance for PRP members in order to clarify expectations, processes and procedures related to the functioning of the Panel.	Agree. The Secretary to the PRP Secretariat will provide a briefing to all new PRP members. This briefing will include an overview of the PRP Terms of Reference (TOR) in order for the PRP members to understand their obligations as well as the differences between MIC, GIF and NBCF programs. The PRP Secretariat will also offer to schedule briefings with the Policy and Communications and the Program Operations Branches as required. This requirement has been included in the PRP TOR and the PRP Management Control Framework as an ongoing responsibility for the PRP Secretariat. An external advisor was recently appointed as a PRP member and received briefings from the PRP Secretary on July 17, 2014, the Policy and Communications Branch on August 20, 2014 and the Program Operations Branch on August 28, 2014. The new Assistant Deputy Minister Program Operations Branch was recently appointed as a PRP member and was briefed on August 12, 2014.	OPI: ADM, Corporate Services Branch Due Date: Completed
2	The Assistant Deputy Minister Policy and Communications Branch should develop a guidance document for analysts that provide and present information to the PRP. The PRP Secretariat should review the guidance documents produced by both Branches.	Agree. Assisted by these audit findings and building upon the suite of tools already utilized in the preparation of materials for project approval, Policy and Communications will develop a guide for analysts with a view to further strengthen overall consistency in the presentation of materials to PRP. This guide, to be reviewed by the PRP Secretariat, will help contextualize, and prescribe the nature of information to be presented.	OPI: ADM, Policy and Communications Branch Due Date: Guide to be prepared by December 31, 2014
3	The Assistant Deputy Minister Program Operations Branch and the Assistant Deputy Minister Policy and Communications Branch in collaboration with the PRP Secretariat should agree on a consistent approach to the presentation of project risks to the PRP.	Agree. Program Operations Branch and Policy and Communications Branch will work together on the development of a consistent approach to the presentation of project risks to the PRP. The development of this approach will take into consideration the differences in the types of risks that may present at the various stages of project development and approvals, as well as the differences in appropriate and applicable mitigation strategies.	OPI: ADMs Program Operations Branch and Policy and Communications Branch Due Date: Approach to be prepared by December 31, 2014