Just-in-Time Audit of PTIF & CWWF - Payments & Reporting

August 2018

Table of Contents

Executive Summary

The overall audit objective is to assure Infrastructure Canada (INFC) senior management of the effectiveness of core controls which will strengthen the implementation of the Public Transit Infrastructure Fund (PTIF) and Clean Water and Wastewater Fund (CWWF) programs.

This just-in-time audit is the third in a series of audits undertaken on PTIF and CWWF programs. The first audit assessed the agreement approval processes, the second assessed governance, and the current audit assesses payments and reporting.

The Audit and Evaluation Branch (AEB) concluded that the PTIF-CWWF programs have processes and procedures in place to manage claims payments and reporting from Provinces and Territories. Opportunities for improvement exist to mature or change program controls and practices and ensure effective and efficient program delivery throughout its lifecycle.

  • Claims process is generally documented and clear guidance is available for INFC claims, regional and finance analysts to perform their required duties. There is a lack of documentation to demonstrate that regional analysts are following the claims process including the review of claim reasonableness. It should also be noted that some claims have been paid to recipients who had not submitted reports as per the Agreement requirements.
  • There is an identified official and complete claims information file, kept by Finance and Administration (F&A), however, it is only available in hard copy and therefore difficult to share to ensure all INFC parties have access to the right information.
  • Reporting processes and procedures have been developed and implemented with recipients. Issues were noted regarding quality and timeliness of reports.
  • Guidelines for claims and reporting were created for recipients, though some did not receive it in a timely manner or found the guidance inadequate. Recipients experienced common challenges around baseline and outcomes reporting.
  • Recipients noted good working relationships with and timely support from INFC analysts.

Three recommendations were made to improve the claims and reporting processes, reduce risk and improve effectiveness:

  1. It is recommended that the Assistant Deputy Minister (ADM) of Program Operations Branch (POB), with support from the ADM of Corporate Services Branch (CSB), assess the oversight of claim processing, including the effectiveness of current controls, and the need for secondary controls.  It is also recommended that documentation requirements and guidance to staff to support the reasonableness of claims be established.
  2. It is recommended that the ADM of POB establish a minimum level of quality required in order for INFC to accept the various types of reports (progress and outcome) and provide guidance to staff.
  3. It is recommended that the ADM of POB, with consultation from the ADM of CSB, consider implementing an electronic depository of claims documentation.

Section I - Background

In Budget 2016, the Government of Canada announced an investment of $120 billion in infrastructure over 10 years, including funding to upgrade and improve public transit systems and for investments in water and wastewater projects. Two new funding programs were introduced, which were to be completed from 2016-17 to 2018-19.

Public Transit Infrastructure Fund (PTIF)

Public transit infrastructure investments aim to strengthen communities and grow the economy. This $3.4 billion program is supporting projects that result in increased capacity, enhanced service or improved environmental outcomes. These projects are contributing to rehabilitating public transit systems, enhancing asset management, modernizing and optimizing transit systems, and supporting planning for future expansion.  This program was extended to March 31, 2020 for all recipients.

Clean Water and Wastewater Fund (CWWF)

This $2 billion program is providing critical project funding for water and wastewater infrastructure to ensure communities have access to safe, efficient, and environmentally responsible water and wastewater systems. Projects are supporting the rehabilitation of water treatment and distribution infrastructure and wastewater and storm water treatment, collection and conveyance infrastructure, as well as initiatives that improve asset management, system optimization, and planning for future upgrades to water and wastewater systems. This program was also extended to March 31, 2020 for all recipients.

Management Control Framework

The Committee of Sponsoring Organizations (COSO) of the Treadway Commission's Internal Control – Integrated Framework broadly defines internal control as "a process, effected by an entity's board of directors, management and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories: efficiency and effectiveness of operations; reliability of financial reporting; and compliance with applicable laws and regulations”.

At INFC, departmental officials developed a management control framework (MCF) for the administration of PTIF/CWWF. The control framework defines the major program risks, the controls in place to mitigate them, the expected outputs generated by the application of the controls, and accountability for each control. This information is available to all program personnel and is used to administer the program.

Relevant Prior Reports

Two just-in-time audits have been previously undertaken on the PTIF and CWWF programs. The first audit assessed the agreement approval processes and the second assessed governance. The second audit, which covered the period June 1, 2016 to March 31, 2017, identified risks related to reporting in the early stages of the programs and recommended that the federal Co-Chairs of Oversight Committees ensure that future progress and outcomes reporting is timely, complete, and sufficiently detailed. While management agreed and implemented an action plan to address the recommendation, the findings of this third just-in-time audit indicate a need to do more.

Section II - Audit Objectives and Scope

The overall audit objective is to assure Infrastructure Canada (INFC) senior management of the adequacy and effectiveness of internal controls over the administration of payments, and compliance to and effectiveness of the reporting process in place for the PTIF/CWWF Projects.

The scope of this audit assessed payments and reporting from December 1, 2016 to March 31, 2018. The following items were considered in-scope:

  • Payments
    • Processes and procedures have been developed and implemented to ensure that claims are being managed and processed effectively and efficiently and that claims are processed in accordance to compliance requirements.
  • Reporting
    • Processes and procedures have been developed and implemented to ensure that reporting is being managed and processed effectively and efficiently and that reporting is in compliance with guidelines and timelines.
    • Appropriateness of reporting as a governance tool.
    • Effectiveness of management action to address reporting issues stemming from the audit on PTIF-CWWF Governance.
  • Adequate tools, guidance and support mechanisms have been developed to provide Provinces, Territories, and recipients with required information and knowledge to complete claims and reporting requirements

Section III - Audit Approach

The audit included various tests, as considered necessary, to provide reasonable assurance on the sound management of the programs.  These tests included, but were not limited to, interviews, observations, sample-based testing, review of supporting documentation, attendance at key meetings, and analytical reviews. Further, a “just in time” auditing approach was leveraged to enable the internal audit function to report on subject matter within a shorter timeframe than under the traditional retrospective approach.

A sample-based approach was used, to ensure good coverage across both programs and jurisdictions:

  • Out of 92 claims available when testing was done, 34 (37%) claims were selected based on:
    • materiality (6 files selected were >$10 million); and
    • random sampling
  • Out of 90 submitted and approved reports available when testing was done, using a judgmental selection, 34 (38%) reports were selected given appropriate distribution across programs, jurisdictions, and timeframe:
    • 10 baseline reports
    • 17 progress reports
    • 7 outcome reports
  • Out of 13 Provinces and Territories, 4 jurisdictions were selected to have 6 Oversight Committees (OCs) provide feedback on the payment and reporting processes of the programs
    • British Columbia and Ontario: separate OCs for PTIF and CWWF
    • Manitoba and Prince Edward Island: joint OCs for PTIF and CWWF

Audit findings were communicated to the auditee to validate facts and to confirm the clarity, accuracy, and completeness of the information reported.

The audit engagement was conducted in accordance with the Treasury Board Policy on Internal Audit and the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing.

Section IV Audit Findings

Management and Processing of Claims

Figure 1: Claims Process Flowchart

Text description of figure 1.

It was expected that processes and procedures have been implemented to ensure that claims are managed and processed effectively and efficiently. Specifically, it was expected that INFC’s internal claims process is implemented in compliance with relevant requirements (e.g., Financial Administration Act, program terms and conditions), and that claims are monitored to ensure they are submitted as per Agreements and appear reasonable given known project progress.

The Agreements with Provinces and Territories (the recipients) require recipients to submit a request for payment to Canada covering the eligible expenditures for all projects, at a minimum, on a semi-annual basis. Each request must include an attestation signed by a delegated senior financial official that eligible expenditures have been incurred. Through interviews, the audit found that claims are a regular topic of discussion between INFC and the recipients; however, requests for payment were not always submitted at the minimum frequency outlined in the Agreements.

The audit found that the claims process is documented and clear guidance is available for the various parties to perform their respective duties related to claims processing (see figure 1 on page 6). The guidance provides a checklist, description of the verification steps, and includes all expected compliance elements. Analysts stated they understand what is required and believe that the overall claims process is operating effectively. Testing of claims files found that they are well reviewed by Claims unit and Finance analysts. Claims status is tracked and monitored through Program Information Management System (PIMS), Oversight Committees (OCs), and ongoing communication between INFC and recipients.

Test of Reasonableness of Claims

  • A governance structure was designed to review and approve claims, subject to the Agreement and the Terms and Conditions of the programs.
  • While regional analysts noted on the claim checklist that they had verified the claims for reasonableness, there was a lack of documented evidence across the regions to support that review, aside from their completion and signature of the claim checklist.
    • Under the claims guideline, test of reasonableness is conducted by the regional analysts and includes verification that costs claimed are consistent with project’s reported progress. Progress should be determined through consideration of various sources such as OC meetings, progress or final reports, site visits, audits, and financial reports.
    • Follow-up interviews with regional analysts and Directors suggested that this verification was primarily based on analysts’ familiarity and experiences with recipients.
    • There is a gap in the guidance for regional analysts on how to document evidence of their reviews of claim reasonableness.
  • Sample testing of claims found that the majority of claims (71%) were paid to recipients without having met all reporting requirements. Three claims (9%) were paid without the most recent six month progress report to verify project progress. While making payments without meeting all reporting requirements was an issue across all regions, making payments without the most recent six month progress report was concentrated in the Western region.
    • The Agreements indicate that Canada will not pay claims until all outstanding reports — as required under the agreement — are received and accepted by Canada (such as baseline, progress, and outcomes reports).
    • The claim checklist requires regional analysts to indicate whether all required reports have been submitted and accepted.
    • Notably, regional analysts indicated receipt of all due reports on the claims checklists despite some being outstanding and provided no documented rationale or explanation as to why this was done.
  • Reports, specifically progress reports, are a key tool to assess the reasonableness of claims. The lack of documentation to demonstrate the review of claim reasonableness by regional analysts, and payment of claims without the required reports indicates the review procedure, including Section 34 certification, is not functioning as designed.

Why is this important?

A lack of evidence to support due diligence efforts by regional analysts may leave the department unable to demonstrate appropriate oversight and management of PTIF and CWWF. This risk is amplified by the high level of employee turnover within the department over the last few years, where corporate knowledge on specific files may easily be lost without proper documentation.

Furthermore, a lack of due diligence efforts in reviewing claims against progress in reporting could lead to improper payments especially given the lack of compensating controls. There has been pressure to provide federal funding on a timely basis to support these programs and there have been challenges in receiving timely reports. In view of this and to minimize future risks, consideration should be given to the appropriateness of current controls and the implementation of secondary controls (compensatory or complementary) or redundant controls (work only if key control fails).

Record Retention

  • Claim data, such as the Provincial-Territorial attested claim summary, completed claims checklist, signed contribution payment form, and supporting financial documents is stored in three places:
    • Program Information Management System (PIMS) contains basic claim information.
    • Internal network drive contains only work completed by the claims analyst.
    • The official and only complete version of claim files are kept as physical copies by the Finance Directorate. These are not easily accessible to the rest of the organization.

Why is this important?

The lack of an established and easily accessible electronic depository for the final claim files reduces efficiency when claims need to be accessed after payment (e.g., to respond to inquiries, or in support of an audit).  Moreover, by only having a single paper copy, the organization is at a risk of being unable to demonstrate appropriate oversight in the event a claim file is damaged or lost. 

Management and Processing of Reporting

It was expected that processes and procedures exist to ensure that reporting is appropriately managed and processed effectively and efficiently. Specifically, it was expected that reporting requirements and timelines are clearly outlined in Agreements, reviewed in accordance with INFC requirements, and reporting is monitored and managed to ensure compliance with Agreements.

Originally, the Agreements required that baseline reports be submitted to INFC within six months of the effective date of the agreement, while progress and outcome reports were required on a semi-annual basis, with due dates based on the effective date of the Agreement. As Agreements were signed on different dates, reports from each recipient were due to INFC at disparate times and did not follow a consistent reporting schedule. This inconsistent reporting schedule created challenges in the roll-up and comparison of information.

INFC recently addressed this issue as part of the extension for PTIF and CWWF. Recipients were required to submit their baseline report, 18 month progress report and 18 month outcome report in order to be eligible for the extension. All Provinces and Territories satisfied the reporting requirement and were granted a one year program extension. This extension amendment also changed the reporting due dates, and going forward all recipients will have the same reporting dates: March 31 and September 30.

Reporting processes and procedures have been developed and implemented with the recipients. The Agreements define high-level requirements for all reporting types (Baseline, Outcomes, and Progress reports), as well as timelines for the submission of those reports. Furthermore, specific guidance was developed for recipients for reporting.

In April 2018, Program Operations Branch (POB)’s PTIF-CWWF working committee conducted a lessons-learned exercise on reporting. This exercise identified both areas that were working well, as well as those that required attention. That process identified similar elements as this audit, such as issues with guidance documents for the recipients, internal challenges and recipients’ feedback on outcomes reports, as well as cases of poor report quality. Key recommendations were developed that are applicable to PTIF and CWWF, as well as for other programs. The recommendations from this exercise have not yet been approved by senior management, but are planned to be presented to the Program Implementation Committee (PIC) in September 2018.

Review of Reporting

  • There is some evidence that recipient reporting is tracked and monitored, and reports are reviewed by regional analysts before being accepted.
  • While there is no formal guidance that outlines the specific process and review requirements for regional analysts, the audit found analysts felt they had a good understanding of what was expected as part of the review. Additionally, interviews found that analysts maintain close working relationships with recipients and are able to leverage that relationship in reviewing reports.
  • Progress reporting is inconsistently completed and quality issues were noted across all regions, resulting in variances in quality of reports:
    • Key quality issues noted include inaccurate dates, omitted sections, and the use of incorrect templates.
    • 4 of 17 progress reports were missing required risk assessment and mitigation sections.
    • 5 of 17 progress reports had partially completed risk assessment and mitigation sections where risk assessments were missing for certain projects, or projects had identified risks without any corresponding mitigation measures.
    • Evidence of analysts’ review and outstanding issues were noted, however, the reports were accepted by INFC without evidence of those issues being addressed.
  • The Audit of PTIF-CWWF Governance found lapses in timeliness and quality of reporting. Management response was to have reporting as a standing item on Oversight Committee meeting agendas, to thoroughly review ongoing progress and outcomes reporting, and to address any reporting deficiencies at OCs. However, while submission of reports was frequently discussed with recipients in consideration of the extension requirements, there is a lack of documented evidence that demonstrates follow up of report quality issues being done at OCs. Outstanding issues noted by analysts often remained unaddressed. Due to the significance of the quality issues, reporting issues, above and beyond timeliness, should be discussed at OCs and documented to ensure reports are at a quality that is acceptable by INFC.
  • Baseline and Outcome reports were submitted as required. Though there is variance in the level of details provided, the reports comply with agreement requirements.
  • Due to the nature of information in baseline and outcome reports, the accuracy of information provided is generally not verifiable by regional analysts. As such, it is important that other controls, such as recipient audits, be used to provide a level of assurance to INFC that information is reliable and accurate.

Late reporting

  • Reporting information is provided by the ultimate recipients to the Provinces and Territories, who review and consolidate the information before it is provided to INFC. The reporting information is captured on an excel spreadsheet, which POB analysts have noted in the lessons-learned exercise as difficult to work with and prone to error.
  • Each report contains all ultimate recipient information under the specific jurisdiction and program. This multi-party and manual process has caused challenges in timeliness of information and in variances in the information submitted.
  • The Agreements require that key reports (Baseline, Outcomes, and Progress reports) be submitted every six months, however, the audit found that there were significant delays in receiving all three types of reports across all regions. 19 of 27 reports tested were late:
    • 8 of 10 baseline reports
    • 9 of 12 six and twelve month progress reports
    • 2 of 5 twelve month outcome reports
  • A further challenge was a result of variances between when claims and when reports were submitted. It can be difficult to assess the reasonableness of dollar values against the progress information if there is a significant gap in timing between the two.

Why is this important?

Reports are a critical element of the management control framemwork for PTIF and CWWF. Issues with report quality or timeliness impact the ability of the department to assess the reasonableness of claims and to monitor project progress. These issues also impact the ability to assess risks and results to ensure achievement of program outcomes.

Tools, Guidance, and Support Provided to Provinces and Territories

It was expected that adequate tools, guidance and support mechanisms have been developed to provide recipients with the necessary information and knowledge to complete claims and reporting requirements.

A guideline was created to provide recipients with information on preparing reports and claims. This guidance includes templates and instructions for completing the baseline, outcomes, and progress reports, as well as the claims form. It also refers to the relevant sections on requirements in the Agreements. Recipients noted issues relating to timeliness and adequacy of these tools.

  • The audit found that guidance provided to recipients is structured and well-articulated.
  • Some recipients did not receive claims or reporting guidance and templates in a timely manner:
    • Some recipients received guidance and template only after having submitted a claim or a report.
    • Several recipients noted that having reporting requirements determined at the agreement phase of the program would have facilitated getting the required information from ultimate recipients.
  • Recipients’ experiences varied depending on the program and existing internal capabilities; some use systems while others use manual processes.
  • Some recipients noted inadequacies in reporting guidance provided, such as a lack of clarity and the need to develop additional training and supporting material to guide their ultimate recipients. This could be a result of late guidance or a lack of awareness of available guidance.
  • Most recipients said that they understood the requirements for progress reporting.
  • Most recipients experienced challenges completing baseline and outcomes reporting. Specifically, there were difficulties in understanding what was required and some indicators were difficult to measure.

Recipients noted the following strengths regarding claim and reporting processes:

  • The claims process is straightforward and there were no issues understanding the process. Most recipients found the claim template simple to complete.
  • The good working relationship and timely support from INFC regional analysts helped facilitate the completion of claims and reporting requirements.

Section V - Recommendations

Audit Opinion

The PTIF-CWWF programs have designed processes and procedures to manage claims payments and reporting from recipients. However, not all procedures are consistently implemented, resulting in the failure of key controls. There is a need to assess current controls to determine their effectiveness in managing program risks and weigh the necessity of implementing secondary controls. Opportunities for improvement also exist to mature program practices and ensure effective and efficient program delivery throughout its lifecycle.

Based on the key observations in the report:

  1. It is recommended that the Assistant Deputy Minister (ADM) of Program Operations Branch (POB), with support from the ADM of Corporate Services Branch (CSB), assess the oversight of claim processing, including the effectiveness of current controls, and the need for secondary controls. It is also recommended that documentation requirements and guidance to staff to support the reasonableness of claims be established.
  2. It is recommended that the ADM of POB establish a minimum level of quality required in order for INFC to accept the various types of reports (progress and outcome) and provide guidance to staff.
  3. It is recommended that the ADM of POB, with consultation from the ADM of CSB, consider implementing an electronic depository of claims documentation.

Statement of Conformance

The audit conforms to the International Standards for the Professional Practice of Internal Auditing and the Internal Auditing Standards for the Government of Canada as supported by the results of the quality assurance and improvement program.

Section VI - Management Response and Action Plan

#

Recommendation

Management Response and Action Plan

OPI and Due Date

1

It is recommended that the Assistant Deputy Minister (ADM) of Program Operations Branch (POB), with support from the ADM of Corporate Services Branch (CSB), assess the oversight of claim processing, including the effectiveness of current controls, and the need for secondary controls. It is also recommended that documentation requirements and guidance to staff to support the reasonableness of claims be established.

(Category: financial management risk)

Management agrees with the recommendation.

In support of the recommendation several actions will be undertaken.

1.a) POB will provide training to PTIF-CWWF analysts on current controls, processes and due diligence expectations.

 1.b) Regional managers will review all PTIF CWWF claims prior to processing to ensure compliance with existing controls and procedures. POB will monitor areas of non-compliance and establish appropriate responses (e.g., additional training, revised guidance or additional controls).

A report on the findings of the monitoring period will be brought to PTIF CWWF squad and to PIC.

1.c) POB, with the support of CSB, will review the findings of the monitoring process (see 1.b) and, as necessary:

  1. produce and deliver additional training
  2. update internal guidance documentation for claims processes for PTIF/CWWF to ensure clarity of processes and due diligence requirements
  3. Establish new and targeted controls to address due diligence compliance issues.

1.a
POB
September 30, 2018

1.b
POB
December 31, 2018

1.c
POB
February 28, 2019

2

It is recommended that the ADM of POB establish a minimum level of quality required in order for INFC to accept the various types of reports (progress and outcome) and provide guidance to staff.

(Category: financial management risk)

Management agrees with the recommendation.

In support of the recommendation the following actions will be undertaken.

2.a) POB will review the existing guidance and establish clear standards on what is considered acceptable in terms of quality and completeness.

2.b) POB will establish approaches and procedures for situations where there are incomplete or inconsistent quality levels within a report provided by a PT.

2.a
POB
by September 30, 2018

2.b
POB
February 28, 2019

3

It is recommended that the ADM of POB, with consultation from the ADM of CSB, consider implementing an electronic depository of claims documentation.

(Category: business process risk)

Management agrees with the recommendation.

Going forward, electronic copies (scans) will be made available so that it becomes more efficient for any party that needs to look at claim information or provide that information to someone else.  Corporate Finance will be responsible for scanning and retaining the full claim docket and save it in the appropriate folder.

In the future, as a new electronic records documentation system (GCDocs) comes online at INFC, the process will be reviewed and adapted to include electronic approvals and file storage within that system.

POB and CSB

Completed
Date modified: