Evaluation of the Horizontal Governance and Reporting – Investing in Canada Plan
PDF Version (Size: 2.12 MB)
Help on accessing alternative formats, such as PDF, PPT and ZIP files, can be obtained in the alternate format help section.
Table of Contents
- Executive Summary
- Program Background
- Evaluation Context
- Major Findings
- Extent to which INFC’s leadership role supports IICP delivery
- Extent to which the horizontal governance structure supports IICP’s delivery
- Extent to which IICP reporting requirements were met
- Extent to which IICP communications supported public reporting
- Resource utilization to meet IICP reporting and governance requirements
- Conclusions and Recommendations
- Annex A: IICP Governance Structure Roles and Membership
- Annex B: IICP Program Background – Logic Model and Funding Allocation Overview
- Annex C: Methodology
- Annex D: Summary of Analysis of Horizontal Initiatives Compared to IICP
- Annex E: Acronym list
- Footnotes
“Some sections within the report were severed in accordance with Access to Information Act”
1.0 Executive Summary
Program Overview
The Investing in Canada plan (IICP, or “the Plan”) is the Government of Canada (GoC)'s comprehensive, long-term plan for building a prosperous and inclusive country through infrastructure investments. The GoC has committed more than $180 billion over 12 years to support the economy, grow the middle class, and improve the quality of life for Canadians today and into the future.
The Plan is an initiative delivered by 14 federal departments and agencies. Infrastructure Canada (INFC) is the lead department responsible for the overall governance and reporting of the IICP and supports the Minister of Infrastructure and Communities’ efforts to coordinate the implementation of the Plan and to communicate the results to Canadians. INFC has established a horizontal governance structure across the 14 partner departments to support the implementation of the Plan.
Evaluation Objective and Scope
This evaluation provides a neutral and objective assessment of the effectiveness of the horizontal governance and reporting structure put in place by INFC to support its leadership role related to the IICP. It also responds to senior management’s information needs and positions the Department to manage the ongoing implementation of the Plan.
The evaluation covers the period from April 1, 2016 to May 31, 2018. The issues that were examined include leadership, governance, reporting, resource utilization as well as communications to the public and across the federal delivery departments.
Key Findings, Conclusions and Recommendations
In terms of leadership, the evaluation found that INFC’s mandate as the lead department to perform IICP horizontal reporting was clear and that its role was understood by the 13 other government departments (OGDs) and central agencies. INFC was effective in its leadership role using the IICP Secretariat, and in creating the governance structure and outlining roles and responsibilities to its OGD partners. While OGD partners’ ability to respond to IICP demands were challenged by competing priorities, they were generally satisfied with the leadership provided by INFC.
The governance, which includes working groups and committees, was generally regarded as useful by OGDs and as an effective mechanism to share information, coordinate horizontal reporting and address cross-cutting issues. As the scope and coverage of the IICP changed, the governance structure adapted to meet the evolving context and emerging needs. INFC fostered a positive, collaborative working relationship with its OGD partners by encouraging them to reach out to the IICP Secretariat with ad-hoc questions, requests for information and seeking advice. Overall, the IICP is comparable in terms of structure with other horizontal initiatives. That said, the IICP horizontal governance structure has some components that are either inactive or without formal Terms of References (ToRs), and further improvements could be made to better support OGDs when they are required to respond to the reporting needs of the Plan, especially ad-hoc requests.
Overall reporting requirements related to the IICP were met. Challenges arose when INFC attempted to report on legacy funding from OGDs given some OGDs are not providing the requested information. Effort will also need to be made to ensure more comprehensive results can be reported. Finally, existing reporting processes and tools introduce risks related to data integrity by relying on data provided via email.
In terms of communicating the results and progress to the public, a variety of communication products were made publicly available, including the Investing in Canada Plan Project Map the IICP website and the IICP placemat. As well, various media announcements about projects associated with the IICP were made. INFC’s messaging surrounding the IICP was consistent and supported by the efforts of the IICP Communications Working Group.
In terms of resource utilization, INFC made efforts to align its internal resources with the increased workload associated with the growth in scope of the IICP and associated reporting and governance functions. This was particularly noted during the rollout of the IICP’s second phase. Resources expended by the IICP Secretariat were in line with horizontal initiatives in other departments, taking into account the differences in the initiatives. However, opportunities for improvement exist regarding INFC’s internal coordination. According to the IICP Secretariat, recent improvements in collaboration between Communications, Data and Research, and the Secretariat have produced positive results.
Recommendations
It is recommended that the IICP Secretariat:
- Review and propose changes, as required, to the governance structure to ensure all components―such as working groups and committees―have a clear purpose and articulate how they are to interact. Attention should also be placed on the processes and tools used by the various committees to better support taskings that arise concerning the IICP.
- Assess whether the improved collaboration between various INFC sectors and branches is sufficient to ensure a more coordinated departmental approach to supporting IICP reporting and governance.
- Consider new reporting tools and processes to efficiently share and view data between all OGDs, and to address risks to data quality.
A Management Action Plan that responds to these recommendations is found in Section 5.3 of this report.
2.0 Program Background
The Investing in Canada plan (IICP)
The IICP is the GoC's comprehensive, long-term plan for building a prosperous and inclusive country through infrastructure investments. The GoC has committed more than $180 billion to support the economy, grow the middle class, and improve the quality of life for Canadians today and in the future. In the initial phase of the IICP, Budget 2016 provided a foundational investment of $14.4 billionFootnote 1. Budget 2017 reaffirmed this commitment and built on the foundation with $81.2 billion of additional funding over 12 years. The IICP also includes existing legacy funding of $92.2 billion, in addition to the commitments of Budgets 2016 and 2017. The details of the funding breakdown by announcement is shown below:
Announcements |
Federal allocations ($ billion) |
---|---|
Budget 2016 |
$14.4Footnote 1 |
Budget 2017 |
81.2 |
Existing programming |
92.2 |
Total: |
$187.8 |
The IICP is a horizontal initiative involving the following 14 federal departments and agencies:
- Infrastructure Canada
- Canada Mortgage and Housing Corporation
- Canadian Heritage
- Employment and Social Development Canada
- Environment and Climate Change Canada
- Health Canada
- Department of Indigenous Services CanadaFootnote 2
- Crown-Indigenous Relations and Northern Affairs CanadaFootnote 2
- Innovation, Science and Economic Development CanadaFootnote 3
- Natural Resources Canada
- Parks Canada
- Public Health Agency of Canada
- Public Safety Canada
- Transport Canada
INFC is the lead department responsible for the overall reporting and governance of the IICP. It supports the Minister of Infrastructure and Communities’ efforts to coordinate the implementation of the Plan and to communicate the results to Canadians.
Horizontal Governance and Reporting Responsibilities
After Budget 2016, INFC was identified as the lead department for the whole-of-government reporting approach regarding the $14.2 billion in new infrastructure investments. Within INFC, the Program Integration Directorate within the Program Operations Branch was initially tasked as the lead to:
- create a reporting framework (including indicators),
- determine how data would be collected and reported to Canadians, and
- coordinate reporting with all partners.
This was done through the 2017-18 Departmental Plan (DP) exercise.
Further to Budget 2017, the scope of the horizontal initiative was broadened and a governance component was added. Beginning in April 2017, responsibility for IICP horizontal governance and reporting progressively transitioned from the Program Integration Directorate to the Policy and Results Branch. In late 2017, the Horizontal Results and Reporting (HRR) division was formed and began functioning as the IICP Secretariat.
The IICP Secretariat leads the horizontal management of the IICP, and coordinates all reporting to Parliament and Canadians on IICP results. INFC regularly requested and collected data from all partnering departments in order to provide regular reports on the results achieved across the Plan. This data was used to inform:
- The Minister of Infrastructure and Communities’ updates to the Ministerial Results Table (MRT)/Stock takes with the Prime Minister on infrastructure;
- cabinet documents related to IICP, including 1) reporting to the Agenda, Results and Communications (ARC) cabinet committee, and 2) reporting on progress towards the Government’s commitments set out in ministerial mandate letters through the Canada.ca/Results portal, led by the Privy Council Office (PCO);
- Canadians on the progress and results of the Plan through events and announcements, project signage, the IICP website, the Investing in Canada Plan Project Map commonly referred to as the GeoMap, annual reports and ad-hoc media briefings by individual departments;
- discussions at any of the committees established to govern the IICP;
- INFC’s Departmental Plans (DP) and Departmental Results Reports (DRR); and
- program audits and evaluations.
Figure 1 depicts below the IICP governance structure as of May 2018
The IICP Secretariat supports the Deputy Ministers’ Coordinating Committee (DMCC) to ensure that Deputy Ministers (DMs) are brought together to have strategic discussions, manage key and emerging challenges, and provide oversight for the Plan. The IICP governance structure also includes various working groups to coordinate IICP governance and reporting activities. This includes an interdepartmental Communications Working Group (led by INFC’s Communications Branch), an interdepartmental Program and Results Working Group (led by INFC’s IICP Secretariat), and an Information Technology (IT) Working Group (led by INFC’s IT Directorate).
Since Spring 2018, the IICP Secretariat has had several discussions with INFC’s Deputy Minister, the Assistant Deputy Minister (ADM) Policy and Results and with IICP partner departments regarding potential revisions to the IICP governance structure; a formal governance review is planned for Fall 2018.
Discussions involving the IICP Secretariat and other internal INFC services and groups (e.g., Finance & Administration; Data & Research) were noted, but were not formally documented.
Full details on the roles of each committee and membership is available in Annex A.
Program Resources
Table 2 presents INFC’s budget for the Horizontal Results and Reporting division, under the Policy and Results Branch, where the IICP Secretariat is housed. Currently, it has 10 approved full-time equivalents (FTEs) for fiscal year 2018-19. Resources that support horizontal governance and reporting in other areas of the Department are not tracked to this degree of specificity.
|
FTEs |
Salary |
Non-salary |
Total |
---|---|---|---|---|
Initial funding |
8 |
$749,984 |
$142,200 |
$892,184 |
Additional approved allocations for 2018-19 |
2 |
187,496 |
- |
187,496 |
Total |
10 |
$937,480 |
$142,200 |
$1,079,680 |
Figure 2 presents the organizational structure of the Horizontal Results and Reporting Division within INFC’s Policy and Results Branch.
3.0 Evaluation Context
Evaluation Objective and Scope
The objective of this evaluation is to provide a neutral and objective assessment of the effectiveness of the governance and reporting structure in place to support Infrastructure Canada’s IICP leadership role. This evaluation responds to senior management’s information needs and positions the Department for the ongoing implementation of the IICP.
This evaluation covers a two-year period from April 1, 2016Footnote 4 to May 31, 2018. Progress made after May 2018 was taken into account, as appropriate.
This evaluation covers all governance structures, reporting requirements, the relationship that INFC had with OGDs, as well as INFC’s resources used to support the Minister’s responsibility to coordinate reporting on the IICP’s results. In order to better define the elements examined, a logic model for the horizontal governance and reporting function of IICP, used for evaluation purposes, is provided in Annex B.
Methodology
The multiple lines of evidence used for this evaluation included both qualitative and quantitative data:
- a document review;
- a survey of the Program and Results and Communications working group members;
- key informant interviews with INFC staff and management, OGDs under the IICP, and central agencies (i.e., PCO, TBS and Department of Finance); and
- a benchmarking exercise with comparable federal horizontal initiativesFootnote 5.
The analytical methods used for this evaluation have been tailored to the nature and availability of the data in relation to the evaluation questions. The multiple lines of evidence were triangulated to substantiate findings and to minimize potential bias. For more information related to the methodologies used and limitations, please refer to Annex C.
4.0 Major Findings
The following sections present the findings related to the horizontal governance and reporting functions for the IICP.
Extent to which INFC’s leadership role supports IICP delivery
This section addresses INFC’s lead role in managing the relationship between all OGD partners and central agencies to meet reporting requirements for the Plan as a whole. It examines the clarity of INFC’s mandate, whether INFC demonstrated sensitivity to emerging needs, and the quality of INFC leadership as assessed by partners.
Finding 1: INFC’s mandate as lead department to perform IICP horizontal reporting was clear to OGDs and central agencies. Competing priorities challenged OGDs’ ability to respond to INFC requests.
Foundation documents and authorities clearly laid out the requirement for INFC to report horizontally for the IICP. They specifically articulated that the Minister of Infrastructure and Communities had the leadership role to ensure that there would be a whole of government approach to reporting and results of the IICP. Additional clarification in subsequent funding and working level documents further defined the requirements for deliverables and clarified and reinforced INFC’s lead role to OGDs.
While OGDs and IICP Secretariat staff reported some initial challenges in establishing INFC’s role and articulating the level of priority of the IICP governance and reporting among OGDs, respondents noted that INFC’s ability to lead has improved since year one. With the recent attention to the IICP that came with the Parliamentary Budget Officer’s (PBO) reportsFootnote 6,Footnote 7, the importance of INFC’s leadership role on horizontal reporting and on the IICP was further emphasized and clarified.
OGDs also confirmed that they understand the mandate of INFC to report horizontally for the IICP. Specifically, when asked if INFC’s leadership roles and responsibilities were clear and if INFC had the appropriate authorities to play a leadership role in the IICP, they tended to agree, as illustrated in Table 3 below.
|
Disagree/Strongly disagree |
Neither agree nor disagree |
Agree/Strongly agree |
Don’t know |
---|---|---|---|---|
Statement |
Counts (%) |
|||
INFC's leadership roles and responsibilities for the IICP are clear |
0 |
4 (19%) |
14 (67%) |
3 (14%) |
INFC has the appropriate authorities to play a leadership role in the IICP |
0 |
4 (19%) |
12 (57%) |
5 (24%) |
While INFC’s mandate is clear, OGDs have their own priorities and mandates that limit their ability to respond to IICP-related requests in a timely fashion. Many of the interviews (including with individuals from OGDs, INFC senior management, IICP Secretariat, and INFC’s Communications Branch) mentioned that OGDs are challenged to meet IICP reporting requirements within the requested deadlines because of competing priorities within their own departments. One respondent noted that INFC had no control over the competing priorities and, as such, could not mandate how OGDs prioritize IICP reporting.
As well, Communications Working Group members in OGDs reported being challenged by competing priorities resulting in slow response rates to INFC’s Communications Branch requests. INFC Communications also noted that engagement from OGDs could be improved. For example, when INFC requested two vignettes per initiative that would demonstrate how these projects positively impacted Canadians, 15 vignettes were provided by OGDs when INFC expected 150.
Finding 2: INFC was effective in its leadership role. OGD partners and central agencies were generally satisfied with INFC’s leadership and the operational support of the IICP Secretariat.
To fulfil its leadership role, INFC created an IICP governance structure that had multiple layers. Governance findings are covered in section 4.2. A Handbook (“IICP Handbook for Governance Horizontal Reporting”) was developed by the IICP Secretariat to assist OGDs in understanding the various roles and responsibilities. The Handbook also outlined and explained the governance structure, including the various committees and membership for each, as well as the regular horizontal reporting requirements for which OGDs were responsible. All of these helped to outline the expectations to all stakeholders involved in the IICP governance and reporting functions.
Feedback from OGDs indicate that they regard INFC’s leadership role as positive and are satisfied with the support provided by the IICP Secretariat. OGDs also appreciated the short response time when OGDs make requests to INFC for information and the clarity given around the requests. They also found that the INFC representatives were clear and concise, approachable and highly engaged; willing to work with OGDs to find solutions; and were also available when needed.
Detailed survey results for key areas of leadership are listed in Table 4.
|
Disagree/ Strongly disagree |
Neither agree nor disagree |
Agree/Strongly agree |
Don’t know |
---|---|---|---|---|
Statement |
Counts (%) |
|||
INFC is accessible when I have questions |
0 |
2 (9%) |
17 (81%) |
2 (9%) |
There is value added in INFC's role in the IICP |
0 |
4 (19%) |
15 (71%) |
2 (9%) |
INFC's leadership of the IICP is satisfactory |
0 |
4 (19%) |
14 (67%) |
3 (14%) |
INFC is responsive to my concerns/needs |
2 (9%) |
2 (9%) |
14 (67%) |
3 (14%) |
INFC IICP Secretariat support is adequate |
0 |
2 (9%) |
13 (62%) |
6 (29%) |
Extent to which the horizontal governance structure supports IICP’s delivery
The governance findings revolve around the changes to the governance structure and whether the current format or organization of the various committees and working groups is effective in supporting and facilitating discussions, resolving reporting issues and providing updates related to the Plan. This does not include how the Secretariat has organized itself internally, as this is considered under section 4.5.
Finding 3: The IICP horizontal governance structure was valued by OGDs for supporting information sharing, reporting coordination and addressing cross-cutting issues.
INFC created the governance structure for the IICP and engaged OGDs to fully implement it. When the HRR division was tasked with creating the governance structure, the Pan-Canadian Framework on Clean Growth and Climate Change horizontal initiative secretariat was consulted. Additionally, its ToRs were used as a basis to inform the IICP governance structure. As well, the IICP’s governance structure was also informed by the TBS guidelines on managing horizontal initiatives, as well as by the commitments made in the related Memorandum to Cabinet.
Meeting materials document the full participation and response of the participants when invited by INFC to attend. Specifically, at the DM level, there is evidence that four Deputy Ministers’ Coordinating Committee (DMCC) meetings took place between July 2017 and May 2018, each attended by 17 DMs and Assistant Deputy Minister (ADM) level participants, including central agencies, and additional observers. At the working level, nine horizontal working level meetings took place in the first year and four Program and Results Working Group meetings were held in the second year.
The governance structure enabled the sharing of information among and coordination of the activities of the 14 partner departments. At the ministerial level, the committee structure facilitated the communication of expectations and direction for IICP reporting.
Meeting materials from the DMCC revealed a similar role to that of the IICP information sharing and coordination. The DMCC was used to provide updates on the implementation of the Plan, discuss horizontal reporting requirements, and to secure commitments and agreement from all involved DMs.
The horizontal IICP governance structure was also used to address emerging issues that cut across the IICP partner departments and central agencies. For example, the emerging need to reduce the reporting gap between the PBO and Government of Canada figures spurred an effort to better understand and streamline requests for integration and dissemination of program information to the public. DMCC meetings facilitated coordination and effort among OGDs to improve reporting. The information requested and its details were also discussed using the governance structure at the Director/ Manager/ Officer level to ensure that everyone involved would agree to use the reporting template provided.
Another example of the horizontal governance structure helping to address cross-cutting issues involved the creation of the Investing in Canada Plan Project Map, also referred to as the GeoMapFootnote 8. Various working groups within the governance structure were leveraged to determine the direction and implementation of the reporting tools for the GeoMap. The original request for a project location map was made. After ministerial direction was obtained, a DMCC meeting took place in May 2017 to discuss the GeoMap and a results webpage. By July 2017, DMCC had been engaged to obtain DM-level agreement in order to move ahead with a website and subsequently, departmental resources: e.g., IT and Communications. By December 2017, the GeoMap was launched highlighting 6,978 IICP projects.
OGDs (including Communications Working Group) respondents believed information sharing and coordination of efforts among federal partners were effective or very effective, as indicated in table 5 below.
Not effective |
Somewhat effective |
Effective/Very effective |
Don’t know |
|
---|---|---|---|---|
Statement |
Counts (%) |
|||
Information sharing among IICP departments |
0 |
1 (7%) |
9 (64%) |
4 (29%) |
Timely resolution of issues |
0 |
1 (7%) |
8 (57%) |
5 (35%) |
Coordination of effort among federal partners |
0 |
2 (14%) |
8 (57%) |
4 (29%) |
Follow-up on action items |
0 |
1 (7%) |
8 (57%) |
5 (35%) |
Supporting the implementation of the IICP |
0 |
3 (21%) |
7 (50%) |
4 (29%) |
Clarifying governance activities and expectations |
0 |
3 (21%) |
7 (50%) |
4 (29%) |
Defining roles and responsibilities of IICP stakeholders |
0 |
3 (21%) |
7 (50%) |
4 (29%) |
Facilitating decision-making |
0 |
3 (21%) |
4 (29%) |
7 (50%) |
According to OGDs, the overall value of the governance structure centered around sharing information, supporting the collection of implementation data, and supporting the publication of GeoMap project data. Specifically, OGDs appreciated INFC’s support for coordinating, running and organizing all the meetings, the regular coordination of reporting (e.g., information call outs and uploading of data), and for providing one voice for the whole plan. As well, OGDs generally had positive feedback for the IICP Secretariat representatives at the director, manager and officer levels.
Finding 4: Over time, the governance structure has been responsive to changing context and emerging needs. Overall, it appeared in line with standards and practices of other similar horizontal initiatives.
In May 2016, INFC began leading a horizontal working group involving nine OGDsFootnote 9to respond to the reporting requirements of Budget 2016. This group helped introduce the reporting responsibilities as well as set up and implement a reporting structure. A total of nine meetings were held during the first year.
The announcement of Budget 2017 brought about new and more complex requirements for the IICP. It increased its:
- number of partners from 9 to 14 OGDs;
- number of funding streams from 3 streams (economic growth, greener infrastructure, improved social inclusion) to 5 streams (public transit stream, green infrastructure, social infrastructure, trade and transport, and rural and northern communities), and programs;
- original funding envelope by $81.2 billion;
- reporting responsibilities. For example, INFC was required to increased its focus on whole-of-Government reporting (e.g., in support of the Investing in Canada Plan Charter) as well as PCO-type reporting (e.g., generate monthly mandate letter tracking and monthly PCO Results and Delivery Unit environmental scanning).
As a result, the IICP horizontal governance and reporting function underwent a series of changes. For example:
- To allow for more horizontal and whole-of-government reporting, the function moved from INFC’s Program Operations Branch (POB) to its Policy and Results Branch (PRB) in spring 2017.
- To strengthen and further formalize the governance structure, the Terms of Reference (ToRs) for both the Investing in Canada Committee (ICC) and the DMCC were endorsed in September 2017.
- In order to provide an overview of the governance structure, a handbook titled the, “IICP Handbook for Governance Horizontal Reporting” was developed. It outlined the roles and responsibilities of the various committees and participants. It was first released in September 2017 and a revised version was released in March 2018.
The impact of these changes on the quality of the relationship between INFC and OGDS was measured through a survey and by interviews. The results indicate that no OGD respondents felt the changes in the IICP governance had negative impacts. Specifically, OGD partners revealed that:
- 7 out of 14 of respondents (50%) felt that the changes made over the last year were a step in the right direction,
- 3 out of 14 of respondents (21%) reported that the changes had no impact, and
- 4 out of 14 (28%) reported that they don’t know what impact the changes may have had.
Overall, the IICP governance structure―in terms of what committees existed and how often they met― appears to be appropriate when compared with other GC horizontal initiatives. Each one of the other governance structures examined reflected its own context with their own complexity of issues and frequency of meetings. For example, the Pan-Canadian Framework on Clean Growth and Climate Change has a rigorous structure with multiple levels of management that meet regularly. It is driven by a group of Deputy Ministers who meet monthly, supported by an ADM-level working group which also meets monthly to prepare for the DM-level meeting. These regular meetings ensure all departments are engaged and prepared to report to the Canadian Council of Ministers of the Environment, as well as the United Nations Framework Convention on Climate Change, on behalf of Canada.
In contrast, the Federal Contaminated Sites Action Plan (FCSAP) has a horizontal board of ADMs that handle strategic program issues as they arise and meet only when the funding requires renewal. It also has a DG-level committee that meets at minimum once a year. The working level Contaminated Sites Management Working Group (CSMWG) consisting of 18 departments meets quarterly.
Finding 5: OGDs wanted more support when responding to the IICP taskings and requests, especially those beyond regularly scheduled tasks.
OGDs reported wanting more predictability surrounding the work that IICP reporting and governance entailed, especially those beyond the regularly scheduled requests. Specifically with respect to governance, they would have liked to have seen more advanced notice of IICP requests and meeting invitations, to avoid last-minute, ad-hoc changes and requests. Interviewees expressed disappointment at this cancellation given there was much work that went into its preparation.
When asked about what improvements could be brought to governance practices, OGDs reported wanting standardized, regular requests and updates. When OGDs were asked to describe gaps or challenges they experienced and suggestions for how governance for IICP could be improved, some mentioned that working group meetings were too irregular and suggested regular updates to promote continued dialogue. Another suggestion was that the IICP Secretariat consider replacing meetings with a brief monthly update or memo when warranted.
Some respondents also recommended that IICP taskings be simplified by leveraging work already done or information that already existed, and to better tailor the requests. A suggestion to avoid duplications and to reduce the reporting burden, was to include more background information as to the purpose of the request to departments participating in IICP reporting or governance taskings. As well, the taskings should also indicate where issues could have already been assessed by INFC (e.g., when the request was already published through the Open Data Portal Initiative), instead of having each department generate its own information. Having tailored requests outlining the exact information required would have avoided duplication of effort by OGDs, but would have required additional analysis by the IICP Secretariat.
While there will always be IICP-related taskings that are unforeseen and emanate from Ministerial, Parliamentary, or central agency requests, there will also be opportunities to help all OGDs better prepare for them. This will be particularly important for OGDs managing their own competing priorities and working through their own internal approvals, as mentioned earlier. Continued correspondence, regular updates or advanced planning of meetings, for example, may also help OGDs prepare themselves for the work ahead, including any last minute requests.
Finding 6: The horizontal governance structure was not formalized below the DM level and lacked guidance on how to collaborate and report back to senior committees.
While the Handbook provided information on roles and responsibilities of the various groups of the horizontal governance structure, as well as their membership, only the ICC, the DMCC and the ADM Data Strategies had formal ToRs. As well, while a ToR was developed for the ICC, it was never approved by Ministers (i.e., the members of the ICC). Finally, despite the need to update the DMCC’s ToR to include all OGDs delivering programs under Phase 2 of the IICP, it has not been addressed to date at DMCC meetings.
This evaluation did not find any formal documentation or articulation of the specific details of how the groups and committees in the IICP governance structure should work together or how information should be exchanged. This clarification would have supported the coordination of all key reporting and communication functions for IICP, outside of the regular reporting mechanisms with OGDs described in Section 4.3. While this evaluation found that that there were working relationships between key players that allowed IICP taskings to be completed, these relationships were active more on a task-by-task basis (e.g., the development and full implementation of the GeoMap), rather than on a regular basis.
The lack of strategic coordination may have contributed to an information gap among key players, especially for the key function of communications. For example, the evaluation found instances of the Secretariat not being aware of certain communication initiatives taking place at the Minister and DM levels, or what IICP information was presented or how. A review of the DMCC membership also revealed that there was no communication working group representation at that committee and, as such, it is unclear how regularly communication issues were discussed or coordinated at the DMCC.
The evaluation also found that some committees of the horizontal governance structure were not serving their intended purpose. For example, the creation of an ADM Data Strategies Working Group was directed by the DMCC on July 18 2017, to improve reporting against the Charter on the impacts of infrastructure investments. It met once in December 2017 and has not met since.
There was also no evidence that the ICC met during the period covered by this evaluation. Instead, ministerial engagement took other forms, including MRT meetings led by PCO. There was no evidence that DMCC had been engaged to advise on the role of Ministers in IICP governance, beyond endorsing ICC ToRs.
Finding 7: Informal collaboration was a key strength in the relationship between INFC and OGD partners.
In addition to the formal horizontal governance structure and meetings, the relationships between INFC and its OGD partners (and between the OGDs themselves) were supported by informal collaborative mechanisms. These included impromptu, ad-hoc emails and phone calls, or bilateral meetings outside the formalized committee meetings, to clarify understanding. Overall, the qualitative responses from interviews and surveys indicate that this was a key strength.
When asked how important these informal collaborations were in coordinating the implementation of the IICP, 9 out of 14 (64%) survey respondents stated that they were important/very important, 2 out of 14 (14%) indicated that they were somewhat important. Interviews with OGDs also revealed generally positive comments on the availability of the IICP Secretariat staff to address questions.
Extent to which IICP reporting requirements were met
This section examines INFC’s reporting role and whether reporting requirements were effectively met. It also includes an assessment of both reporting tools and templates used to collect information from all partners, as well as the partners’ assessment of the relationship built by INFC to collect regular information and complete reports. Table 6 lists the reporting products for the IICP.
Item |
Description / frequency |
---|---|
Project level reporting |
Monthly
|
Program Implementation Tracking and dashboard |
Monthly
|
Annual report (first one to be completed Spring 2019) |
Annually
|
TBS Requirement: Departmental Plan (DP)/Departmental Results Report (DRR) (horizontal initiative supplementary table) |
Annually
|
INFC Results and Delivery Charter |
Developed by INFC’s Policy Team, the Charter was approved by the Minister in February 2017. It provides the framework for the Horizontal Appendix. |
TBS Requirement: horizontal appendix |
Annually
|
PCO requirement: mandate letter tracker |
Monthly
|
PCO requirement: Results and Delivery Unit (RDU) scan |
Monthly
|
Finding 8: The basic reporting requirements related to IICP were met for the period covered by this report. Going forward, concerted effort should be made to ensure more results can be reported. it is difficult to predict the extent to which these projects tend to achieve the expected outcomes of the Program.
Basic reporting requirements outlined by TBS and PCO have been met over the last two years, including products listed in Table 6 above. In addition, various ad hoc reports and briefing requests for senior management, PCO and TBS were produced. The regular data collection, particularly the implementation and project data, helped inform managerial briefings that took place, as well as any ad-hoc requests for information. However, some of these requests could not be responded to by previously provided data, requiring the IICP Secretariat to reach out OGDs for additional data. For example, a ministerial request was made for information regarding the spending for all types of bilateral agreements under the IICP by province and territory. The IICP Secretariat did not have the full set of data needed to respond, requiring them to reach out to OGDs to provide additional information. Another example is the status of programs launched under the second phase of the IICP. INFC had to frequently report on this metric, and was required to reach out to OGDs to get updates on an ad-hoc basis.
The expectations surrounding and complexity of data reported monthly have grown over time. As well, definitions for data collected were refined and improved over time. An examination of the reporting templates in 2016 indicates that the data for each OGD included four data fields per initiative: program name, budget value, value of agreements signed, and money flowed. As a result of shifting data needs in 2017, the data fields for OGD monthly reporting increased to 12 fields for each initiative in the program implementation data report, in addition to 27 fields for each project for the GeoMap.
OGD respondents were mostly satisfied with INFC’s leadership on reporting requirements, in the quality of the reporting templates and tools, and in the adequacy of the timelines to submit reports. The lowest level of satisfaction was found with the alignment of IICP reporting requirements with other internal or ad hoc external requests (e.g., PCO, TBS) where 6 of the 14 respondents indicated that they were only “somewhat satisfied.”
|
Not satisfied |
Somewhat satisfied |
Satisfied/Very satisfied |
Don’t know |
---|---|---|---|---|
Statement |
Counts (%) |
|||
Leadership/direction provided by INFC on reporting requirements |
0 |
1 (7%) |
12 (86%) |
1 (7%) |
Reporting templates and tools |
0 |
2 (14%) |
11 (79%) |
1 (7%) |
Adequacy of timelines to submit reports |
0 |
3 (21%) |
10 (71%) |
1 (7%) |
Quality of the relationship established with INFC for reporting |
0 |
3 (21%) |
10 (71%) |
1 (7%) |
Relevance of the performance information that departments are asked to collect |
0 |
2 (14%) |
10 (71%) |
2 (14%) |
Clarity of reporting expectations |
1 (7%) |
3 (21%) |
9 (64%) |
1 (7%) |
Flexibility of timelines to accommodate unforeseen challenges |
0 |
3 (21%) |
7 (50%) |
4 (29%) |
Alignment of IICP reporting requirements with other internal or external requests |
0 |
6 (43%) |
6 (43%) |
2 (14%) |
When asked to describe aspects of IICP reporting that they felt were working well overall, OGD respondents (14) reported that the IICP Secretariat was flexible and open to suggestions. When asked to describe ways to improve the way IICP information is reported to INFC, OGDs suggested that more time be provided to respond to requests for ad hoc reporting, as well as encouraging and establishing a single INFC point of contact for each department to streamline the provision of information.
Interviews also found that reporting requirements were generally met, despite initial challenges. For the regular monthly reporting requirements (e.g., project and implementation data from OGDs), there were challenges initially in clarifying the expectations (e.g., frequency, data definitions, and determining how to extract business intelligence from departmental databases such as accounting, geomatics and/or program information). The 2017 and 2018 PBO report focused extra attention on the availability of information and reporting requirements that influenced the IICP Secretariat to improve its data collection processes with partner OGDs.
To date, reporting has focused on tracking how well the IICP has been implemented, including monitoring when agreements were signed, projects approved, and funding allocated. The move from output-based information to results and impact-type information will require changes and discussions with OGDs to ensure future reporting requirements are met. Central agencies and OGDs reported that more work is needed to ensure that INFC has the ability to obtain sufficient information to tell the “results story.” For example, there is a concern about the absence of long-term outcome results. That said, information presented to the DMCC in May 2018 indicates that Canada’s Core Public Infrastructure survey (CCPI) and Infrastructure Economic Accounts (INFEA) information will be considered as an important source of information to develop baseline and evidence-based results information. Specifically, 4 out of 7 IICP long-term outcomes will rely on CCPI or the INFEA. The information from CCPI and INFEA became available in September 2018, based on data collected by Statistics Canada in 2016.
Going forward, reporting tools will need to be more robust with the addition of Budget 2017 projects and the move towards reporting of outcome and results data in addition to output data. Work is underway in INFC’s Data and Research Division to coordinate information with Statistics Canada and provide relevant analysis. However, this work is in its early stages. While INFC’s Data and Research Division and the IICP Secretariat are both part of the Policy and Results Branch, during the period covered by this evaluation, there was no formal relationship established between the two divisions. However, in October 2018, the Data and Research and Horizontal Results and Reporting divisions were combined to form the Economic Analysis and Results Directorate. This reorganization is intended to facilitate the building of the analytical foundation necessary to provide evidence-based advice in terms of where investments should be made and support data-driven reporting on the results of the investments.
Public reporting on results will be increasingly important. INFC’s new Minister’s mandate letter, announced on August 28, 2018, emphasized the need and expectation to communicate results to the public. This will be especially important as the IICP moves from launching initiatives and into the later stages of implementation.
Finding 9: There is a lack of clarity on the reporting scope with respect to some programs under the IICP.
IICP authority documents specify the requirement for INFC to report on Budget 2016 and Budget 2017 IICP initiatives for the whole of government. Existing legacy programsFootnote 10 within INFC are considered part of the IICP, in addition to funding for new or increased programs announced in Budget 2016 and Budget 2017. Therefore, reports produced must also contain information on those initiatives. However, authority documents related to Budget 2016 and Budget 2017 do not mention legacy funds for OGDs, and, as a result, the requirement for reporting OGD legacy programs is unclear.
Staff within the Department of Finance and TBS as well as INFC’s DM have expressed their desire, as a good practice, to have the IICP Secretariat report on whole of government infrastructure spending―including legacy programs delivered by all departments under the IICP.
A spreadsheet prepared by the Department of Finance detailed the full funding status of infrastructure programs authorized through Budget 2016, Budget 2017 and previously approved legacy infrastructure funding. The total for all approved infrastructure programs is $187.8 billion. The IICP Secretariat notes that as of October 2018, additional information has been provided by the Department of Finance and further information is slowly trickling in on the status of these funds.
Given the fact that this is a vast horizontal initiative, some clarification is required with regards to legacy infrastructure funding programs allocated to other departments prior to Budget 2016 as well as the complete picture of IICP allocations. Ongoing effort will be required among INFC Corporate Finance and central agencies to obtain a complete understanding of all program information related to IICP funding.
Finding 10: There are differences between various information sources, including reports produced by INFC, Privy Council Office and Parliamentary Budget Officer.
There was a significant gap between the current GeoMap which displays 10,000 projects and the full implementation data set, which covers 28,000 projects (as of May 2018). Clarification provided by IICP Secretariat staff reveal that the bulk of this difference can be explained by the varying ways in which each OGD counts projects. For example, if a department provides funding for repairs or renovations to 40 housing units at the same location, this department may decide to count this as 40 projects in implementation reporting, but given the single location, it would appear as one project on the GeoMap.
The PCO’s results tracking website reports on infrastructure priorities in a different way. Its website tracks the priority of “Sustainable Infrastructure”, which contains information on IICP as well as from other initiatives unrelated to IICP. Given the different approaches used by PCO and INFC to present infrastructure spending, it is difficult for Canadians to understand exactly how much is being spent and where.
Further potential confusion may have been introduced when in February 2017, the Parliamentary Budget Officer (PBO) reportedFootnote 11 on infrastructure for the first time. The report stated that there were 32 departments involved in infrastructure spending. The first phase of the IICP, as defined in Budget 2016 included only 9 departments. As such, the report’s coverage differed significantly from INFC’s area of leadership, but this may not have been easily understood by Canadians.
The purpose of the PBO is to provide independent, authoritative and non-partisan financial and economic analysis and supports parliamentarians in scrutinizing the raising and spending of public funds. As such, PBO reports were not necessarily aligned with official Government of Canada figures. After close collaboration between INFC and PBO personnel, the PBO released a third infrastructure report in August 2018. It focused only on the IICP, listed the correct departments and initiatives and accurately reflected the scope of the Plan, as tracked by INFC.
In order to avoid potential confusion for Canadians, it will be important to reconcile and make transparent the information reported through the GeoMap and in other reporting mechanisms.
To date, the full implementation data, while updated monthly, is used only for internal briefing purposes, central agency reports and other briefings and has not been distributed publicly. The IICP Annual Report, when published, will need to take this into account to avoid potential confusion.
Finding 11: Current reporting processes and tools have limited opportunities for quality control and introduce risks to IICP data quality.
The graphic below summarizes the typical IICP reporting process followed by the 14 OGD partners
Some working group members reported that there have been challenges trying to use the INFC templates given they may need to pull data from several different department specific systems or corporate functions (e.g., accounting, geographic information system (GIS) information) to complete. As depicted in the graphic, a review of the data collection process shows that monthly data provided by OGDs are primarily collected in Excel spreadsheets, transmitted by OGD partners to INFC via email, integrated through a series of manual data transfers before being checked for quality by an analyst within the IICP Secretariat. This data is then used as the basis for multiple reports that are created by the IICP Secretariat. This evaluation found that these multiple manual data transfers are both labour intensive and introduce potential risk to the data integrity and quality due to the reliance on manual checks. Once the transfers are complete, there is a limited opportunity for OGDs to view, verify or correct their information as the data is held internally at INFC.
As part of this evaluation, a benchmarking exercise found that other federal horizontal initiatives have addressed data transfer and sharing barriers by using online collaborative platforms. For example, the Pan-Canadian Framework on Clean Growth and Climate Change uses SharePoint to share documents among various players; the Ocean Protection Plan (OPP) uses GCcollab. Reporting authorities contacted in Pan-Canadian Framework and OPP were satisfied with their own approaches.
Use of a shared data repository could simplify operations and the handling of data elements, thus reducing the potential for human errors when transferring data. In addition, it would allow for greater transparency, allowing all OGD partners to see what has been reported by other OGDs. This approach would also encourage OGDs to assess their progress and data quality more closely.
Technological issues regarding the GeoMap will also need to be addressed. As of July 2018, the online map responded slowly due to the large dataset that must be uploaded, despite the fact that not all Budget 2016 and Budget 2017 projects have yet been entered. Budget 2017 represents more than $95 billion of funding over ten years and will result in an exponential growth of data points. INFC will need to consider how the GeoMap will perform.
Extent to which IICP communications supported public reporting
This section examined how well the communications function produced communications products for public consumption. It includes the assessment and perception of communications staff on the quality and consistency of the communications products.
Finding 12: Various IICP communications products are publicly available to inform Canadians on the progress of the IICP.
A variety of IICP public communications products were produced in collaboration with the Horizontal Reporting and Results team over the course of the evaluation period, including:
- the GeoMap released December 2017. This interactive map on an INFC-hosted website shows the geographical location of IICP funded projects and investments;
- the Investing in Canada plan webpage launched in April 2018. This website serves as the portal to all IICP-related information including an overview and links to both the GeoMap and the Investing in Canada: Canada’s Long-Term Infrastructure Plan document;
- the Investing in Canada two-page placemat that describes/illustrates the IICP. It breaks down infrastructure investments into the 5 streams of funding and outlines how INFC’s Budget 2016 and Budget 2017 investments are divided among all the provinces and territories;
- various media announcements and news releases at both the Plan and project level. For example, the Minister communicated the IICP’s progress to Canadians through an April 19, 2018 news release; and
- multiple online social media campaigns. For example, INFC profiled Infrastructure Weeks (May 12-25). – which promoted the IICP, including a series of funding announcements for various projects such as the Port de Montreal project, the Chignecto Isthmus Study and the Calgary Green Line.
All of the above products were designed to ensure that the IICP’s progress and results are communicated to the public across various media. For the online materials, views and visits were tracked by the INFC Communications Branch to measure interest. Between January and June 2018:
- the GeoMap received 12,893 total visits and 15,220 views,Footnote 12
- the main Investing in Canada Plan splash page received 16,942 visits and 26,586 views, and
- the placemat received 1,568 visits and 2,063 views.Footnote 13
The Communications Working Group also tracked and analyzed media uptake and how easily Canadians were able to access their materials. For example, according to a media analysis done by INFC’s Communications Branch, the Minister of Infrastructure and Communities April 19, 2018 update on the IICP generated 24 articles and 2 broadcasts by media outlets. A social media campaign, Infrastructure Week May 12-25, garnered significant online, print and broadcast media attention that was largely positive. Media analysis indicated that this campaign generated 259 retweets, 257 likes on Facebook and 5,218 click-throughs.
Overall, evidence suggests that the various materials produced by the Communications Working Group is available online and was being accessed by Canadians. As well, INFC’s announcements were used by media outlets for the development of their stories.
Finding 13: The IICP Communications Working Group supports consistent messaging to the public across the 14 OGDs.
The responsibility of communicating the results of each OGD’s own portfolio rests with each OGD’s respective Communications branches. However, the communication of the IICP as a whole is the responsibility of INFC’s Communications Branch. It leveraged the Communications Working Group to help develop and share key messages and Question & Answer documents to support consistent communications efforts around the IICP. The working group held regular meetings every 2-3 months to ensure that messages about the progress and results of the IICP are as consistent as possible. At present, the Communications Working Group is able to support the consistency of their message.
INFC’s Communications Branch reported that they are generally satisfied with the consistency of messages communicated to the public, including those communicated by OGDs. They explained that, generally, they are able to detect inconsistencies in public messaging when they occur through ongoing media monitoring. INFC’s Communications Branch reported that, over the past two years, there were no inconsistencies detected in the media coverage that warranted a letter to the editor to correct substantial discrepancies.
Resource utilization to meet IICP reporting and governance requirements
This section examined the internal resources allocated to the reporting and governance function to determine whether the resources available were adequate to fulfil the mandatory IICP requirements.
Finding 14: Resources utilized by INFC’s IICP Secretariat are comparable to other federal horizontal initiatives and demonstrates responsiveness to change.
During its first year of operation, coordination of the IICP at INFC was assigned to a team comprised of one director and two students. They prepared reports summarizing the activities of 9 OGD partners responsible for $14.4 billion in IICP funding. As previously mentioned in section 4.3, each department’s reporting requirements were limited to four fields per initiative. For the second year of reporting, the number of partners increased to 14 and the total number of initiatives to report on increased to over 70. Additional coordination with central agencies was added. As well, a monthly GeoMap reporting requirement was added. Specifically, the GeoMap reporting structure introduced 27 fields per project, and the implementation reporting requirements increased to 12 fields per project instead of six fields during the first year.
To address these new elements, between December 2017 and May 2018, the IICP Secretariat doubled in size to 10 employees, and was managing one additional employee on a risk basis (e.g. without a formal budget) to match the additional workload. The IICP Secretariat is now working with a dedicated cost center to track its use of financial and human resources. The operating budget for fiscal year 2018-19 was $1.1 million which covers employee salaries and non-salary expenditures.
Various secretariats for other federal horizontal initiatives were reviewed to try to assess INFC’s approach against accepted practices. Secretariat resources ranged from 6 to 13 FTEs. However, activities and governance approaches varied, which limited comparability with the IICP Secretariat. Two other comparable horizontal initiatives, the Pan-Canadian Framework on Clean Growth and Climate Change and the Federal Contaminated Sites Action Plan (FCSAP), coordinate with 19 and 18 departments respectively and have 13 and 12 FTEs in their respective secretariats. While the materiality of the initiatives are not comparable (IICP totals $187 billion, the other initiatives do not exceed $4.5 billion), the level of complexity and/or regular activity is. See Annex D for more information.
As such, the level of INFC resources for the management of the governance and reporting functions is considered reasonable for an initiative of this magnitude.
Both the IICP Secretariat and INFC’s Communications Branch demonstrated their efforts to optimize their resources by adopting a portfolio-based work management approach. For example, the IICP Secretariat team is transitioning to a portfolio approach where each analyst will be responsible for a group of departments. The objective stated by the Secretariat is that this approach will allow analysts to become very familiar with the programs of their assigned departments, and will provide departments with a central point of contact for IICP requests. As well, INFC’s Communications Branch restructured itself to have dedicated staff on various IICP-related files to build expertise in order to undertake IICP communications more efficiently. For example, one analyst is assigned to Smart Cities Challenge to develop expertise with the program.
While not related to this finding, it may be interesting to note that OGDs were mixed in terms of the adequacy of their internal resources to support IICP efforts: 11 departments claimed to be adequately resourced or well resourced, and 8 departments claimed to be under resourced.
Finding 15: There appears to be a lack of internal coordination among INFC areas of expertise to effectively respond to IICP responsibilities.
INFC is responsible for supporting the Minister of Infrastructure and Communities in reporting and publicly communicating the progress and results of the IICP on behalf on the Government of Canada, including for its own programming. To do this well, a range of functional areas with specific skills and knowledge have a role. These areas include, but are not limited to: policy, programs, corporate services, and communications.
There is no documentation of any formalized governance structure established to coordinate the IICP internally within INFC. While internal governance structures already exist―such as the Departmental Management Committee (DMC) chaired by the DM, and the Program and Policy Committee (PPC) ―a review of the DMC and PPC agendas show few instances where IICP issues are discussed. Moreover, despite PPC’s documented mandate to “provide strategic coordination of core departmental policy and program issues” and, specifically, “complex and horizontal issues involving multiple branches,” when IICP issues are discussed, they are discussed in discrete parts rather than holistically.
At the time of the writing of this report, the IICP Secretariat staff reaches out at the working level to INFC areas of expertise, when required to respond to IICP taskings that require internal input and coordination. For example, Corporate Services’ Finance and Administration team is consulted as needed to better understand and present IICP funding. As well, interactions with Policy and Results Branch’s Results and Data division were undertaken to better understand how to report results. There is a well-established monthly reporting and submission process involving the INFC Program and Operations Branch, which occurs regularly.
Interviews with INFC’s Communications team members indicate that the IICP Secretariat is not always aware of all communication initiatives taking place at the departmental level, involving the Minister and DM, in regards to the IICP. Improved integration between INFC’s Communications Branch and the IICP Secretariat should help INFC when asked to craft a single, appropriate story line for the IICP and avoid conflicting public messages and presenting an incomplete story. For example, to better prepare the DM to brief the Minister on public appearances and announcements, it would be more beneficial for the INFC Communications Branch and IICP Secretariat to discuss and agree upon the same strategic presentation beforehand.
Enhanced collaboration would allow all INFC key players to better understand what is needed to respond to IICP requirements, particularly at the executive level, and how best each functional area could contribute their expertise. With better coordination and involvement of key internal players, the IICP Secretariat will be even better equipped to produce annual reports and update the results website.
While outside the period covered by this evaluation, it would be useful to mention that, since June 2018, the IICP Secretariat, Communications and Research and Data teams have had weekly meetings to discuss reporting, data and communication issues related to IICP. The storyline and IICP narrative have been key issues discussed at these meetings.
5.0 Conclusions and Recommendations
Conclusions
INFC demonstrated effective leadership when governing and reporting horizontally for the IICP. The Department had a clear mandate that was understood by all OGDs and central agencies. In addition, the interviews and survey results done for this evaluation showed that federal stakeholders were satisfied with INFC’s leadership and its ability to create and build the collaborative relationships needed to coordinate the reporting and meetings required to govern and report on the IICP.
Each department has their own authority to manage their part of the IICP, as well as their own concurrent priorities and processes to approve and to communicate updates on their programs under the IICP. This necessitates building a good relationship with an understanding of the OGDs’ contexts. It also demands strong leadership when it comes to planning and delivering IICP taskings. Advance notice of what’s required, even if not fully defined or without confirmed due dates, along with regular sharing of information along with clear direction would help OGDs plan their workload ahead of time. It would also improve their ability to deal with competing priorities. However, it is recognized that there will always be external demands that create unforeseen taskings for OGDs that will need to be dealt with on a regular, if unpredictable, basis.
The governance structure created and managed by INFC helped to gather OGDs and central agencies to discuss issues at various levels, including Ministerial, DM, senior management and officer levels. The structure was robust enough to resolve emerging issues such the creation of the GeoMap. OGDs found that INFC created a governance structure that successfully shared information formally with them, but also established strong informal relationships to answer ad-hoc issues and questions as they arose. However, there is a need to re-examine the structure to ensure all committees and their purpose are clear and, more importantly, how they function together to meet IICP reporting and communication requirements.
The reporting activities reviewed during this evaluation met all the necessary products to fulfil IICP’s reporting requirements, even with the growing complexity of monthly reporting for OGDs. Communications products, as with reporting products, were made available to Canadians. As with leadership and governance findings, OGDs are also generally satisfied with INFC’s reporting processes. Effort will need to be made to ensure that INFC is ready and able to report to the Canadian public on the impacts of the IICP over the life of the initiative, and meet corporate reporting requirements. A challenge to reporting is the lack of clarity related to the scope of legacy funding included under the IICP and the inability of current practices to give a complete funding and spending status of the IICP progress and results. All of these, in addition to public reporting differences between initiatives (e.g., PCO vs. INFC), highlight the challenge of providing a clear message to the public.
To address risks to data quality, new reporting tools and processes should be considered to better share and view data among all OGDs.
INFC resources used over time echoed the amount of work and the growth in the scope in the governance and reporting function. When comparing the IICP to other comparable horizontal initiatives, the level of staffing was coherent given the extensive governance coordination, the large number of departments involved and the various coordination activities required. INFC, in leading the IICP, demonstrated several ways it improved the use of internal resources by restructuring the teams in both the IICP Secretariat and the INFC Communications Branch to create expertise. However, improved collaboration between all areas of expertise within INFC needs to be addressed. There are opportunities to better coordinate internal experts from the various areas to better facilitate information exchanges and better support IICP responsibilities for the entire plan.
Recommendations
It is recommended that the ADM, Policy and Results:
- Review and propose changes, as required, to the IICP governance structure to ensure all components (e.g., working groups and committees) have a clear purpose and are able to articulate how they are to interact. Attention should also be placed on the processes/tools used by the various committees to better support ad hoc IICP taskings.
- Assess if improved collaboration among INFC sectors and branches is sufficient to ensure a more coordinated departmental approach to supporting IICP reporting and governance.
- Consider new reporting tools and processes to better share and view data among all OGDs, and to address risks to data quality.
Investing in Canada Plan (IICP): Evaluation of the Horizontal Governance and Reporting
Management Response and Action Plan
Recommendation 1: |
Management Response: Agreed. |
|||||
Strategy:
|
||||||
Action Plan |
Planned Completion Date |
Office of Primary Interest |
||||
---|---|---|---|---|---|---|
The Policy and Results Branch will conduct a review of IICP governance and will make recommendations to the Deputy Minister on the need for potential revisions to the IICP governance structure, membership and associated terms of reference. |
December 2019 |
ADM-Policy and Results Branch |
||||
Recommendation 2: |
Management Response: Agreed. |
|||||
Strategy:
The Policy and Results Branch will also consider options to enhance engagement within INFC on framing the IICP results narrative. |
||||||
Action Plan |
Planned Completion Date |
Office of Primary Interest |
||||
|
April 2019
|
ADM-Policy and Results Branch |
||||
The Policy and Results Branch will conduct a review of IICP governance and will make recommendations to the Deputy Minister on the need for potential revisions to the IICP governance structure, membership and associated terms of reference. |
December 2019
|
ADM-Policy and Results Branch
|
||||
Recommendation 3: |
Management Response: Agreed. |
|||||
Strategy:
|
||||||
Action Plan |
Planned Completion Date |
Office of Primary Interest |
||||
Continue working collaboratively with IICP delivery departments to produce the first IICP Progress Report; Implement an improved standardized reporting template for monthly reporting that will reduce the manipulation burden and human error risk, and ease the burden on departments. Roll-out will include workshop training sessions for all IICP delivery partners. |
June 2019
September 2019 |
ADM-Policy and Results Branch ADM-Policy and Results Branch |
Annex A: IICP Governance Structure Roles and Membership
Title |
Level |
Role |
Membership |
---|---|---|---|
Investing in Canada Secretariat |
INFC-internal |
Responsible for coordinating the governance committees and horizontal reporting activities under the Investing in Canada plan. |
Led by the Director of Horizontal Reporting and Results |
Committee on Agenda, Results and Communications (ARC) |
Ministerial |
A Cabinet Committee that tracks progress on the Government’s agenda and priorities, considers strategic communications issues related to advancing the government’s agenda, and all aspects of Parliamentary business. Meeting schedule is set by the Privy Council Office. As part of its mandate to track progress on results, the ARC reviews Results and Delivery Charters. |
|
Ministerial Results Table |
Ministerial |
Discusses progress towards achieving the government’s desired results, and problem-solves where risks are identified. The MRT role is currently under review. |
Cabinet Ministers, government officials and external experts |
Cabinet Committee on Growing the Middle Class and Inclusion |
Ministerial |
A Cabinet Committee that considers initiatives that is responsible for initiatives that will strengthen and grow the middle class through innovation, inclusive economic growth, employment and social security, as well as issues concerning the social fabric of Canada and the promotion of Canadian pluralism. |
|
Investing in Canada Committee (ICC) |
Ministerial |
Serves as a consultation and collaboration forum for Ministers responsible for delivering the IICP. The Committee aims to ensure a whole of government approach to the implementation of the IICP and facilitate the delivery of results. This Committee provides advice and support to the Minister of Infrastructure and Communities regarding progress updates on the IICP at ARC and MRT. This committee has not met during the period covered by the evaluation. |
|
Deputy Ministers’ Coordinating Committee (DMCC) |
Deputy Minister |
Serves as a forum at the senior official level that provides strategic oversight of, and direction on, the implementation of the IICP. It supports the ICC and guides results and reporting on the IICP as set out under the Charter. Meetings are convened roughly every quarter. |
|
ADM Data Strategies Working Group |
ADM (interdepart-mental) |
Provides a forum to discuss and address the methodologies behind the data strategies used to collect data in support of reporting on IICP results under the Charter. The group supports the DMCC, as appropriate, and meetings will be convened every two months, in advance of DMCC meetings. The Working Group is chaired by the Assistant Deputy Minister, Policy and Results, Infrastructure Canada. |
|
Programs and Results Working Group |
Director, manager and officer levels (Interdepart-mental) |
Facilitates coordinated horizontal results reporting on the IICP. It is also a forum for providing updates on implementation of IICP programs and horizontal reporting. The group provides support to the DMCC and the ADM Data Strategy Working Group, as necessary. The Working Group meets monthly and is led by the Horizontal Results and Reporting team at INFC. |
Members consist of director, manager and officer levels at departments delivering programs under the IICP. |
Communications Working Group |
Director, manager and officer levels |
Shares information on the status and public communications aspects of programs that departments deliver under the IICP. The work of the committee helps ensure consistent and coordinated messaging while increasing the reach and impact of communications activities conducted by each department. The group supports the Programs and Results Working Group and provides direction regarding the content and structure of horizontal reporting. The group meets on an ad hoc basis and is led by INFC’s Communications Branch. |
Members include director, manager and officer levels in the areas of communications in departments delivering programs under the IICP. It is led by INFC’s Communications Branch. |
IT Working Group on Reporting Data |
Director, manager and officer levels |
Ensures coordination across departments with respect to the geo-location of projects for horizontal reporting. The group supports the Programs and Results Working Group, as necessary. The group meets on an ad hoc basis and is led by INFC’s Information Technology team. |
Members consist of director, manager and officer levels involved with geo-location of projects in departments delivering programs under the IICP. |
Annex B: IICP Program Background – Logic Model and Funding Allocation Overview
Annex C: Methodology
Document Review
The document review included INFC documents to understand the program design and delivery model and the context in which the program was developed. This included reviewing data within performance, financial, program and administrative areas (e.g., meeting minutes/notes and data received from OGDs). It also included financial commitment documents. Program data from Communications, including the E-communications division, was used to determine the availability and reach of public communications of IICP related websites and communication products (e.g., website page hits, downloads).
The document review also included material outlining the structures and committees created as well as any records of meetings that took place to govern the IICP. This evaluation also noted the types of discussions that have taken place in these committees to determine how the decision-making and/or problem solving-processes guided the IICP. For the reporting function, the document review outlined the types of reporting templates and subsequent data aggregation processes. The types of reports were outlined by their respective audience and purpose. Notes on timeliness of the data received were also added.
Survey
Surveys were used to reach two groups: (1) OGD representatives that actively participate in regular IICP reporting and/or governance activities; the survey was sent to all representatives from the 13 OGD partners that were not interviewed, and (2) the Communications Working Group were surveyed to determine the relationship INFC, as the lead department, had established with the partner OGDs and the leadership INFC has demonstrated in producing IICP communications products.
Type of Stakeholder |
# of Respondents |
Response rate |
---|---|---|
OGD partners |
19 |
10 (53%) |
Communications Working Group |
49 |
11 (22%) |
TOTAL |
68 |
21 (31%) |
Interviews
This evaluation included key informant interviews. The objective of the interviews was to gather in-depth information including views, explanations and specific examples. Opinions from OGDs, TBS, PCO and Department of Finance on the effectiveness, satisfaction of the roles, processes and products related to both the governance and reporting functions were collected. Internal INFC staff were interviewed for additional points of analysis, including asking their perspectives about the relationships built with OGDs, central agencies, challenges to date and suggestions for improvement. Staff from POB that formerly managed the function (i.e: in year one) were interviewed to determine the reporting and governance responsibilities at program outset.
While the survey will collect general information about the quality of the relationship, interviews were used to collect more detailed and nuanced answers. Stakeholder groups were divided as follows, with their respective response rates:
Type of Stakeholder |
Planned Interviewees |
Interviewees |
---|---|---|
Central Agencies (TBS, PCO, Department of Finance) |
6 |
4 (66.67%) |
INFC Staff (POB, Policy/IICP Secretariat, Communications) |
16 |
15 (93.75%) |
INFC Management (DG, ADM, Associate DM, DM) |
6 |
6 (100%) |
OGD IICP partners |
13 |
12 (92.30%) |
TOTAL |
41 |
37 (90.24%) |
Benchmarking
The benchmarking exercise was designed to compare the IICP to several similar horizontal projects within the Government of Canada to determine if there were any best practices that the IICP could utilize. Specifically, the analysis will help to identify challenges and mitigation strategies used by comparable projects that could be adopted by the IICP.
The benchmarking exercise combined various methods of data collection―including document review and interviews with lead secretariats―to confirm the information in the documents and to gather qualitative answers about best practices, challenges and recommendations. The final list of initiatives chosen for benchmarking were: Oceans Protections Plan (lead: Transport Canada), Pan-Canadian Framework for Clean Growth and Climate Change (lead: Environment and Climate Change Canada), and Federal Contaminated Sites Action Plan (lead: Environment and Climate Change Canada).
Limitations and Mitigation Strategies
Achievement of Outcomes:
The extent to which the evaluation can assess the results of the reporting and governance functions are limited as there were no pre-developed indicators for the collected data. Moreover, the two functions have continually changed to become more efficient and respond to emerging needs since the IICP was originally announced in 2016. The responsibility for the governance and reporting functions were also transferred from the Programs and Operations Branch (POB) to the Policy and Results Branch in May 2017, making it difficult to focus on measuring the effectiveness of the lead department’s role. Given the above, there is limited impact data available.
However, the purpose of the evaluation was primarily formative in nature, focusing on implementation aspects: the delivery of outputs and very early immediate outcome information in terms of impact on target groups only. For example, participant perspectives were investigated to ensure their perspectives on the effectiveness of the governance and reporting products and processes were assessed. The triangulation of various perspectives and the use of clear examples ensured that any conclusions made are substantiated.
Efficiency:
The efficiency analysis is also limited to INFC resources that are part of the IICP Secretariat. While the initiative involved not only the work of the IICP Secretariat, but other resources in INFC, such as the Communications Branch, as well as the resources of 14 OGD partners, several representatives of central agencies and various higher level committees that were involved in governing the IICP, detailed expenditure or FTE data was not available. To address this, the appropriate analysis was performed on any data available about financial information of the comparative horizontal projects to evaluate efficiency.
Lack of quantitative information:
Given the nature of the evaluation questions, qualitative data sources provided the primary source of data. Assessments, perceptions and opinions highlight the successes and challenges of the governance and reporting functions as there were limited quantitative measures on the how well a large horizontal initiative is coordinated. To mitigate the risk of bias of qualitative data, quantitative data from document reviews and statistics was used to substantiate opinions. Also, views and perspectives were triangulated from multiple responses to ensure they are validated and well supported. Preliminary findings were presented to the program area for their validation to ensure perspectives were accurately depicted.
Summary Evaluation Matrix
Evaluation Questions |
Indicators |
Method |
---|---|---|
Q1. To what extent does the INFC leadership role support progress towards delivering the IICP government-wide initiative, in partnership with OGD federal partners and central agencies (PCO, and TBS and Department of Finance)? |
1.1 Clarity of INFC’s role, responsibility and mandate to all partners and details including:
|
|
1.2 Quality of INFC’s leadership, from the perspective of key partners (including central agencies, OGDs) and INFC staff, in terms of:
|
|
|
Q2. To what extent does the formal IICP governance structure effectively support the implementation of the IICP? What can be done to improve it? What informal collaboration approaches support the formal governance structure and are they effective? |
2.1 The extent to which the governance structure changed between 2016-2018 and the purpose of these changes, including the purpose/structure/meeting frequency |
|
2.2 Effectiveness/value added of the INFC-led governance committees/processes in providing updates and facilitating discussion, as reported by key partners, including:
|
|
|
2.3 Extent to which informal coordination/communication occurs to support formal governance structure, including description of the types/frequency of informal communication above/beyond the formal committee meetings |
|
|
Q3. To what extent does the IICP governance structure support an effective decision-making process? How this can be improved? |
3.1 Examples (#/type) of discussions/issues, decision made, by the type of meetings and with who (if at all possible, track % of or degree to which issues raised that were adequately addressed, or barriers removed/reduced) |
|
3.2 Extent to which governance structures/processes effectively and efficiently come up with solutions for identified issues/challenges, as reported by key partners, including:
|
|
|
Q4. To what extent is the reporting function effectively managed internally, in collaboration with OGDs and central agencies (e.g., PCO and TBS)? What can be done differently to improve it? |
4.1 Extent to which IICP reporting requirements have been met between 2016-2018, including reporting obligations to TBS, PCO, OGDs, and INFC senior management |
|
4.2 The value of/satisfaction with the established reporting/data collection relationship and/or data aggregation process, as reported by INFC and key partners, including: satisfaction overall, challenges, and ways to improve the relationship/process. |
|
|
Q5. How effective are the reporting tools in place? What can be done to improve them in order to facilitate the consolidation for various reporting purposes? |
5.1 The quality of reporting tools/templates, including: satisfaction, utility/clarity, effectiveness, type of tool/template, as reported by key partners and INFC staff |
|
5.2 The quality/utility of data collated by OGDs, as reported by INFC and key partners, to meet various reporting requirements from TBS, PCO, OGDs, and INFC senior management, including: satisfaction, utility/clarity, effectiveness. |
|
|
Q6. To what extent is the IICP’s information publicly available, easily accessible and useful for stakeholders (and parliamentary organizations, such as the Parliamentary Budget Office)? |
6.1 Perception of effectiveness of public communication products (e.g., implementation tracker website), including any challenges, and/or ability to produce data/products for public consumption. |
|
6.2 Number of communications products published between 2016 and 2018 and evidence of reach. |
|
|
6.3 Extent to which IICP information/data is available and accessible for public/parliamentary information, use or consumption. |
|
|
Q7. What are the lessons learned and good practices that other horizontal initiatives can provide, in support of horizontal governance and reporting? To what extent can they be considered in the ongoing implementation of the IICP? |
7.1 Comparison with similar horizontal governance initiatives across the Government (e.g., Pan-Canadian Framework on Clean Growth, Ocean Protection Plan) to determine how IICP’s governance and reporting function compares on key variables |
|
Q8. What resources are dedicated to the Horizontal Reporting and Governance Initiative in INFC and OGDs? How can resources best be optimized? |
8.1 Resources utilized between 2016 and 2018 to perform INFC’s governance and reporting role (including financial, FTE, other resources). |
|
8.2 Perspectives of key partners on 1) the level of effort/resources that required to them to align to IICP initiative requirements and 2) what improvements could be made to minimize costs and/or use existing resources more cost-effectively. |
|
Annex D: Summary of Analysis of Horizontal Initiatives Compared to IICP
|
IICP |
Federal Contaminated Sites Action Plan |
Oceans Protection Plan |
Pan- Canadian Framework on Clean Growth and Climate Change |
---|---|---|---|---|
Lead department |
INFC |
ECCC |
TC |
ECCC |
Overall budget of initiative |
$187 billion |
$4.5 billion |
$1.7 billion |
Not clear |
# of FTEs in lead department secretariat |
10 |
12 |
6 |
13 |
# of reporting departments |
14, plus central agencies |
18 |
4 |
19 departments and agencies, provinces and territories, indigenous communities |
Key reporting features |
|
|
|
|
Years in operation |
2016 to 2028 |
2005 to 2020 |
2016 to 2021 |
2016; ongoing program with Long Term targets(i.e. 2030) |
Annex E: Acronym list
ADM |
Assistant Deputy Minister |
ARC |
Cabinet Committee on Agenda, Results and Communications |
CCPI |
Canada’s Core Public Infrastructure |
DM |
Deputy Minister |
DMCC |
Deputy Ministers’ Coordinating Committee |
DP |
Departmental Plan |
DRR |
Departmental Results Report |
FCSAP |
Federal Contaminated Sites Action Plan |
FTEs |
Full-time equivalent |
HRR |
Horizontal Results and Reporting |
ICC |
Invest in Canada Committee |
IICP |
Investing in Canada plan |
INFC |
Infrastructure Canada |
INFEA |
Infrastructure Economic Accounts |
IT |
Information Technology |
MRT |
Ministerial Results Table |
OGDs |
Other government departments |
PBO |
Parliamentary Budget Officer |
POB |
Program Operations Branch |
TBS |
Treasury Board Secretariat |
ToR |
Terms of reference |
Footnotes
- Footnote 1
-
The original Budget 2016 allocation was $14.4 billion. However, approximately $250 million was re-profiled to the Budget 2017 allocation.
- Footnote 2
Formerly Indigenous and Northern Affairs Canada (INAC)
- Footnote 3
-
Regional Development Agencies also received funding.
- Footnote 4
-
The effective start date of the 2016 Budget announcement.
- Footnote 5
Working with the program area, three initiatives were chosen: Oceans Protection Plan, Pan-Canadian Framework on Clean Growth and Climate Change, and Federal Contaminated Sites Action Plan.
- Footnote 6
-
Status Report on Phase 1 of the Investing in Canada plan, Feb 2017, http://www.pbo-dpb.gc.ca/en/blog/news/Phase_1_Investing_in_Canada_Plan
- Footnote 7
-
Canada’s New Infrastructure Plan: 1st Report to Parliament- Following the money, Mar 2016, http://www.pbo-dpb.gc.ca/web/default/files/Documents/Reports/2017/NIP/New%20Infrastructure%20Plan_EN.pdf
- Footnote 8
- Footnote 9
-
INAC, ESDC, CMHC, HC, PCH, RDAs, NRCan, ECCC and NRC.
- Footnote 10
-
Legacy programs are infrastructure related funding initiatives announced in federal budgets prior to Budget 2016.
- Footnote 11
-
Canada’s New Infrastructure Plan: 1st Report to Parliament – Following the Money, February 2017
- Footnote 12
-
Each time someone looks at a page on the GeoMap website, it is counted as a visit to that site. However, that same person may view multiple pages during that single visit; each click increases the number of views for that site.
- Footnote 13
-
Source: Data provided by INFC Communications Branch.
- Date modified: