Five-Year Departmental Evaluation Plan 2020-2021 to 2024-2025

2.1 Departmental transition

In August 2017, the Prime Minister announced the dissolution of Indigenous and Northern Affairs Canada (INAC), and the creation of two new departments, ISC and Crown-Indigenous Relations and Northern Affairs Canada (CIRNAC), designed to better meet the needs of the people they serve, to accelerate self-determination and the closing of socio‑economic gaps, and to advance reconciliation. ISC was created to ensure a consistent, distinctions-based approach to the delivery of services to Indigenous peoples. ISC believes that transferring control to Indigenous partners is the best way to improve services and support their inherent right to self-determination with flexible approaches that are inclusive and responsive to the diversity of Indigenous peoples. Its status as a distinct department was formalized in July 2019 with the Department of Indigenous Services Act.

With that priority in mind, the transition has been a continuing process that has affected both departments as resources and responsibilities are divided to align and support the respective mandates of each department.

From an operational perspective, the evaluation function continued as a common service shared between ISC and CIRNAC since the initial announcement was made in 2017. In July, 2019, the Evaluation function was then divided so that each department would have a dedicated organization. The ISC Evaluation function now reports to the Assistant Deputy Minister (ADM) Strategic Policy and Partnerships, who is also designated by the Deputy Minister (DM) as the Head of Evaluation.

Positioning the evaluation function within the Strategic Policy and Partnerships Sector of the Department is a catalyst for closer alignment with the overall strategic direction of the department and will more proactively help to shape and support the re-design of the department's policies and programs with a view to transferring control to Indigenous partners.

2.2 Departmental mandate and vision

ISC's mandate is to work with First Nations, Inuit and Metis to improve access to high quality services, improve well-being in Indigenous communities across Canada, and support Indigenous peoples in assuming control of the delivery of services at the pace and in the ways they choose. Its vision is to be a trusted partner that respects Indigenous perspectives and supports First Nations, Inuit and Métis in controlling the delivery of high-quality services for healthy, prosperous and thriving communities.

2.3  Strategic Policy and Partnerships Sector

SPP Sector supports the Department's legislative mandate by advancing, in an integrated, co-developed, sustainable and evidence-based way, the improvement and phased transfer of services to Indigenous peoples, for Indigenous peoples, by Indigenous peoples. SPP Sector will serve as an internal catalyst, driver, and agent of change, supporting sectors in improving and transferring Indigenous services, and building partnerships that pave the way for Indigenous jurisdiction and control.

2.4 Evaluation and Policy Re-Design Branch

The role of the Branch is to lead horizontal policy integration to advance departmental priorities, identify innovative policy interventions and partnerships to improve and transfer services, evaluate the success of programs in a neutral manner, and support the advancement of Indigenous evaluation capacities and functions.

The Branch further articulates its role to identify forward-looking, innovative, cross-cutting, horizontal policy interventions to support ISC sectors, other government departments and Indigenous partners in accelerating the transfer of Indigenous services, and leveraging partnerships to advance and evaluate progress on Indigenous-centric service design and delivery.

2.5 Evaluation Directorate

Aligned with the Department and the SPP sector visions, the role of ISC Evaluation is to offer robust, timely and meaningful evidence to help shape and support the re-design of the departments' policies and programs, and to inform and influence improved service delivery and the transfer of services to Indigenous partners.

In this context, the evaluation function is anchored in three main areas: 

1. Delivering and managing core work on program evaluations and activities in accordance with the TBS Policy on Results (2016) and the Financial Administration Act (FAA);
2. Discovering and implementing innovative evaluation methods and approaches with Indigenous partners that specifically support service transfer; and
3. Exploring and implementing ways to support Indigenous evaluation capacity and advance Indigenous-led evaluation functions. This work is crosscutting to (1) and (2) above and underpins all of the evaluation work in this 5-year evaluation plan.

2.6 Transition and impacts

One of the consequences of the transition is a broadened scope of work, and increased workload because of the additional programs for which the department is now responsible. In 2017, the First Nations and Inuit Health Branch (FNIHB) was transferred from Health Canada to ISC, as the health programs complement the Department's responsibilities around income assistance, education and infrastructure; fourteen programs were transferred to ISC with no additional resources for evaluation activities. The refreshing of the Departmental Results Framework for 2020-2021 resulted in the transfer of five programs from CIRNAC to ISC's Program Inventory: Individual Affairs (includes Treaty Annuities); Indigenous Entrepreneurship and Business Development; Economic Development Capacity and Readiness; Land, Natural Resources and Environmental Management; Statutory, Legislative and Policy Support to First Nations Governance.

The machinery of government changes and the organizational changes over the past year, as well as the increased scope (coverage) of evaluation work, has presented some challenges in completing the evaluations planned for 2019-2020. These challenges are expected to continue, as some work will be carried forward into 2020-2021 and more time is needed to stabilize the organization and the planning cycles. There are also important considerations regarding  the additional challenge of conducting evaluations during the COIVD-19 pandemic. These are addressed in section 4.6 below.

2.7 Program structures

ISC's activities are largely structured by funding arrangements or formal agreements with individual First Nations, Inuit and Metis and/or provincial or territorial governments. Most departmental programs, representing a majority of spending, are delivered through partnerships with Indigenous communities and in very few cases federal-provincial or federal-territorial agreements are in place. However, other funding structures have been established that are expected to become increasingly common as the department moves to transfer services to Indigenous entities. For example, the British Columbia Tripartite Health Governance program involves a joint agreement between the Province of British Columbia, the First Nations Health Authority and ISC on health governance. Under this agreement, the parties agree to a tripartite evaluation, led by the First Nations Health Authority, with the final report being shared between the parties. In the future, more non-standard funding arrangements can be expected, with ISC Evaluation examining the 'readiness' of Indigenous governments or institutions to assume program delivery.

Currently, ISC has 33 authorities (voted Grants and Contributions) to support the 34 programs in its program inventory. However, a one-to-one relationship between the authorities listed in the Main Estimates and the Program Inventory of ongoing programs of Grants and Contributions deemed a 'program' by the Department exists in very few instances.

A number of programs draw on multiple grants and contributions; and conversely, there are grants and contributions that contribute to multiple programs. This plan has been developed using the Treasury Board's Interim Guide on Results, which was updated most recently in August 2018, where there is the recognition that a 'program' of Grants and Contributions can cut across two or more programs in the Program Inventory and the relationship is not always one‑to-one.

This structure has presented the department with some challenges in ensuring evaluation coverage for all authorities because of the segregation of a single authority over multiple programs with different evaluation timelines.

2.8 Evaluation planning methodology

The planning methodology is a multi-step process that begins with a review of the previous year's planning approach to update the steps where necessary, and to review all relevant guidance and planning documents, such as the Departmental Plan and Treasury Board commitments.

For 2020-2021, ISC refreshed its Departmental Results Framework (DRF) – used to report its high level, core responsibilities and outcomes – and its Program Inventory. Evaluation planning was conducted in alignment with that framework. As noted, the updated 2020-2021 Program Inventory includes five additional programs that were previously delivered by CIRNAC.

The 2020-2021 Core Responsibilities for ISC are:

• Services and Benefits to Individuals – with four supporting programs
• Health and Social Services – with fourteen supporting programs
• Governance and Community Development Services – with thirteen supporting programs
• Indigenous Self-Determined Services – with three supporting programs

Appendix A presents the entire Program Inventory for ISC by core responsibility, with additional programs noted.

The Plan is also expected to align with the 2020-21 Risk-Based Audit Plan, which is prepared by the Audit and Assurance Services Branch (AASB). It should be noted that the AASB is part of the Audit and Evaluation Sector of CIRNAC and it serves both CIRNAC and ISC. As has been past practice, consideration will continue to be given to timing audits and evaluations in a way that does not over burden programs. In addition, steps will be taken to consider the Corporate Risk Profile, which is also updated every year.

Each evaluation included in the Plan has been further assessed as to the type of evaluation required (e.g., impact, formative) based on the review of Financial Administration Act or Treasury Board requirements and stakeholder consultations, as well as noting whether it is a horizontal or non‑horizontal evaluation.

According to Section 42.1 of the Financial Administration Act, government departments and agencies must review, at least once every five years, the relevance and effectiveness of each ongoing Grants and Contributions program. Credit for evaluation coverage is granted upon deputy head approval of an evaluation report, under the premise that approval allows for the sharing of information in support of management action and the sharing of results with the public. Evaluation planning is conducted with that requirement in mind, reviewing the last fiscal year in which a program was evaluated, and scheduling completion and approval of the program evaluation five years later. Note that programs that are not funded through voted Grants and Contributions are not subject to the Financial Administration Act requirements.

However, as per the Policy on Results (Section 2.5), programs with five-year average actual expenditures of less than $5 million per year can be exempted from Section 42.1 and do not need to be evaluated every five years. Guidance from Treasury Board Secretariat states the five-year average is to be calculated on three years of actuals (Public Accounts) and two years of planned spending (Main Estimates). These calculations were conducted as part of the planning process and, based on these parameters, the following programs are not scheduled for evaluation:

The Evaluation and Policy Redesign Branch conducted individual consultations in June-July 2020 with ISC senior management (Assistant Deputy Ministers – ADMs) on the draft five-year schedule of evaluations to obtain their input on priorities for their sector and on the timing of evaluations. Their input was synthesized and considered as part of the finalization process for the five-year schedule. The following senior management offices were consulted:

• Senior ADM Regional Operations
• Senior ADM First Nations and Inuit Health Branch
• Chief Finances, Results and Delivery Officer
• ADM Child and Family Services Reform
• ADM Lands and Economic Development
• ADM Education and Social Development Program and Partnership

The planning methodology is largely driven by the requirements of the TBS Policy on Results, the Financial Admiration Act (FAA) as well as identified risks. Going forward, ISC Evaluation will also engage National Indigenous Organizations on future iterations to ensure greater transparency of the planning process and to take into consideration the priorities of Indigenous partners.

3.1 Status of evaluation completion in 2019-2020

Looking back on the previous year, four (4) evaluations were completed and two (2) are in the final report stage, awaiting formal approval. These evaluations were:

• Evaluation of Assisted Living
• Evaluation of Home and Community Care
• Evaluation of Nutrition North Canada (Horizontal; CIRNAC-led)
• Evaluation of the Aboriginal Health Human Resources Initiative

1. Evaluation of Elementary and Secondary Education (awaiting formal approval)
2. Evaluation of Post-Secondary Education (awaiting formal approval)

Also completed during the fiscal year was an unplanned project: The Rapid Assessment of the Ring of Fire Community Well-Being Pilot Project. The Ring of Fire deposits represent the most significant chromite discovery in North America and are expected to create more 26,000 direct and 50,000 in-direct jobs. Indigenous communities located in the Ring of Fire are among the lowest-scoring on the Community Wellbeing (CWB) Index in Ontario^{Footnote 1}, which threatens their ability to leverage the opportunities for training, employment, business development and infrastructure offered by the Ring of Fire developments. Recognizing that basic health and social conditions needed to be significantly improved before communities could agree to or benefit from these opportunities, Indigenous Services Canada, Health Canada and Employment and Social Development Canada officials decided to pilot a new approach to community development in order to accelerate improvements in individual and community well-being. The Ring of Fire Community Well-Being Pilot Project was initiated in the Spring of 2016 by the Federal Government along with the Province of Ontario in three Matawa First Nations communities: Webequie, Marten Falls, and Neskantaga First Nations. ISC Evaluation was asked by the Ontario Regional office, as well as the Director of the Land's and Economic Development Sector, to conduct a neutral Rapid Assessment of the pilot project. The rapid assessment was conducted between July 2019 – October 2019 and focused on the relevancy of the Community Well-Being Pilot Project, the effectiveness of the place-based, whole-of-government approach to community development and lessons learned to inform future programming using a similar approach.

3.2 Other ISC Evaluation activities in 2019-2020

In addition to conducting evaluations, ISC Evaluation undertakes a range of activities to support and strengthen evaluation across the department. These include corporate planning and reporting at the departmental level, supporting capacity building and outreach activities and developing research and special studies as opportunities or needs arise. In 2019-20 ISC evaluators engaged in the following:

4.1 Planned spending

Based on the 2020–2021 Main Estimates, total planned spending for 2020-21 is outlined in the table below according to the Department's core responsibilities.

4.2 COVID-19 emergency funds

As of August 20, 2020, over $2 billion has been committed in specific support to Indigenous and northern communities and organizations in response to the COVID-19 pandemic. Of this amount, approximately $1.8 billion will be delivered by ISC. At this time, there is not a specific evaluation project identified to evaluate these emergency funding initiatives. Instead, as part of ongoing and future evaluations, ISC will take into consideration and/or address what impact COVID-19 emergency funds have had on the activities and results of core programming. For example, ISC is undertaking a combination of survey, focus group and case study work over the next few months to determine early lessons learned on the department's response in the context of the planned Evaluation of the Communicable Disease Control and Management program. Similar efforts for other planned evaluations are being made going forward. In addition, while the Plan is updated annually, it is evergreen. ISC Evaluation will monitor progress and adjust the plan as the pandemic evolves and as priorities and needs change.

4.3 Coverage

ISC plans for 34 evaluations for the five-year planning cycle covering 99 percent (99%) of planned departmental spending from 2020-21 to 2024-25. The remaining one percent (1%) represents Internal Services ($170,639,148). Departmental planned spending includes salary, operations and maintenance, capital, statutory payments and the costs of employee benefit plans, as well as grants and contributions funding.

There is one program (Statutory, Legislative and Policy Support to First Nations Governance) below the $5 million threshold that exempts the program from Section 42.1 of the Financial Administration Act.

Of the 34 evaluation projects planned, 31 are being conducted to meet mandatory evaluation requirements, including fulfilling evaluation commitments specified in TB submissions and addressing the statutory requirements of the FAA (section 42.1), taking into consideration the exemptions identified in the TBS Policy on Results. Two (2) projects are discretionary evaluations (non-mandatory under the Policy on Results). One was identified during previous consultations with senior management and the Deputy Ministers as being of particular value to support decision making. The remaining evaluation is being led by the Program to suit its own decision-making processes. Two (2) of the projects are horizontal evaluations that will address initiatives across multiple departments and organizations. ISC is a partner department in these horizontal evaluation projects.

ISC covers 100 per cent of its voted Grants and Contributions amounts.

4.4 Resources

The Evaluation Directorate has the following planned expenditures for 2020-21 to deliver on the commitments made in the Departmental Evaluation Plan. In addition to the figures in Table 5 below, the Directorate is in the process of accessing additional Vote 10 (Grants and Contributions) to work in partnerships with Indigenous communities and organizations on the co-development of evaluations.

4.5 Challenges