Evaluation of the Contaminated Sites On-Reserve (South of the 60th Parallel) Program

Overview

• This Management Response and Action Plan, developed together with the ISC Evaluation Directorate in March 2024, was developed to address recommendations resulting from the Evaluation of the Contaminated Sites On-Reserve (CSOR) Program.
• The Lands and Environmental Management Branch (LEMB) recognizes the findings outlined in the evaluation regarding the performance and delivery of the Contaminated Sites On-Reserve. Specifically, ISC-LEMB recognizes the need to:
  • strengthen the delivery of the program by identifying gaps and challenges of existing internal human and financial capacity for contaminated sites management;
  • work with First Nations partners to ensure on-going access to qualified and dedicated human resources within their communities;
  • maximize socio-economic benefits from the contaminated sites projects to Indigenous communities and businesses;
  • identify and leverage funding mechanisms to provide financial support for activities not currently funded under the CSOR Program; and,
  • revise and update the Program's performance measurement framework to ensure that it accurately captures and reflects the tangible impact of the CSOR Program on First Nations' priorities.
• This evaluation provides five recommendations to improve the delivery and effectiveness of the CSOR Program. All recommendations are accepted by the CSOR Program, and the attached Action Plan identifies specific activities to move towards meeting these recommendations.
• The evaluation encompasses the timeframe spanning from 2014 to 2019, offering a comprehensive assessment of the CSOR Program's performance and outcomes during this period. Since then, the CSOR Program team has proactively initiated several measures aimed at addressing specific recommendations that were identified through the evaluation process. There are actions from the evaluation's recommendations that will be addressed in the coming years.
• In recognizing the complexity and dynamic nature of the CSOR Program's objectives, there is acknowledgment that certain actions may necessitate additional time, resources, or strategic planning for effective implementation. Over the next 3 years, the CSOR Program commits to address outstanding recommendations in a comprehensive manner. This involves developing specific action plans, collaborating with stakeholders, allocating resources, and ensuring continuous monitoring and evaluation to align with best practices and emerging needs. ISC`s Evaluation team will conduct quarterly reviews of the Management Response and Action Plan, sharing updates with the ISC Performance Management and Evaluation Committee (PMEC) to monitor progress and activities.

Assurance

• The Action Plan presents appropriate and realistic measures to address the evaluation's recommendations, as well as timelines for initiating and completing the actions.

Key players

Implementation of ISC's CSOR Program is supported by Environment and Climate Change Canada's FCSAP, which provides funding for the CSOR Program on a cost-share basis. The FCSAP was established in 2005 as a 15-year program with funding of $4.54 billion from the Government of Canada. The program was renewed for another 15 years (2020 to 2034) with $1.16 billion announced in Budget 2019 for the first five years (Phase IV, 2020 to 2024)^{Footnote 8}.

The FCSAP provides the majority of funding while ISC's Regional Operations' Capital Facilities and Maintenance Program provides the balance. More specifically ISC contributes 20% of assessment costs and 15% of remediation costs for its projects, unless the project's value is $90 million or higher, in which case cost-share requirement for remediation is waived. During the evaluated period, ISC did not have any projects that would meet the $90 million threshold to waive cost-share requirements.

ISC's Lands and Economic Development (LED) Sector coordinates and flows funding to First Nations, liaises with the FCSAP Secretariat and Treasury Board Secretariat (TBS), and works with First Nations to develop and secure contracts for work to be undertaken on reserve.

The roles and responsibilities for implementing the CSOR Program can be broadly divided into three categories:

• First Nation recipients, who are responsible for managing ISC funding through grants and contributions funding agreements, and running a 'request for proposals' procurement process where they will select and oversee the work of a qualified consultant;
• ISC Regional CSOR Program staff, who manage and deliver on individual projects with the support of First Nations. Regional delivery models vary, and in some regions the link between LED Environment Units and Capital/ Infrastructure staff (Capital or Technical Services Officers) is closer than in others; and
• ISC Headquarters, who are responsible for the effective administration and oversight of the program, providing national level reporting and coordination with federal partners under the FCSAP Horizontal Initiative.

Context

Information on all known federal contaminated sites is contained in the Federal Contaminated Sites Inventory (FCSI). FCSI is a searchable online database managed by the TBS and includes a public-facing component. This component enhances accessibility, allowing the public to access relevant information about contaminated sites across the country.

As of March 2020, FCSI listed 23,714 contaminated sites from all custodian departments, including ISC. Of these, 1,795 were suspected sites, 4,860 sites were being assessed or remediated and 17,059 sites were closed and require no further action^{Footnote 9}.

The CSOR Program manages information related to its sites through the Integrated Environmental Management System (IEMS), which is an internal-facing project management database overseen by CSOR Program staff within ISC. IEMS facilitates the input of the CSOR Program's information to FCSI. IEMS includes site-specific detailed work plans and quarterly site reports that inform both planning and reporting processes.

Between April 2014 and March 2020, the ISC CSOR Program managed 1,013 active contaminated sites on reserves south of latitude 60˚N. On March 31, 2020, the reported liability of these sites was estimated at $249.1 million. Table 1 below shows the distribution of sites in the IEMS across the site classifications by number, percentage of total sites, and percentage of total expenditures made by the CSOR Program.

The CSOR Program's inventory of sites were spread throughout Canada's provinces, which is shown in Figure 2 below. Among the 1,013 contaminated sites that had CSOR expenditure between April 2014 and March 2020, the majority were in British Columbia (45%), followed by Manitoba (23%), and then Ontario (17%).

Among the Class 1 sites with expenditure, sites in Ontario made up the greatest share, with 40% (80 of 202) of all Class 1 contaminated sites nationally. Regarding sites that were either suspected or had insufficient information to classify them between April 2014 and March 2020, Manitoba, Saskatchewan, Alberta, and British Columbia had a higher representation of sites compared to other regions.

Text alternative for Figure 2. Sites by province and Class, between April 2014 and March 2020 (n=1,013)

Figure 2 is a stacked bar chart that shows the regional distribution of potentially contaminated and contaminated federal sites on-reserve that have had assessment or remediation activities between April 2014 to March 2020, divided across different site classifications. Sites classifications include: suspected, insufficient information to classify, not a priority for action, class 3, class 2, and class 1. A total of 1,013 sites had CSOR activities during the temporal scope of the evaluation. British Columbia had 452 sites with CSOR activities. Manitoba had 233 sites with activities. Ontario had 174 sites with activities. Alberta had 60 sites with activities. Saskatchewan had 49 sites with activities. Quebec had 42 sites with activities. The Atlantic region had 3 sites with activities.

Unique context

A majority of ISC interviewees agreed that the FCSAP funding is an important component of the CSOR Program, and that the funding creates opportunities to assess and remediate sites that ISC could not otherwise afford to address. There are, however, some challenges associated with relying on the FCSAP funding structure. FCSAP is structured to support a variety of federal departments and agencies to reduce liabilities related to contaminated sites, and ISC, along with Crown-Indigenous Relations and Northern Affairs Canada (CIRNAC), has a unique context that is not common to other custodians under FCSAP. One of the major challenges is that for other custodial departments, the control over the contaminated sites rests with that department. Once a site has been assessed and remediated, it can be 'closed' and there is a low risk for recontamination due to future misuse. As well, in many instances, it is the custodian department who was responsible for creating the contamination in the first place, and the custodian department is aware of the suspected contamination. In contrast to other custodian departments, ISC (and CIRNAC) do not have control over contaminated sites, as the management of reserve lands is primarily the responsibility of First Nations. New contamination may arise due to new activities caused by third parties, but also older, previously unreported contaminated sites are still being discovered.

This has meant that while other custodians experienced a sharp rise in liabilities with the initial FCSAP phases assessments and then have seen a reduction over time, for ISC and CIRNAC, there are new suspected sites reported year after year. While the FCSAP Secretariat has appropriately responded to the bulk of its custodians by focusing its funding support on remediation rather than assessment in FCSAP Phase III (2016 to 2019) and Phase IV (2020 to 2024), this has left ISC in a position where the funding structure is no longer meeting the unique needs and demands of the CSOR Program. While specific limitations related to funding are outlined throughout the rest of this section, more information on the impacts of the relationship between FCSAP and the CSOR Program is discussed in Finding 9.

Inadequate funding

Overall, the evaluation found that there is a greater demand for the CSOR Program than the available funding can address. ISC interviewees often mentioned that ISC lacked the funds to satisfy FCSAP cost-share requirements. Because of inadequate access to funding, First Nations are consistently unable to assess and remediate all the contaminated sites within their jurisdiction. There are specific requests from both ISC staff and First Nation interviewees for the CSOR Program's funding to reflect the increase in costs (due to the COVID-19 pandemic, climate change, inflation, etc.) and the high costs to mobilize equipment and consultants in the north and in remote communities.

ISC interviewees mostly emphasized the insufficiency of CSOR Program funds available for assessment. Between April 2014 and March 2020, 10% of CSOR expenditures were directed to assessment activities, while 90% was directed towards remediation activities. While some regions and communities were able to secure funding from other sources to assess sites, this was on an ad hoc basis. The challenge related to inadequate assessment funding stems from an overall insufficiency of ISC funds to fill the gap between the CSOR Program's needs, and the FCSAP funding's prioritization of remediation over assessment.

Without funding to assess the expected remediation cost of sites that are suspected to be contaminated, the Program and communities are not able to comprehensively plan and budget for future remediation. On one hand, some interviewees mentioned that the extent of contamination is often more than expected because of a lack of or out-dated of assessment, resulting in remediation costs that are over-budget. On the other hand, some sites do not require any remediation at all, possibly resulting in the artificial inflation of the CSOR Program's inventory of sites and the community' unnecessary concern about contamination. Among the sites that closed between April 2014 and March 2020, 44% of sites (157 out of 353) closed without any remediation.

Because FCSAP prioritizes sites that have been assessed to be high risk, a few regional officers also shared that insufficient assessment funding hindered the ability of First Nations to secure funding for the high number of suspected sites on their reserve land. During the evaluated period, funding for the remediation of sites was also highlighted by a few First Nation interviewees as inadequate, as the insufficient funds for remediation resulted in delays between the assessment and remediation for some sites that were not prioritized for funding, but which still impacted their communities.

Annual funding

Some First Nations interviewees and survey respondents explained that ISC's allocation of funds on an annual rather than multi-year basis hindered long-term comprehensive planning in their communities. In order to develop a community plan for the assessment and remediation of their contaminated sites, First Nations requested that the FCSAP Assistant Deputy Minister's Committee define a multi-year budget, indicating that this would have the dual benefit of facilitating multi-year contracts with consultants and support the self-determination of First Nations^{Footnote 26}. As the program does not have dedicated and permanent A-base funding^{Footnote 27}, the CSOR Program secured its funding from the Capital and Facilities Maintenance Program on an annual basis during the evaluated period. An internal document review revealed concerns that this could make entering multi-year agreements with First Nations more risky. A few ISC interviewees also mentioned that because Regional capital funding was used for various capital projects in addition to contaminated sites, competing priorities could make it challenging to secure funds for the CSOR Program. Though beyond the temporal scope of the evaluation, in FCSAP Phase IV (2020 to 2024) the CSOR Program moved to a five-year funding allocation, and it is anticipated that this will also be the case for FCSAP Phase V.

The reactive nature of the CSOR Program

While they are within the eligible activities under the CSOR Program, activities to prevent pollution do not currently receive funding given the limited financial resources available. This results in the ongoing creation of new liabilities. One ISC staff interviewee shared: "If we're not investing in the prevention and helping educate and increase capacity of those Indigenous leaders within those communities to understand what's happening, we're just going to continually do this cycle to the end of time." Some First Nation interviewees shared their desire for more public education related to causes and effects of contamination, while a few ISC interviewees identified small investments to upgrade infrastructure, such as replacing fuel storage tanks and related hoses or equipment, as beneficial prevention activities. In fact, the leading causes of contamination for all sites as recorded in the IEMS (n=3,512 as of August 2023) include fuel related practices (50%), and inadequately managed landfill and waste sites (29%). The evaluation heard from both ISC staff and First Nation interviewees that providing an operations and maintenance budget to staff facilities at high-risk for contamination, such as bulk fuel storage facilities, would be a beneficial investment for the CSOR Program. The Program also does not have many enforcement mechanisms to support the "polluter pays" principle or regulations to prohibit contamination on reserve, with one ISC staff interviewee sharing that they write strongly worded letters to the polluter and hope that this will spur action. While legal action is possible, the court cases can be lengthy and do not always result in the third party being forced to pay, which can delay remediation of some sites for decades. In addition to the lack of enforcement options, a gap in federal and provincial/territorial responsibility for environmental protection means that not all First Nations have sufficient regulations to prohibit contamination on reserve^{Footnote 28}. Without prevention or enforcement activities, the root cause of contamination is not addressed.

Spotlight on prevention efforts and enforcement challenges

After One Arrow First Nation closed down a dumpsite and replaced it with a solid waste transfer site, the Nation experienced challenges in enforcing laws against illegal dumping. Leading up to the closed landfill, there was garbage strewn up and down a closed road. Illegal dumpers exacerbated contamination by burning their trash so that others would not be able to identify them through their garbage.

The Nation's administration tried numerous methods, such as signs and physical barriers to block community members and nearby non-First Nations neighbors from accessing the site and illegally dumping their waste in the closed facility. The contamination along the road to the dumpsite can be seen in satellite images.

While the Indian Act allows for a fine of up to $100 if an individual is caught dumping illegally on reserve, representatives from the Nation say this fine has never been issued in their region.

Key Finding 5: Human and technical capacity in both ISC and in First Nations is a challenge in delivering the CSOR Program effectively. Having a sufficient number of staff who are, dedicated, qualified, and appropriately compensated is necessary to achieve good results.

The evaluation team found that, as with many ISC programs, capacity challenges hindered the CSOR Program's effective delivery. In particular, human, and technical capacity were raised as challenges within ISC, and within the First Nations responsible for delivering the CSOR Program's projects. In general, the evaluation found that ensuring staff members in both ISC and First Nations are appropriately-compensated, qualified, and have some historic or contextual understanding of communities is necessary for the Program to achieve positive results.

Staffing in the Regional Offices

The evaluation team heard that adequate capacity within ISC is needed to ensure projects are appropriately prioritized, tracked and monitored. As discussed in Finding 4, this means having appropriate financial capacity to address high-risk sites and assess suspected sites as they are reported. It also means having the appropriate human resources within the department to manage projects internally, update internal databases, and action requests for data and information from both within and outside of ISC. One ISC interviewee shared, "There's a bunch of work that has to be done [such as going] to community meetings, before [the project] even gets started; [then] when it gets started, after it gets started, [and] after it's finished. It's very heavy on [full time employees] and that was and continues to be something we have in fairly limited supply." Many interviewees were concerned about turnover within the CSOR Program, and how it impacted program delivery to be constantly onboarding new employees and have new employees trying to follow processes that are regionally defined or defined on an office-by-office basis.

A majority of ISC interviewees shared that they face operational challenges due to understaffing in their regional office, raising concerns around burnout. Some ISC interviewees discussed the challenges they faced in managing workloads for the CSOR Program when their time was being divided between that and other related program areas, such as litigation, First Nation Land Management, solid waste, community infrastructure, climate change impacts, and the department's response to the COVID-19 pandemic.

While generally ISC interviewees agreed that having science-based staff at both the working and managerial level is helpful to ensure everyone understands the projects and the process, one ISC interviewee mentioned the challenge of hiring the appropriate expertise within their unit, given that the compensation package for the staffing group and level allocated, along with the actual workload and high-profile nature of the work to be completed, was not competitive in their high-cost-of-living urban centre.

Community-level capacity

At a community-level, the evaluation heard that First Nations cannot always prioritize the assessment and remediation of contaminated sites, given other more pressing issues or capacity challenges within their communities. The evaluation team heard from both ISC and First Nation interviewees that communities may have priorities outside of the contaminated site that take up their time, resources and attention; and that getting the Nation on board with a project can be a challenge. One First Nation interviewee shared about the challenges they have experienced getting their fellow community members to discussions around their contaminated site project: "Contaminated sites is not buzzworthy. There needs to be some new terms to get people excited and interested, you've got to 'green it'." Other First Nation interviewees spoke about how there is always something happening in the community. Another First Nation survey respondent shared that "the serious nature of social issues overshadows concerns for the environment." Some ISC interviewees also shared that CSOR projects can be politically sensitive to take on, given that there may be a long lag between assessment and remediation which may reflect poorly on a First Nations' leadership, though the actual timeline for remediation is not under the Nation's control.

The evaluation heard that having the appropriate human resources within First Nations to respond to contaminated site projects, and to act as stewards for the projects within the community, is a key component of success. A few First Nation interviewees shared that their communities did not have a dedicated environmental specialist and this made it challenging to move forward on comprehensive environmental planning or to have a full picture of how contaminated sites projects will impact the Nations moving forward. One First Nation interviewee stressed the importance of having someone within the community who has the formal and technical knowledge to understand the nuances of ESAs, and can also speak in a way that the community can trust and understand, and shared the perspective that these positions must be appropriately compensated in order to both recruit and retain that specialized knowledge.

Considerations surrounding appropriate capacity within First Nations communities is further elaborated under Finding 8 related to service transfer.

Areas of efficiency

The evaluation heard that internally, ISC has achieved some efficiencies in the CSOR Program by ensuring that funding is nimble. First, ISC interviewees shared that the practice of transferring unused, surplus funds from FCSAP custodian departments, ISC regions, and other contaminated sites projects to other prioritized and 'shovel-ready' projects facilitated efficiency by maximizing resources. However, some ISC interviewees from Headquarters mentioned that they faced some challenges with transferring funds between departments as the processes had a greater administrative burden than internal ISC transfers. Furthermore, ISC is not always able to come up with the required cost-share portion needed to access surplus FCSAP funds.

Additionally, a 2016-17 transition to a "flex" funding model enabled the transfer of unused funds from one fiscal year to the following fiscal year. Again, however, despite this transition, some respondents mentioned they had been unable to take advantage of this "flex" funding model as FCSAP rules do not allow transfers between funds allocated for assessment and those allocated for remediation.

Updates to the FCSAP terms and conditions also created efficiencies for the CSOR Program. In Phase III of FCSAP (2016 to 2019), federal contaminated sites were only eligible for remediation funding if the site was classified as Class 1 (high priority) or as a Class 2 (medium priority) where remediation had began prior to 2011 (in Phase 1) and if the contamination action occurred prior to April 1, 1998. Criteria for funding eligibility was expanded in Phase IV of FCSAP (2020 to 2024), and the terms and conditions were updated to include the ability to bundle multiple sites into one project. While technically outside the temporal scope of the evaluation, this was identified by a majority of interviewees as an extremely efficient change to the Program. Bundling expands FCSAP's funding eligibility to all Class 2 and 3 sites located on a reserve, regardless of their proximity to a Class 1 site^{Footnote 29}. Bundling has allowed communities to take advantage of high cost resources that have been mobilized for high-risk Class 1 sites, such as equipment and consultants, and use them for lower risk Class 2 and 3 sites that are geographically close but may not have been able to access FCSAP resources otherwise. Bundling projects can be particularly advantageous in remote communities, where the cost to mobilize resources is higher. Furthermore, bundling sites has been reported to reduce the administrative burden on First Nation communities, as only a single, consolidated funding application is required rather than multiple funding applications for the different sites. With multiple sites included in one budget, bundling can further help communities to better forecast project spending and mitigate against the uncertainty involved when budgeting for remediation.

Finally, involving First Nations directly in projects is a source of efficiency. Most First Nation interviewees expressed that staff from the community can act as liaisons between the community and the CSOR Program's project teams, fostering local buy-in for projects and the importance of clean up, establishing credibility for projects, and developing trust and collaboration between the CSOR Program staff and communities. This can reduce potential disruptions and barriers to site assessment and remediation. Staff from the community are also better situated to respond to the needs and concerns of their nation, as community members may have existing background knowledge about the site and can establish project continuity. A few First Nation interviewees also shared that they can use the CSOR Program's reporting process to ensure their project has good value and their leadership is on board.

Ensuring that CSOR assessment and remediation projects are awarded to local firms in remote First Nation communities can also reduce wait times if projects experience delays, as the resources can be mobilized more quickly than if they are coming from an urban centre located hours away from the contaminated sites.

A few First Nation interviewees and survey respondents stressed that feasibility studies should be done with every assessment to ensure that the most appropriate course of action is being taken; and that the technology selected for the project will be compatible with the environment in which it is being used.

One interviewee from a consulting firm shared that working with a community which had the land manager on-site and directly engaged in the project had allowed them to complete more work in a much shorter time period than they had initially forecasted, and that finding additional resources within the community was generally easy given the land manager's involvement and pre-existing relationships.

Areas of inefficiency

The CSOR Program experiences some areas of inefficiency that impede productivity. First, almost all ISC interviewees expressed challenges with the Program's administrative and reporting requirements for FCSAP and ISC. ISC interviewees shared that they face a high burden of reporting. Some ISC interviewees spoke to challenges of updating databases that are not user-friendly, such as the IEMS. They mentioned how this time-consuming process can either take their attention away from completing other work, or how it may be left undone for months at a time. While reporting importantly contributes to transparency, tracking project progress, and accountability to the public, ISC staff shared that the detail and frequency of reporting can lack utility and result in duplicated work. For example, while bundling projects is a source of project efficiency, IEMS cannot accommodate this practice and individual sites must have expenditures tracked individually. This means that an officer is inputting the same information multiple times for the same project. These reporting requirements can increase ISC workloads, reduce the already limited capacity of ISC staff, and take resources away from CSOR Program delivery. Other sources of delays and administrative burdens resulting from internal processes are the extensive contracting processes, the need for ISC staff to manage multiple funding sources, and complex processes to access FCSAP funds.

Moreover, IEMS is not an effective performance measurement tool for the CSOR Program and is limited in its functions to optimize the internal processes and support decision-making. The IEMS is a program management and reporting tool used to track and report on contaminated site project and financial information. The IEMS is primarily used for financial planning and forecasting. Though, nearly all internal ISC interviewees experienced challenges and limitations with using the IEMS to conduct useful data analyses for informed decision-making. To start, the IEMS reflects funding allocations for contaminated site projects rather than actual expenditures, which makes it challenging to reflect on the concrete progress and performance of the CSOR Program's projects. The tool has also been reported to contain unreliable information which cannot be used to accurately measure performance. Many ISC interviewees shared that the unreliability and inaccuracy of IEMS stems from regional differences in breaking down project and site costs, human errors when entering data into the numerous fields within the database, time lags between contaminated site data collection and entry into the IEMS, and difficulties with accurately measuring certain datapoints such as contamination volume. A few ISC interviewees also shared challenges with extracting IEMS data to upload for use into other systems such as the Treasury Board's FCSI due to the errors in the data which must be manually corrected. A planned systems update promises more accuracy, consistency, and the ability to update site data live, though the update has been delayed past its expected date of March 2024.

Furthermore, where progress on social and environmental outcomes of CSOR projects can be tracked, the data is also not always helpful for performance measurement. The IEMS lacks the geolocation data that could facilitate place-based analysis, and there are currently no systemic means of tracking project outcome data related to GBA Plus outcomes around individual identity characteristics like sex, gender, religion, etc., or health-related outcomes as a result of the CSOR Program's interventions. As ISC is not the primary service provider, and the responsibility for managing contaminated sites projects lies directly with First Nations, GBA Plus data collection is particularly challenging for the Program. GBA Plus data that is collected is limited to FCSAP requirements to report on diversity in CSOR project employment and hiring, rather than on the impacts of the Program's outcomes on diverse groups. There is also no systematically collected data on Indigenous employment or the benefits of the Program's projects to the Indigenous community members. Finally, as noted in the "External Forces" section of this evaluation, some data on the impacts of climate change on contaminated sites has only recently been collected as a FCSAP requirement. Examining further data on social and environmental outcomes from the CSOR Program could contribute to a more holistic understanding of the program's progress towards its outcome that "Land and natural resources in Indigenous communities are sustainably managed," beyond the program's ability to reduce liabilities. However, a noted challenge is that requiring reporting on topics of GBA Plus, health outcomes, and Indigenous employment could introduce further administrative burdens on ISC and First Nations staff and that assuring the quality of the resulting data would be a significant challenge.

ISC's Internal Program Structure

The CSOR Program is structured as a lateral relationship between the CSOR Program staff at Headquarters and in the Regional Offices. At Headquarters, the CSOR Program falls under the Lands and Economic Development sector. In regions, CSOR Program delivery falls under the Regional Operations sector. Contaminated site management work spans the two sectors and is also affected by interrelated programs, such as the First Nations Waste Management Initiative, community infrastructure programs, capital projects, environment programs, and land-related programs.

First Nations

For the First Nation communities that the evaluation team visited, working with ISC Regional offices was generally a positive experience, as the regional ISC staff knew their unique context and their history. ISC and First Nation interviewees shared that great relationships between First Nations partners and ISC are built on trust, transparency, respect, responsiveness, and partnership. Interviewees put the most emphasis on the importance of trust, mentioning that ISC regional staff and project teams must build the trust of community leaders and members to mitigate impacts of historic mistrust with the Government and encourage community member's cooperation with remediation projects. "As Indigenous communities are often disproportionately affected by industrial development, meaningful inclusion of Indigenous knowledges into project development processes is an essential step toward more accurately representing Indigenous exposure to risks and harm^{Footnote 30}."

The evaluation heard from both ISC and First Nation interviewees that involving communities in leading the bidding and tendering process to choose consultants (ideally from First Nations-owned companies) contributes to community ownership, economic benefits, capacity building, and alignment with community needs. Some First Nation interviewees mentioned that hiring local contractors enables solutions that make sense locally.

The evaluation team heard that ensuring a community member is involved in all aspects of the project can facilitate local support for projects by aligning projects with community priorities and by facilitating trust from community members. The evaluation also found literature that supported the idea of involving Indigenous peoples in assessment and remediation projects to provide local socio-economic opportunities, build community capacity, result in better overall project outcomes and project design, and could even create smoother regulatory processes^{Footnote 31}.

The evaluation identified a best practice, where regional ISC employees participate directly with community members in project implementation through project management teams. The teams generally include representatives from the First Nation, ISC, and the consultant. Project management teams are a forum to discuss the progress and direction that the contaminated site project can take. The evaluation team heard that these teams created good relationships with First Nation partners through high engagement, transparency, collaboration, and technical competence.

FCSAP

The evaluation identified positive collaboration between the CSOR Program and other custodian departments within FCSAP. Within FCSAP's structure, a selection of custodian departments (Environment and Climate Change Canada, Fisheries and Oceans Canada, Health Canada and Public Services and Procurement Canada) act as 'expert support departments' who provide a peer-review service to the classification ratings given by departments to Class 1 sites in their portfolio^{Footnote 32}. ISC also connects with the other FCSAP custodians^{Footnote 33} through the FCSAP Secretariat's Regional Integrated Planning Boards, which functions as an effective knowledge-sharing body when properly resourced. The evaluation found that many ISC interviewees generally agreed that the custodian departments could better mobilize resources, and reduce duplication, as currently there are very few examples of custodial departments sharing resources to assess or remediate adjacent contaminated sites. The evaluation heard that improved integration with other FCSAP custodians is an area that could be further developed.

Training and staffing

A majority of First Nation and ISC interviewees mentioned training and capacity development when discussing the CSOR Program. In general, the evaluation found that ISC resources are not adequate to provide the appropriate level of support to meet the training and capacity development needs of communities. Many ISC interviewees shared that the CSOR Program had limited resources for training and that where training was funded or resourced, it was generally ad hoc training provided when other complementary programs, such as First Nations Land Management, did not have the budget to fulfill community's training requests. Some ISC interviewees also shared that community representatives are expressing a desire for more specialized or localized training in the CSOR Program. Some First Nation interviewees expressed that they were not aware of any or many CSOR Program-related training opportunities.

The evaluation heard about many types of methods to integrate training opportunities into the CSOR Program. Of the training desired, ISC interviewees expressed a greater need for more training on prevention efforts and project management for First Nations. On the other hand, First Nation interviewees were most likely to discuss contaminated site management process or technical training needs, with one First Nation survey respondent sharing a desire for regional CSOR Program training. Many First Nation interviewees also expressed that when they had accessed training through the CSOR Program, this training was generally experiential or "on the job" learning. The value of other modes of training and learning, however, were highlighted. Many First Nation interviewees were hopeful that they could continue to share knowledge within and between First Nation communities. A few First Nation interviewees stressed the importance of formal, classroom-based education for First Nations involved in contaminated site management to ensure that First Nations' have the technical and project management skills required to carry out projects successfully. Some regional ISC interviewees shared that they have endeavoured to include training for First Nation partners in CSOR projects by partnering with a local college to ensure First Nation students could get real work experience through the CSOR Program, or by accessing funds from other ISC programs to organize community workshops.

For First Nations to achieve self-sufficiency with assessing and remediating contamination on reserve, all interviewees and survey respondents highlighted the importance of ensuring that First Nations have adequate financial resources. Interviewees stressed that dedicated financial support is particularly important to ensure that First Nations have ongoing technical capacity, and can recruit and retain talent in their communities. Both ISC and First Nation interviewees shared that having a consistent contact for environmental work in communities was a beneficial practice for the effective delivery of projects. In Ontario, the CSOR Program provides a 1.25% administration allocation that can be used by communities for staff salary, though a few ISC staff interviewees shared that this can be a challenge when their community contact is working on CSOR projects as well as many other projects at the same time. For a particularly complex multi-year remediation project, one ISC interviewee shared that they were able to use these funds to cover the salary for a full-time staff member in the community, as well as funding for associated community engagement costs. With community buy-in and ownership, First Nation interviewees shared that they have led projects by setting standards for remediation, getting involved in project management teams, and deciding how to use remediated land based on their community's needs. By providing additional financial resources to First Nations to ensure they can hire and retain technical expertise within their communities, First Nation and ISC interviewees shared that the CSOR Program could support First Nations' capacity for service transfer.

Business development and procurement

For service transfer to occur for in the CSOR Program, the evaluation evidence suggests a need for a dedicated First Nations technical group of environmental professionals to support the CSOR Program's work, including the training of local staff to both complete work directly and manage external contractors, and to improve First Nation firms' access to contracting opportunities.

One First Nation interviewee expressed the difficulty in getting training opportunities for their members, as the community is quite small and their members can face a variety of barriers to access training, such as transportation issues, remoteness, and availability for work. One ISC staff interviewee mentioned that a good practice for training through a dedicated technical group can be found in the Circuit Rider Training Program^{Footnote 34}, a long-term capacity building program where individuals are trained on how to operate First Nations drinking water and wastewater systems and then visit different communities in a region on a cyclical basis.

ISC interviewees speculated that dedicated technical groups could be responsible for overseeing contracting and related work within each region, which would reduce the work required by individual First Nations on their projects. First Nation interviewees suggested that a technical organization could also function as training and knowledge-sharing body responsible for disseminating knowledge about contaminated site projects between neighbouring First Nations. Good examples of the training and knowledge-sharing supports in the lands context are the National Aboriginal Land Manager's Association's Regional Lands Associations, and the First Nation Land Management Resource Centre's regional bodies.^{Footnote 35}

One challenge that many ISC interviewees identified within the CSOR Program is the limited number of qualified consulting firms to hire for contaminated site assessment and remediation projects. Because there are only a few large firms in the environmental consulting sector who can conduct the work, ISC interviewees shared that they have experienced varying levels of expertise and professionalism in contractors, as well as increased project costs as the companies have greater leverage to control market prices with limited competition. Moreover, the evaluation heard that ISC regional staff and First Nations have had pre-existing challenges in their relationships with the consulting firms who do complete bids, and that they have continued to work with the same few firms due to a lack of alternatives.

A dedicated First Nations technical group of environmental professionals could help address contracting challenges, both through training, but also through supporting First Nation environmental consultant businesses. To address gaps in technical capacity and support community development, both First Nation and ISC interviewees shared that First Nation environmental consultant businesses could join larger firms and take advantage of the economic opportunities available through involvement in contaminated site management processes^{Footnote 36}. Further, many First Nation interviewees shared that hiring from a community's own local talent can both build capacity for their businesses and be beneficial for the First Nation beyond the remediation of the contaminated site.

Respecting a potential dedicated First Nations technical group of environmental professionals, concerns were raised that, in order to be effective, it must be sustainable and not limited to individual communities. For example, some ISC interviewees expressed frustration about how the remediation of a contaminated site is often a "one off" project for many communities, so the skills developed are only used for a short period of time.

Ultimately, by appropriately defining and resourcing a process to provide the level of training and business development opportunities desired by First Nations' organizations and communities, and by focusing procurement efforts to maximize benefits to Indigenous communities, the evaluation heard that ISC could support service transfer and enhance the technical and business capacity available in the environmental sector.

Performance measurement

Throughout the evaluation, the team heard about liabilities as a measure of the CSOR Program's success, and the challenges that rise when success is tracked by the reduction of liabilities. The CSOR Program's currently tracks progress based on metrics such as "reduction of liabilities" and "number of high-risk (Class 1) sites with remediation or risk management activities^{Footnote 37}." The CSOR Program reduces liabilities through remediation activities. Liabilities increase with identification of new contaminated sites, identification of additional contamination at sites, and through any increase in the estimated remediation cost. For example, a site's liability can increase if the contract bids come in higher than the cost estimate. Ideally, the reduction of liabilities is supposed to act as a proxy for the Program's efforts to clean up and close sites that no longer pose health risks, making them available for development.

However, a majority of ISC interviewees stressed that while their teams have been successful in reducing liabilities through remediation, using this as the sole measure of their performance is not an accurate reflection of their efforts and successes. This is primarily due to the various forces that can impact total liabilities which are beyond the Program's control. ISC interviewees shared that contamination and associated liabilities can change due to economic, social and environmental forces such as inflation, changes in project costs, climate change disasters that could create new contaminated sites, or changes in the available funding for the CSOR Program activities. For example, some ISC interviewees shared that the discovery of contamination on sites that have not been fully assessed makes identifying accurate liability challenging. This is due to the unknown state of contamination for sites with limited or outstanding ESA work, and progress on the assessment of sites can depend on available funding. Paradoxically, in fact, additional funding for assessments have increased liabilities due to the identification of contaminated sites that were previously classified as suspected sites. This classification effectively concealed the successful remediation work and reduction in liabilities that the CSOR Program had already accomplished. Ultimately, although fulfilling Canada's legal obligations is an appropriate goal of the program, the evaluation heard from a majority of ISC interviewees that the reduction of liabilities is not an accurate proxy for the CSOR Program's successes or efforts to sustainably manage lands and natural resources in Indigenous communities.

Beyond relying on reducing ISC's liabilities as an indicator of success, the evaluation team heard of other general and specific concerns with how the program's performance is measured. For example, many ISC interviewees shared that the Program does not have a good strategy to measure progress on the social outcomes of projects, such as decreased risk to public health and safety, and the availability of First Nations land for development. ISC interviewees shared that the proxy indicators for these outcomes are expenditures on high-risk sites and anecdotal evidence, respectively.

The evaluation also reviewed literature that raised a concern around the standards for remediating contaminated sites on reserve. To achieve program goals, the CSOR Program is intended to remediate reserve lands to a minimum acceptable standard, in order to incur the least amount of cost to taxpayers. However, these minimum standards may not always align with a First Nations' future plans for their lands, or in absence of these plans, may leave a Nation with a large tract of land that cannot be leveraged to its full socio-economic potential^{Footnote 38}. The literature reviewed by the evaluation team spoke to the importance of understanding the holistic approach to environmental management inherent to many traditional Indigenous knowledge systems^{Footnote 39}, and that using western risk assessment tools and frameworks may not readily apply to Indigenous contexts^{Footnote 40}. According to one First Nation interviewee's views, "Success is based on what do you do for the people in the community." To illustrate, the evaluation heard of cases where ISC regional offices had gone above and beyond the minimum required to ensure communities were satisfied with the scope and work related to contamination on their land; but the success of these efforts were not captured through the CSOR Program's current performance measurement approach.

Ultimately, the evaluation team heard that, while environmental liabilities constitute a federal obligation that ISC must continue to report on, by restructuring the Program's performance measurement strategy to focus on First Nations' definitions of success, the CSOR Program could be better prepared for eventual service transfer to First Nation partners, and could also tell a more comprehensive story to Canadians about the actual outcomes of the Program.

Key Finding 9: To achieve transformative change and facilitate service transfer, the CSOR Program should better reflect First Nations priorities, but this is complicated by the status of sites as federal liabilities.

The evaluation found that the priorities of First Nations are not fully integrated into the decision-making process for contaminated site management. With insufficient internal funds from ISC to assess and remediate the sites identified by First Nations, the CSOR Program relies on funding from FCSAP through cost-sharing. In this context, FCSAP's funding priorities and site eligibility conditions pose significant barriers to the service transfer of the Program. These limitations constrains the ability of First Nations to address their own priority sites as FCSAP prioritizes high-risk sites and remediation activities over assessments. Consequently, First Nations are not always able to assess suspected sites, or to access remediation funding for sites in their communities that have been deemed as lower-risk through the National Classification System for Contaminated Sites. For example, one First Nation interviewee shared that one of their sites that had been deemed low-risk by ISC, was prioritized by the community due to its higher importance and health impact, considering the proximity of the contamination to wild-growing traditional medicines. While the 'bundling' of sites has reportedly helped with addressing lower-risk sites prioritized by First Nations, one ISC staff interviewee pointed out that securing funding for suspected sites is a particular challenge.

In addition to sites that are eligible but not prioritized through the FCSAP criteria, the evaluation heard about sites that are not currently eligible but that still impact First Nations' communities. As discussed above, the CSOR Program and the FCSAP provide funds for contaminated sites that have been accepted as a federal liability. For example, FCSAP funding cannot be used to address sites that are not on reserve lands, but are within First Nations' traditional territories, or on sites purchased by a First Nation through the 'additions to reserve' process. The FCSAP funding accessed by the CSOR Program also cannot be used to address capital costs, such as the removal of contaminated buildings. During one site visit, the evaluation team saw a former gas station where the lands were being remediated under the CSOR Program, but the community had to access funding from other ISC programs to remove the building on the contaminated land. Other ineligible sites include sites where pollution has been caused by a 'private party' such as when reserve lands are held under a Certificate of Possession (CP)^{Footnote 41}. One First Nation interviewee mentioned that since some First Nations have more Certificates of Possession on their land than other Nations, the ineligibility of sites on CP lands affects some First Nations much more than others.

Sites that are ineligible under the FCSAP are supposed to be assessed and remediated at the expense of the owner or purchasing First Nation, in the case of 'additions to reserve' lands. However, one ISC staff interviewee noted that the original polluters may depart or go bankrupt, or may not be willing to pay for remediation. Without the CSOR Program's funding, First Nations may not be able to allocate their resources to site assessment or remediation, leaving these sites to continue to pose risks to community health and well-being. Finding a mechanism to support First Nations to assess and remediate contaminated sites that are currently ineligible under the Program's funding structure can amplify First Nation's authority over their lands, and could contribute to transformative change for the CSOR Program.

When the federal government is responsible for contamination, or the polluter cannot be held accountable under the "polluter pays" principle, the federal government accepts the liability. The CSOR Program and the FCSAP provide funds for contaminated sites that have been accepted as a federal liability. ISC interviewees mention that the presence of federal liability is a barrier to the full transfer of the CSOR Program to First Nations: to the extent that the Government of Canada is liable for a contaminated site, it must remain accountable for the site's management.

However, some ISC interviewees emphasized that transferring responsibility for cleaning up federal contaminated sites from ISC to First Nation partners, is not feasible, as it would conflict with the accountability system in place for the polluter pays principle. Effectively, this sort of full service transfer to First Nations would make First Nations liable for cleaning up pollution that they were not responsible for creating and had no mechanisms to prevent. For example, during one community site visit the evaluation team heard about a historic industrial polluter who had signed a long-term lease with Canada for a parcel of their reserve, without the Nation's input or approval. Many ISC interviewees shared the view that the Government of Canada should remain responsible for environmental liabilities on reserve lands, and that it is unjust to devolve responsibility for environmental damage caused by third parties to the First Nations.

Some ISC staff emphasized that, even if First Nations do not take on federal liabilities for contaminated sites, they have a right to set priorities and make decisions about contaminated site management on their lands.

5.1.1 Relevance

The evaluation sought to understand why the CSOR Program is needed and to assess the extent to which the Program meets those needs. The evaluation concluded that the CSOR Program is highly relevant to both the Government of Canada and ISC's priorities, and that it does meet an ongoing need in First Nations' communities. The CSOR Program addresses the remediation of contaminated sites on First Nations' lands, aiming to reduce Canada's liabilities and support environmental reconciliation. The program impacts First Nations' communities holistically, including their health, well-being, economy, and land use.

The evaluation also looked to understand the CSOR Program's operating context and how it has changed over the scope of the evaluation. The evaluation concluded that since April 2014, there is a greater focus by the federal government on First Nations' self-determination, and increasing efforts by First Nations to restore their governance over their lands. The CSOR Program's main funding partner, FCSAP, has also undergone changes over the scope of the evaluation. Since 2014 it has progressed into Phase III (2016-19) and Phase IV (2020-24) of its activities, broadening the scope for contaminated sites on reserve, and changing in both funding and site eligibility criteria. The evaluation also took a Gender-based Analysis Plus lens to the CSOR Program's activities. Although data limitations did not allow for an in-depth analysis, the evaluation found some evidence that the CSOR Program's outcomes have a particularly beneficial impact on the most vulnerable members of First Nation's communities, such as Elders, youth, pregnant women, and people experiencing houselessness.

5.1.2 Performance

With regard to performance, the evaluation sought to understand the extent to which the CSOR Program is achieving its expected results, including the Program's contributions to the ISC departmental result that "Indigenous communities have sustainable land management and infrastructure"; and to understand what challenges might prevent the Program from achieving its expected results.

The evaluation has concluded that the CSOR Program is effective in addressing contaminated sites, though it is limited in its ability to reduce liabilities by the need for additional resources and greater proactive measures. Where projects occur, the CSOR Program is contributing to both a reduced risk to public health and safety, and to the availability of First Nations' land for development. The evaluation identified a need for adequate human resource capacity, and improved staff continuity in both ISC and First Nations to ensure effective project management. The evaluation identified a significant need for additional funding to address the sites in the CSOR Program's inventory, with a particular focus on the need for resources to assess suspected sites.

The evaluation also sought to assess the extent to which the CSOR Program is delivered efficiently, and whether the appropriate performance measurement systems are in place. The evaluation found that internally within ISC, the processes for delivering funding to Regional Offices, and reassigning surplus funding, is both nimble and responsive to regional needs and realities. The evaluation also found that accessing surplus FCSAP funds from other departments was an area where additional efficiencies could be produced, though the FCSAP reallocation process is not always straightforward and can come with a heavy administrative burden. The evaluation concluded that there are some processes that impact the CSOR Program's delivery: manual data entry, duplication of data entry work, and lack of automation contribute to inefficiencies in processes. These inefficiencies also contribute to challenges in measuring the performance of the CSOR Program, given that the performance data is not always available, accurate, timely, or in a format that is easily accessible. Overall, the CSOR program faces challenges in data collection, reporting, and systems integration. Improvements in performance measurement systems, data management tools, and processes are needed to address these issues and enhance the efficiency of the program.

Finally the evaluation looked into the process for First Nations and tribal councils to receive training and capacity development. The evaluation has concluded that the CSOR Program does not have an appropriately defined or resourced process to provide the level of training opportunities desired by First Nations, and that there is no national standard or guidelines to provide this training within ISC or from FCSAP. The evaluation concluded that First Nations are looking to access relevant training related to technical capacity, project management and process, and pollution prevention, and that First Nations organizations would be well positioned to provide these opportunities if adequately supported.

5.1.3 Relationships

The evaluation sought to assess the appropriateness of the roles within and between the ISC national office, ISC regional offices, FCSAP, and delivery partners including First Nations and consultants. The evaluation has concluded that FCSAP is seen as a good partner for the CSOR Program, though the FCSAP cannot always be responsive to ISC's unique context. Internally, ISC Headquarters' role in supporting reporting requirements and developing national guidelines is seen as appropriate, though there is a desire for more understanding of the regions' unique and technical contexts and to have stronger national guidelines. The role of ISC Regional offices is appropriate given that they work both horizontally and vertically within ISC to ensure projects are prioritized appropriately according to the Program's funding terms and conditions; and that their role in working directly with First Nations to support project delivery is useful where good relationships have been established by ISC regional officers.

The evaluation has also concluded that the CSOR Program benefits from direct First Nation involvement in both governance and delivery of projects, given that First Nation representatives are well-positioned to communicate results between ISC and First Nations' leadership and members. Ensuring a knowledgeable member from the First Nation is involved in all aspects of the project can facilitate local support for projects by aligning projects with community priorities; enhancing communication between the Nation and ISC; and facilitating effective and timely project reporting. The evaluation found that First Nations preferred to access environmental consultants who are Indigenous-owned or who are willing to hire local community resources; and that doing so can reduce timelines and overall project costs.

5.1.4 Best practices

The evaluation sought to highlight the best practices and lessons learned from the management and implementation of the CSOR Program. Key themes identified by the evaluation include the importance of communication, community engagement, trust-building, and transparency in fostering successful relationships between First Nations and ISC. In Saskatchewan, project management teams consisting of representatives from the ISC regional office, the First Nation, and the consultant support long-lasting partnerships. These partnerships have benefits that extend beyond the individual project into other areas of the relationship between the First Nation and ISC. ISC regional officers and First Nations representatives emphasized the need for early engagement with First Nations, aligning ISC's projects with First Nations' priorities, and ensuring community buy-in for efficient and successful project outcomes. Additionally, the evaluation heard that the historic relationship of distrust between First Nations and the government must be considered when undertaking work on First Nations' land, and must consider the impact of discrimination and racism when considering the scope and long-term management of contaminated sites. Overall, the value of trust, collaboration, and technical competence cannot be overstated in achieving the goals of the CSOR Program.

5.1.5 Service transfer

One major lesson learned from the CSOR Program's management and implementation is that the reliance on FCSAP for the majority of the Program's funds causes challenges for both effective implementation, and for eventual service transfer. Contaminated sites are unique in the ISC context, as opposed to other custodial departments working with FCSAP, and reliance on FCSAP funding structure creates challenges for ISC to respond to its own unique context in terms of both the types of activities undertaken and the processes to fund activities. ISC is distinct from other FCSAP custodians as it is First Nations themselves, rather than ISC, that manage the assessment and remediation processes on reserve. To achieve transformative change, the CSOR Program has a need for a mechanism to support First Nations in assessing and remediating sites that First Nations identify as priorities, but which are outside of the current funding eligibility requirements. Evaluation evidence also suggested that securing A-based funding for the CSOR Program outside of FCSAP would allow ISC to share decision-making power on the Program's site prioritization more equitably with First Nations, and be a path toward transformative change and eventual service transfer. Finally, the evaluation concluded that further that revising the Program's logic model to ensure that its terms and conditions are responding to First Nations' priorities and needs would support service transfer, and that updated performance measurement indicators would allow for more accurate reporting to Canadians on the actual outcomes of the Program.

5.1.6 External forces

The evaluation sought to understand how climate change and the COVID-19 pandemic have impacted the CSOR Program and its outcomes. The evaluation concluded that climate change adds urgency to the work of addressing contaminated sites and highlights the need for adaptive approaches and flexibility in budgeting and timelines, while the pandemic has highlighted the need for flexibility and creative solutions in navigating the program's operations.

The evaluation found that some sites may not be significantly affected by climate change, but others are at risk of flooding, forest fires, coastal erosion, and water contamination. These impacts can increase costs and make it more challenging to close and manage contaminated sites. The unpredictability of weather patterns and extreme events also pose difficulties in project planning and risk management, particularly in working on sites which are only accessible via seasonal roads. There is a growing emphasis on incorporating climate change considerations into the design and resilience of remediation projects. However, there are challenges in collecting and analyzing climate change data, as well as determining the actual impact of climate change on site contamination.

The evaluation also found that COVID-19 pandemic has caused the CSOR Program to experience a backlog of work due to project delays. The increased costs of equipment, consultant fees, and transportation have strained budgets, leading to reduced purchasing power for the CSOR Program. Supply chain disruptions and inflation have further contributed to project delays and increased expenses. Remote work affected First Nations' ability to engage their communities, however, virtual meetings and remote communication tools did allow for some level of continuity in project delivery. The CSOR Program adapted and found workarounds for these challenges, such as implementing pandemic-related safety measures, developing project contingency plans, and reallocating budgets to remote projects where work could occur.