Audit of the Mental Wellness Program

#1 Continuous communication and collaboration among Mental Wellness Program offices

The audit found that organizational structures and regular meetings were in place in the Mental Wellness Program (MWP). In interviews with the MWP offices, several regular meetings are consistently scheduled between National Office and regional offices (e.g., Annual 2 Day face-to-face meeting, Annual Bi-lateral meetings between National Office and each regional program, monthly Mental Wellness Cluster Leads (MWCL) and Focal Point meeting). This was further confirmed through the review of sample meeting minutes, when available, as well as by Internal Audit's attendance of one (1) of the MWCL monthly meetings, a key MWP-led meeting that serves as an open forum for the National Office (NO) and all seven (7) regional offices to discuss updates and challenges related to their portfolios. This suggested continuous collaboration and communication between key stakeholders of the program to raise updates, risks, and challenges of the program, in a timely manner.

#2 Lack of formal governance documentation and meeting minutes

The audit found that MWP offices attended several committees, partnership discussions, and roundtables, hosted by Indigenous partners and communities. Several of these meetings were deemed "informal". However, they served key purposes, such as planning, monitoring, and oversight, and are sources of key decision-points related to the program and funding decisions. Manually written listings of attendees for some key committees, partnerships, and roundtables were provided for MWP-led meetings. However, Terms of Reference (TOR) and formal process documentation were not available for most of the MWP offices sampled. Additionally, documented evidence of meetings was not available for most of the MWP-led meetings. Through interviews with all the MWP offices sampled, it was further confirmed that formal documentation pertaining to key MWP-led meetings, such as agendas, meeting minutes, and/or Records of Decision (RoD) were not consistently documented.

In interviews with MWP offices, it was further confirmed that agendas, meeting minutes, and RoD were not consistently documented due to limited resources and time constraints during the pandemic and were not reinstituted after the pandemic. During these interviews, it was also noted that some meetings were not captured in the form of meeting minutes as the subject matter can be sensitive. MWP offices noted that some meeting attendees may not be as forthcoming if they knew that meetings were being documented.

A lack of formal governance process documentation, related to MWP-led governance structures and meetings, could result in loss of accountability and transparency and result in misunderstandings, misalignment, and inconsistent execution of processes due to a lack of clear and MWP tailored roles and responsibilities by those charged with governance roles. This finding could also result in negative effects on collaboration and strategic alignment, resulting in hinderances to the program's ability to adapt to changing needs and circumstances within the community-based mental wellness landscape caused by a lack of clear alignment on program-wide goals and objectives. The finding may result in increased risk of inefficiency and ineffectiveness caused by a lack of governance documentation articulating who is to complete certain tasks and how and potentially leading to duplicated efforts, overlooked tasks, or conflicting priorities.

Additionally, limited documentation of key meetings could result in increased risk of misaligned operations caused by lack of consistent capturing and tracking of meeting minutes. There is a risk of losing valuable insights and information discussed during these meetings, such as evidence related to key decisions, action items, and follow-up activities. This lack of tracking and documentation may hinder the program's ability to accurately assess progress, identify areas for improvement, and make informed decisions. Lastly, results could also include the loss of institutional knowledge caused by lack of record-keeping and recent turnover, resulting in the inability of recalling what was discussed, what tasks were assigned, and what deadlines were set.

#3 Insufficient clarity on roles and responsibilities for governance and oversight functions of the Mental Wellness Program

The audit found that there are generic job descriptions in place and reviewed them for the sampled MWP offices. Internal Audit acknowledged that the Government of Canada (GoC) Directive on Classification outlines the necessity for departments to include, at a minimum, Generic Job descriptions. While the MWP offices had adopted the Generic GoC job descriptions from the National Occupational Classification (NOC) website, these descriptions were not tailored to the program's evolving mandate. The job description for the new role of DG of the MWP is in development with EX classification. There may be an opportunity to further refine and customize these descriptions as well as advance detailed workplans to clearly articulate the roles and responsibilities of individuals within the MWP offices.

Over the past three fiscal years (2021–22 to 2023–24), new positions and turnover at different levels has occurred, resulting in continued vacancies and posing challenges in maintaining effective operational capacity. Particularly, key roles, such as the DG of the MWP was created on November 6, 2023, to replace the Mental Wellness division reporting to the DG Office Of Population & Public Health (OPPH), as well as staffing changes for two (2) Regional Directors, among others. Role changes and turnover were brought forward in interviews with the National Office and the regions, as well as at the Mental Wellness Cluster Leads (MWCL) meeting that Internal Audit attended. It was noted that each region generally has two (2) representatives at MWCL and focal point meeting to facilitate information sharing. However, the vacancies along with the lack of updated organizational chart and tailored job descriptions, could create challenges in assessing if the program has adequate staff to continue operating effectively, and who is responsible for what responsibilities within the MWP.

Additionally, for most of the MWP offices sampled, training materials and guidance were not consistently available and/or implemented. It is uncertain if all responsibilities are being performed as there is no operational manual, training materials, nor specific job guidance established, which could provide a baseline for comparison.

During interviews, it was also noted that MWP offices have faced considerable time and resource constraints due to the pandemic and other on-going emergencies. Some MWP offices reported that they were overburdened with the amount of work and did not have time and resources to keep up with documentation and key administrative and organizational tasks.

Limited guidance documentation, such as lack of updated organizational charts, MWP-tailored job descriptions, and training materials, could result in (1) loss of talent and institutional knowledge and an increase in the program's vulnerability to disruptions in Program operations in case of turnover. The finding could lead to (2) increased risk of errors caused by a lack of clear policies, procedures, and job responsibilities to guide employees. Furthermore, it could lead to (3) reduced clarity and consistency around the roles and responsibilities of new employees of the MWP, resulting in tasks not being appropriately allocated and executed by new employees of the program.

#4 Continuous communication and collaboration between Mental Wellness Program offices and Indigenous partners and communities

The audit found that that Mental Wellness Program (MWP) offices attend several committees, partnership meetings, and roundtables which include representation from Indigenous partners and communities. Attendance of these regular meetings was outlined during interviews with all MWP offices sampled. Additionally, select attendee listings, agendas, and meeting minutes from these Indigenous partner-led meetings were reviewed, when provided. This suggested open communication channels between the MWP offices and Indigenous partners and communities which could allow for a free exchange of information and increased collaboration.

#5 Lack of data collection instruments (DCIs) and key performance indicators (KPIs) that better represent measures of success in Indigenous communities.

The audit reviewed the data collection instruments (DCIs) and key performance indicators (KPIs) that are currently used across the Mental Wellness Program (MWP). It was found that standardized performance measures were in place and are used throughout FNIHB. However, these DCIs and KPIs were not developed with Indigenous partners and were not adapted to specifically reflect measures of success as defined by the communities that the program services. In interviews, all MWP offices acknowledged the need for more community specific DCIs/KPIs, however, there are currently no updates to these templates underway.

In interviews with all MWP offices, it was also noted that the Indigenous partners they collaborate with have been developing KPIs and DCIs that better represent the needs of their communities. Samples of DCIs and KPIs created by and for Indigenous communities were reviewed to validate this.

One of the challenges around adapted performance measures is that tailoring DCIs/KPIs to the distinct Indigenous communities that program serves, could reduce consistency and standardization across funding arrangements. Lack of standardization makes it difficult to assess and compare results on a wholistic level for the MWP. MWP offices are responsible for their own annually mandated reporting which requires the inclusion of data requested in existing, standard DCIs/KPIs. As such, this could be a challenge from a reporting perspective.

The absence of Key Performance Indicators (KPIs) that have been specifically adapted to the individual needs of Indigenous communities, as well as a lack of set indicators in flex funding requirements, may result in inaccurate assessment of funding progress and effectiveness. Without tailored indicators that reflect the unique characteristics and priorities of Indigenous communities, the assessment may fail to capture the full extent of project or service outcomes and impact. As the MWP is the funder, not the service provider, the date received is the key driver for any assessments made on projects and services provided. This deficiency is particularly pronounced due to the diverse nature of Indigenous communities served by these projects, where indicators of success may vary significantly based on unique cultural, social, and environmental contexts. This finding may also result in potential disconnection from community needs caused by assessments that inadvertently overlook or misinterpret the unique needs and aspirations of Indigenous communities. This disconnect may result in interventions that fail to address underlying systemic issues or adequately empower community members, limiting the long-term sustainability and relevance of project interventions. If the finding was not corrected, this could potentially lead to decreased stakeholder confidence caused by the inability to effectively measure project success which may erode stakeholder confidence in the program's ability to achieve its objectives. Without clear evidence of Program outcomes, stakeholders may question the value and relevance of the program, potentially leading to decreased support and investment over time.

#6 Lack of a formally documented process to incorporate feedback from Indigenous communities, as well as a lack of a formal process to capture lessons learned from a program perspective.

The audit found that there was no formally documented process to periodically request, retrieve, collect, analyze, synthesize, and action on feedback from Indigenous Communities, as it relates to the MWP. However, MWP offices noted that there were several meetings in which verbal feedback from Indigenous partners and communities were sought. Additionally, the FNIHB Mental Wellness Cluster Leads (MWCL) Face-to-Face (F2F) Meeting was outlined as a meeting in which the MWCL meet to discuss progress, barriers, and potential solutions to barriers are outlined.

Notes from a sample MWCL F2F were reviewed. This document included a summary of feedback/barriers, as well as sample solutions. However, there was no clear indication of the responsible parties, action items, and/or timelines to complete any of the updates. Additionally, no documentation was presented to confirm if the barriers outlined were actioned on and if changed were made to the MWP. Due to a lack of detailed, formal documentation, it could not be assessed if a holistic review of the funding arrangements in the MWP was conducted to make improvements to the activities of the MWP. No other lessons learned documentation was available for the period 2021–2024.

In interviews, it was outlined that meetings are used to discuss feedback from Indigenous communities and to reflect on the activities of the MWP. As noted, some of these discussions occur at scheduled meetings, however, these meetings are currently not consistently documented in the form of meeting minutes. It was also noted that some meetings are not documented as the subject matter can be extremely sensitive. Some meeting attendees may not be as forthcoming if they knew that meetings were being documented. ISC also noted that surveys are not executed by ISC but that ISC provides funding to Indigenous partners to execute surveys. As such, documented feedback was limited, and the lessons learned process could not be formally evidenced.

MWP is guided by the First Nations Mental Wellness Continuum Framework which include the commitment to Indigenous-led frameworks, programs, and services (e.g., Honouring Our Strengths, the National Inuit Suicide Prevention Strategy, and Pathfinding Towards a Flourishing Future Awareness and Advocacy Guide, etc.), which were developed through strong community engagement. While the foundation of the MWP includes Indigenous-led frameworks, a lack of a formally documented feedback mechanism and lessons learned process, could deter the MWP from reaching its objectives.

In absence of formal feedback requests led by MWP offices, and limited, documented communications between MWP and Indigenous communities (e.g., meeting minutes, RoD, etc.), the extent to which stakeholder input has been requested, collected, processed, cannot be clearly assessed. Furthermore, there is no documented evidence indicating the incorporation of stakeholder feedback into program adjustments which could result in failure to uphold principles of reconciliation. Reconciliation requires meaningful engagement, respect for Indigenous rights and perspectives, and a commitment to addressing historical injustices. Without documentation of these efforts, it may not be clear how Indigenous knowledge, experiences, and priorities are being considered. Another result of this finding may be diminished community engagement. Without documented guidance for soliciting and acting on feedback, community engagement may reduce with the MWP, and other government endeavors aimed at enhancing mental wellness. Without robust community engagement, the potential for achieving positive outcomes and fostering lasting change becomes constrained, hindering progress towards shared goals of mental wellness and reconciliation. The finding may also result in undermined trust and engagement due to a lack of tangible evidence of genuine engagement or responsiveness to community input. Additionally, the inability to demonstrate a systematic approach to feedback utilization may raise concerns about the program's accountability and adherence to ISC mandates and policies on Indigenous engagement and reconciliation.

Limited documentation related to the lessons learned process may result in missed opportunities for learning and improvement which are essential for advancing mental wellness outcomes and achieving its reconciliation objectives. A lack of a formalized process for collecting and acting on feedback, may raise uncertainty regarding the program's adaptive capacity to address community needs. Additionally, the finding may result in limited effectiveness in addressing mental wellness needs. Without structured feedback mechanisms, including the implementation of changes, the MWP may struggle to adequately address the diverse and nuanced mental health needs of Indigenous communities. Consequently, the effectiveness of interventions may be compromised, hindering efforts to improve mental wellness outcomes.

#7 Flexibility in funding allocations through increased use of Flexible and Block Funding approaches

The audit found increased flexibility in funding reallocation and surplus management within the MWP, through the use of Flexible and Block Funding approaches. Through interviews and review of Government of Canada Policies, and sample funding agreements, the adoption of Flexible and Block Funding approaches was confirmed. These approaches, available across various programs within FNIHB, empower recipients to exercise greater control in the management of their funding. Flexible Funding allows recipients to reallocate funds in the same Program Authority, carry over program funding annually for the duration of their agreement, with reimbursement of unspent funds from the recipient to the government at the expiry of the agreement. Block Funding is additionally flexible, in that recipients may reallocate funds across authorities (except for specifically identified programs), as well as retain surpluses to reinvest in other priorities. Funding arrangements can contain a mix of funding with different funding approaches as well as multiple projects. The proportion of Mental Wellness funding flowed through Flexible or Block Funding has continuously increased in the MWP, from 68% in 2021–22 to 77% in 2023–24, as identified in MWP project listings that outlined the approaches in which the projects were funded, allowing MWP recipients to take advantage of the flexibility they offer.

Similarly, with respect to funding allocations, the audit found that there was increased flexibility in the submission of reporting mandated by Block Funding approaches. Through interviews, review of Government of Canada Policies, and reporting for project samples, more flexible reporting was confirmed for MWP funding recipients. Recipients that receive their funding through Block Funding approaches may provide uniquely created reports to their MWP regional office or NO, instead of following the rigid reporting required by the other funding approaches. However, Block funding recipients are required to submit an annual report to the Minister (DCI HC P001 Annual Report for Block) that covers multiple programs and replaces the DCI reporting requirements for those programs.

#8 Inconsistent tracking of reporting and limited visibility over projects

The audit reviewed existing reporting tracking mechanism and found that the majority of the Mental Wellness Program (MWP) offices sampled had no clear and consistent processes to track mandated reporting from its recipients. Agreements between the MWP offices and their recipients clearly outline reporting requirements for all funding recipients. Reporting may vary in its frequency and format, depending on the funding agreement and approach used, however, it is a requirement for recipients to submit reporting to continue their eligibility to receive funding. It is the responsibility of the MWP offices in collaboration with other FNIHB teams to follow-up with the recipients if reporting is missing and/or if there are any errors or issues related to the reporting received.

All MWP offices have access to the Grants and Contribution Information Management System (GCIMS), a repository that houses documentation such as mandated recipient reporting. However, only a few of the MWP offices used this repository, along with communications with the recipients, to create their own manual tracking tools, note missing reporting, note reasons for missing report, note errors in reporting, and/or note issues regarding the reporting that could impact recipients' eligibility to receive funding. In the instances that manual tracking tools existed, within the tracking documents available, there was no clear indication of who tracked the reporting, if reporting was reviewed and who reviewed it, nor if missing reporting or errors and/or issues with reporting was followed up on.

Notably, reporting expectations during the pandemic were removed to allow service providers to deal with emergencies and to focus on their operations first. Therefore, reporting between 2021–22 and 2022–23 was not consistently submitted. During this time, reporting deficiencies were also attributed to resource and timing constraints. It was noted that MWP offices prioritized their relationships and communication with the Indigenous communities, with reporting becoming a secondary activity.

Additionally, Block Funding requirements became more flexible to accept any reasonable form of report and/or update from clients (e.g., verbal phone calls, site visits, alternate reporting, among others). While this approach was used to adapt to the needs of Indigenous communities, it was noted in interviews that this approach impedes the programs' ability to compare different projects as the reporting can differ significantly. Submitted recipient reporting from sample projects under Block Funding approaches was reviewed and it was confirmed that the formatting and content varied among the majority of the samples.

A lack of systematic tracking and review of partners' community reporting by the regions, combined with the inconsistent use of GCIMS reporting, may result in challenges in monitoring and assessment of projects, resulting in the inability to clearly track missing reporting and eligibility/ineligibility for funding. The funding may also result in a lack to track progress, risks, challenges, and lessons learned throughout the fiscal year. The finding could also lead to limited visibility into project progress caused by inherent flexibility of reporting requirement of Block Funding approaches, causing potential hinderances on the program's ability to make informed decisions and adjustments to optimize outcomes, as well as to intervene in a timely manner, if needed. Without mitigation, the issue may lead to a lack of oversight regarding the alignment between projects and program objectives. This could result in difficulties in monitoring and assessing projects to ensure their outcomes effectively align with program goals and priorities.