2.0 EVALUATION OBJECTIVES, SCOPE AND APPROACH

2.1 EVALUATION OBJECTIVES

Aligned with the Treasury Board approved RMAF, the objectives of this evaluation include assessing the relevance, success and cost-effectiveness of the SAR NIF contribution program, as well as the adequacy of the NSS and departmental design and management of the program and monitoring activities. These objectives are also consistent with the core issues to be addressed by departments in an evaluation as per the recently issued TB Directive on the Evaluation Function ⁴.

2.2 SCOPE

The delineation of the specificity of the scope is important. As mentioned in the previous section, the scope of the evaluation is limited to the Vote 10 SAR NIF contribution funds managed directly by the National Search and Rescue Secretariat. The renewed terms and conditions for the contribution component of SAR NIF were approved on August 30, 2005. The revised SAR NIF contribution terms and conditions are applicable for the period from April 1, 2005 to March 31, 2010. Therefore, the period of coverage subject to the evaluation is accordingly from the start of the period noted to the date of the end of the evaluation fieldwork phase, May 2009.

The SAR NIF contribution program evaluation included the review of the following program aspects and / or activities as per the evaluation framework:

• Setting program objectives and guidelines;
• the development and approval of MOUs and contribution agreements;
• the establishment and communication of agreed funding categories;
• proposal assessment activity;
• the project approval and funding process;
• project monitoring activity;
• the processing of project claims and payments;
• the measurement and communication of project effectiveness; and,
• the adequacy of the project evaluation selection process and evaluation conduct.

The SAR NIF contribution program evaluation also addressed the following Expenditure Review Committee issues:
– Public Interest, Role of Government, Federalism, Partnership, Value-For-Money, Efficiency, and Affordability.

2.3 APPROACH AND METHODOLOGY

The program evaluation was conducted through a review of documents, selected interviews and the use of survey questionnaires. Documents reviewed included among others, MOUs between the National Search and Rescue Secretariat and other Federal SAR Partner departments and agencies, contribution agreements with provinces and with recipients, the SAR NIF Partners’ Manual, Applicant’s Guide, and Treasury Board terms and conditions. Also looked at were the contribution project files, the NSS and SAR NIF Website, administrative data, performance measurement results as available, previous program and project evaluation and audit reports that were available and data obtained through selected interviews and surveys. Interviews were conducted with key staff within the National Search and Rescue Secretariat (NSS), and with the current and recent past federal Merit Board members in a focus group session and as well with the provincial Merit Board representative – see Annex B. The survey questionnaire for SAR NIF stakeholders is included in Annex C. Based on the survey questionnaire results, follow-up telephone interviews were conducted with selected recipient stakeholders.

• 4. Much of the program description in this section has been obtained from the Treasury Board approved Terms and Conditions of the Contribution Program in Support of the SAR New Initiatives Fund.

 Table of Contents

3.0 FINDINGS AND RECOMMENDATIONS -> Next Page