![]() ![]() ![]() |

|
'96 SAR Prevention Review -"SMART Future" SAR PREVENTION REVIEWApproved by ICSAR July, 1997 Results in Brief Results in BriefThe SAR Prevention Review revealed that there is no reliable information base by which to inform management of the impacts and the results of prevention activities. Moreover, the review indicates there is a need for improved interfacing of the separate prevention practices through increased information sharing. Prevention practitioners were very receptive to sharing lessons learned, best practices and receiving expert advice on other prevention activities. The acronym S.M.A.R.T. embodies the future approach to SAR prevention recommended in this report. In short, the future of SAR prevention should be: Specific -for future prevention activities to succeed -clear objectives and formal design procedures are needed along with a clear and detailed understanding of the target audiences. Measurable -no clear picture of the impact and results of SAR prevention activities was discernible in the review. However, progress in the evaluation of prevention activities is being made in other areas such as crime prevention, AIDS prevention and health promotion. Thus, the techniques for conducting good evaluation do not need to be invented, two decades of evaluation research have provided a basic conceptual framework for undertaking such efforts. Every effort should be made by the SAR Program to build on past progress to ascertain results and impact information. The evaluation of SAR Prevention activities is not an easy task. It will take time and it will also require a long-term commitment of effort and resources. However, given the objective of reducing loss of life and injury, the commitment of adequate resources for careful evaluations of the effectiveness of SAR prevention programs should be viewed as an investment in the future. Attainable -resources and expertise are needed for prevention activities to succeed now and in the future. However, within limited budgets it is not always possible to have the ideal resources for proposed education and promotion activities. Several solutions are offered as remedies for this problem: continued sharing of information and ideas on best practices amongst and between SAR departments should be pursued with vigor; using experts from other prevention areas to advise on efficient and economical approaches; referral at the design stage of a prevention activity to policy and front line staff in order to ensure that the proposed intervention is necessary and will make a difference; establishing partners to serve as alternate message delivery agents and as sources of funding; and understanding the culture and behavior of the targeted audience is key to attaining success in the "influencing" game. Realistic -prevention in the form
of education and information is a long term affair. Management cannot
expect to see "attitudinal or behavioral" change overnight as
most prevention efforts require a long-term commitment. Carefully Timely -the business adage, that
you have to produce the right product in the right manner at the right
time and provide it to the right people, holds true for prevention activities.
This review urges SAR managers to adopt a social marketing model in order
to produce timely products which will provide the desired change in attitude/
behaviour sought. Review ScopeThe SAR Prevention Review focuses on the non-regulatory prevention activities
of education and promotion undertaken by the Interdepartmental Committee
on Search and Rescue (ICSAR) departments. Appendix A, is a summary of
the individual ICSAR departments' responsibilities for non-regulatory
prevention activities. These activities cost approximately 7 million operating
dollars per year out of the total 235 million annual program budget. Study ObjectivesOn October 10, 1996, three study objectives were identified and approved by ICSAR: Objective 1 Objective 2 Objective 3 Led by the three study objectives, the Review primarily focused on how
to develop the SAR Prevention Program in the future. Review MethodologyMultiple lines of evidence were used in this review study. A comprehensive
document review and interview program as well as an Expert Opinion Panel
and a Focus Group (ICSAR Sub-Committee workshop) were conducted. However,
the usual data analysis associated with reviews was not possible because
of lack of data. Summary of FindingsA key element of each of the review methodologies was to capture information on the 'best practices' of prevention services in general. The inclusion of a 'best practices' methodology in this review was a new initiative for the ICSAR Review Sub-Committee and will hopefully serve as a basis for developing further expertise in this area. 1 The following summarizes the findings contained in the document review, interviews, expert panel and workshop session. Technical papers on the document review, interview sample and expert panel are available and provide detailed information against the study objectives. These findings reflect the level of the study effort requested by ICSAR which was that objectives 1 & 2 should consume no more than 30% of the review study effort and objective 3 should consume the remaining 70%. Objective 1 The focus of this study objective was to provide information on issues and current practices of SAR Prevention efforts. Although, narrowly focussed, this study objective provides the starting point for looking toward the future. Approach (Overall conceptual vision and strategy) No coordinated vision regarding SAR Prevention and no clear understanding of individual departmental roles within the larger SAR Prevention framework currently exists. As such, little if any, sharing of information regarding prevention theory and practice takes place between SAR prevention practitioners. Design (Formal and detailed program planning)
Delivery (Implementation) Evaluation (Formative, Process, Impact) Objective 2 Building on an understanding of the issues and concerns outlined in study Objective 1, Objective 2 focuses on how SAR prevention activities at present, can be improved. Suggestions were drawn from the SAR prevention community, but also, suggestions were gathered from other prevention activities exemplifying the 'best' in their processes and practices. In short, study Objective 2 offers some solutions to present problems. Approach (overall conceptual vision and strategy)
Design (Formal and detailed program planning)
Delivery (Implementation) Who delivers the message can be a critical factor in successfully influencing the targeted audience's attitude or behaviour. For example, using members of peer groups or front-line departmental staff are examples of this 'best practice'. Partnerships between prevention program practitioners and private sector organizations have proven to be effective as an alternate or complimentary forms of information dissemination. 5 In order to accomplish program goals and objectives it is important to have sufficient resources with which to implement prevention programs. This may entail either procuring alternate resources such as sponsorships, or restructuring goals and objectives to meet available resources. 6
Evaluation (Formative, Process, Impact Evaluations) The following is a synthesis of their 'best practice' tenets in evaluation.
Without question, these suggestions require resources and effort; however,
this commitment should be viewed as an investment in the future.
Objective 3 The following outlines, in the opinion of the review team, "what" should be done (Observations) and "how" it should be done (Recommendations). Approach Planning forum needed To this end, several possible forums were discussed at the review workshop including creating a separate ICSAR prevention sub committee and the establishment of a planning forum to discuss and coordinate issues and planned interventions relating to SAR prevention. Recommendation 1: Approach improve planning information For SAR prevention, the requirement for the future should be to improve
"needs" information and the challenge is to overcome any additional
resource implications that improving data may pose. A best practice example
in this regard is the Lifesaving /Red Cross initiative to create an economical
and effective surveillance system using coroner's information. The SAR
program has a unique strength in this regard in that pooling the efforts
of six departments with a view to improving the information system has
a better Recommendation 2: It is further recommended that the ICSAR, acting as a partnership, agree on the use of an analytic tool such as the Haddon Matrix which will provide a common planning framework for understanding how prevention activities should use the data collection. Approach develop a program based objective and an overall
message Specifically, actions such as the pooling of ideas and approaches to prevention activities and the sharing of best practices and available expertise were identified as useful and value added activities. However, a framework is needed to develop where practical, the overall goal and objective of SAR Prevention and to provide assistance on how to better coordinate efforts. The start of a more communicative and cooperative approach should be agreement on an overall common goal (e. g. of a safer culture? or to minimize or eradicate loss of life?) and an overall message ( for example, one prevention program examined in the review announces that 90% of accidents are preventable and asks that people not cross the "stupid line"). Developing the overall objective and message should not be viewed as a bureaucratic "nicety" but a concrete step towards solidifying the enthusiasm of SAR prevention practitioners to work together. Recommendation 3: Design better access to expertise and the use of generic models and improved design processes Designing an intervention to influence and change attitudes/ behaviours is a process requiring specific expertise in social psychology and social marketing. An equally important ingredient in designing SAR prevention activities is knowledge of different sub populations in the SAR world and applicable policies and regulations which may affect these groups. Access to this expertise and knowledge at the design stage is critical and has to be accomplished within the current confines of having to provide timely products with limited resources. Obtaining adequate access to expert knowledge can therefore be a major challenge. Within the review of best practices, one innovative practice was identified which may address this particular challenge. Natural Resources Canada's development of a Research and Development Impact Assessment Network between itself, other departments and industry, has some direct applicability to the SAR prevention world. If ICSAR departments were able, as a group, to finance a similar venture, which would allow, at the design stage, coordination through on-line computer access or contacts to in person consultations with experts in social marketing and social psychology, a very useful service would be created. Preliminary analysis of the cost of financing such a network appear to be reasonable and the benefits high. Access to, and use of other successful models in similar areas in designing a SAR prevention intervention are equally useful. Among the best practices identified in the review which should be used as generic design models are: applying cost benefit analyses to assess programming options, making use of checklists to ensure program design is on track and adopting 'social marketing' processes to ensure a holistic approach to SAR Prevention program design. Recommendation 4: In addition generic models proven to work in other fields, then tailored to SAR Prevention, should be utilized and the funding of future SAR Prevention initiatives (e. g. NIF, individual ICSAR departmental funding, etc.) should be contingent on the inclusion of a formal design phase, including mechanisms to evaluate the impact of the program. Evaluation an under used and under-valued tool The reality is that given the right tools and used in appropriate ways,
evaluation should, in the future, be an essential tool for SAR prevention
managers. A review of how evaluation is utilized in other prevention practices
validates that with a bit of innovation and commitment, evaluation can
reach the deserved status of "useful". Firstly, the review revealed
that evaluation should not be viewed as only a "post action"
tool for assessing impact and results. Rather, successful prevention practitioners
see it as an element in the total management process (i. e. evaluation
mechanisms should be used for formulating the design of the intervention,
monitoring the intervention once it is implemented and for assessing impact).
However, impact evaluation is still a contentious subject for many managers
of prevention. In today's fast paced Investment in improved evaluation for SAR prevention activities must address the constraints of a lack of resources and available expertise and a paucity of measurement data. These challenges are being met in other areas in a variety of innovative ways. For example, meeting the resource challenge in one area means that evaluations are conducted at reduced costs by utilizing graduate students vice expensive staff resources. Other prevention areas are attempting to address the shortfall (and cost of collecting) information for evaluation purposes by obtaining representative samples of project impacts versus wider program surveys. Despite the innovations and progress made in other areas, findings in this review indicate that improving evaluation of SAR prevention will inevitably entail some investment of additional resources. Suggesting expenditure of more resources will likely be unpopular; but the reality is that obtaining information for both short term and long term results on prevention activities is essential. As noted earlier in the section under "approach", information on causality and type and severity of incidents is important in the design of projects. Such information is equally important for establishing long term trends and results. Without this type of information, it will be virtually impossible to know if attitudes and behaviour patterns have changed and that the incident rate is correspondingly down. Recommendation 5: It is further recommended that no prevention project be funded unless
it contains an evaluation plan. Appendix A -
|
|
Personal
|
Equipment
|
Environment
|
|
| Pre-Incident (Prevention) |
Avoid alcohol consumption |
Boat with safety equipment in order |
Frequent weather reports on local radio |
| Incident (Survival) |
swimming ability | PFD's | |
| Post-Incident (Response) |
knowledge of CPR | First-aid kit | Communication & transportation network |
For more details on Haddon's Matrix and its applications consult
the, "National Drowning Report", Canadian Red Cross Society,
1996.
4. A psychographic analysis of the targeted population
is generally the most effective means of identifying what issues are important
to the audience and what messages they will likely respond to. This analysis
should be combined with consumer/ client behavioural research, as well
as research relevant to changing risk-behaviours.
5. In some cases, prevention programs may be 'piggy-backed'
onto already existing mediums thereby reducing costs and ensuring the
distribution of the message to an already established audience.
6. Sponsorship is a relationship in which one party,
the sponsor, supports the activities, program, or cause of the other partner,
in return for some type of recognition. For more on this topic see "Corporate
Sponsorship" by J. Mintz and G. Wallace (1994).
7. A formative evaluation involves a small-scale effort
to identify and resolve intervention issues before a program is largely
implemented. "What communication's medium works best for the targeted
audience?" is the salient question for formative evaluation.
8. A process evaluation addresses two broad questions,
"What was done?" and "To whom and how?" When interventions
continue over a long period of time, as may be the case for some SAR prevention
activities, measurements at several times are warranted to ensure that
components of the intervention continue to be delivered to the right people
at the right time.
9. This type of evaluation assesses the effectiveness
of a prevention program and is used to answer the question, "Do the
interventions make a difference?" Like process evaluation, this evaluation
can also be conducted at intervals during an on-going program. There are
several ways in which these evaluations can be conducted. Ideally to assess
the effect of a prevention program on program participants, one would
like to know what would have happened to the same participants in the
absence of the program. Because it is not possible to make this comparison
directly, inference strategies that rely on proxies have to be used. The
most common of these strategies is the before and after study (i. e. pre-test,
post-test strategy). In this design, pre-intervention measurements are
compared with post-intervention measurements to detect change in the outcome
variables that the intervention was designed to influence. Another strategy
is the control group design in which the control group is selected by
matching non-participants to participants in the group exposed to the
program on the basis of selected characteristics. The groups are then
tested in order to attribute program effects. Possibly the best strategy,
in terms of resource efficiency and accuracy, is that of randomized experiments.
In such experiments, one singly constituted group is established for study.
A subset of the group is then randomly chosen to receive the prevention
activity, with the other sub-set becoming the control group. Randomized
experiments provide for clear causal inferences and effectively answer
the questions "Does the intervention work? ' and " What works
better?"
10. Technical Report-Expert Opinion Panel Proceedings,
SAR Prevention Review (February, 1997) pg. 13. 16
|
|||||||