DEPARTMENT OF HEALTH AND HUMAN SERVICES PUBLIC HEALTH SERVICE SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION CENTER FOR MENTAL HEALTH SERVICES COMMUNITY ACTION GRANTS FOR SERVICE SYSTEMS CHANGE Short Title: Community Action Grants Program Announcement (PA) No. PA 00-003 Catalog of Federal Domestic Assistance No. 93.230 Under the authority of Section 520(A) of the Public Health Service Act, as amended (42 U.S.C. 290aa), and subject to the availability of funds, SAMHSA�s Center for Mental Health Services, will accept applications in response to this announcement for the receipt dates of May 10 and September 10 each year. _______________________________ Bernard S. Arons, M.D. Director Center for Mental Health Services _______________________________ _______________________________ Ruth Sanchez-Way, Ph.D. Joseph H. Autry III, M.D. Director Acting Administrator Center for Substance Abuse Prevention Substance Abuse and Mental Health Services Administration _______________________________ H. Westley Clark, M.D., J.D., M.P.H. Director Center for Substance Abuse Treatment Date of Issuance: February, 2001 PART I - PROGRAMMATIC GUIDANCE Table of Contents Note to Applicants: In order to prepare an application, Part II, �General Policies and Procedures Applicable to all SAMHSA Applications for Discretionary Grants and Cooperative Agreements� (February 1999 edition) must be used in conjunction with this document. Section I - Overview Purpose........................................................................................................................................1 Eligibility...............................................................................................................................1 Availability of Funds..............................................................................................................1 Period of Support.................................................................................................................2 Section II - Program Description Supporting Documentation....................................................................................................2 Target Population..................................................................................................................2 Program Plan........................................................................................................................3 Goals - Phase I Grants...........................................................................................................3 Design - Phase I Grants.........................................................................................................6 Goals - Phase II Grants..........................................................................................................6 Design - Phase II ..................................................................................................................6 Evaluation.............................................................................................................................6 Definitions............................................................................................................................6 Section III - Project Requirements Phase I..................................................................................................................................7 Phase II..............................................................................................................................11 Section IV - Review of Applications Guidelines........................................................................................................... ......... .......13 Review Criteria - Phase I Grants............................................................................... ......... .14 Review Criteria - Phase II Grants.........................................................................................17 Section V - Special Considerations/Requirements SAMHSA Inclusion Policy.................................................................................................18 Government Performance Monitoring ........................................................................ .......18 Healthy People 2000 ................................................................................................ ......... ..18 Consumer Bill of Rights ................................................. ......... ......... ......... ......... ......... .......18 Promoting Nonuse of Tobacco .................... ......... ......... ......... ......... ......... ......... ......... .......18 Supplantation of Existing Funds .................... ......... ......... ......... ......... ......... ......... ......... .....18 Letter of Intent ..................................................................................... ......... ......... ......... ...18 Coordination with Other Federal/Non-Federal Programs ......... ......... ......... ......... ......... ....18 Public Health System Reporting Requirements .......... ......... ......... ......... ......... ......... .........18 Confidentiality/SAMHSA Participant Protection ..............................................................18 Section VI - Application Procedures Application Receipt and Review Schedule ............... ......... ......... ......... ......... ......... ......... ...19 Consequences of Late Submission ............................................. ......... ......... ......... ......... ...20 Application Requirements/Component Checklist ............................................................ 20 Terms and Conditions of Support ................................ ......... ......... ......... ......... ......... ........ 27 Award decision Criteria ........................................................ ......... ......... ......... ......... ........ 27 Contacts for Additional Information ............................. ......... ......... ......... ......... ......... ....... 27 References ..........................................................................................................................28 1 SECTION I - OVERVIEW PURPOSE The Substance Abuse and Mental Health Services Administration's (SAMHSA) Center for Mental Health Services (CMHS) announces the availability of funds through this Program Announcement (PA) to support the adoption and implementation of exemplary practices related to the delivery and organization of services for children with serious emotional disturbance or adults with serious mental illness. The target population may also have co-occurring disorders, such as substance abuse or other mental, emotional or behavioral disorders. This Program entitled �Community Action Grants for Service Systems Change,� and hereinafter referred to as the �Action Grant Program,� is made up of two types of grants: Phase I Grants: Consensus Building and Decision Support and Phase II Grants: Implementation Support. Phase I Grants: Consensus Building and Decision Support: assist communities in building consensus around planning for the adopt ion of exemplary practices that meet the needs of a defined target population. Phase II Grants: Implementation Support: are available only to successful Phase I grantees and support implementation of the exemplary practices adopted by key stakeholders in Phase I. ELIGIBILITY Phase I applications may be submitted by units of State or local governments, by tribal governments and organizations, and by domestic private nonprofit organizations such as community-based organizations, faith-based organizations, provider and consumer groups, universities, colleges, and health care organizations. SAMHSA encourages applications from consumer and family organizations. Phase II applications are restricted to past or current Phase I grantees. To be eligible, Phase II applicants must demonstrate they have met the Phase I requirements and are ready to implement their exemplary practices. This restriction is due to limited funding available to support a second phase of the Program and to SAMHSA�s inte t to document, monitor, and evaluate the process and outcome of both the consensus building and implementation phases. AVAILABILITY OF FUNDS Approximately $5.1 million will be available to support 24 to 36 Phase I awards and 10 to 15 Phase II awards. The award amounts are expected to range from $50,000 to $150,000 in total costs (direct+indirect). 2 PERIOD OF SUPPORT The period of support for Phase I and Phase II projects will be for up to 1 year each. SECTION II - PROGRAM DESCRIPTION SUPPORTING DOCUMENTATION SAMHSA seeks to promote the implementation of exemplary practices in communities nationwide. The goal of the Community Action Grant Program is to stimulate communities� acceptance of exemplary practices by convening partners, building consensus, helping to eliminate barriers, supporting decision making and adapting service models to meet local needs. The Community Action Grant Program is part of a larger set of SAMHSA program activities known as Knowledge Development and Application (KDA). The KDA process seeks to stimulate service system change that increases positive outcomes for consumers, inclusive of adults and children and their families. The �life cycle� of the KDA program involves thefollowing activities: exchange of exemplary practice information, preparing communities for exemplary practice and adoption of exemplary practice. The process includes several phases in a continuum that begins with the field�s identification of service priorities through research and evaluation into what works. Research and evaluation questions are driven by community needs, rather than by investigator interests. Knowledge gained becomes the subject of active discussion and analysis by decision makers and other key stakeholders. The process is successfully concluded when the knowledge developed about an exemplary practice is actually implemented by a community. Aggressive and sustained effort throughout the life of KDA activity is the essence of SAMHSA�s KDA program. In this context, the CMHS Community Action Grant Program sits at the left - the adoption end of the KDA Life Cycle. TARGET POPULATIONS Phase I applicants must focus on one of two subgroups in the target population: (a) adults with serious mental illness or (b) children and adolescents with serious emotional disturbance and their families. Individuals in each of these two groups may have a co-occurring disorder of substance abuse. Special consideration will be given to proposals of exemplary practices that address needs of especially vulnerable subgroups of this target population, such as homeless children or children in foster care placements, and racial or ethnic minorities. In addition to this general announcement, applicants are advised to check any addenda that may be issued. Phase II must continue to focus on the target population identified in the Phase I grant program. 3 PROGRAM PLAN Goals - Phase I Grants The goal of the Phase I projects is to obtain a decision to adopt an exemplary practice in mental health services in the target community. The decision to adopt includes approval of a financing plan for the exemplary practice. The following are examples of activities that may be supported within the applicant�s work plan to achieve consensus for implementing the exemplary practice: �Convening of key stakeholders (e.g., consumers, family members, providers, advocates, legislative representatives, public authority representatives) and the facilitation of the consensus building and decision support process; �Providing monetary support for travel and other logistical costs necessary to ensure participation by consumers, family members and others needing financial assistance; �Providing consultation and training for consumers, family members and others on project goals, objectives and processes within the project; �Providing education, training and technical assistance that will promote the understanding and awareness of the proposed exemplary practice, the pros and cons for adopting the practice, the mechanics of the consensus building processes, and models for technology transfer and application; �Providing education and training on the dynamics of organizational change; the dissemination of information to the community-at-large; and the expert consultation on mental health and substance abuse issues; �Providing consultation for implementing strategies that result in local or state changes essential for the adoption and maintenance of the exemplary practice; �Identifying and organizing financing strategies for the exemplary practice; �Providing expert consultation on conducting a community needs assessment, developing service models and adapting exemplary practices to unique community requirements, and; �Providing expert consultation on conducting an evaluation of the consensus building process and outcomes. Design - Phase I Grants In order to accomplish the goals for Phase I, grantees will be expected to (1) identify an 4 exemplary practice and (2) conduct a consensus building process that will build support among all key stakeholders for the adoption of an exemplary practice. Exemplary Practice: An exemplary practice is a consistently applied service delivery mechanism which meets the criteria listed below and is intended to improve outcomes for adults with serious mental illness or children with serious emotional disorder or persons with a co-occurring substance abuse disorder. It might be specific, such as a precise clinical or related service protocol designed to ameliorate one aspect of an individual�s disorder, or it might be general, such as a set of principles and criteria for treating individuals within the target population, such as a system of care for children with serious emotional disturbance. In each case, the practice must be fully documented with a detailed description of key operational components. To be exemplary, the practice must meet the following criteria: (1) it has been validated as an exemplary practice by one or more of the following means: �formal evaluation or research as evidenced by the availability of empirical findings that appear in relevant peer-reviewed literature; �meta-analysis results on a body of literature demonstrating effectiveness of the exemplary practice; �evidence of significant consensus among experts, including evaluators, policy makers, providers, consumers, and families; (2) it has been implemented in at least two different communities, i.e. , in two different geographic areas, by two different service provider organizations, and; (3) it has been fully documented in manual or other written forms that fully describe all key operational components. An exemplary practice may involve an organizational configuration of services or supports, as well as its delivery. It may include a clinical intervention (e.g., the Peer Assertive Treatment Model that works with persons who are difficult to engage, a particular therapeutic foster care model for children and families), an organizational practice (for example, a particular case management model), or a support service (e.g., a particular self-help model, respite care for families with children with serious emotional disturbances). Consensus Building: Consensus building is a process that identifies all key stakeholder and addresses all issues (e.g., key stakeholder attitudes, needs, lack of understanding, buy-in; delivery systems policies, reorganizat ion, staff training or education, appropriations, funding and maintenance of the exemplary practice) necessary to reach agreement among key stakeholders that the exemplary practice can and should be implemented. 5 Goals - Phase II Grants The goal of the Phase II projects is to implement the exemplary practice in the target community. The following are examples of activities that may be supported within the applicant�s work plan for the implementation of the exemplary practice: �Maintaining and strengthening support for the decision by key stakeholders to implement the exemplary practice; �Providing consultation and training for consumers, family members and others on project goals, objectives and processes within the project; �Providing monetary support for travel and other logistical costs necessary to ensure participation by consumers, family members and others needing financial assistance in the implementation process; �Disseminating information to stakeholders and members of the community-at-large regarding the impact of the exemplary practice on services; �Planning and conducting any necessary reorganizat ion of an agency or its components, hiring or reassignment of staff; development of job descriptions and program operation manuals; �Providing education, training and technical assistance for staff in areas such as policy development, management and service delivery to support the implementation of the exemplary practice model; �Providing consultation for implementing strategies that result in local or state changes essential for the implementation and maintenance of the exemplary practice; �Facilitating the negotiation of agreements between or among collaborative and other participating agencies or providers, �Updating and refining a financing plan for maintaining the implemented initiative, and; �Conducting an evaluation of the implementation process and outcomes. Design - Phase II Grants In order to accomplish the goals of Phase II, the grantee is expected to implement a plan that will successfully adopt the exemplary practice into a system of care, including funding and maintenance of the practice supported by permanent funding sources. The plan will: 6 �Identify all barriers with approaches to overcome them; �Ensure consumer and family involvement in the planning, implementation, and initiative monitoring strategies. �Assure the exemplary practice is implemented with fidelity to all service recipients. �Include any necessary orientation, training or consultation to support the implementation of the exemplary practice. �Assure that evaluation process is formulated and understood by all constituents. EVALUATION Each applicant must provide a plan for conducting a process and outcome evaluation of the project by an experienced, objective evaluator. The evaluation should document the implementation of the project and identify factors that contributed to the success or failure of the project�s implementation. There should also be frequent feedback of evaluation findings to the participants, including key stakeholders, on the progress that is being made in their decisions to act toward adopting the exemplary practice. DEFINITIONS �Consensus is agreement among key stakeholders that the exemplary practice can and should be implemented. It must be in sufficient detail that it resolves critical concerns, including financing, and represents a commitment to adopt the practice within a certain time frame. Consensus must also address sustainability of the practice once it is adopted. �Decision to adopt exists when consensus has been achieved and documented (e.g., written agreements, contracts, administrative memos, budgetary recommendations and letters of commitment) among key stakeholders and is accompanied by a feasible implementation plan that includes timetables, a financing plan, evaluation plan, and a firm commitment from permanent funding sources that support the implementation and maintenance of the exemplary practice. �Financing Plan describes the cost in time, labor and dollars for the implementation and maintenance of the exemplary practice. It identifies the permanent funding sources and includes a description of the reliability of the commitment by the permanent funding sources. �Exemplary Practice Implementation is the incorporation of an exemplary practice into a system of care, including funding and maintenance of the pract ice supported by permanent funding sources. 7 �Key Stakeholders are all those entities and individuals whose approval and support are needed for an exemplary practice to be implemented and sustained in a community. Key stakeholders include, but are not limited to, program managers, consumers of mental health and substance abuse treatment and prevention services, families of consumers, advocates for consumers and families, elected and appointed officials who oversee programs and make funding decisions, service system managers, service providers, and other systems such as legal, educational, welfare, and social services that must support implementation. �Permanent Funding Sources may include State and local (e.g., city, county) governments; and private funding sources such as foundations, charitable organizations, private non-profits, lending institutions, and private individuals which reasonably can be expected to support delivery of the exemplary practice for the foreseeable future. SECTION III - PROJECT REQUIREMENTS All applicants should provide a brief abstract for the purpose of publications, reporting to Congress, and press releases, should the application be funded. The abstract should be no longer than 5 lines, 72 characters per line, and could be the first 5 lines of the required Project Abstract, see Section VI Application Procedures. All applicants must provide the information specified below under the proper section heading. The informat ion requested relates to the individual review criteria in Sect ion IV of this PA. PHASE I This section of the application is intended to provide a comprehensive framework and description of all aspects of the proposed project. It should be written in a manner that is self-explanatory to reviewers unfamiliar with the prior related activities of the applicant. It should be succinct and well organized, should use section labels that match those provided in the table of contents. It must contain all the information necessary for reviewers to understand the proposed project including how it addresses age, race/ethnic, cultural, language, and gender issues. In defining the target population, provide just ificat ion for any exclusions under SAMHSA�s Inclusion Policy, Page 1 in Part II. Description of Proposed Exemplary Practice(s) There will be two levels of review. The first level of review includes an evaluation of whether the proposed exemplary practice meets the criteria specified in the PA. Only applications that pass the first level of review will be considered for a technical review. To assure that sufficient information is included to assess the proposed exemplary practice, the application should contain a concise description of the specific exemplary practice proposed for implementation and the 8 target population for the practice. In addition, the application should specify the expected impact on the target population and include a review of the literature for both the proposed exemplary pract ice and the effectiveness of the practice for target populat ion. Applicants may identify more than one exemplary practice. However, such instances should be limited to cases where the exemplary practices are linked in a fashion that suggests the feasibility and desirability of addressing them through one consensus development process. The applicant must present a cogent rationale and justification for the choice of multiple practices. SAMHSA anticipates that most applications will be limited to one exemplary practice, because achievement of the project goal, actual adoption of the exemplary practice, would be compromised if the project was too ambitious. Finally, the application should address each of the following criteria for qualifying a practice as exemplary: A detailed discussion of the basis for claiming that the proposed practice is exemplary, specifying the means for validation. It should address both the empirical evidence supporting the proposed practice and the extent of consensus among experts on the subject. Particular attention should be given to describing why the practice is exemplary for the target population and the circumstances that exist in that target community. A description of previous implementations of the exemplary practice including descriptions of the communities where the practice has been implemented. A detailed discussion that describes and defends the basis for claiming that the exemplary practice has been fully documented as required under Section IV Review of Applications, Review Criteria, Phase I, Level One: Review Of The Exemplary Practice, Part III.� Technical Review Note that the headings A-D in this section match review criteria A-D in Section IV. To assure that sufficient information is included to assess Level Two, the application should contain the following information: A. Project Impact And Feasibility Applicants should describe in detail the significance of implementing the proposed exemplary pract ice in the community and the evidence that the expected results are likely to occur if the grant is awarded. Specifically, the applicant should include the following information: A description of the extent to which key stakeholders indicate support for the project. Categories of key stakeholders, including consumers and families, should be identified and their place in 9 decision making explained. A key element of a successful application will be that it assures that key decision makers are willing to engage in discussion. (Letters of support and other documents showing stakeholder interest in the project should be included in Appendix No. 1 entitled, �Lett rs of Support�.) A description of the potential barriers to adopt the exemplary practice and the methods that will be employed to overcome them. Issues currently blocking implementation of the �ex mplary- practice� (e.g., delivery system�s policy and human resource needs, alternate funding sources, new State legislation, systems, provider and consumer �readiness� status) should be identified and described and accompanied by a discussion of how the proposed project will overcome the barriers and result in the adoption of the exemplary practice. A description of how any anticipated adaptations to the proposed exemplary practice meet community needs while preserving the potential for effectiveness as evidenced in the literature or in other settings. The anticipated changes or adaptations to the exemplary practice should be discussed clearly enough to convey how the balance between preserving the exemplary characteristics of the practice and accommodating local needs will be achieved. A description of the anticipated impact of the proposed exemplary practice on the target population. B. Project Approach/Plans Applicants must submit a work plan that describes the processes and milestones for developing agreement to implement an exemplary practice. The following information should be included: �A description of the objectives of the project and how they will be achieved. �Identification of the elements for systems change in the proposed exemplary practice and the methodology for adaption to local needs. �A description of the applicant�s understanding of the mental health and/or substance abuse issues related to the target populat ion. �A description of the process for identifying and convening key stakeholders and expert resources; for providing necessary orientation, training and consultation for the participants. �A description of the proposed consensus-building and decision-support methodology and an explanation of how implementation of this methodology will result in decisions to adopt the practice. �A detailed description of the financing plan that will support the adoption or implementation 10 of the exemplary practice with identification of the actual or potential permanent funding sources. �A detailed description of the steps to ensure consumer and family involvement in the decision- making process, including methods for convening key stakeholders to initiate discussion regarding all pertinent issues. �A description of the age, culture, language, gender issues of the target population as they relate to the proposed exemplary practice. C. Evaluation Design And Analysis Plan Evaluation designs should ensure the best possible assessment of the intervention and include: �A detailed description of the evaluation plan�s objectives to document implementat ion of the project. �A description of the design to evaluate the consensus building among key stakeholders as part of the decision support process. �A description of the qualitative and quantitative data which will be collected, the instruments to be used, any adaptations/modification to instruments for special populations, the schedule for data collection, who will collect the data, and how it will be analyzed. �A description of the plan to collect information and data on project implementation and to provide feedback from the evaluation to the participants. D. Management Plan And Staffing Management/Staffing of the project should be clearly specified. In particular, the application should include the following: �A description of the qualifications and experience of the project director and other key personnel with respect to the diversity of the population in the project�s community. Also, include key staff for the evaluation and management of the process evaluat ion study. �A description of the capability and experience of the applicant organization with similar projects and populations. �A description of the relevant experience, capability and commitment of proposed collaborators, consultants, and subcontractors. This documentation should be included in Appendix No.2 entitled, �Documentation of experience, capability and commitment of collaborators, consultants, and subcontractors.� 11 �A description of the project management plan including timelines and staffing patterns (e.g., rat ionale for percent of time for key personnel and consultants - with attention to cultural, language, and gender issues). �A description of the relevant resources available (e.g., computer facilities). PHASE II In general, Phase II applicants must demonstrate that they have obtained a reliable decision to adopt an exemplary practice, that a reliable source of funds exist and that a Phase II award increases the chances of successful implementation for all members of the target population. Applicants must provide assurances that any adaptations of the exemplary practice does not erode the potential for effectiveness of the exemplary practice. Applicants must also assure that the practice will be implemented with fidelity to its implementation procedures. Phase II applicants must provide a statement that the target population remains the same as proposed in Phase I. The following specific requirements must also be met: A. Project Impact And Feasibility Applicants should describe in detail their degree of readiness to adopt the exemplary practice. Specifically, evidence should be presented by the applicant that includes the following information: �A brief description of the process evaluation conducted in Phase I and its results. A copy of the evaluation report submitted to CMHS must be included as Appendix No. 3 entitled, �Process and Outcome Evaluation.� �A concise descript ion of the exemplary practice and, if relevant, any adaptations to the exemplary practice. The exemplary practice should be discussed sufficiently enough to convey how the balance between preserving fidelity to procedures of the exemplary practice and accommodat ing local needs will be achieved. (Note: If there are no adaptations to the exemplary practice, a statement cert ifying that the exemplary practice approved for the Phase I program remains the same must be included in Appendix No. 4 entitled, Exemplary Practice Certification. E.g., �I certify that the exemplary practice presented in Phase I remains the same.�) �Identify all the key stakeholders, including consumers and families, along with well- documented evidence indicating key stakeholder support and commitment toward the implementation of the exemplary practice. This evidence should be submitted in Appendix No. 5 entitled, �Letters and Documents� (e.g., letters of commitment, contracts, memoranda of agreement, contracts, administrat ive memos, budgetary recommendations). This 12 documentation should indicate that necessary decisions to implement the exemplary practice under Phase I have been agreed to and are likely to be honored throughout implementation and maintenance of the exemplary practice. �A description of the Phase I Financing Plan that ident ifies the funding sources that will support the exemplary practice and provide evidence of the availability and reliability of the commitment from the identified funding source that will fund implementation and maintenance of the exemplary practice. The application should describe why the funding source is reliable for the foreseeable future. B. Project Approach and Plan Applicants must submit a well-developed implementation plan for how the exemplary practice will be adopted in the new setting and include the following information: �The identification of anticipated barriers and a discussion on how these barriers will be overcome (e.g., needed reorganization of systems or services, new job descriptions new training or education, development of manuals, community education), and a discussion of any known or anticipated implementation problems. �A description of the process to be used to ensure continued consumer and family involvement in the planning and implementation and monitoring of the implemented project strategy. �A description of the methods that will be employed to assure the exemplary practice is implemented with fidelity to all service recipients. �A description of the plan to identify expert resources, and to provide necessary orientation, training or consultation to support the implementation of the exemplary practice. C. Process Evaluation Methodologies and Data Sources The evaluation plan should ensure the best possible documentation of the project�s implementation as well as an analysis plan for identifying factors that contribute to the success or failure of the project assessment of the intervention. This plan should include: �. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An explicit statement of the purpose and object ives of the process evaluation and the questions that will be addressed by the evaluation (e.g., What is the strategic importance that key stakeholders represent that contributes to implementation? What is the reliability of commitment for funding, maintenance, materials and personnel necessary for implementation and maintenance of the exemplary practice?). �A description of the methodology for documenting the project�s implementation. The plan 13 should identify the proposed data sources (e.g., staff observation, meeting minutes, records and reports), instruments/measures, if any, to be used, and the methods of data collection for each data source. D. Organization, Staffing, Resources and Other Support Applicants must describe the expected project management and provide an Implementation Plan Time Line that includes specific activity, target date for completion, and responsible person. (This may be presented in a table.) In particular, the application should include the following: �A description of the qualifications and experience of the project director and other key personnel with respect to the diversity of the population in the project�s community. Also, include key staff for the evaluation and management of the process evaluat ion study. �A description of the relevant experiences, capability and commitment of collaborators, consultants, and subcontractors. This documentation should be included in Appendix No. 6 entitled, �Documentation of experience, capability and commitment of collaborators, consultants, and subcontractors.� �A description of the relevant resources available (e.g., computer facilities). SECTION IV - REVIEW OF APPLICATIONS GUIDELINES Applications submitted in response to this PA will be reviewed for technical merit in accordance with established PHS/SAMHSA review procedures outlined in the Review Process section of Part II. Applicants must review the Special Considerations/Requirements and Application Procedures sections that follow, as well as the guidance provided in Part II, before completing the application. It is important for applicants to carefully follow the outline, headings, and subheadings when providing the requested information. The following information provided in this section corresponds to Section III, subsections Description of Proposed Exemplary and Technical Review A-D. Reviewers will respond to each review criterion on the basis of the information provided by the applicants in their program narrative. Applications will be reviewed and evaluated according to the review criteria that follow. The points noted for each criterion indicate the maximum number of points the reviewers may assign to that criterion if the application is considered to have sufficient merit for scoring. The bulleted statements that follow each review criterion do not have weights. The assigned points will be used to calculate a raw score that will be converted to the official priority score. Peer reviewers will be instructed to review and evaluate each relevant criterion in relation to cultural competence. Points will be deducted from applications that do not adequately address 14 the cultural aspects of the criteria. (See Appendix D in Part II, for guidelines that will be used to assess cultural competence.) REVIEW CRITERIA - PHASE I GRANTS There will be two levels of review. At Level One, reviewers will evaluate the extent to which the exemplary practice meets the specified criteria noted in the PA for exemplary practices. Only those applications that pass the Level One review will receive further review at Level Two. At Level Two, the reviewers will conduct a technical merit review of criteria A-D. Level One: Review Of The Exemplary Practice The following three criteria will be used for the Level One review. The maximum possible points are noted for each. All applications must score a minimum of 5 points per criterion and at least 70 points total within Level One to be eligible for further review at Level Two. (1) The extent to which the practice has been evaluated for effectiveness in community settings by one or more of the following means: (45 points) ! Formal evaluation or research as evidenced by the availability of empirical findings that appear in relevant peer-reviewed literature; ! Meta-analysis results on a body of literature demonstrating effectiveness of the exemplary practice; ! Evidence of significant consensus among experts, including evaluators, policy makers, providers, consumers, and families. (2) The extent of evidence that the practice has been implemented in at least two different communities, i.e., in two different geographic areas, by two different service provider organizations (25 points). (3) The extent to which the practice is fully documented (e.g., by a manual or other written descriptions (30 points). Level Two: Technical Merit Review A. Project Impact and Feasibility (35 points) ! The extent to which key stakeholders indicate support for the project. ! The level and degree of difficulty in overcoming barriers for successful adoption of the 15 exemplary practice. ! The degree of concurrence of the proposed project and possible adaptations with community needs. ! The extent of the anticipated impact of the proposed exemplary practice on the target population. B. Project Approach and Plans (40 points) ! The extent to which the objectives of the project are achievable and realistic. ! The extent to which the project plan incorporates the elements of systems change in the proposed practice, including the methodology for adaptation to local needs. ! The extent to which the project plan demonstrates an understanding of the mental health, substance abuse or co-occurring issues related to the target populat ion. ! The appropriateness of the plan for identifying and convening key stakeholders and expert resources; for providing necessary orientation, training and consultation for the participants. ! The feasibility of the proposed consensus building plan and the appropriateness of the decision support methodology. ! The feasibility of the financing plan supporting adoption and implementat ion of the exemplary practice. ! The extent of consumer and family involvement in the decision making process. ! The extent to which the project plan addresses age, culture, language, and gender appropriateness in the proposed exemplary practice and demonstrates the involvement of representatives of the target population in the conception and planned implementation of the project. C. Evaluation Design and Analysis (10 points) ! The appropriateness of the plan to conduct an evaluation to document the implementation of the project. ! The appropriateness of the design to assess consensus building among key stakeholders. ! The appropriateness of evaluation measures selection; that is, validity and reliability of existing measures selected or strategies for obtaining validity and reliability of measures to be developed, and the appropriateness of the aforementioned measures for the target 16 population. ! The appropriateness of the plan to collect information and data on project implementation, and the plan to provide feedback from the evaluation to the participants. D. Management Plan and Staffing (15 points) ! The qualifications and experience of the project director, evaluator, and other key personnel, and the extent to which the staffing plan reflects appropriate attention to the diversity of the population/community to be served. ! The capability and experience of the applicant organization with similar projects and populations. ! The capability, experience, and evidence of commitment of proposed collaborators, consultants and subcontractors. ! The feasibility of accomplishing the project in terms of (1) time frames, (2) adequacy and availability of resources (e.g., staffing, consultants with attention to cultural, language and gender issues , collaborating agencies, facilities, equipment), and (3) management plan. ! The extent to which the applicant and collaborators commit available and relevant resources to the project (e.g., computer facilities). REVIEW CRITERIA - PHASE II GRANTS A. Project Impact and Feasibility (50 points) ! The extent of evidence of the success of the consensus process as indicated in the Phase I Process Evaluation. ! The degree of concurrence among key stakeholders with any plans for adaptations of the exemplary practice necessary to meet unique community needs. ! The extent to which key stakeholders are committed to implementation of the exemplary pract ice and the strength of the evidence that all necessary decisions to implement the exemplary practice have been made. ! The reliability of the permanent funding source that will support the exemplary practice. B. Project Approach and Plan (30 points) 17 ! The comprehensiveness and reasonableness of the plan for implementing the exemplary practice in the new setting, including attention to anticipated barriers and problems. ! The extent of consumer and/or family involvement in the planing and implementation process. ! The extent to which the exemplary practice is implemented with fidelity to service recipients. ! The extent to which the plan presents an understanding of the barriers and problems facing implementation of the exemplary practice. ! The adequacy of resources identified and proposed for overcoming implementation barriers. C. Process Evaluation Methodologies and Data Sources (10 points) ! The extent to which the design assesses the roles of key stakeholders, degree of commitment of funds, materials, and personnel supporting implementation of the exemplary practice. ! The extent to which the evaluation plan describes appropriate assessment of project activities. D. Organization, Staffing, Resources and Other Support (10 points) ! The appropriateness of the qualifications and experience of the project director, evaluator, and other key personnel, and the extent to which the staffing plan reflects the diversity of the target community. ! The capability, experience, and evidence of commitment of collaborators, consultants and subcontractors. ! The feasibility of accomplishing the project in terms of (1) time frames, (2) adequacy and availability of resources (e.g. , staffing, consultants with knowledge and experience in cultural, language and gender issues , collaborating agencies, facilities, equipment), and (3) management plan. ! The extent to which the applicant and collaborators commit available and relevant resources to the project (e.g., computer facilities). 18 NOTE: Although the reasonableness and appropriateness of the budget for the project is not a review criterion for this PA, the Initial Review Group will be asked to consider it after the merits of the application have been considered. SECTION V - SPECIAL CONSIDERATIONS/REQUIREMENTS SAMHSA�s policies and special considerations/requirements related to this program include: ! SAMHSA�s Inclusion Policy ! Government Performance Monitoring ! Healthy People 2000 ! Consumer Bill of Rights ! Promoting Nonuse of Tobacco ! Supplantation of Existing Funds ! Letter of Intent ! Coordination with Other Federal/Non-Federal Programs ! Intergovernmental Review (E.O. 12372) ! Public Health System Reporting Requirements ! Confidentiality/SAMHSA Participant Protection. The SAMHSA CMHS Director has determined that projects funded under this program must meet SAMHSA Participant Protection requirements. Specific guidance and requirements for the application related to these policies and special considerations/requirements can be found in Part II, General Policies and Procedures applicable to all SAMHSA Applications for Discretionary Grants and Cooperative Agreements in the section by the same name. SECTION VI - APPLICATION PROCEDURES All applicants must use the Public Health Service (PHS) Grant Applicat ion form 5161-1 (Rev. 6/99) and follow the requirements and guidelines for developing an application presented in Part I Programmatic Guidance and Part II General Policies and Procedure Applicable to all SAMHSA Applications for Discretionary Grants and Cooperative Agreements. The following must be typed in Item Number 10 on the face page of Standard Form 424 (found in PHS 5161-1) of the application form:�PA 00-003 - Community Action Grant Program�. For more specific information on where to obtain application materials and guidelines, see the Application Procedures section in Part II. Complete application kits for this program may be obtained from the Knowledge Exchange Network (KEN), phone number: 800-789-2647. The address for KEN is provided in Part II. 19 Completed applications must be sent to the following address: SAMHSA Programs Center for Scientific Review National Institutes of Health Suite 1040 6701 Rockledge Drive MSC-7710 Bethesda, MD 20892-7710* *Applicants who wish to use express mail or courier service should change the zip code to 20817 APPLICATION RECEIPT AND REVIEW SCHEDULE Applications will be received and reviewed two times per year according to the following schedule: Receipt Date IRG Review Council Review Earliest Start Date May 10 June September September September 10 October January February Applications must be received by the above receipt date(s) to be accepted for review. An application received after the deadline may be acceptable if it carries a legible proof-of-mailing date assigned by the carrier and the proof-of-mailing date is not later than 1 week prior to the deadline date. Private metered postmarks are not acceptable as proof of timely mailing. (NOTE: These instructions replace the "Late Applications" instructions found in the PHS 5161-1.) If the receipt date falls on a weekend, it will be extended to Monday; if the date falls on a holiday, it will be extended to the following work day. Applicants are advised that certain aspects of this program and one or more of the above receipt dates may be withdrawn, depending on the availability of funds. The SAMHSA Center for Mental Health Services will annually publish in the Federal Register a Notice of Funding Availability (NOFA) and a statement of the applicable receipt dates for this program. Applicants are strongly encouraged to verify receipt dates and terms of funding before preparing and submitting applications. CONSEQUENCES OF LATE SUBMISSION Applications received after the specified receipt dates are subject to assignment to the next review cycle or may be returned to the applicant without review. APPLICATION REQUIREMENTS/COMPONENT CHECK LIST 20 All applicants must use the Public Health Service (PHS) Grant Applicat ion form 5161-1 (Rev. 6/99) and follow the requirements and guidelines for developing an application presented in Part I Programmatic Guidance and Part II General Policies and Procedure Applicable to all SAMHSA Applications for Discretionary Grants and Cooperat ive Agreements. The application should provide a comprehensive framework and description of all aspects of the proposed project. It should be written in a manner that is self-explanatory to reviewers unfamiliar with the prior related activities of the applicant . It should be succinct and well organized, should use section labels that match those provided in the table of contents for the Program Narrative that follows, and must contain all the information necessary for reviewers to understand the proposed project. To ensure that sufficient information is included for the technical merit review of the application, the Programmatic Narrative section of application must address, but is not limited to, issues raised in the sections of this document entitled: 1. Program Description and Project Requirements 2. Guidelines and Review Criteria for Applicant Note: It is requested that on a separate sheet of paper the name, title, and organization affiliation of the individual who is primarily responsible for writing the application be provided. Providing this information is voluntary and will in no way be used to influence the acceptance or review of the application. When submitting the information, please insert the completed sheet behind the application face page. A COMPLETE application consists of the following components IN THE ORDER SPECIFIED BELOW. A description of each of these components can be found in Part II. ___FACE PAGE FOR THE PHS 5161-1 (Standard Form 424 - See Appendix A in Part II for instructions.) ___OPTIONAL INFORMATION ON APPLICATION WRITER (see note above) ___ABSTRACT (not to exceed 30 lines) ___TABLE OF CONTENTS (include page numbers for each of the major sections of the Program Narrative, as well as for each appendix) ___BUDGET FORM (Standard Form 424A - See Appendix B in Part II for instructions.) ___PROGRAM NARRATIVE (The information requested for Review of the Exemplary Practice and sections A-D of the Program Narrative is discussed in the subsections with 21 the same titles in Section III - Project Requirements and Section IV - Review of Applications. This section may not exceed 25 single-spaced pages. Applications exceeding these page limits will not be accepted for review and will be returned to the applicant.) ___Review of the Exemplary Practice ___A. Project Impact and Feasibility ___B. Project Approach and Plans ___C. Evaluation Design and Analysis Plan ___D. Management Plan and Staffing There are no page limits for the following sections E-H except as noted in G. Biographical Sketches/Job Descriptions. Sections E-H will not be counted toward the 25 page limitation for sections A-D. __E. Literature Citations (This section must contain complete citations, including titles and all authors, for literature cited in the application.) __F. Budget Justification/Existing Resources/Other Support Sections B, C, and E of the Standard Form 424A must be filled out according the instructions in PartII, Appendix B. A line item budget and specific justification in narrative form for the first project year�s direct costs AND for each future year must be provided. For contractual costs, provide a similar yearly breakdown and justification for ALL costs (including overhead or indirect costs. All other resources needed to accomplish the project for the life of the grant (e.g., staff, funds, equipment, office space) and evidence that the project will have access to these, either through the grant or, as appropriate, through other resources, must be specified. Other Support (�Other Support� ref rs to all current or pending support related to this application. Applicant organizations are reminded of the necessity to provide full and reliable information regarding "other support," i.e., all Federal and non-Federal active or pending support. Applicants should be cognizant that serious consequences could result if failure to provide complete and accurate information is construed as misleading to the PHS and could, therefore, lead to delay in the processing of the application. In signing the face page of the application, the authorized representative of the applicant organization cert ifies that the application information is accurate and complete. 22 For your organization and key organizations that are collaborating with you in this proposed project, list all currently active support and any applications/proposals pending review or funding that relate to the project. If there are none, state "none." For all active and pending support listed, also provide the following information: 1. Source of support (including identifying number and title). 2. Dates of entire project period. 3. Annual direct costs supported/requested. 4. Brief description of the project. 5. Whether project overlaps, duplicates, or is being supplemented by the present application; delineate and justify the nature and extent of any programmatic and/or budgetary overlaps. ___G. Biographical Sketches/Job Descriptions. A biographical sketch must be included for the project director and for other key positions. Each of the biographical sketches must not exceed 2 pages in length. In the event that a biographical sketch is included for an individual not yet hired, a letter of commitment from that person must be included with his/her biographical sketch. Job descriptions for key personnel must not exceed 1 page in length. The suggested contents for biographical sketches and job descriptions are listed in Item 6 in the Program Narrative section of the PHS 5161-1. ___H. Confidentiality/SAMHSA Participant Protection. The information provided in this section will be used to determine whether the level of protection of participants appears adequate or whether further provisions are needed, according to SAMHSA Participant Protection (SPP) standards. Adequate protection of participants is an essential part of an application and will be considered in funding decisions. Projects proposed under this announcement may expose participants to risks in as many ways as projects can differ from each other. Following are some examples, but they do not exhaust the possibilities. Applicants should report in this section any foreseeable risks for project participants, and the procedures developed to protect participants from those risks, as set forth below. Applicants should discuss how each element will be addressed, or why it does not apply to the project. Note: So that the adequacy of plans to address protect ion of participants, confidentiality, and other ethical concerns can be evaluated, the information requested below, which may appear in other sections of the narrative, should be included in this section of the application as well. 1. Protection from Potential Risks: 23 (a) Identify and describe any foreseeable physical, medical, psychological, social, legal, or other risks or adverse effects, besides the confidentiality issues addressed below, which are due either to participation in the project itself, or to the evaluation activities. (b) Where appropriate, describe alternative treatments and procedures that might be advantageous to the subjects and the rationale for their nonuse. (c) Describe the procedures that will be followed to minimize or protect participants against potential risks, including risks to confidentiality. (d) Where appropriate, specify plans to provide needed professional intervention in the event of adverse effects to participants. 2. Equitable selection of participants: Target population(s): Describe the sociodemographic characteristics of the target population(s) for the proposed project, including age, gender, racial/ethnic composition, and other distinguishing characteristics (e.g., homeless youth, foster children, children of substance abusers, pregnant women, institutionalized individuals, or other special population groups). Recruitment and Selection: (a) Specify the criteria for inclusion or exclusion of participants and explain the rationale for these criteria. (b) Explain the rat ionale for the use of special classes of subjects, such a pregnant women, children, institutionalized mentally disabled, prisoners, or others who are likely to be vulnerable. (c) Summarize the recruitment and selection procedures, including the circumstances under which participation will be sought and who will seek it. 3. Absence of Coercion: (a) Explain whether participation in the project is voluntary or mandatory. Identify any potentially coercive elements that may be present (e.g., court orders mandating individuals to participate in a particular intervention or treatment program). (b) If participants are paid or awarded gifts for involvement, explain the remuneration process. 24 (c) Clarify how it will be explained to volunteer participants that their involvement in the study is not related to services and the remunerat ion will be given even if they do not complete the study. 4. Appropriate Data Collection: (a) Identify from whom data will be collected (e.g., participants themselves, family members, teachers, others) and by what means or sources (e.g., school records, personal interviews, written questionnaires, psychological assessment instruments, observation). (b) Identify the form of specimens (e.g., urine, blood), records, or data. Indicate whether the material or data will be obtained specifically for evaluative/research purposes or whether use will be made of existing specimens, records, or data. Also, where appropriate, describe the provisions for monitoring the data to ensure the safety of subjects. (c) Provide, in Appendix No. 7, entitled "Data Collection Instruments/Interview Protocols," copies of all available data collection instruments and interview protocols that will be used or proposed to be used in the case of cooperative agreements. 5. Privacy and Confidentiality: Specify the procedures that will be implemented to ensure privacy and confidentiality, including by whom and how data will be collected, procedures for administration of data collection instruments, where data will be stored, who will/will not have access to information, and how the ident ity of participants will be safeguarded (e.g., through the use of a coding system on data records; limiting access to records; storing identifiers separately from data). Note: If applicable, grantees must agree to maintain the confidentiality of alcohol and drug abuse client records in accordance with the provisions of Title 42 of the Code of Federal Regulations, Part 2 (42 CFR, Part 2). 6. Adequate Consent Procedures: (a) Specify what information will be provided to participants regarding the nature and purpose of their participation; the voluntary nature of their participation; their right to withdraw from the project at any time, without prejudice; anticipated use of data; procedures for maintaining confidentiality of the data; potential risks; and procedures that will be implemented to protect participants against these risks. (b) Explain how consent will be appropriately secured for youth, elderly, low literacy and/or for those who English is not their first language. 25 Note: If the project poses potential physical, medical, psychological, legal, social, or other risks, awardees may be required to obtain written informed consent. (c) Indicate whether it is planned to obtain informed consent from part icipants and/or their parents or legal guardians, and describe the method of documenting consent. For example: Are consent forms read to individuals? Are prospective participants questioned to ensure they understand the forms? Are they given copies of what they sign? Copies of sample (blank) consent forms should be included in Appendix No. 8, entitled "Sample Consent Forms." If appropriate, provide English translations. Note: In obtaining consent, no wording should be used that implies that the participant waives or appears to waive any legal rights, is not free to terminate involvement with the project, or releases the institution or its agents from liability for negligence. (d) Indicate whether separate consents will be obtained for different stages or aspects of the project, and whether consent for the collection of evaluative data will be required for participation in the project itself. For example, will separate consent be obtained for the collection of evaluation data in addition to the consent obtained for participation in the intervent ion, treatment, or services project itself? Will individuals not consent ing to the collection of individually identifiable data for evaluative purposes be permitted to participate in the project? 7. Risk/Benefit Discussion: Discuss why the risks to subjects are reasonable in relation to the anticipated benefits to subjects and in relation to the importance of the knowledge that may reasonably be expected to result. APPENDICES (Only the appendices specified below may be included in the application. These appendices must not be used to extend or replace any of the required sections of the Program Narrative. Except for Appendix 3 which is limited to 20 pages, the total number of pages in all other appendices CANNOT EXCEED 30 PAGES, excluding all instruments.) ___Appendix 1: Letters of Support (Phase I Applicants) ___Appendix 2: Documentation of experience, capability and commitment of collaborators, consultants, and subcontractors (Phase I Applicants) ___Appendix 3: Process and Outcome Evaluation (Phase II Applicants) ___Appendix 4: Exemplary Practice Certification (Phase II Applicants) ___Appendix 5: Letters and Documents (Phase II Applicants) 26 ___Appendix 6: Documentation of experience, capability and commitment of collaborators, consultants, and subcontractors (Phase II Applicants) ___ Appendix 7: Data Collection Instruments/Interview Protocols (Phase I and Phase II, if applicable) ___Appendix 8: Sample Consent Forms (Phase I and Phase II, if applicable) ASSURANCES NON-CONSTRUCTION PROGRAMS (STANDARD FORM 424B) CERTIFICATIONS DISCLOSURE OF LOBBYING ACTIVITIES CHECKLIST PAGE (See Appendix C in Part II for instructions) TERMS AND CONDITIONS OF SUPPORT For specific guidelines on terms and conditions of support, allowable items of expenditure and alterations and renovations, applicants must refer to the sections in Part II by the same names. In addition, in accepting the award the Grantee agrees to provide SAMHSA with Government Performance Reporting Act (GPRA) Client Outcome (if applicable) and Evaluation Data. Reporting Requirements: For the SAMHSA policy and requirements related to reporting, applicants must refer to the Reporting Requirements section in Part II. Lobbying Prohibitions: SAMHSA's policy on lobbying prohibit ions is applicable to this program; therefore, applicants must refer to the section in Part II by the same name. AWARD DECISION CRITERIA Program applications will be considered for funding on the basis of overall technical merit as determined through the IRG and CMHS National Advisory Council review process. Award criteria will include: ! Availability of funds, ! Overall program balance in terms of geography (including rural/urban/suburban areas), race/ethnicity of proposed project population, and project size, ! Balance among projects in terms of types of exemplary practices, ! Balance among projects in terms of types of applicant organizations, e.g., 27 community based organizations, State or community agencies, provider groups, consumer groups, universities, criminal justice organizations, health care organizations, etc.), and ! Balance among projects in terms of target populations with special needs, e.g., children that reside in foster care or homeless, people with HIV/AIDS. CONTACTS FOR ADDITIONAL INFORMATION Questions concerning program issues may be directed to the appropriate program staff: Santo J. (Buddy) Ruiz Community Support Programs Branch Center for Mental Health Services Substance Abuse and Mental Health Services Administration 5600 Fishers Lane, Room 11C-22 Rockville, MD 20857 (301) 443-3653 Michele Herman Children, Adolescent and Family Branch Center for Mental Health Services Substance Abuse and Mental Health Services Administration 5600 Fishers Lane, Room 11C-16 Rockville, MD 20857 (301) 443-1333 G.T. (Gigi) Belanger Homeless Programs Branch Center for Mental Health Services Substance Abuse and Mental Health Services Administration 5600 Fishers Lane, Room 11C-05 Rockville, MD 20857 (301) 443-3706 Patricia Gratton Office of the Director Center for Mental Health Services Substance Abuse and Mental Health Services Administration 28 5600 Fishers Lane, Room 11C-26 Rockville, MD 20857 (301) 443-3603 REFERENCES Agency for Health Care Policy and Research. (1992). An annotated bibliography: Effective dissemination to health care practitioners and policy makers. Rockville, MD: Author. AHEC/Community Partners. (1995). From the ground up: A workbook on coalition building and community development. Amherst, MA: Author. Backer, T.E. (1995a). Assessing and enhancing readiness for change: Implications for technology transfer. In Backer, T.E., David, S.L. & Soucy, G. (Eds). Reviewing the behavioral science knowledge base on technology transfer. Rockville, MD: National Institute on Drug Abuse. Backer, T.E. (1995b). Dissemination and utilization strategies for foundations; Adding value to grantmaking. Kansas City, MO: Ewing M. Kauffman Foundation. Backer, T.E., Brown, B.S. & Howard, E.A. (1994). Evaluating drug abuse technology transfer: An analytic review. Los Angeles: Human Interaction Research Institute. Backer, T.E., Liberman, R.P. & Kuchnel, T. (1986). Dissemination and adoption of innovative psychosocial interventions. Journal of Clinical & Consulting Psychology, 54(1), 111- 118. Baum, H.S. (1987). The Invisible Bureaucracy. New York: Oxford Kavanaugh, K.H. (1995). Collaboration and diversity in technology transfer. In Backer, T.E., David, S.L. & Soucy, G. (Eds). Reviewing the behavioral science knowledge base on technology transfer. Rockville, MD: National Institute on Drug Abuse. Kiresuk, T.J. (1993). The evaluation of knowledge utilization: Placebo and nonspecific effects, dynamical systems, and chaos theory. Journal of the American Society for Information Science, 44(4), 235-241. Klein, S.S. (1993a). Are there better ways to identify and share the best within and among federal education programs? Evaluation and Program Planning, 16(3), 227-239. Klein, S.S. (1993b). Sharing the best: Finding better ways for the federal government to use 29 evaluation to guide the dissemination of promising and exemplary education solutions. Evaluation and Program Planning, 16(3), 213-217. Kochlar, C. & Erickson, M.R. (1993). Business-education partnerships for the 21st century: A practical guide for school improvement. Gaithersburg, MD: Aspen Publishers. Mattessich, P.W. & Monsey, B. R. (1992). Collaboration: What Makes It Work. St. Paul, Minnesota: Amherst H. Wilder Foundation. Rogers, E.M. (1995) Diffusion of Innovat ions. New York: Free Press. Fourth edition. Sechrest, L., Backer, T.E., Rogers, E.M., Campbell, T.F., & Grady, M.L. (Eds). (1994). Effective dissemination of clinical and health information. AHCPR Pub. No. 95-0015. Rockville, MD: Agency for Health Care Policy and Research. Tenkasi, R.V., & Mohrman, S., (1995). Technology Transfer as Collaborative Learning. Reviewing the behavioral science knowledge base on technology transfer. Rockville, MD: National Institute on Drug Abuse. ADDITIONAL RESOURCES Cross, T., and Deserly, K., (1996), American Indian Children�s Mental Health Services, an . Assessment of Tribal Access to Children�s Mental Health Funding and a Review of Tribal Mental Health Programs, National Indian Child Welfare Association. Kauffman and Associates, Gathering of Native Americans (GONA) Facilitator Guide, Substance Abuse Prevention Curriculum, (scheduled release of condensed version, 2000) U.S. Department of Health and Human Services, Indian Health Service, and Substance Abuse and Mental Health Administration. Original release in1992 by Substance Abuse and Mental Health Administration, Center for Substance Abuse Prevention). Manson, S., ed.(1982) New Directions in Prevention among American Indian and Alaska Native Communities, National Center for American Indian and Alaska Native Mental Health Research, Oregon Health Sciences University. U.S. Congress, Office of Technology Assessment, Indian Adolescent Mental Health, (1990). U.S. Department of Health and Human Services, Indian Health Services, Office of the Director. (1998). Key Facts about American Indian and Alaska Native Youth. U.S. Department of Health and Human Services, Substance Abuse and Mental Health Administration. (1998). Substance Abuse and Mental Health Statistics Source Book, �Suicide, Homicide, and Total Death Rates by Race/Ethnicity�, pp. 228-237. 30 30 U.S. Department of Health and Human Services, Substance Abuse and Mental Health Administration, SAMHSA Fact Sheet, �Prevalence of Substance Abuse among Racial/Ethnic Subgroups in the United States, 1991-1993". U.S. Department of Health and Human Services, Substance Abuse and Mental Health Administration, Center for Substance Abuse Prevention, Substance Abuse Resource Guide: American Indians and Native Alaskans, (1998) U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics, American Indians and Crime, (1999).