Topics on this page: Goal 2. Objective D | Table of Related Performance Measures | Analysis of Results | Plans for the Future | FY 2014 Strategic Review Objective Progress Update Summary
Goal 2. Objective D: Increase our understanding of what works in public health and human services practice
Working together with its public and private partners, HHS is committed to improving the quality of public health and human service practice by conducting applied, translational, and operations research and evaluations. HHS uses these studies to inform policy and program implementation efforts. HHS has identified approaches that help people make healthy choices, assist communities as they work to improve the health and well-being of their residents, support safety and stability of individuals and families, and help children reach their full potential. HHS also monitors and evaluates programs to assess efficiency and responsiveness and to inform the effective use of information in strategic planning, program or policy decisions, and program improvement.
HHS investments in public health and human service research have yielded many important findings about what works. The Department will work to identify promising, effective approaches that are culturally competent and effective for populations with varying circumstances and needs.
A number of HHS agencies promote the adoption of evidence-based programs and practices including ACF, ACL, AHRQ, CDC, HRSA, NIH, OASH, and SAMHSA. CDC conducts systematic reviews of scientific literature that form the basis for evidence-based Community Preventive Services Task Force recommendations about effective programs, services, and policies for improving health and preventing many chronic and infectious diseases and injuries. ACF and SAMHSA both maintain “What Works” clearinghouses of research in the areas of family and youth support and mental health and substance abuse services to facilitate evidence based decision making. Below are representative measures which HHS and its components will use to guide performance. The Office of the Secretary led this Objective’s assessment as a part of the Strategic Review.
Objective 2.D Table of Related Performance Measures
Increase the percentage of Community-Based Child Abuse Prevention (CBCAP) total funding that supports evidence-based and evidence-informed child abuse prevention programs and practices. (Lead Agency - ACF; Measure ID - 7D)
| FY 2012 | FY 2013 | FY 2014 | FY 2015 | FY 2016 | FY 2017 | |
|---|---|---|---|---|---|---|
| Target | 65.3 % | 76.7 % | 71.4 % | 92.4 % | Prior Result +3PP | Prior Result +3PP |
| Result | 73.7 % | 68.4 % | 89.4 % | Oct 31, 2016 | Oct 31, 2017 | Oct 31, 2018 |
| Status | Target Exceeded | Target Not Met | Target Exceeded | Pending | Pending | Pending |
Increase access to and awareness of the Guide to Community Preventive Services, and Task Force Findings and Recommendations, using page views as proxy for use (Lead Agency - CDC; Measure ID - 8.B.2.5)
| FY 2012 | FY 2013 | FY 2014 | FY 2015 | FY 2016 | FY 2017 | |
|---|---|---|---|---|---|---|
| Target | 973,724 | 1,032,147 | 1,400,000 | 1,400,000 | 1,420,000 | 1,420,000 |
| Result | 1,220,956 | 1,359,772 | 1,339,561 | 1,301,832 | Oct 31, 2016 | Oct 31, 2017 |
| Status | Target Exceeded | Target Exceeded | Target Not Met | Target Not Met | Pending | Pending |
By 2018, identify three effective system interventions generating the implementation, sustainability and ongoing improvement of research-tested interventions across health care systems. (Lead Agency - NIH; Measure ID - SRO-8.7)
| FY 2012 | FY 2013 | FY 2014 | FY 2015 | FY 2016 | FY 2017 | |
|---|---|---|---|---|---|---|
| Target | Complete target by identifying three effective implementation strategies that enhance the uptake of research-tested interventions in service systems such as primary care, specialty care and community practice. | Identify three key factors influencing the sustainability of research-tested interventions in service systems such as primary care, specialty care, and community practice. | Identify three effective implementation strategies that enhance the sustainability of research-tested interventions in service systems such as primary care, specialty care and community practice. | Identify three (3) key factors influencing the scaling up of research-tested interventions across large networks of services systems such as primary care, specialty care and community practice. | Initiate testing of hypothesized mechanism of treatment effect of one novel intervention, and determine whether the intervention should progress further to clinical testing. | Establish one research-practice partnerships to improve dissemination, implement-ation, and continuous improvement of evidence-based mental health care services. |
| Result | NIH identified three approaches that enhance the uptake of research-tested interventions in service delivery systems addressing child mental health, attention deficit hyperactivity disorder, and depression. | NIH researchers identified three influences on sustainability of research-tested interventions in service systems: Community Development Teams in child mental health service systems; barriers and facilitators to evidence-based interventions to control blood pressure in community practice; and a set of factors to enhance sustainability of health care interventions across multiple settings. | NIH researchers identified three effective implementation strategies that enhance the sustainability of research-tested interventions in service systems including: strategies to overcome these barriers and to enhance the sustainability of research-tested interventions; development of specific scales on sustainability as a strategy to identify factors affecting ongoing use as diagnostics for system action; and, strategies to scale-up and sustain HIV prevention interventions within low and middle income countries. | NIH researchers identified three key factors that influence the scaling up of research-tested interventions across large services systems including: the utilization of technological approaches to enhance validation and scale-up; optimization of treatment fidelity in the delivery of research-based treatment; and the development of research community partnerships to promote research-tested interventions. | Dec 31, 2016 | Dec 31, 2017 |
| Status | Target Met | Target Met | Target Met | Target Met | In Progress | Pending |
Analysis of Results
The most efficient and effective programs often use evidence-based and evidence-informed practices. Currently, ACF’s Children’s Bureau and its National Resource Center for the Community-Based Child Abuse Prevention (CBCAP) program are working closely with states to promote more rigorous evaluations of their funded programs. The CBCAP program developed an efficiency measure to gauge progress towards programs’ use of these types of practices. For the purposes of this efficiency measure, the Children’s Bureau defines evidence-based and evidence-informed programs and practices along a four level continuum (from least to most): Emerging and Evidence Informed; Promising; Supported; and Well-Supported. The funding directed towards these types of programs (weighted by “evidence-informed” or “evidence-based” practices level) will be calculated over the total amount of funding used for direct service programs to determine the percentage of total funding that supports evidence-based and evidence-informed programs and practices. HHS selected the target of a three percentage point annual increase in the amount of funds devoted to evidence-based practice as a meaningful increment of improvement through FY 2017. This performance expectation takes into account the fact that this is the first time that the program has required grantees to target their funding towards evidence-based and evidence-informed programs, and it will take time for states to adjust their funding priorities to meet these new requirements. In FY 2014, the percentage of total funding increased significantly to 89.4 percent. This increase is due to significantly more funding being used for programs on the higher continuum of evidence, especially in the "supported" category.
The Community Preventive Services Task Force (Task Force) is an independent, nonpartisan, nonfederal, unpaid panel of public health and prevention experts. The Task Force's mandate is to identify population-based programs, services, and policies that are effective in saving American lives and dollars, increasing longevity, and improving quality of life. Task Force recommendations provide information about evidence-based options that decision makers and stakeholders can consider when determining what best meets the specific needs, preferences, available resources, and constraints of their jurisdictions and constituents. Task Force recommendations are compiled in The Guide to Community Preventive Services (The Community Guide). The Community Guide website (http://www.thecommunityguide.org) is the primary dissemination tool used to 1) provide information about Task Force-recommended options to individuals, organizations, agencies, and communities who are making their own decisions about what is best for their circumstances, and 2) assist those who request help in implementing Task Force recommendations that best meet their needs. In FY 2015, CDC received 1,301,832 page views on the Community Guide website, a decline over the previous year’s result and missing the target. CDC maintained the majority of expected page views (~96 percent) from its high in FY 2013 by utilizing processes, strategies, and web-based products developed and tested during 2011-2013. CDC expects modest growth in page views in FY 2016 with a likely plateau in FY 2017 due to the release and promotion of enhancements to the Community Guide website, which provide customized decision and implementation support for a range of user audiences.
NIH has broadened its portfolio of implementation research by encouraging teams of scientists and practice stakeholders to work together to overcome barriers to implementing research-tested interventions. In FY 2015 NIH researchers identified three key factors that influence the scaling up of research-tested interventions across large services systems such as child welfare, primary care, specialty care, and community practice. These key factors include the utilization of technological approaches to enhance validation and scale-up; optimization of treatment fidelity in the delivery of research-based treatment; and the development of research community partnerships to promote research-tested interventions. As documented in the past several decades, many barriers exist to the successful implementation of effective clinical practices, and sustainability of those practices over time is even more difficult to achieve. These barriers exist at individual, organizational, system, and policy levels. NIH-funded investigators have developed a number of strategies to overcome these barriers and to enhance the sustainability of research-tested interventions.
Plans for the Future
Over time, the ACF Community-Based Child Abuse Prevention (CBCAP) program expects to increase the number of effective programs and practices that are implemented, maximizing the impact and efficiency of CBCAP funds. ACF is committed to continuing to work with CBCAP grantees to invest in known evidence-based practices, while continuing to promote evaluation and innovation, so as to expand the availability of evidence-informed and evidence-based practice over time.
CDC will continue to conduct systematic reviews of the evidence on the effectiveness of community preventive programs, services, and policies. It will also expand the reach of the Community Guide website, improve users' satisfaction with the usefulness of the website, multiply options for technical assistance in implementing Task Force findings, and increase dissemination of Task Force findings in public health practice through partnerships and collaborations. CDC will expand the reach and usefulness of the Community Guide website including the release and promotion of enhancements to the website (developed in 2012-2014) that provide customized decision and implementation support for a range of user audiences.
NIH has developed and will implement a series of process steps to identify three effective system interventions generating the implementation, sustainability, and ongoing improvement of research-tested interventions across health care systems by 2018.
FY 2014 Strategic Review Objective Progress Update Summary
Please note that this section summarizes the result of the FY 2014 HHS Strategic Review process, limiting the scope of content to that available prior to spring of 2015. Due to this constraint, the following may not be the most current information available.
Conclusion: Progressing
Analysis: HHS is committed to improving the quality of public health and human services practice by conducting applied, translational, and operations research and evaluations as well as using data for decision-making. Accomplishments related to furthering this objective included the publication of an ACF evaluation policy which identifies five important principles to guide evaluation: rigor, relevance, transparency, independence, and ethics. Notable examples of recent publications include findings from Head Start CARES, a randomized trial of the effectiveness of three approaches to improving preschoolers’ social and emotional competence; data from the three-year follow-up survey of the National Survey of Child and Adolescent Well-Being, a paper on integrating human services and health programs under the Affordable Care Act, and a journal article on “Sustained Effects of the Communities That Care System on Prevention Service System Transformation.”
Early outcomes from the Partnership in Employment (PIE) Systems Change grant have shown improvements in a variety of approaches within states including California’s Program for Transition Age Youth no longer permitting sub-minimum wages for transitioning youth as well as efforts in Missouri which led to increases in Business Partnerships and the number of businesses providing employment opportunities for youth and young adults with I/DD. The Bridgespan Group released in November 2013 a report entitled, “What Does It Take to Implement Evidence-based Practices? A Teen Pregnancy Prevention Program Shows the Way,” in which they identified the Teen Pregnancy Prevention program as a model for implementing evidence-based programs with fidelity and quality. In addition, several HHS Divisions partnered to host the 2014 HHS Teen Pregnancy Prevention (TPP) Conference, Bridging the Gaps: Eliminating Disparities in Teen Pregnancy and Sexual Health. This collaborative effort brought together close to 1,000 people, sparked previously unavailable opportunities for coordination of state and local efforts to prevent teen pregnancy, reduced duplication, and demonstrated good stewardship of federal funds.
The Minority AIDS Initiative addresses the disproportionate impact of HIV/AIDS on communities of color. In FY 2014, HRSA funded seven new or ongoing MAI projects that increase understanding of what works in the delivery of comprehensive, culturally, and linguistically appropriate HIV/AIDS care and treatment. Examples include: Reaching Low Volume Clinician Providers Through Telehealth Training Centers; UCARE4 (The Use of Mobile Texting to Improve Retention in Care and Medication Adherence in Youth and Adult Minorities Living with HIV in Southern States); and Health Literacy Project Targeting Adult and Young Black MSM.
HHS is working to build evidence about what works though available procurement vehicles, which lack the flexibility needed to match the dynamic nature of the high-quality evaluations and statistical surveys that are essential to building evidence about what works. Barriers exist that prevent the adoption and implementation of newly devised and research-tested interventions into service systems. These barriers may occur at the individual level, practice level, or broader organizational level. For example, an evidence-based program may require extensive clinical training, the cost of which may be prohibitive for a provider. Other financial barriers may exist such as the inability to get reimbursed for providing a specific intervention.
