FY 2019 Annual Performance Plan and Report - Data Sources and Validation
Fiscal Year 2019
Released April, 2018
Data Sources and Validation
Reporting HHS Agencies: ACF | ACL | AHRQ | ASA | ASPR | CDC | CMS | FDA | HRSA | IHS | NIH | SAMHSA
HHS FY 2019 Annual Performance Plan and Report Data Validation Table
Administration for Children and Families (ACF)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 3A (ACF) |
Classroom Assessment Scoring System (CLASS: Pre-K) | CLASS: Pre-K is a valid and reliable tool that uses observations to rate the interactions between adults and children in the classroom. Reviewers, who have achieved the standard of reliability, assess classroom quality by rating multiple dimensions of teacher-child interaction on a seven point scale (with scores of one to two being in the low range; three to five in the mid-range; and six to seven in the high range of quality); low range is defined as any CLASS review with a domain scoring below 2.5 for purposes of this performance measure. ACF will implement ongoing training for CLASS: Pre-K reviewers to ensure their continued reliability. Periodic double-coding of reviewers is also used, which is a process of using two reviewers during observations to ensure they continue to be reliable in their scoring. |
| 4A (ACF) |
The Runaway and Homeless Youth - Homeless Management Information System (RHY-HMIS) | In FY 2015, ACF entered into a Memorandum of Understanding with HUD, SAMHSA, and VA to use Homeless Management Information Systems (HMIS) as primary information technology systems to enter data on clients served by federally-funded homeless assistance services. Since FY 2015, RHY grantees have been using local HMIS systems to upload de-identified client-level data to the RHY national data repository called RhyPoint. Following each upload, grantee data are validated by RhyPoint and a report is sent to grantees to monitor and improve data completeness and quality. The aggregate data are then cleaned and validated using a set of business rules developed by Family and Youth Services Bureau to make sure that records are accurate and relevant using a number of logic checks. |
| 7B (ACF) | National Child Abuse and Neglect Data System (NCANDS) | States report child welfare data to ACF through the NCANDS. Each state’s annual NCANDS data submission undergoes an extensive validation process which may result in revisions to improve data accuracy. To speed improvement in these data, ACF funds a contractor to provide technical assistance to states to improve reporting and validate all state data related to outcome measures. The Children’s Bureau, in ACF, and the NCANDS project team are working with states through national meetings, advisory groups, and state-specific technical assistance to encourage the most complete and accurate reporting of these data in all future submissions. All of these activities should continue to generate additional improvements in the data over the next few years. |
| 7D (ACF) |
State Annual Reports | States are required to submit an Annual Report addressing each of the Community-Based Child Abuse Prevention (CBCAP) performance measures outlined in Title II of the Child Abuse Prevention and Treatment Act. One section of the report must “provide evaluation data on the outcomes of funded programs and activities.” The 2006 CBCAP Program Instruction adds a requirement that the states must also report on the OMB performance measures reporting requirements and national outcomes for the CBCAP program. States were required to report on this efficiency measure starting in December 2006. The three percent annual increase represents an ambitious target since this is the first time that the program has required programs to target their funding towards evidence-based and evidence-informed programs, and it will take time for states to adjust their funding priorities to meet these requirements. |
| 14D (ACF) |
Family Violence Prevention and Services Program Performance Progress Report Form | Grantees submit this data in an aggregated format (non-client level data). When the grantees submit their reports in the Online Data Collection System, there are automatic data validation and error checks that run before the grantees are able to submit their reports. The Family Violence Prevention and Services Act (FVPSA) Office provides a check of each grantee’s data by comparing the current year’s data to prior years and checking for inconsistencies or typos. The grantee is then given a short amount of time to confirm the submitted data or revise the report. In addition, performance report data are used to inform grant monitoring by state administrators and federal staff. |
| 16C (ACF) | Matching Grant Progress Report forms | Data are validated with methods similar to those used with Performance Reports. Data are validated by periodic desk and on-site monitoring, in which refugee cases are randomly selected and reviewed. During on-site monitoring, outcomes reported by service providers are verified with both employers and refugees to ensure accurate reporting of job placements, wages, and retentions. All of the grantees use database systems (online or manual) for data collection and monitoring of their program service locations. Beginning with FY 2016, grantee data points for each enrolled individual is loaded into the Office of Refugee Resettlement’s (ORR’s) iRADS database system biannually. The data presented here is generated by iRADS. |
| 22B (ACF) |
National Directory of New Hires (NDNH) | Beginning with performance in FY 2001, the above employment measures – job entry, job retention, and earnings gain – are based solely on performance data obtained from the NDNH. Data are updated by states, and data validity is ensured with normal auditing functions for submitted data. Prior to use of the NDNH, states had flexibility in the data source(s) they used to obtain wage information on current and former Temporary Assistance for Needy Families (TANF) recipients under high performance bonus (HPB) specifications for performance years FY 1998 through FY 2000. ACF moved to this single source national database (NDNH) to ensure equal access to wage data and uniform application of the performance specifications. |
Administration for Community Living (ACL)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| ALZ.3 (ACL) |
ACL’s Dementia Capability System Quality Assurance tool | Each fall grantees complete the tool to assess improvements in the dementia capability of their long-term services system. Technical assistance liaisons review grantee data for completeness and accuracy. The new on-line system will facilitate grantee completion of the tool, review and analysis. |
| 8F (ACL) |
Protection and Advocacy for Individuals with Developmental Disabilities (PADD) Annual Program Performance Report (PPR) | Outcome data for each fiscal year are reported in PPRs submitted in January of the following fiscal year. Verification and validation of data occur through ongoing review and analysis of annual reports. Data collected in the PADD PPR is validated and verified by comparing the data against parameters of that field and also compared with previous year’s data. In case of any outlier data, grantees are asked to verify and/or validate and provide ACL with an explanation and/or supporting documents. |
Agency for Healthcare Research and Quality (AHRQ)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.3.8 (AHRQ) |
Internal AHRQ performance management systems | Tools included in this measure will be made publicly available. |
Assistant Secretary for Administration (ASA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.6 (ASA) |
Office of Personnel Management Employee Viewpoint Survey. https://www.fedview.opm.gov/ | This federal survey is a self-administered web survey that is offered to all full-time and part-time employees by the Office of Personnel Management. Data collected from the survey respondents were weighted to produce survey estimates that accurately represent the agency population. |
| 2.8 (ASA) |
Human Resources Employment Processing System and Business Intelligence Information System | The HHS Office of Human Resources Director of Analytics reviews and validates the data. |
| 3.3 (ASA) |
Risk Management Framework Portal | The HHS Office of Chief Information Director of Information Security validates the data. |
| 3.4 (ASA) |
OGR Biannual FITARA Scorecard | The Subcommittee on Government Operations validates the data. |
| 3.5 (ASA) |
PhishMe Solution and PhishMe Report | The HHS Office of Chief Information Director of Information Security validates the data. |
| 3.6 (ASA) |
RiskVision: Ad Hoc Reports | The HHS Office of Chief Information Director of Privacy Office validates the data. |
Assistant Secretary for Preparedness and Response (ASPR)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.4.13a (ASPR) |
For all performance measures related to licensure, emergency use authorization, and/or commercialization of medical countermeasures are captured either through approval from appropriate regulatory agencies such as the United States Food and Drug Administration (FDA) and/or associated host country regulatory licensing board. This information is publically available and has gone through rigorous review approval for the safety, efficacy, tolerability and immunogenicity of such medical countermeasure for the advancement of pandemic preparedness and critical lifesaving interventions. During emergency times, Emergency Use Authorization’s (EUA) are assigned by the FDA to move forward certain lifesaving technologies in order to meet pandemic preparedness and response timelines. All EUAs are made public on the FDA website: https://www.fda.gov/EmergencyPrepar edness/Counterterrorism/MedicalCoun termeasures/MCMLegalRegulatoryandPo licyFramework/ucm182568.htm#current |
All data are checked against multiple databases to ensure accuracy and validation of the numbers reported. Contracts awarded and draft requests for proposal for industry comment are negotiated and issued, respectively, in accordance with Federal Acquisition Regulations (FAR) and the HHS Acquisition Regulations (HHSAR). Interagency Agreements are developed with federal laboratories to address specific advanced research questions. Contractors and awardees are required by contract terms and conditions to report on inventions, discovery, and other advancements in the advanced development of medical countermeasures. This information is used for quality assurance and control purposes to ensure data reported is accurate. |
| 2.4.15b (ASPR) | Data sources for performance measure 2.4.15b will be collected and reported from the number of executed awards made during the fiscal year as it relates to the advanced research and development of influenza vaccines and broad-spectrum therapeutics. Data sources will include www.USASpending.gov, www.fbo.gov, UFMS, and other government systems. | All data are checked against multiple databases to ensure accuracy and validation of the numbers reported. Contracts awarded and draft requests for proposal for industry comment are negotiated and issued, respectively, in accordance with Federal Acquisition Regulations (FAR) and the HHS Acquisition Regulations (HHSAR). |
Centers for Disease Control and Prevention (CDC)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 1.3.3a (CDC) |
Behavioral Risk Factor Surveillance System (BRFSS) | BRFSS is an ongoing state-based monthly telephone survey which collects information on health conditions and risk behaviors from randomly selected people ≥ 18 years among the U.S. population. BRFSS respondents were asked if they had received a flu vaccine in the past 12 months, and if so, in which month and year; this information was self-reported and not verified by medical records. Starting in 2011, BRFSS methods changed by adding persons in households with only cellular telephone service and improving weighting procedures; these changes were reflected in the 2011-12 and subsequent flu vaccination coverage estimates. |
| 3.2.4b (CDC) |
CDC’s National Healthcare Safety Network (NHSN) and CDC’s Emerging Infections Program (EIP) ’s Healthcare-Associated Infections Community Interface (HAIC) activity surveillance for community-onset Clostridium difficile infections (CDI) reduction | NHSN data are validated by the Centers for Medicare & Medicaid Services and state/local health departments. EIP data undergoes annual audits to ensure accuracy |
| 3.2.5 (CDC) |
National Healthcare Safety Network (NHSN) facility survey | Extensive cross-field edit checks are used for validation and incomplete records cannot be reported. Detailed instructions for completion of report forms ensure consistency across sites. Process and quality improvements occur through email updates and annual meetings. |
| 3.3.3 (CDC) |
National Healthcare Safety Network (NHSN) | Extensive cross-field edit checks are used for validation and incomplete records cannot be reported. Detailed instructions for completion of report forms ensure consistency across sites. Process and quality improvements occur through email updates and annual meetings. |
| 3.5.2 (CDC) |
Electronic Laboratory Reporting Repository – automated | The ELR Implementation Support and Monitoring team (collaboration between (NCEZID and OSELS) will analyze data for anomalies. |
| 4.6.2a (CDC) |
US Census and Treasury; Alcohol Tobacco Tax and Trade Bureau (TTB), Monthly Statistical Reports, and the Census Bureau Annual Census Estimates | Data are pulled from public reports from US Census and Treasury, and validated through HHS and CDC calculations. |
| 4.11.10a (CDC) |
National Health and Nutrition Examination Survey (NHANES), CDC, NCHS | Data are validated by NCHS |
| 4.11.10b (CDC) |
National Health and Nutrition Examination Survey (NHANES). | NHANES data are validated by quality control standards. |
| 7.2.6 (CDC) |
CDC/NCHS, National Vital Statistics System, Mortality | See http://www.cdc.gov/nchs/nvss/aboutnvss.htm. NVSS data are provided through contracts between NCHS and vital registration systems operated in the various jurisdictions legally responsible for the registration of vital events including deaths. |
| 8.B.1.4 (CDC) |
National Notifiable Disease Surveillance System (NNDSS) | Data are validated by calculations at CDC based on the format of data transmissions received by CDC. The frequency of calculation and monitoring is at least yearly. |
| 13.5.3 (CDC) |
Self-reported data from 62 PHEP grantees. | Quality assurance reviews with follow-up with grantees |
Centers for Medicare & Medicaid Services (CMS)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| MCR23 (CMS) |
The Prescription Drug Event (PDE) data | CMS has a rigorous data quality program for ensuring the accuracy and reliability of the PDE data. The first phase in this process is on-line PDE editing. The purpose of on-line editing is to apply format rules, check for legal values, compare data in individual fields to other known information (such as beneficiary, plan, or drug characteristics) and evaluate logical consistency between multiple fields reported on the same PDE. On-line editing also enforces business order logic which ensures only one PDE is active for each prescription drug event. The second phase of our data quality program occurs after PDE data has passed all initial on-line edits and is saved in our data repository. We conduct a variety of routine and ad hoc data analysis of saved PDEs to ensure data quality and payment accuracy. |
| MCR30.1 (CMS) |
Medicare Shared Savings Program Financial Reconciliation Reports; Master Data Management (MDM) System; Integrated Data Repository (IDR); TAP files; CCW claims data; CMS Office of the Actuary (OACT) annual Part A and B expenditure data Numerator: Model payment actuals based on model specific data, such as the number of aligned beneficiaries and annual per beneficiary spending. Denominator: The CMS Office of the Actuary (OACT) actual or estimated annual Part A and B expenditure. To monitor the movement of payments to more advanced payment models, HHS developed the following payment taxonomy to describe health care payment through the stages of transition from pure fee-for-service to alternative payment models and, ultimately, population based payments. CMS is using this framework to measure Medicare payments tied to alternative payment models. This framework classifies payment models into four categories according to how providers are paid: category 1—fee-for-service with no link of payment to quality; category 2—fee-for-service with a link of payment to quality; category 3—alternative payment models built on fee-for-service architecture; category 4—population-based payment. CMMI models in categories 3 and 4 that are directly testing how providers are paid, and for which we can measure these payments, are considered for this measure. For example, ACOs, including Pioneer ACO and the Medicare Shared Savings Program, are considered for this measure because the model allows us to understand how providers are receiving FFS payments. Advanced primary care medical homes, including CPC and Multi-Payer Advanced Primary Care Practice, are also considered in this total for this same reasoning. In contrast, models such as the Health Care Innovation Awards that provide funding to organizations to support providers in moving toward this goal are not included in this total, because the model is not necessarily explicitly testing how providers are paid. |
Numerator: Model payment actuals based on model specific data, such as the number of aligned beneficiaries and annual per beneficiary spending. Denominator: The CMS Office of the Actuary (OACT) actual or estimated annual Part A and B expenditure. CMS staff and contractors provide beneficiary alignment and expenditure data to CMMI. Model teams and contractors use quality assurance measures and data cleaning, including an audit and validation process of the programs that calculate the results to ensure the reliability of the results. |
| MCR31 (CMS) |
Medicare Shared Savings Program. Accountable Care Organizations (ACOs) Consumer Assessment of Healthcare Providers and Systems (CAHPS) | CMS-approved survey vendors are required to follow data collection protocol. CMS approved vendors must meet minimum business requirements, attend training, complete a post-training assessment, provide a quality assurance plan, submit test data, participate in site visits and quality monitoring activities. The Survey Administration Contractor then validates CAHPS scores through a multi-step process. The first step ensures that vendor submitted data are correct and complete survey items are programmatically checked to verify the correct response scale is used, the correct disposition status is assigned to respondents, and that all beneficiaries in the ACO’s original sample are accounted for. If errors are detected during this step, the survey vendor is required to re-submit a corrected data file. The second step applies “forward cleaning logic” to the respondent-level data. In certain cases, a beneficiary may answer “No” to a screener question, but subsequently answers the dependent questions (this is against protocol). In these cases, we programmatically set the responses for the dependent questions as missing. After forward cleaning, the respondent-level dataset is run through the CAHPS scoring macro. The CAHPS scoring macro is developed and maintained by Harvard, it has been used for over 10 years to correctly calculate case-mix adjusted (CMA) CAHPS scores. The macro outputs 0-100 CMA scores and reliabilities for each CAHPS item and Summary Survey Measures (SSM). Scores are produced at both the ACO and national level. These scores then undergo a final programmatic check to ensure 1) CMA item and SSM scores are produced for every ACO, 2) the scores and reliabilities are within scale and, 3) reliability flags are correctly assigned to items and SSMs.” |
| MIP1 (CMS) |
The Comprehensive Error Rate Testing (CERT) Program selects a random sample of Medicare Fee-for Service (FFS) claims from a population of claims submitted for Medicare FFS payment. Complex medical review is performed on the sample of Medicare FFS claims to determine if the claims were properly paid under Medicare coverage, coding, and billing rules. | The CERT program is monitored for compliance by CMS through monthly reports from the contractors. In addition, the HHS Office of the Inspector General (OIG) conducts annual reviews of the CERT program and its contractors. |
| MIP5 (CMS) |
The Part C Error Rate estimate measures errors in clinical diagnostic data submitted to CMS by plans. The diagnostic data are used to determine risk adjusted payments made to plans. | Data used to determine the Part C program payment error rate is validated by several contractors. The Part C program payment error estimate is based on data obtained from a rigorous Risk Adjustment Data Validation (RADV) process in which medical records are reviewed by independent coding entities in the process of confirming that medical record documentation supports risk adjustment diagnosis data submitted by Medicare Advantage Organizations for payment. |
| MIP6 (CMS) |
The payment error measurement in the Part D program is a rate that measures payment errors from errors in Prescription Drug Event (PDE) records. A PDE record represents a prescription filled by a beneficiary that was covered by the plan. | For the Part D payment error rate, the data to validate payments comes from multiple internal and external sources, including CMS’ enrollment and payment files. Data are validated by several contractors. A key data source is CMS’ PDE Data Validation process, which validates PDE data through contractor review of supporting documentation submitted to CMS by a national sample of Part D plans. |
| MIP9.1 (CMS) |
As part of a national contracting strategy, adjudicated claims data and medical policies are gathered from the States for purposes of conducting medical and data processing reviews on a sample of the claims paid in each State. | CMS and CMS contractors are working with the 17 States to ensure that the Medicaid and CHIP universe data and sampled claims are complete and accurate and contain the data needed to conduct the reviews. In addition, the OIG conducts annual reviews of the PERM program and its contractors. |
| MIP9.2 (CMS) |
As part of a national contracting strategy, adjudicated claims data and medical policies are gathered from the states for purposes of conducting medical and data processing reviews on a sample of the claims paid in each state. | CMS and CMS contractors are working with the 17 states to ensure that the Medicaid and CHIP universe data and sampled claims are complete and accurate and contain the data needed to conduct the reviews. In addition, the OIG conducts annual reviews of the PERM program and its contractors. |
| MMB2 (CMS) |
CMS Geographic Variation Database (Foundation of the Chronic Conditions Warehouse). This performance measure defines a readmission as a case of a full-benefit Medicare-Medicaid enrollee in fee-for-service who is discharged from an acute care hospital and admitted to the same or another acute care hospital within thirty days from the date of the index admission discharge. The formula is the number of readmissions per 1000 eligible beneficiaries. CMS uses a hybrid method of extracting readmissions data on Medicare-Medicaid enrollees, which incorporates elements of the Partnership for Patients readmission measure and the Medicare Hospital Readmissions Reduction Program (HRRP) measure methodologies (see MCR26 for more information). The methodology differs from MCR26 in that readmission data on all full-benefit Medicare-Medicaid enrollees in FFS is analyzed, as opposed to only those 65 years old and older, in order to capture the experience of those with disabilities under age 65 years. |
Data are validated using parallel coding, reasonableness checks on each file, version-to-version changes by variable and service types, and year-over-year comparisons. |
| MSC5 (CMS) |
CMS reports the percentage of long-stay nursing home residents that received an antipsychotic medication with a quality measure (QM) derived from the Minimum Data Set (MDS). | The MDS is the source of the data used to calculate this measure. The MDS is considered part of the medical record. The nursing home must maintain the MDS and submit it electronically to CMS for every resident of the certified part of the nursing home. |
| QIO7.2 (CMS) |
Nursing Home Compare Data | Data for nursing home compare are validated as part of the process to display nursing home compare 5 star rating scores, and are comprised of Medicare claims data and MDS data. For this measure, underlying data for the 5 star rating were analyzed, and baseline and targets were set to focus improvements on current one star value nursing homes to raise the overall quality of care for nursing homes assessed, specifically one star homes. |
| QIO11 (CMS) |
Medicare Patient Safety Monitoring System (MPSMS) | The Agency for Healthcare Research and Quality (AHRQ) National Scorecard data comes mostly from independent clinical chart abstractions of a statistically representative sample of United States Prospective Payment System hospitals. These charts are collected by the CMS Clinical Data Abstraction Center (CDAC) through the Medicare Patient Safety Monitoring System (MPSMS), which uses software-guided chart review of inpatient records performed by nonclinical analysts to identify 21 types of adverse events. We apply the MPSMS methodology to a multi-stage stratified random sample each year of approximately 30,000 inpatient charts from about 400 acute care hospitals eligible for Medicare’s inpatient prospective payment system (IPPS). In addition, the AHRQ National Scorecard draws on two other measurement systems to capture additional adverse event types not captured by the MPSMS: select Centers for Disease Control (CDC) National Healthcare Safety Network (NHSN) and AHRQ Patient Safety Indicator (PSI) measures, which are included in order to generate a more comprehensive set of preventable patient harms. Over 90% of this dataset is not dependent on coding or coding practices, making it a highly reliable account of patient safety harms occurring on a national scale. Preliminary data are collected and provided to AHRQ for validation and analysis, followed by their delivery of finalized compiled results. Earlier data are used to fill-in gaps for preliminary estimates. As all the data for a given year become available, a final number is produced. For example, preliminary results for 2015 are based on quarters 1-3 of 2015 from the MPSMS. The rate for the 2015 Q4 MPSMS data has been estimated as 93 percent of the 2015 Q1–Q3 rate, based on the mean rate for Q1–Q3 compared with Q4 rates for the MPSMS data from 2010 to 2014. Data from HCUP and NHSN included in the preliminary 2015 number are actually from 2014. More details are available at: https://www.ahrq.gov/professionals/quality-patient-safety/pfp/index.html AHRQ and CMS are in the process of updating the MPSMS data source to which new clinical measures were last added in 2005. For example, opioid-related adverse drug events are not currently monitored by MPSMS. An updated baseline is anticipated to accommodate major updates to the measurement system during the performance period. |
Food and Drug Administration (FDA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 292202 (FDA) |
Sentinel uses a distributed data approach in which Data Partners maintain physical and operational control over electronic data in their existing environments. The distributed approach is achieved by using a standardized data structure referred to as the Sentinel Common Data Model. The combined collection of datasets across all Data Partners is known as the Sentinel Distributed Database (SDD). | The Sentinel Data Quality Review and Characterization Programs are used by the Sentinel Operations Center (SOC) for data quality review and characterization of the Sentinel Distributed Database (SDD). To create the SDD, each Data Partner transformed local source data into the Sentinel Common Data Model (SCDM) format. The SOC created a set of data quality review and characterization programs to ensure that the SDD meets reasonable standards for data transformation consistency and quality and that the SDD data meets expectations needed for a distributed health data network. |
| 291101 (FDA) |
The Office of Scientific Program Development (OSPD) produces annual evaluation reports which offer a detailed summary of the outcomes, including the number of applications and selections, demographics, research contributions to FDA product centers, and yearly percentage of FDA hires. | Recruitment and graduation records are created, maintained and verified by FDA’s Office of Scientific Program Development (OSPD). |
Health Resources and Services Administration (HRSA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 4.I.C.2 (HRSA) |
HRSA Bureau of Clinician Recruitment Service's Management Information Support System (BMISS) | BMISS is internally managed with support from the NIH which provides: Data Management Services, Data Requests and Dissemination, Analytics, Data Governance and Quality, Project Planning and Requirements Development, Training, and Process Improvement. |
| 16.III.A.4 (HRSA) |
The RWHAP Services Report (RSR). The RSR contains client-level data and enables the Program to un-duplicate the estimated number of people who received at least one RWHAP-funded service within the reporting period. | This web-based data collection method communicates errors and warnings in the built-in validation process. To ensure data quality the Program conducts data verification for all RSR submissions. Recipients receive reports detailing items in need of correction and instructions for submitting revised data. The web system has an array of reports available through which the grantees and their funded providers can identify data issues that need to be resolved. In addition, the Program provides technical assistance and training during and after the submission period to address quality issues. |
| 29.IV.A.3 (HRSA) |
Reported by grantees through the Program's Performance Improvement Measurement System. | The data are reviewed and validated by HRSA project officers. |
Indian Health Service (IHS)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 81 (IHS) |
IHS Integrated Data Collection System Data Mart | IHS conducts a monthly review of reports for completeness regarding full participation and monitoring of outliers. |
| MH-1 (IHS) |
Indian Health Service Performance and Evaluation System (IHPES). | Reports generated from the IHS Performance and Evaluation System (IHPES) are reviewed and verified periodically to assure data quality control and monitor percent change outliers which may indicate error. |
National Institutes of Health (NIH)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| SRO-2.1 (NIH) |
Publications, databases, administrative records and/or public documents | NIH staff review relevant publications, databases, administrative records, and public documents to confirm whether the data sources support the scope of funded research activities. The most common data sources are articles in peer-reviewed journals, as well as presentations and progress reports. Scientific journals use a process of peer review prior to publishing an article. Through this rigorous process, other experts in the author’s field or specialty critically assess a draft of the article, and the paper may be accepted, accepted with revisions, or rejected. |
| SRO-2.9 (NIH) |
Publications, databases, administrative records and/or public documents | NIH staff review relevant publications, databases, administrative records, and public documents to confirm whether the data sources support the scope of funded research activities. https://clinicaltrials.gov/ct2/show/NCT02568215?term=HPTN+081&cond=HIV&rank=1 https://clinicaltrials.gov/ct2/show/NCT02716675?term=HPTN+085&cond=HIV&rank=1 |
| SRO-2.12 (NIH) |
Publications, databases, administrative records and/or public documents | NIH staff review relevant publications, databases, administrative records, and public documents to confirm whether the data sources support the scope of funded research activities. The most common data sources are articles in peer-reviewed journals, as well as presentations and progress reports. Scientific journals use a process of peer review prior to publishing an article. Through this rigorous process, other experts in the author’s field or specialty critically assess a draft of the article, and the paper may be accepted, accepted with revisions, or rejected. |
| SRO-4.9 (NIH) |
Publications, databases, administrative records and/or public documents | NIH staff review relevant publications, databases, administrative records, and public documents to confirm whether the data sources support the scope of funded research activities. The most common data sources are articles in peer-reviewed journals, as well as presentations and progress reports. Scientific journals use a process of peer review prior to publishing an article. Through this rigorous process, other experts in the author’s field or specialty critically assess a draft of the article, and the paper may be accepted, accepted with revisions, or rejected. |
| SRO-5.1 (NIH) |
Publications, databases, administrative records and/or public documents | NIH staff review relevant publications, databases, administrative records, and public documents to confirm whether the data sources support the scope of funded research activities. Strategy 1: http://cdrewu-dcrt.org/research.aspx?page=partnerships; http://medschool.ucla.edu/body.cfm?id=1158&action=detail&ref=860; http://grantome.com/grant/NIH/U54-CA143931-04S1 Strategy 2: Hawai‘i Cancer at a Glance, 2009-2013; http://www.uhcancercenter.org/images/pdf/HTR_cancer_booklet.pdf; http://tritonscall.com/uog-cancer-research-grant/; Guam Cancer Facts & Figures 2008-2012; http://www.guamcrc.org/wp-content/uploads/2016/10/GuamCancerFactsFigure2008%E2%80%942012.pdf |
| SRO-5.3 (NIH) |
Manuscripts: Analysis of the ADSP Discovery Phase data has yielded a publication which describes sample selection, project design, and analysis plans for ADSP data. The manuscript is a bench mark paper to support ADSP data analysis by the worldwide research community. Seven additional manuscripts are under peer review or are being readied for submission to journals. Analysis of WGS data from two subsequent ADSP study components is underway: 1]. The ADSP Discovery Family-Based Extension Study: To further assess the genomes in multiply affected families, an additional 428 samples from ethnically diverse cohorts were whole genome sequenced. 2]. The ADSP Discovery Case-Control Based Extension Study: An additional 3,000 subjects were whole genome sequenced. This included 1,466 cases and 1,534 controls. Of these 1,000 each of Non-Hispanic White (NHW), Caribbean Hispanic (CH), and African American (AA) descent were sequenced. Of these a total of 739 autopsy samples were sequenced [568 cases (500 NHW cases and 68 AA cases) and 171 controls (164 NHW and 7 AA)]. Analysis of genomic regions of interest for the ADSP Discovery Extension Phase was begun in 2017. The majority of variants identified to date are rare. Genetic factors identified for late onset AD likely modulate risk and age at onset; identification of a number of genetic risk factor genes in an individual may help clinicians in disease prediction. The next phase of the study, the ADSP Follow-Up Study (PAR-16-406 and PAR-17-214) was launched in FY 17, and emphasizes the study of ethnic minorities. Whole genome sequencing of up to 10,000 subjects is in progress. Pathway Analysis: Over the past year, several cellular pathways emerged as being implicated in the etiology of AD. The pathways include genes related to amyloid beta metabolism, cholesterol metabolism, the innate immune system, and glial function. These pathways may serve as guides in the search for therapeutic targets for AD. See related studies published by ADSP members: http://onlinelibrary.wiley.com/doi/ 10.1002/ajmg.b.32499/full; https://www.nature.com/ng/journal/v 49/n9/pdf/ng.3916.pdf; https://jamanetwork.com/journals/ja maneurology/fullarticle/2645746. |
NIH staff review relevant publications, databases, administrative records, and public documents to confirm whether the data sources support the scope of funded research activities. The most common data sources are articles in peer-reviewed journals, as well as presentations and progress reports. Scientific journals use a process of peer review prior to publishing an article. Through this rigorous process, other experts in the author’s field or specialty critically assess a draft of the article, and the paper may be accepted, accepted with revisions, or rejected. Manuscript publication: “The Alzheimer's Disease Sequencing Project: Study design and sample selection”. Beecham et al. Published online October 13, 2017doi: http:/?/?dx.?doi.?org/?10.?1212/?NXG.?0000000000000194 Neurol Genet October 2017 vol. 3 no. 5 e194 http://ng.neurology.org/content/3/5/e194.short?rss=1; MS ID# NG/2017/005587. Additional key publications by AD geneticists who are members of the ADSP are: Rare coding variants in PLCG2, ABI3, and TREM2 implicate microglial-mediated innate immunity in Alzheimer's disease. Sims R, et al. Nat Genet. 2017 Sep;49(9):1373-1384. https://www.nature.com/ng/journal/v49/n9/pdf/ng.3916.pdf Early-Onset Alzheimer Disease and Candidate Risk Genes Involved in Endolysosomal Transport. Kunkle et al. JAMA Neurol. 2017 Sep 1;74(9) https://jamanetwork.com/journals/jamaneurology/fullarticle/2645746 Genomic variants, genes, and pathways of Alzheimer’s disease: An overview. Naj et al, Am J Med Genet B Neuropsychiatr Genet 2017 Jan;174(1):5-26. http://onlinelibrary.wiley.com/doi/10.1002/ajmg.b.32499/full An NIH policy statement was issued to broaden the definition of Alzheimer’s disease consistent with the National Alzheimer’s Project Act (NAPA) PLAW-111publ375.pdf. to include Alzheimer’s disease and Alzheimer’s disease related dementias in AD genetic studies: “Notice of Information: The Alzheimer's Disease Sequencing Project Policy (ADSP) on the Publication of Study-Related Data” NOT-AG-16-033 GCAD - http://grants.nih.gov/grants/guide/ rfa-files/RFA-AG-16-001.html: the Genome Center for Alzheimer’s Disease [U54 AG05242, www.adgenomics.org; Schellenberg and Wang-University of Pennsylvania (RFA-AG-16-001)] assembled ADSP data quality control measures and completed data harmonization on all available ADSP data; and will perform meta-analysis of these data. NIAGADS - https://www.niagads.org/: Rapid data sharing is enhanced by NIA Genetics of Alzheimer’s Disease Data Storage Site. The ADSP data release for the Discovery and Discovery Extension phase is on track for an early 2018 to share with the worldwide research community: https://www.niagads.org/adsp/conten t/home. NIAGADS currently stores nearly one petabyte of data (a quadrillion bytes), up from 64 Tb in 2016. Reports and updated milestones have been received and accepted for the following awards: Funded under PAR-12-183 (ADSP Cooperative Agreements): UF1 AG047133-01; U01 AG049506; U01 AG052409; U01 AG052411; and U01 AG049507. Funded under PAR-15-356 (ADSP-related): R01 AG054002; R01 AG054076; RF1 AG054074; R01 AG054060; RF1 AG053959; RF1 AG054023; RF1 AG054052; R01 AG054047; R01 AG054058; RF1 AG054080. Newly funded applications: Under PAR-16-205: “The NIA Late Onset of Alzheimer’s Disease Family Based Study” [U24] to collect large multiply affected AD families, with well phenotyped subjects. Data collection for up to an additional 5,000 families is in progress. Under PAR-16-406: “Limited Competition: Additional Sequencing for the Alzheimer's Disease Sequencing Project (U01)”. The study will perform WGS on up to 10,000 subjects with emphasis on ethnically diverse populations. Sequencing was started in October 2017 by the Department of Defense (DoD) funded Uniformed Services University of Health Sciences (USUHS) - “The American Genome Center (TAGC).” Under PAR-15-358: RF1 AG055477 aimed at the identification of AD-related cellular pathways. Under RFA-AG-17-010: R01 AG056279 and AG056476 aimed to identify functional targets in the Alzheimer’s genome. Under peer review: Applications under PAR-17-214: “Limited Competition: Analysis of Data from NIA's Alzheimer's Disease Sequencing Project Follow-Up Study (U01)” submitted under peer review for funding in FY 18/19. |
Substance Abuse and Mental Health Services Administration (SAMHSA)
| Measure ID | Data Source | Data Validation |
|---|---|---|
| 2.3.19K (SAMHSA) |
The National Directory of Drug and Alcohol Abuse Treatment Facilities is a listing of federal, state, and local government facilities and private facilities that provide substance abuse treatment services. It includes treatment facilities that (1) are licensed, certified, or otherwise approved for inclusion in the Directory by their State Substance Abuse Agencies, and (2) responded to the previous year’s N-SSATS. | The National Survey of Substance Abuse Treatment Services (N-SSATS) is an annual census designed to collect information from all facilities within the 50 States, the District of Columbia, and the U.S. territories, both public and private, that provide substance abuse treatment. N-SSATS provides the mechanism for quantifying the dynamic character and composition of the United States substance abuse treatment delivery system. |
| 2.3.19L (SAMHSA) |
National Survey on Drug Use and Health (NSDUH) | NSDUH uses audio computer-assisted self-interviewing to provide the respondent with a highly private and confidential mode for responding to questions in order to increase the level of honest reporting of illicit drug use and other sensitive behaviors. |
| 2.3.19O (SAMHSA) |
National Survey on Drug Use and Health (NSDUH) | NSDUH uses audio computer-assisted self-interviewing to provide the respondent with a highly private and confidential mode for responding to questions in order to increase the level of honest reporting of illicit drug use and other sensitive behaviors. |
