Data Science Research and Methods │ Data Science

DARPA’s World Modelers Program (2017–2022)—Global

National security threats such as conflict, displacement and migration, malnutrition, WASH (water, sanitation, and hygiene), access to local markets, food and fuel prices, food insecurity, and population growth, to name a few are myriad in the East African region. Several challenges impede policymakers’ ability to work against these development issues. Analysis of national security questions can take months or even years. Data is sparse, missing, disbursed, and unprocessed. There was no infrastructure to not only build models but to integrate them so that the prediction of one model could feed in as an explanatory variable for other development predictions. The Defense Advanced Research Projects Agency’s (DARPA) investment in the “World Modelers” project created the opportunity to provide sophisticated, near real time, integrated modeling infrastructure to help policy makers quickly respond to or prevent humanitarian disasters. As a result, AIR built 13 models across the East African region: processing every type of data from rainfall and soil viability to the price of potatoes and petrol to survey data around access to clean water and toilets. Several models are integrated, for example, crop yields predicted by the crop model are one of the variables predicting malnutrition. The infrastructure is completely open source and development will be continued by the Bill & Melinda Gates Foundation.

UNICEF Innovation Fund: Methods for Extremely Rapid Observation of Nutritional Status (MERON) (2017–2018)—Kenya

Over 3 million children under five die of malnutrition each year. Capturing childhood malnutrition is imperative for understanding the context in poor countries, tracking progress, and directing aid funding and nonprofit workers. Traditional conceptualizations of the metric include weight for height and mid upper-arm circumference. These methods involve the physical handling of children, bulky equipment, are susceptible to inaccuracies due to subjective interpretations and can be time consuming. AIR developed MERON, an application that allows for a noninvasive, time-efficient, and tamper-proof approach to assessing the malnutrition status of an individual by using a facial recognition and processing algorithm. Using 3,500 images of children under age 5 that were collected alongside UNICEF's Standardized Monitoring and Assessment of Relief and Transitions (SMART) survey in Kenya, this tool achieved a preliminary classification accuracy level of 60%.

Entity Resolution for USPTO Inventors, Applicants, and Organizations

The U.S. Patent and Trademark Office (USPTO) does not monitor inventor and assignee (organization) activities over time. Instead, when inventors and organizations apply for a patent, it is a singular event. Such inventors are not required to follow a consistent format for how they record their name, their co-inventors’ names, their organizations’ name, or even the locations with which they or their organizations are associated. AIR’s PatentsView platform uses complex entity resolution and disambiguation techniques to allow patent data users to longitudinally track the patenting activity of inventors, their organization, and their locations. This enables researchers, policymakers, and other users to understand trends in the patenting landscape over time.

AIR is constantly evaluating and updating its disambiguation process to incorporate recent developments emerging from computer and information sciences, as well as reported issues from patent users. See patentsview.org for more information.

Text Analysis and Clustering for CareerOneStop Certification Database

Sub-baccalaureate certificates are a growing postsecondary award category in the United States. However, the titles given to certification programs are not currently standardized. Equivalent certificates are often named differently across institutions. This lack of consistency can make it difficult for students to compare offerings across institutions. It can also make it challenging for employers to evaluate applicants’ credentials. Our data scientists worked with researchers to analyze certification titles and descriptions in the CareerOneStop database. Our text analysis identified prevalent terminology while our clustering analysis identified certifications sharing this terminology. These results enabled the team to create a taxonomy for classifying these certificates.

Applying Machine Learning Models to Predict MIPS Participation

In January 2017, the Centers for Medicare & Medicaid Services (CMS) implemented the Merit-based Incentive Payment System (MIPS). MIPS allows physicians to participate in the CMS Quality Payment Program (QPP), which aims to improve the quality of medical care by offering quality-based payments to physicians. Only 5–15% of eligible physicians participated in the program in 2017. This resulted in a class imbalance problem, where the target class (participating physicians) was severely under-represented in the data, and thus impacted the model’s prediction capability. To overcome this challenge, the team used the latest developments in both data-level and algorithmic-level methods to correct and improve the predictive performance. With this work, the team identified physician characteristics that predicted physician participation in MIPS with the highest accuracy, and offered insights, such as optimizing the eligibility criteria and the rate of downward adjustment for nonparticipants, to improve the MIPS program design.