Data Collection and Acquisition │ Data Science

Leveraging conventional and non-traditional data sources

AIR is proficient in implementing all forms of data collection using both novel and well-established methods. In addition to using our access to highly reputable data, we leverage emerging technologies and software to extract data from less structured websites and sources that would otherwise be unavailable to traditional researchers.

We leverage these conventional and unconventional data sources to gain more nuanced insights. We also identify, obtain, and transform data from surveys, administrative sources, websites, social media platforms, sensors, and other sources. To maximize the value of open data initiatives, we integrate different interfaces to automate the collection and sharing of information within and across organizational lines. At the same time, our work is informed by our keen understanding of the limitations of emerging data sources for research and evaluation.

Contact
Christina Jones

Christina Jones

Principal Data Scientist

Featured AIR Projects

Coronavirus Aid, Relief, and Economic Security (CARES) Act

On March 28,2020, after the onset of the COVID-19 pandemic, the federal government signed the CARES Act into law. The purpose of this law was to provide economic support to thousands of colleges and universities across the United States to assist in overcoming the economic challenges imposed by the pandemic. In accordance with this law, these institutions were required to report on their websites seven types of information, including the total amount of funds they received, the number of students these funds would go to, and their plans for distributing funds to all eligible students.

Since there was no standard or prescribed format for posting this information on websites, institutions used a variety of them to present this information, such as PDFs, Word documents, and raw HTML code. This made the process of collecting and parsing the information highly challenging. Our team developed methods that were able to extract these data in the various formats they were presented. The team implemented the algorithm in Python and used text analysis techniques to extract the responses to all seven questions. These techniques included searching for key features in the text such as numbered lists, bullet points, keywords, and phrases, and parts-of-speech tagging to find the text that fit with the relevant section. The team then used similarity scores and other validation methods to assess the likelihood that the information extracted were correctly filed under the seven sections prescribed under CARES Act.

Early-Concept Grants for Exploratory Research (EAGER)

Small high-tech companies are often considered the engine of economic growth. Under a grant from the National Science Foundation, AIR evaluated the utility of information available on entrepreneurial firm websites in industries with the potential to transform our economy and way of living. Following the extraction of data from these websites, AIR investigators applied text analytics, clustering, and classification algorithms to find differences in firm capabilities and online customer-facing communication strategies.

Processing Comments Received on DOL Regulations

To support the U.S. Department of Labor’s Wage and Hour Division (WHD) in its regulatory reform efforts, AIR provided a variety of services, including conducting economic impact analyses, producing written reports, effectively managing the collection and analysis of public comments, and preparing other pertinent written materials. For this contract, in addition to providing applied economic analysis support, we used text analytics based on natural language processing (NLP) to analyze public comments received on the regulation under study. We used NLP and text analytics tools to summarize comments, extract themes and understand sentiments. We used a variety of text-based metrics to combine comments, detect substantial comments, and identify and extract organizations and institutions who submitted their opinions on the regulation.

Related Work
Teacher checking in on students writing

Improving Pandemic Recovery Efforts in Education Agencies

Researchers from AIR's CALDER, Harvard's Center for Education Policy and Research, and NWEA are partnering with a coalition of districts across the country to help determine which COVID recovery interventions are working (or not working), which students they are helping, and why.
Doctors looking at graphs on tablet

Supplemental Payment Reporting Requirements and Guidance Development

States have the flexibility to set Medicaid payment rates for providers consistent with federal regulations that require payments promoting the Medicaid program’s economy and efficiency. As supplemental payments are now a major source of Medicaid reimbursement for providers in many states, both the U.S. Government Accountability Office and the Medicaid and ...

Healthcare Data Analytics and Statistical Products (HDASP)

AIR supports AHRQ in increasing the utility of the Medical Expenditure Panel Survey, Healthcare Cost and Utilization Project, and National Healthcare Quality and Disparities Report. The project’s goal is to develop data and statistical products, including public use files, to expand and diversify how data can be applied for research ...