Department of Health and Human Services
Statistical Software for Healthcare Oversight Challenge
"Each year HHS handles hundreds of millions of Medicare and Medicaid claims valued at over a trillion dollars. Due to the high volume of claims, statistical sampling provides a critical tool to ensure effective oversight of these expenditures. Sampling is used by the providers in their own efforts to monitor their performance and by the various organizations within HHS... The objective of the current challenge is to develop the foundation for an upgraded version of RAT-STATS. The current version of RAT-STATS is well validated; however, its user interface can be difficult to navigate and the underlying code makes the software costly to update. OIG needs a new, modern version of the software that is easy to use and can be extended in a cost effective manner. In addition, the new version of the software must be 508 compliant." - Contest description.
Outcome: SRLLC was a prize winning finalist.
Bureau of Reclamation
Water Supply Forecast Rodeo
"Accurate seasonal water supply forecasts are crucial for effective water resources management in the Western United States. This region faces dry conditions and high demand for water, and these forecasts are essential for making informed decisions. They guide everything from water supply management and flood control to hydropower generation and environmental objectives.
Yet, hydrological modeling is a complex task that depends on natural processes marked by inherent uncertainties, such as antecedent streamflow, snowpack accumulation, soil moisture dynamics, and rainfall patterns. To maximize the utility of these forecasts, it's essential to provide not just accurate predictions, but also comprehensive ranges of values that effectively convey the inherent uncertainties in the predictions...
... By improving the accuracy, explainability, and uncertainty characterization of seasonal streamflow forecasts, water resources managers will be better equipped to operate facilities for high flows, mitigate impacts of drought, improve hydropower generation, and meet environmental targets." - Contest description.
Outcome:
| Hindcast Development | Hindcast Evaluation | Forecast | Final |
|---|---|---|---|
| 3rd place | 12th place | 6th place | 1st place |
Streamflow Forecast Rodeo
"Streamflow forecasts are integral to water management. With higher skill forecasts, water managers can likely better operate facilities for high flows, to mitigate drought impacts, and to achieve other improved outcomes (e.g. hydropower generation).
This challenge seeks to improve the skill of short-term streamflow forecasts (10 days) via a year-long competition. Solvers will develop and implement their methods for locations across the western United States, attempting to outperform state of practice streamflow forecasts." - Contest description.
Outcome:
| Previous months | April | May | June | July | Aug |
|---|---|---|---|---|---|
| Not a participant | 22nd place | 4th place | 1st place | 3rd place | 2nd place |
Department of Homeland Security
DHS / TSA Passenger Screening Algorithm Challenge
"As part of their Apex Screening at Speed Program, DHS has identified high false alarm rates as creating significant bottlenecks at the airport checkpoints. Whenever TSA’s sensors and algorithms predict a potential threat, TSA staff needs to engage in a secondary, manual screening process that slows everything down... Currently, TSA purchases updated algorithms exclusively from the manufacturers of the scanning equipment used. These algorithms are proprietary, expensive, and often released in long cycles. In this competition, TSA is stepping outside their established procurement process and is challenging the broader data science community to help improve the accuracy of their threat prediction algorithms. Using a dataset of images collected on the latest generation of scanners, participants are challenged to identify the presence of simulated threats under a variety of object types, clothing types, and body types." - Contest description.
Outcome: SRLLC was in the top 9% of participants.
National Institute of Justice
Real Time Crime Forecasting Challenge
"The Real-Time Crime Forecasting Challenge sought to harness the advances in data science to address the challenges of crime and justice. It encouraged data scientists across all scientific disciplines to foster innovation in forecasting methods. The goal was to develop algorithms that advance place-based crime forecasting through the use of data from one police jurisdiction... Specifically, the Challenge tested how effectively and efficiently contestants’ crime forecasting algorithms could forecast police calls-for-service in four crime categories in Portland, Oregon, for five forecast periods." - Contest description.
Outcome: SRLLC won.
Recidivism Challenge
"The National Institute of Justice’s (NIJ) “Recidivism Forecasting Challenge” (the Challenge) aims to increase public safety and improve the fair administration of justice across the United States. In accordance with priorities set by the DOJ, NIJ supports the research, development, and evaluation of strategies to reduce violent crime, and to protect police and other public safety personnel by reducing recidivism. Results from the Challenge will provide critical information to community corrections departments that may help facilitate more successful reintegration into society for previously incarcerated persons and persons on parole." - Contest description.
Outcome: SRLLC placed 2nd for male parolees and 4th for fairness (accounting for racial bias, year 1.)
Office of the Under Secretary of Defense for Intelligence
ODNI-OUSD(I) Xamine Challenge
"Machine-based approaches to generating and evaluating analytic products from disparate structured and unstructured data types are emerging areas of research for the U.S. Intelligence Community (IC). As these approaches mature beyond demonstration systems with controlled data sources, such IC systems will require a means for inspecting and ensuring the integrity of the data that are ingested by these systems. These considerations will become particularly critical as the information available to the IC’s analytic community continues to exceed the ability for traditional, human vetting. Accordingly, the ODNI and OUSD(I) are seeking ideas and descriptions of a viable technical approach for enabling the automated validation of information prior to the dissemination of machine-generated intelligence products." - Contest description.
Outcome: SRLLC won.
Office of the Director of National Intelligence
ODNI-OUSD(I) Xtend Challenge
"The evaluation of analytic products is an area ripe for exploring new technological capabilities and approaches. Currently, intelligence products are reviewed—prior to publication—by numerous levels of management and edited against an Intelligence Community (IC) agency’s signature style using essentially the same methods as publishers have traditionally used. The ODNI and OUSD(I) are seeking ideas and descriptions of a viable technical approach for enabling the automated evaluation of finished intelligence products." - Contest description.
Outcome: SRLLC won.
CosmiQ Works, et al.
SpaceNet 5 Challenge
"Determining optimal routing paths in near real-time is at the heart of many humanitarian, civil, military, and commercial challenges... In a disaster response scenario, for example, pre-existing foundational maps are often rendered useless due to debris, flooding, or other obstructions. Satellite or aerial imagery often provides the first large-scale data in such scenarios, rendering such imagery attractive in the quest to aid disaster response. This public competition will challenge competitors to automatically extract road networks from satellite imagery, along with travel time estimates along all roadways, thereby permitting true optimal routing." - CosmiQ Works blog.
Outcome: SRLLC placed 18th.
Centers for Disease Control and Prevention
Text Classification Challenge
"Every day, work-related injury records are generated. In order to alleviate the human effort expended with coding such records, the Centers for Disease Control and Prevention (CDC) National Institute for Occupational Safety and Health (NIOSH), in close partnership with the Laboratory for Innovation Science at Harvard (LISH), is interested in improving their NLP/ML model to automatically read injury records and classify them according to the Occupational Injury and Illness Classification System (OIICS)." - Contest overview.
Outcome: SRLLC placed 8th (top 9% of participants.)
European Food Safety Authority
Natural Language Recognition Challenge
Implemented an end-to-end natural language processing package for the European Food Safety Authority in the domains of animal and plant research journals: machine learning models and software were developed to enhance the content information available during meta-analyses and systematic reviews.
Outcome: SRLLC was one of the top 5 participants, however, no detailed rankings were provided.
