RESEARCH

OVERVIEW

A call for project proposals in 2019 resulted in 10 projects that encompass a total of 15 research workstreams. The projects engage more than 140 faculty, researchers, and students, who are affiliated with more than 20 different organizational units across MIT Campus and MIT Lincoln Laboratory. All project teams involve Air Force personnel, who are embedded in the research teams and serve as liaisons between projects and Department of Defense stakeholders. The projects started in January 2020 and advance AI research in a broad range of areas, including weather modeling and visualization, optimization of training schedules, and enhancement of autonomy for augmenting and amplifying human decision-making. The research activities of the AI Accelerator have been successfully expanding, including seed research projects in collaboration with the Naval Postgraduate School (NPS) and an AI Education Research project that started in January 2021.

AI Accelerator Projects

Guardian Autonomy for Safe Decision Making

Air Guardian aims to advance AI and autonomy by developing algorithms and tools for augmenting and amplifying human decision making. The AI Guardian assists humans by suggesting actions using data from the past and fusing inputs from sensors and information sources. Support from an AI Guardian system is especially useful in the presence of surprises and complex situations. Guardian’s end-to-end machine learning algorithms learn from experts how to respond with common sense reasoning in highly dynamic and surprising situations. Our goal is to enable an agent to perceive its environment, identify short-term risks, reason about intentions and behaviors of its operator, and other cooperative and adversarial agents to determine the best course of action. This will lead to Guardian autonomy systems capable of anticipating potential hazardous situations in the future.

Team
– Daniela Rus (MIT PI)
– Ross Allen (MIT Lincoln Laboratory Lead)
– Ho Chit Siu (MIT Lincoln Lab Lead)
– Kyle Palko (Air Force Liaison)

Prof. Daniela Rus, PI
Ross Allen
Ho Chit Siu
Kyle Palko

Transferring Multi-Robot Learning through Virtual & Augmented Reality for Rapid Disaster Response

This project seeks to develop a new framework and class of algorithms that allow unmanned aerial systems to learn complex multi-agent behavior in simulator environments, then seamlessly transfer their knowledge from simulation to real-world field environments. The team envisions a first responder system where a swarm of autonomous aircraft are virtually trained on how to navigate and cooperate in a simulation of a novel disaster area. The system then transfers the learning gained in the simulation to the real autonomous aircraft swarm. An aircraft deploys a large “mothership” ground station which releases these trained autonomous aircraft to automatically perform time-critical, labor-intensive tasks like surveying disaster areas and locating and identifying survivors.

Team
– Sertac Karaman (MIT PI)
– Luca Carlone (MIT Co-PI)
– Daniel Griffith (MIT Lincoln Laboratory Lead)
– Victor (Air Force Liaison)
– Armando Cabrera (Air Force Liaison)
Sertac Karaman
Luca Carlone
Daniel Griffith
Victor
Armando Cabrera

Multimodal Vision for Synthetic Aperture Radar

Synthetic Aperture Radar (SAR) is a radar imaging technology capable of producing high-resolution images of landscapes. Due to its ability to produce images in all weather and lighting conditions, SAR imaging has advantages in Humanitarian Assistance and Disaster Relief (HADR) missions compared to optical systems. This project aims to improve human interpretability of SAR images, performance of SAR object detection and Automatic Target Recognition (ATR) by leveraging complementary information from related modalities (e.g., EO/IR, LiDAR, MODIS), simulated data, and physics-based models. Project findings and resulting technologies will be shared across the government enterprise to be beneficial in the HADR problem space where multiple partners across services may be able to exploit developed technology.

Team
– Phillip Isola (MIT PI)
– Taylor Perron (MIT Co-PI)
– Bill  Freeman (MIT Co-PI)
– Miriam Cha (MIT Lincon Laboratory Lead)
– Armando Cabrera (Air Force Liaison)
– Victor (Air Force Liaison)
Philiilp Isola
Taylor Perron
Bill Freeman
Miriam Cha
Armando Cabrera
Victor

AI-Assisted Optimization of Training Schedules

In order to improve the immensely complex and time-consuming process of manually scheduling aircraft flights, this project aims at automating aircraft flight scheduling to improve scheduling efficiency and robustness in the presence of uncertainty. This will optimize training flight schedules while providing explainability and removing silos in decision-making. This technology enables schedulers to quickly and effectively re-build schedules in the presence of rapidly changing circumstances, vastly accelerating planning and decision cycles. While initially focused on aircraft flight scheduling, this technology applies to all complex resource-allocation tasks across many sectors.

Team
 – Hamsa Balakrishnan (MIT PI)
– Mike Snyder (MIT Lincoln Laboratory Lead)
– Ronisha Carter (Air Force Liaison)
– Kyle McAlpin (Air Force Liaison)
Hamsa Balakrishnan, Associate Professor of Aeronautics and Astronautics at MIT in Cambridge, MA on 7/1/16. © Bryce Vickmark. All rights reserved. www.vickmark.com 617.448.6758
Mike Snyder
Ronisha Carter
Kyle McAlpin

Fast AI: Data Center & Edge Computing

Fast AI: Quick Development of Portable High-Performance AI Applications
The AI revolution has been enabled by the availability of vast amounts of labeled data, novel algorithms, and computer performance. But long computer-in-the-loop development cycles inhibit humans from inventing and deploying creative AI solutions. Moreover, the end of Moore’s has curtailed the historical ability of semiconductor technology to deliver performance. AI performance increasingly relies on hardware architecture, software, and algorithms. The Fast AI project focuses on developing a foundation for quickly building AI solutions, enabling performance and portability on both modern and legacy hardware platforms. We innovate in the areas of programming languages, compiler technologies, comprehensive instrumentation, analytical productivity tools, and parallel algorithms.
Team
– Charles E. Leiserson (MIT PI)
– Tao B. Schardl (MIT)
– Neil Thompson (MIT)
– Vijay Gadepally (MIT Lincoln Laboratory Lead)
– Allan Vanterpool (Air Force Liaison)
– Lindsey McEvoy (Air Force Liaison)
Charles Leiserson
Vijay Gadepally
Tao B. Schardl
Neil Thompson
Allan Vanterpool
Lindsey McEvoy
ML-Enhanced Data Collection, Integration & Outlier Detection
A core requirement for AI techniques to be successful is high quality data. Preparing systems to be “AI ready” involves collecting and parsing raw data for subsequent ingest, scan, query and analysis. This project will develop ML-enhance database technologies to reduce storing and processing costs while enabling data sharing amongst various database silos. Additionally, we will develop an outlier detection engine to identify temporal anomalies amongst complex event streams from multiple sources.
Team
– Tim Kraska (MIT PI)
– Manya Ghobadi (MIT Co-PI)
– Michael Stonebraker (MIT Co-PI)
– Samuel Madden (MIT Co-PI)
– Benjamin Price (MIT Lincoln Laboratory Lead)
– Allan Vanterpool (Air Force Liaison)
– Lindsey McEvoy (Air Force Liaison)
Tim Kraska
Manya Ghobadi
Michael Stonebraker
Samuel Madden
Benjamin Price
Allan Vanterpool
Lindsey McEvoy

Conversational Interaction for Unstructured Information Access and Language Learning ​

Conversations Interaction for Unstructured Information Access

The AI Accelerator Natural Language Processing project aims to advance conversational agents, knowledge representation, and prediction algorithms on flat/text image data and on Air Force missions. As the field of artificial intelligence advances, as we memorialize more of our work in data, and find more devices in our homes, it’s crucial that people are able to interact with the technology in meaningful ways – as humans, language matters – especially in discovering information on digital systems. The goal is to advance the AI community with conversational interaction and knowledge extraction for open-domain conversation and into unstructured information.

Team
– Jim Glass (MIT PI)
– Boris Katz (MIT Co-PI)
– Charles Dagli (MIT Lincoln Laboratory Lead)
– Michael Kanaan (Air Force Liaison)
– Ronisha Carter (Air Force Liaison)
– Kyle McAlpin (Air Force Liaison)
Jim Glass
Boris Katz
Charles Dagli
Michael Kanaan
Ronisha Carter
Kyle McAlpin
AI for Personalized Foreign Language Education

The AI Accelerator Natural Language Processing for foreign language project focuses on building personalized foreign language education framework, which includes a model of the language knowledge to be acquired. This work leverages current foreign language, tailored to the expected level of knowledge by the learner at various stages of coursework, and sets to standardized proficiency test measures. The model will help personalize the learning experience and illuminate when and/or where learning outcomes are unfavorable to the student.

 

 

Team
 – Shigeru Miyagawa (MIT PI)
– Emma Teng (MIT Co-PI)
– Doug Jones (MIT Lincoln Laboratory Lead)
– Michael Kanaan (Air Force Liaison)
– Ronisha Carter (Air Force Liaison)
– Kyle McAlpin (Air Force Liaison)
Shigeru Miyagawa
Emma Teng
Doug Jones
Michael Kanaan
Ronisha Carter
Kyle McAlpin

The Earth Intelligence Engine

The Earth Intelligence Engine
The Earth Intelligence (EI) Engine for weather and climate includes a novel AI testbed platform to support rapid, effective decision-making and long-term, strategic planning and operations for USAF. Advances in AI help close the gap between AI researchers and available Earth systems data via a platform connecting data and models, novel algorithms, and image gap-filling tasks to bridge lower-quality to higher-quality weather and climate data sets. The EI Engine will provide the USAF with improved algorithms for anomaly detection; critical remote access to centralized Earth intelligence data; intuitive supercomputer visualizations of Earth intelligence for mission support; improved nowcasting weather forecasting for mission operations; and strategic location identifications affected by climate change to enhance resource allocation. The goal is to develop a platform, which provides global scale to high-resolution local scale Earth weather and climate data and visceral visualizations to better inform policy decision-makers and leaders in government and business.
Team
– Dava J. Newman (MIT PI)
– Christopher Hill (MIT Co-PI)
– Stephanie Dutkiewicz (MIT Co-PI)
– Mark Veillette (MIT Lincoln Laboratory Lead)
– John Radovan (Air Force Liaison)
– Lindsey McEvoy (Air Force Liaison)
– Allan Vanterpool (Air Force Liaison)
Dava Newman
Christopher Hill
Stephanie Dutkiewicz
Mark Veillette
John Radovan
Lindsey McEvoy
Allan Vanterpool
Explainable Machine Learning
Despite their incredible performance, machine learning models remain inscrutable–we do not understand how or why they arrive at their conclusions. Consequently, it is impossible for us to attain confidence in models’ decisions, and to debug them should they malfunction. This “black-box” nature limits our ability to deploy and frugally maintain machine learning systems, especially in high-stakes contexts. This project approaches machine learning explainability with a radically new mindset: combining machine learning and human-computer interaction methodologies to make actionability for the actual users the primary objective. The goal is to identify criteria for explainable machine learning that will enable development of models with human- and task-aligned data representations and decision-making interfaces.
Team
– Aleksander Madry (MIT PI)
– Arvind Satyanarayan (MIT Co-PI)
– Antonio Torralba (MIT Co-PI)
– Steven Gomez (MIT Lincoln Laboratory Lead)
– Theo Tsiligkardis (MIT Lincoln Laboratory Lead)
– Kevin Nam (MIT Lincoln Laboratory Lead)
– John Radovan (Air Force Liaison)
– Lindsey McEvoy (Air Force Liaison)
– Allan Vanterpool (Air Force Liaison)
Aleksander Madry
Arvind Satyanarayan
Antonio Torralba
Steven Gomez
Theo Tsiligkardis
Kevin Nam
John Radovan
Lindsey McEvoy
Allan Vanterpool
Continual and Few-Shot Learning
AI techniques have proven very successful in many critical applications such as object recognition, speech recognition, and others. However, these successes have relied on collecting enormous datasets and careful manual annotations. This process is expensive, time-consuming, and in many scenarios, enough data is not available. Transfer learning offers a solution to these problems by leveraging past data seen by a machine to solve future problems using only few annotated examples. This research focuses on challenges in transfer learning and aims at developing algorithms that can fundamentally learn from multiple heterogeneous tasks, moving beyond low-level task similarity to enable broader transfer across distinct tasks. Such algorithms will find general applicability in several areas, including computer vision and natural language processing, and will substantially reduce the dependence on large amounts of annotated data and consequently reduce costs and time for deployment and maintenance of AI systems.
Team
– Pulkit Agrawal ( MIT PI)
– Regina Barzilay (MIT Co-PI)
– Marin Soljacic (MIT Co-PI)
– Olga Simek (MIT Lincoln Laboratory Lead)
– John Radovan (Air Force Liaison)
– Victor (Air Force Liaison)
– Armando Cabrera (Air Force Liaison)
Pulkit Agrawal
Regina Barzilay
Marin Soljacic
Olga Simek
John Radovan
Victor
Armando Cabrera
Robust AI Development Environment
AI and machine learning (ML) methods have demonstrated enormous promise for USAF. Many existing ML algorithms often fail catastrophically, however, when data inputs or task objectives change from those encountered during algorithm training. This lack of reliability combined with the opaque nature of modern ML techniques makes it impossible to deploy machine learning systems confidently in mission-critical environments. Furthermore, the inability of a model to adapt to changing environments translates into the need for (often costly and difficult) model retuning whenever the environment changes. This research will focus on a robustness-centered approach to developing ML algorithms. Robust AI Development ENvironment (RAIDEN) prioritizes ML reliability, versatility, and adaptability. The models, framework, and algorithms provided by our effort will enable streamlined deployment of truly reliable and efficient ML systems.
Team
– Asu Ozdaglar (MIT PI)
– Aleksander Madry (MIT Co-PI)
– Pablo Parrilo (MIT Co-PI)
– Olivia Brown (MIT Lincoln Laboratory Lead)
– John Radovan (Air Force Liaison)
– John Radovan (Air Force Liaison)
– Lindsey McEvoy (Air Force Liaison)
– Allan Vanterpool (Air Force Liaison)
Asu Ozdaglar
Aleksander Madry
Pablo Parrilo
Olivia Brown
John Radovan
Lindsey McEvoy
Allan Vanterpool

Objective Performance Prediction & Optimization Using Physiological and Cognitive Metrics

This project brings together experts in biomedical instrumentation, signal processing, neurophysiology, psychophysics, computer vision, Artificial Intelligence (AI), and Machine Learning (ML), as well as Air Force pilots, to develop and test AI-based, multi-modal physiologic sensor fusion approaches for objective performance prediction and optimization. The project will leverage immersive virtual environments to train pilots and unobtrusively measure predictors of performance. A series of Challenge Datasets developed from the program will be used to engage the community. Partnering with multiple governmental research efforts and Air Education and Training Command’s myriad pilot training units, the team seeks to provide proof-of-concept by demonstrably accelerating pilot training timelines, producing “better pilots faster.” These methods for accelerating training can then be transferred to all modes of learning across many disciplines and any task requiring significant cognitive effort.

Team
– Thomas Heldt (MIT PI)
– Tamara Broderick (MIT)
– Vivienne Sze (MIT)
– Hrishikesk Rao (MIT Lincoln Laboratory Lead)
– Laura Brattain (MIT Lincoln Laboratory)
– Kyle McAlpin (Air Force Liaison)
– Ronisha Carter (Air Force Liaison)
Thomas Heldt
Tamara Broderick
Vivienne Sze
Hrishikesh Rao
Laura Brattain
Kyle McAlpin
Ronisha Carter

Robust Neural Differential Models for Navigation and Beyond

There are several different GPS-alternatives being researched across the DoD and civilian sectors to address a GPS alternative; however, each alternative comes with additional costs and use cases. Magnetic Navigation presents an alternative GPS system that relies on magnetic resonance of the Earth – a system that is largely known and unchanging – to navigate. Some of the current problems with magnetic navigation involve 1) reducing excess noise on the system, such as magnetic outputs from the Aircraft itself, 2) determining position at a real-time pace or speeds consistent with military systems, and 3) combining with other systems to present a full-alternative GPS system.  The present project looks into using robust neural differential models to solve magnetic navigation shortcomings and provide a viable alternative to GPS.

Team
 – Alan Edelman (MIT PI)
– Chris Rackauckas (MIT Co-PI)
– Michael O’Keeffe (MIT Lincoln Laboratory Lead)
– Jonathan Taylor (MIT Lincoln Laboratory Lead)
– David Jacobs (Air Force Liaison)
– Kyle Palko (Air Force Liaison)
Alan Edelman
Chris Rackaukas
Michael O’Keeffe
Jonathan Taylor 
David Jacobs
Kyle Palko

AI-Enhanced Spectral Awareness and Interference Rejection

This project seeks to apply AI to enhance the USAF’s ability to detect, identify, and geolocate unknown radiofrequency (RF) signals, while providing tools for adaptive interference mitigation and smart spectrum analysis. These capabilities enhance Air Force Intelligence Surveillance and Reconnaissance (ISR) missions, communications, signals intelligence (SIGINT), and electronic warfare. Results will increase bandwidth utilization efficiency and spectrum sharing, improve Air Force communications performance in high interference environments, produce higher-quality RF signals intelligence, and improve system robustness to adversarial attacks and interference.

Team
 – Gregory W. Wornell (MIT PI)
– Yury Polyanskiy (MIT Co-PI)
– Jarilyn Hernandez Jimenez (MIT Lincoln Laboratory Lead)
– Binoy Kurien (MIT Lincoln Laboratory Lead)
– Lindsey McEvoy (Air Force Liaison)
– Allan Vanterpool (Air Force Liaison)
Greg Wornell
Yury Polanskiy
Jarilyn Hernandez Jimenez
Binoy Kurien
Lindsey McEvoy
Allan Vanterpool

AI Education Research: Know-Apply-Lead (KAL)

KAL is an exploratory research project that aims to advance educational research activities that promote maximum learning outcomes at scale for learners with diverse roles and educational backgrounds, ranging from Air Force and Department of Defense (DoD) personnel to the general public. The project team will research and evaluate various pedagogical practices and learning benefits associated with training Air Force personnel in AI topics over a variety of existing courses, map out the landscape of educational needs and competencies, and pilot experimental learning experiences with the goal of outlining early prototypes for innovative technology-enabled training and learning. The research is expected to provide insights that will benefit AI learners across the nation while supporting the DoD’s objective to develop elite and world-class AI-ready services.

Team
 – Cynthia Breazeal (MIT PI)
– Katerina Bagiati (MIT)
– Andrés Felipe Salazar Gomez (MIT)
– Kathleen Kennedy (MIT)
– Lori Glover (MIT)
– Diane Staheli (MIT Lincoln Laboratory Lead)
– John Radovan (Air Force Liaison)
– Kyle Palko (Air Force Liaison)
Cynthia Breazeal
Katerina Bagiati
Andrés Felipe Salazar Gomez
Kathleen Kennedy
Lori Glover
Diane Staheli
John Radovan
Kyle Palko
Research was sponsored by the United States Air Force Research Laboratory and the United States Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-19- 2-1000. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the United States Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.