Jon Hollenbach is will be presenting his thesis proposal on November 13th. Please see the details below:
Date and Time: November 13th, 1PM
Location: Malone 137
Zoom Link: https://JHUBlueJays.zoom.us/j/7428546704?pwd=YlVFcVVoUk5SVDR0RjJsQTROSDhUUT09
Advisor: Prof. Mitra Taheri
Title: Data-Driven Autonomous Electron Microscopy: Enabling low-latency control and machine learning for defect engineering in 2D Materials
Abstract: Transmission Electron Microscopy (TEM) offers an exciting platform for the autonomous discovery and processing of materials through multi-modal sensing, precise electron-photon optics, and low-latency controls. The focused electron probe of the TEM, combined with in situ capabilities and direct detection systems, is uniquely equipped for manipulating point defects in two-dimensional materials such as graphene, transition metal dichalcogenides (TMDs), and hexagonal boron nitride (hBN). By enabling atomically precise creation and manipulation of defect arrays within these materials, TEM facilitates the exploration of structures that are critical for the development of quantum devices and single-photon emitters. Achieving reliable processing at the atomic scale necessitates low-latency control over the instrument, as well as rapid analysis and decision-making of large, multi-modal datasets. This work highlights advancements that address these challenges as part of a cohesive framework for autonomous TEM control.
Machine learning (ML) has proven to be an effective approach for analyzing information-dense detection modes, including hyperspectral imaging, 4D STEM, and in situ experimentation, particularly for feature extraction, classification, and decision-making. We demonstrate how deep learning methods can provide unsupervised analysis of experimental data, linking clusters within datasets to computationally generated structures. In this context, decision-making based on ML results can either follow a human-in-the-loop approach defined by scientists or leverage custom-built agentic models to generate hypotheses and guide experimental actions.
Control frameworks for TEM are rapidly evolving to meet the needs of laboratory automation. Current frameworks often struggle with latency and flexibility, necessitating the development of a new autonomy framework presented in this work. Given the complexity of TEM systems, which involve hardware and software from multiple vendors and limited application programming interfaces (APIs), our framework aims to unify all control endpoints and streamline data analysis and feedback control. This is achieved through an asynchronous control interface that abstracts endpoint-specific functions into generalized instrument commands, a dedicated inference server that ingests data from all detectors and automates the analysis process, and an experiment database for tracking actions, data, and analyses across experiments. Consequently, this framework serves as both an automation platform for scientist-defined experimentation and a testing ground for autonomous control models, including reinforcement learning and large language model (LLM) agent control models.