Internal Revenue Service (IRS) online services
One of the priorities of the Internal Revenue Service (IRS) is to improve the experience of using its online services. MITRE's project here focuses on a few areas:
Taxpayer Digital Communication (TDC) system:
Interview IRS customer service representatives (CSRs) about their experience using the chat and secure messaging functions of TDC and propose ways for IRS to improve the TDC experience for their CSRs
Analyze best practices on chat and secure messaging from industries that are relevantly analogous to the IRS
Users with limited English proficiency (LEP):
Interview managers and employees at IRS offices that predominantly serve LEP taxpayers to learn about the challenges they face at work and how they can be better supported; supporting LEP taxpayers is not just about translations, but about understanding their deeper needs (e.g., technology know-how, prioritizing services, distrust of government, colloquial language, geographical logistics)
Perform an empirical experiment on the IRS's translated webpages to evaluate their usability by LEP taxpayers
Blind and visually-impaired (BVI) users: Test the effectiveness of IRS webpages on how well they serve BVI taxpayers through usability sessions and interview questions
COVID-19 vaccine distribution process
The effort to vaccinate Americans for COVID-19 involves many entities (federal and state governments, public institutions, vendors, logistics companies, vaccination sites, general public, etc.) and processes (manufacturing, inventory, ordering, record-keeping, shipping, etc.) that can be hard to keep straight. MITRE is currently modeling the entire effort to aid government officials in assessing areas for improvement.
Website usability: Adaptive Acquisition Framework (AAF)
MITRE developed a website to help government acquisition personnel navigate the mounds of policies, regulations, paperwork, and procedures that they have to deal with on a regular basis for their work (using an interactive website is way easier than digging around thousands of PDF pages!). I was part of a team that used surveys, interviews, and heuristic evaluations to improve the usability of this website. Improvements were related to issues such as structural consistency, visibility of system and user status, and affordances that intuitively suggest methods of interaction with elements. The recommendations will be under consideration for implementation by the Defense Acquisition University (DAU) and Under Secretary of Defense for Acquisition and Sustainment (OUSD A&S).
FAA: UAS Traffic Management (UTM), Remote ID (RID)
Unmanned aerial systems (UASs) and drones are increasingly being used to facilitate many aspects of civilian life (deliveries, law enforcement, recreation, etc.). As these vehicles become more popular and our skies become more populated, regulations become important for ensuring safe and efficient operations. The FAA is interested in devising these regulations and has asked MITRE to help them describe UAS-related entities and the processes that those entities commonly perform. I am using modeling software to capture these relevant entities and processes in a visually orderly manner that enables detailed understanding of the state space (and quick experimentation on various proposed processes, as well).
The Office of the Under Secretary of Defense for Research and Engineering (OUSD R&E) is aiming to modernize the targeting process (and close air support, in particular) by reducing human workload, applying AI and machine learning to optimize decision-making, and streamlining the flow of information on the battlefield. My roles in the projects (Fires Support Next; Air Space Total Awareness for Rapid Technical Execution, or ASTARTE) include using process models to describe and evaluate current and proposed future targeting processes, applying human factors and human-computer interaction principles to the process where relevant, and learning about the experiences of military personnel with the process. The eventual goal is a smarter, faster, and more accurate targeting process that enables soldiers to effectively call the air support they need in the theater. This project is a component of Autonomy Roadmap, an initiative focused on improving the use of autonomy across many parts of the Department of Defense; the components I have worked on include A) targeting processes (described previously), B) command and control of autonomous systems, and C) airspace deconfliction.
Fishing video game for training P-8 instructors
The U.S. Navy wants to improve the way that P-8 anti-submarine warfare (ASW) instructors are trained, focusing on improvements in training outcomes and player engagement. SoarTech is producing a video game in which the tasks required of players (controlling a fleet of drones to catch fish in a lake) are substantively analogous to the tasks required of P-8 ASW instructors during teaching periods. Through controlled experiments, this game will also serve as an opportunity to examine the effectiveness of various game interface transparency levels and various speech recognition grammar sets. UPDATE: The game (called DroneFisher) was named a finalist at the 2019 I/ITSEC Serious Games Showcase and Challenge.
Lifelong learning portal
SoarTech is building a tool that military personnel, students, and others can use to organize and better achieve a lifetime of learning. The production of this tool (the lifelong learning portal) is being funded by the Advanced Distributed Learning Initiative (ADL); my role on the team included designing user experience and outlining functional requirements.
Defining complex cognitive skills (CCSs)
Navigating cognitive complex domains brings unique challenges, whether in training, skill evaluation, environmental factors, or team-related concerns. I am working on a SoarTech team (sub-contracted to Northrop Grumman) tasked with creating a framework that outlines current CCS-related research and provides a road map for future research and training in this area. In particular, we are investigating the domain of cyber security as a relevant framework use case for Northrop Grumman and other military end-users.
Adaptive simulator for marksmanship training
During my time at the Army Research Laboratory, I worked on a simulated shooting range designed to teach cadets the fundamentals of marksmanship: positioning, trigger control, aiming, and breath control. The training system, powered by the Generalized Intelligent Framework for Tutoring (GIFT), provides feedback to cadets that is adapted to address weaknesses in performance (when compared to expert models). The work from this project has been published at various outlets such as the Interservice/Industry Training, Simulation, and Education Conference and the International Journal of Artificial Intelligence in Education (see "Publications" section).
Centers for Disease Control immunization schedule
I led a project through the HFES student chapter in which we used human factors and display principles to re-design the CDC's immunization schedule (2015 version here). We recommended some changes through a prototype schedule design and some of these changes were implemented in the 2017 schedule, with more changes possibly being implemented in the 2018 version of the schedule. The immunization schedule is a document referenced by doctors in their offices and available online for the public to use. This project was presented at the 2018 iteration of the HFES conference and a PDF of the proceedings paper can be found here.
Education research (grad school at Georgia Tech)
Productive failure: Educators often argue about the amount of guidance students should be provided when learning - some believe that a high amount of guidance ensures that students learn exactly what they are supposed to learn, while others believe that allowing learners to explore with less guidance helps them to create more meaningful connections to the material. The productive failure instructional methodology posits that students will learn more effectively with low amounts of guidance initially as they explore the problem space, and high amounts of guidance later. During the initial period of less guidance, students will often fail to solve the problems in front of them, but the failure will require them to think in useful ways (e.g., connecting new information with pre-existing knowledge, attacking problems from many different angles) that will serve them better in the long run and when solving novel problems. In the later periods with more guidance, they can correct their misconceptions and thereby make powerful connections to their unique experiences during exploration.
Interactive multimedia: Technology is fast becoming a large part of modern education, and with the rise in educational technologies is an increase in the use of interactive materials. People often assume that interactive materials are inherently better than non-interactive materials, but the effects of interactivity might depend on learning domains, task difficulty, and pre-existing individual differences in the users
Metacognition: Metacognition, the monitoring of one's own learning, is one of the best ways a student can regulate his or her learning processes to achieve positive outcomes. However, there are many ways to facilitate learners' metacognitive processes, and the job of researchers is to find ways that are effective but not mentally burdensome.
Paper vs. screen: The way we read has undergone drastic changes in recent years; where there used to be only printed books, now there are electronic forms such as e-readers. Does this change in delivery medium affect how we read? The answer might surprise you (Scientific American article on this topic, an article in HP's The Garage for which I was interviewed on this topic)
Psychology of Card Magic
Magicians are often able to deceive people by exploiting human psychology, whether it's memory fallibility, attention misdirection, or audience expectations. As a (very) amateur magician studying psychology, I use card tricks as a mechanism for illustrating basic psychology principles in a fun way.
"The Psychology of Card Magic" lectures (at Georgia Tech)
Course: Sensation and Perception (5x)
Course: Engineering Psychology (5x)
Course: Cognitive Psychology (2x)
CEISMC high-school summer camp (July 2017)
Throughout the summer of 2014, I worked with members of the Georgia Tech HFES student chapter (led by Thomas Gable) on a submission to the "Voting System of Tomorrow" design competition hosted by the Human Factors and Ergonomics Society (HFES). Our submission was a website prototype that acted as a "one-stop shop" for users to accomplish all of their voting-related tasks including registration, browsing logistical information, and even the actual voting itself. I led the team during the writing and ballot interface design phases of the project. The team was honored with a first-place distinction among academic teams during the annual HFES conference, and reported its findings in the Ergonomics in Design journal (listed as second author on manuscript; see "Publications" section) to make recommendations about how the American voting process can be improved through the use of human factors.
Submission document as a tech report (PDF)
Online Collaboration Applications Evaluated by Ease of Use
With so many different work collaboration tools to choose from nowadays (Google, Skype, Cisco, etc.), it can be difficult to choose one to use, particularly when there are so many different functions you might need a tool to be capable of doing. The HFES chapter at Georgia Tech put together a team of students (led by Lauren Margulieux) to evaluate the most popular tools in terms of their adherence to classic usability heuristics. A paper summarizing the outcomes of this project was published by Ergonomics in Design, on which I was listed as the second author (see "Publications" section).
Medication tray redesign
During my senior year at the University of Michigan (Ann Arbor), I worked with three other industrial engineering seniors to redesign the medication trays at the Ann Arbor Veterans Affairs Hospital. We used human factors and lean engineering principles to create user-friendly trays that improved nurses' drug retrieval speed and accuracy. This work was presented at the American Society of Health-System Pharmacists (ASHP) conference in 2013 and is now published in the Joint Commission Journal of Quality and Patient Safety (listed as first author; see "Publications" section). The project was also named a finalist for the ASHP's national Award for Excellence in Medication Safety during the summer of 2017 (2-minute video about award finalist recognition from ASHP)