A control-room operator looks directly at a warning icon, then makes a decision that suggests the warning was not understood. An eye tracker records a fixation; a supervisor records a mistake. Neither observation alone tells us what the operator knew about the unfolding situation. Situation awareness concerns a person’s grasp of relevant elements in a dynamic environment, their meaning, and what those elements imply next. Measuring it is difficult because the state changes as events unfold and because every method reaches it through behavior, a report, or a probe. The study must define what relevant knowledge is for the task before it asks whether an operator had it.
Start with a task-specific model
Endsley’s influential model distinguishes perceiving relevant elements, comprehending their significance, and projecting likely near-future states. These are useful analytic levels, not three sealed compartments inside the mind. A person can notice a number without understanding whether it is dangerous, or understand the present state without anticipating a transition. The model directs a researcher to write questions about information required for safe action, rather than ask a vague question about general awareness. Other theories emphasize distributed cognition and the interaction between people and artifacts, so the chosen measurement approach should make its theoretical stance explicit.
What counts as relevant depends on the operator’s goal and the system state. A pilot, nurse, dispatcher, and automated-system supervisor may all need different information even when viewing the same display. A valid probe set therefore begins with task analysis: which cues should be available, which relationships matter, and which future developments could be projected at that moment? If the researcher selects only easy-to-query facts, the instrument may reward memory for display labels while missing the information that actually supports a decision. Expert review and pilot testing can improve content coverage, though expertise is not a guarantee of one correct mental model.
Situation awareness should be separated from performance. A good outcome may follow a lucky choice despite poor understanding; a bad outcome may follow a sensible action when the environment changes unpredictably. Performance can provide indirect evidence, but it includes motor skill, decision policy, constraints, and chance. Equating awareness with success makes the construct circular: an operator is called aware because they succeeded and successful because they were aware. A measurement study needs evidence about the person’s information state that is at least partly independent of its outcome measure.
It should also be separated from confidence. A person may feel certain while holding an inaccurate model of the situation. Endsley’s meta-analysis found divergence between objective and subjective awareness measures across studies, with potential roles for calibration and workload confounds. This does not make self-ratings useless. They answer a different question: what the person believes about their own understanding. Reporting both accuracy and confidence can reveal a dangerous mismatch that either score alone would hide.
Freeze probes and their assumptions
The Situation Awareness Global Assessment Technique, or SAGAT, pauses a simulated task, removes or masks task information, and asks questions about the current situation. Endsley’s 1995 measurement paper discusses evidence and limits for this query-based approach. Its strength is that an answer can be checked against the simulation state at a known moment. It also imposes a particular definition of awareness: information that the participant can answer when probed after an interruption. Researchers should not silently extend that definition to every kind of skilled, embodied or team-based understanding.
Probe content must match the moment. A question about an aircraft’s altitude is useful only if altitude mattered to the operational task and the simulation state gives a clear correct answer. A projection question needs a defined horizon and assumptions about what may change. If the task allows multiple reasonable forecasts, scoring only one as correct can punish sound judgment. Before collecting participants’ data, develop answer keys and rules for ambiguity. Document why each query belongs to perception, comprehension, or projection and whether it tests information available to the operator.
Freezing the task protects the probe from ongoing changes, yet it may alter the very process being measured. An interruption can disrupt memory and action plans; participants may learn to anticipate freezes and monitor queryable facts. Endsley’s original studies addressed intrusiveness under their conditions, while later reviews and critiques note that usability and transport to other settings remain open questions. A study should report freeze frequency, duration, timing, and whether the display was hidden. It should examine whether performance or behavior changes around freezes instead of assuming the probe is invisible to the task.
One freeze is a thin sample of a dynamic state. A person may miss a cue at one moment and recover awareness seconds later, or appear accurate just before a critical change. Multiple probes can cover different phases, but repeated interruptions increase burden and predictability. Sampling should follow the research question: awareness at a transition, sustained tracking across a scenario, or response to a particular design feature. Aggregate scores can conceal meaningful variation by query level and time. Report those patterns rather than only a total percentage correct.
Self-ratings, observers, and live queries
Self-ratings are attractive in settings where freezing the task is impossible. They can capture how aware an operator felt during a scenario, but they require introspection and may be influenced by outcome and effort. Someone who just completed a hard task may rate awareness low because the task felt demanding, even if probe accuracy would be high. Someone who never noticed a missing cue may rate awareness high. This is why subjective and objective measures should not be treated as interchangeable. The wording, timing, and response scale need to be preserved when results are interpreted.
Observer ratings can reflect visible behavior, communication, and errors, but an observer cannot see every inference the operator made. A person may quietly maintain a correct model without saying it aloud; another may recite information without using it. Training observers and checking agreement can improve reliability, yet agreement about visible behavior is not proof that the underlying awareness state has been captured. A strong observational scheme states the behavior it codes and avoids turning that code into a direct claim about private knowledge unless there is supporting evidence.
Live queries preserve task flow more than a full freeze in some settings, but they compete for attention and may be constrained by response time. Delayed answers can mean uncertainty, high workload, or a reasonable decision to prioritize the primary task. A probe accepted or rejected during work is therefore a joint outcome of awareness and task management. Researchers can model response availability and accuracy separately. They should also make clear whether the participant was permitted to consult displays while answering, because that changes the question from retained awareness to ability to retrieve current information.
Eye movement and physiological proxies
Eye tracking reveals where and when the eyes were directed. It is valuable for testing whether relevant information was sampled, especially when combined with a precise map of what the display showed at each moment. De Winter and colleagues argue for relating gaze to the task environment rather than interpreting gaze metrics in isolation. Fixation on a warning is not proof of comprehension; no fixation does not always prove ignorance, because peripheral vision and prior knowledge can matter. The strongest gaze analysis asks which cues were available, when they became relevant, and whether looking predicted later answers or decisions under the task’s conditions.
A gaze heat map averaged over a long scenario can be especially misleading. Two operators may spend equal total time looking at a gauge, but one looks when the value changes and the other looks only after the danger passes. Time locking to events and considering the sequence of information seeking are necessary. Pupil size, EEG, or heart-rate measures may speak to demand or arousal, not directly to situation awareness. A model trained to predict high versus low SAGAT scores from such signals still needs validation across displays, tasks, and populations before it can be called a measure of awareness itself.
Workload is a major confound and a substantive neighbor. A difficult interface can increase effort while making key information harder to integrate. A self-rating that asks whether the situation felt manageable may partly measure workload. Conversely, an operator with excellent understanding may expend effort to maintain it. Workload measures can quantify effort, but they do not directly validate a measure of situation awareness. A study interested in both constructs should measure them separately and test whether a proposed design changes one, the other, or both.
Teams and automation
Many operational situations are handled by teams. No individual may hold every relevant fact; knowledge can be distributed through handoffs, shared displays, and communication. A set of individual probe scores does not automatically describe team awareness. Team measures need to examine whether critical information reached the people who needed it at the right time and whether members held compatible interpretations. Communication logs can help, but frequent talk is not the same as shared understanding. A quiet, well-coordinated team may be effective, while a talkative team may repeatedly exchange outdated facts.
Automation adds another layer. A system can execute a control action while leaving the human uncertain about its goal, current mode, or next move. Accurate performance during routine periods can coexist with poor awareness when the automation fails or hands control back. A simulation designed to test awareness under automation should include transitions and abnormal events, not only smooth operation. Questions can target what the automation is doing, why it is doing it according to displayed information, and what it is likely to do next. The answer key must follow the actual system state, not a designer’s assumption about what the interface communicates.
In high-stakes settings, measurement should not create preventable risk. Freeze probes belong in simulation or other controlled circumstances, not unprotected live care or control tasks. Field methods can use retrospective reconstruction, observation, or carefully placed debriefs, acknowledging the memory and outcome biases they introduce. The ethical choice of method is part of construct validity: if a measure changes the task, excludes particular operators, or forces attention away from safety, its results may be less applicable to the setting it was meant to improve.
Design and reporting for valid inference
A clear report lists the scenario’s critical events, the operator goals, the available cues, and the query set or coding rules. It explains how correct answers were established, how ambiguous responses were handled, and how the measure was timed. It reports awareness scores separately by type and phase where those distinctions matter. It also gives performance and workload outcomes without using either as a substitute for awareness. This allows readers to assess whether a design improved comprehension, simply made the task easier, or changed what the measure happened to sample.
Reliability requires more than identical scores on repeated trials. Scenarios differ, and participants learn. A probe set may be easy in one event sequence and impossible in another because information was unavailable. Tests of internal consistency can be misleading if queries represent different pieces of a situation that need not covary. Reproducibility depends on well-specified scenarios and scoring; generalizability depends on sampling different relevant situations. The 2026 meta-review highlights that methods vary in psychometric evidence across domains. Validation must be local enough to support the actual use, especially when translating a cockpit instrument to healthcare or consumer interfaces.
Analysis should resist turning a complex time series into a moral judgment about operators. A low probe score can arise from display design, task priorities, ambiguous information, or a query about a fact that was not needed. Before concluding that a person failed to pay attention, ask what the system made perceptible and what the task rewarded. Individual differences may matter, but a measurement program is most useful when it identifies improvable conditions rather than merely labeling people. The purpose is to understand information flow and decision support.
What the evidence does not show
No method directly reads situation awareness from the mind. Freeze probes assess answers at selected moments under an interruption; self-ratings assess perceived understanding; gaze assesses visual sampling; observers assess visible behavior. None alone proves that a person grasped the whole situation or would act correctly in every future event. The cited studies also do not establish a universal score that transfers across aviation, healthcare, and automated consumer products. The construct is task-dependent, and the measure must be tested against the information and decisions that matter locally.
Situation awareness measurement is most useful when it answers a precise design question. Did the new display help operators detect a changed state in time? Did they understand the implication, or merely look at the icon? Did they anticipate a transition before a handoff? The evidence grows stronger when a task analysis, timed probes, behavioral record, and honest account of interruption all point in a coherent direction. A well-measured moment of awareness is still only one moment, but it can show where a system helps people remain oriented as the world changes around them.
Research references
- Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. View source
- Endsley, M. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37(1), 65–84. View source
- Salmon, P. M., Stanton, N. A., Walker, G. H., & Green, D. (2006). Situation awareness measurement: A review of applicability for C4i environments. Applied Ergonomics, 37(2), 225–238. View source
- Endsley, M. R. (2020). The divergence of objective and subjective situation awareness: A meta-analysis. Journal of Cognitive Engineering and Decision Making, 14(1). View source
- de Winter, J. C. F., Eisma, Y. B., Cabrall, C. D. D., Hancock, P. A., & Stanton, N. A. (2019). Situation awareness based on eye movements in relation to the task environment. Cognition, Technology & Work, 21, 99–111. View source
- Moens, L. L., Lydon, S., Cucurachi, S., O’Connor, P., Sauter, T. C., Töndury, G.-A., & Manser, T. (2026). Measuring situation awareness: A meta-review across domains. Human Factors, 68(5), 632–672. View source