Understanding the HED Hierarchy

Why HED is hierarchical

The HED schema is a controlled vocabulary organized as a tree. Each tag belongs to a path, and each child tag is a more specific version of its parent.

For example:

Item/Object/Geometric-object/2D-shape/Rectangle/Square

This means:

Square is a type of Rectangle
Rectangle is a type of 2D-shape
2D-shape is a type of Geometric-object
Geometric-object is a type of Object

This matters because HED tools can search at different levels. A search for 2D-shape can also find events tagged with Square, Rectangle, Cross, or Triangle.

HED tools also support short-form tags. In annotation, you usually write only the final node name, such as Square, not the full path. The tool can expand it to the long form when needed.

How to read the schema browser

When you open the schema browser, the top-level branches are broad conceptual categories. These are not random word lists. They answer different annotation questions.

Top-level branch	Main question it answers	Typical use
`Event`	What kind of event is this?	Stimulus, response, recording marker, trial structure
`Agent`	Who or what is involved?	Participant, software, device, experimenter
`Action`	What happened or what was done?	Press, move, speak, read, track
`Item`	What object, stimulus, or thing is involved?	Face, image, sound, body part, text
`Property`	What characteristics describe it?	Color, label, ID, task role, environment
`Relation`	How are things linked?	Connected-to, part-of, above, equal-to

For beginners, this is the easiest way to think about HED:

A good HED annotation often answers: what happened, who did it, what was involved, what properties matter, and how things are related.

Example: A beginner-friendly recipe for choosing tags

When annotating an event, go through these questions in order:

Question	HED branch to check	Example answer
What kind of event is this?	`Event`	`Sensory-event`
Who is involved?	`Agent`	`Human-agent`, `Experiment-participant`
What happened?	`Action`	`Press`, `Saccade`, `Speak`
What item or stimulus is involved?	`Item`	`Face`, `Cross`, `Tone`, `Word`
Which properties matter?	`Property`	`Target`, `Correct-action`, `Label`, `Condition-variable`
Are relations important?	`Relation`	`Performed-by`, `Above`, `Part-of`

1. Event: what kind of event happened?

The Event branch is usually the best place to start. The HED quickstart recommends that you first choose one of the major event types before adding more detail.

Main Event categories

Event tag	Meaning in plain language	Typical neuroscience / psychology example
`Sensory-event`	Something was presented to the participant or perceived by the participant	A face image appears on the screen; a tone is played
`Agent-action`	A person, device, or software agent does something	The participant presses a button; the experimenter speaks
`Data-feature`	A meaningful feature appears in the recorded signal or derived data	An EEG artifact marker; a blink feature detected in the data
`Experiment-control`	The system controls the run or recording process	Start recording; stop task; pause acquisition
`Experiment-procedure`	A procedural step in the experiment takes place	Calibration, setup, instruction phase
`Experiment-structure`	The event marks the structure of the design	Trial start, block start, rest period
`Measurement-event`	Something happens specifically related to measurement or acquisition	MRI acquisition begins; impedance check

Practical rule

For each event row, ask first:

What is this row mainly describing?

a presented stimulus -> start with Sensory-event
a response or behavior -> start with Agent-action
a trial or block marker -> start with Experiment-structure
a calibration or setup step -> start with Experiment-procedure
something in the signal itself -> start with Data-feature

2. Agent: who or what is involved?

The Agent branch describes the entity that acts, responds, or participates.

Major Agent categories

Agent tag	Meaning	Typical example
`Animal-agent`	A non-human animal acts	Rodent in a behavioral task
`Avatar-agent`	A virtual or represented actor acts	Avatar in VR or game-like task
`Controller-agent`	A control mechanism or controller is the acting entity	Joystick controller, controller logic
`Human-agent`	A human is the acting entity	Participant, clinician, experimenter
`Robotic-agent`	A robot acts	Robotic arm in an interaction task
`Software-agent`	Software performs an action	Presentation software logs a trigger

Why this matters

In neuroscience and psychology datasets, the difference between participant action, software action, and device action can be crucial.

Example:

Participant presses a key -> Human-agent, Agent-action, Press
Presentation software displays an image -> Software-agent, Sensory-event, Visual-presentation

3. Action: what was done?

The Action branch is one of the richest parts of the schema. It contains many concrete terms that are highly useful in behavioral experiments.

Main Action families

Action family	What it covers	Example tags from the browser
`Communicate`	Expressive or communicative behavior	`Speak`, `Laugh`, `Cry`, `Shout`, `Whisper`, `Wave`, `Smile`, `Wink`
`Move`	Whole-body or body-part movement	`Walk`, `Run`, `Jump`, `Turn`, `Blink`, `Saccade`, `Stretch`, `Sit-down`
`Perceive`	Acts of sensing	`Hear`, `See`, `Smell`, `Taste`, `Sense-by-touch`
`Perform`	Generic task performance actions	`Read`, `Write`, `Play`, `Open`, `Close`, `Repeat`, `Operate`, `Rest`
`Track`	Following or tracking something	Eye tracking, cursor tracking, target tracking

Tag	Meaning in plain language	Useful in
`Speak`	Producing spoken language	Speech tasks, interview paradigms
`Laugh`	Laughing vocalization	Naturalistic behavior studies
`Whisper`	Speaking quietly	Speech production tasks
`Smile`	Facial expression of smiling	Emotion tasks, video annotation
`Wave`	Hand wave gesture	Social interaction tasks
`Nod-head`	Nodding movement	Nonverbal response annotation

Tag	Meaning in plain language	Useful in
`Blink`	Eyelid closure movement	EEG / eye tracking / artifact annotation
`Saccade`	Fast eye movement	Eye tracking, visual attention studies
`Fixate`	Maintain gaze on a target	Visual tasks
`Press`	Pressing something	Button press tasks
`Grab`	Reaching and taking hold	Motor tasks
`Walk`	Walking movement	Gait or movement studies
`Jump`	Jumping action	Motor control tasks
`Turn-head`	Rotating the head	Motion tracking, VR
`Lift-head`	Raise the head	Movement annotation
`Kick`	Leg action	Motor paradigms
`Touch`	Touching an object or surface	Tactile tasks

Tag	Meaning in plain language	Useful in
`See`	Visually perceive	Gaze or visibility contexts
`Hear`	Auditory perception	Auditory task annotation
`Sense-by-touch`	Tactile perception	Somatosensory experiments

4. Item: what object, stimulus, or thing is involved?

The Item branch is where you describe what was presented, manipulated, or referenced.

This branch is extremely important in stimulus annotation.

Major Item families

Item family	What it covers	Example tags from the browser
`Biological-item`	Body parts, organisms, anatomical structures	`Face`, `Eye`, `Brain`, `Hand`, `Foot`, `Human`, `Animal`
`Language-item`	Linguistic units	`Word`, `Sentence`, `Syllable`, `Phrase`, `Paragraph`
`Object`	Physical or abstract objects	`Cross`, `Triangle`, `Arrow`, `Book`, `Tool`, `Machine`, `Robot`
`Sound`	Acoustic stimuli	`Tone`, `Click`, `Buzz`, `Siren`, `Instrument-sound`, `Crowd-sound`

Biological-item examples

Tag	Meaning	Common use
`Body`	Whole body	Body motion, posture, embodiment tasks
`Head`	Head as an anatomical item	Head movement, face/head stimuli
`Brain`	Brain as an anatomical item	Neuroanatomical references
`Eye`	Eye structure	Eye movement, gaze, facial stimulus annotation
`Face`	Face as an anatomical or visual item	Face perception experiments
`Mouth`	Mouth structure	Speech, facial expression
`Hand`	Hand or upper extremity part	Motor tasks, gesture tasks
`Foot`	Foot or lower extremity part	Gait and motor studies
`Human`	Human organism	Social stimuli, social cognition
`Animal`	Non-human animal	Animal image or sound stimuli

Language-item examples

Tag	Meaning	Common use
`Character`	Single character	Reading and orthography tasks
`Word`	Word stimulus	Lexical decision, reading
`Pseudoword`	Word-like nonword	Psycholinguistic paradigms
`Sentence`	Sentence stimulus	Language comprehension
`Syllable`	Syllable stimulus	Speech and phonology tasks
`Phoneme`	Speech sound unit	Auditory language tasks
`Paragraph`	Multi-sentence text	Reading comprehension

Object examples relevant for experiments

Tag	Meaning	Common use
`Cross`	Cross shape	Fixation cross
`Arrow`	Arrow shape	Cueing tasks
`Triangle`	Triangle shape	Shape discrimination
`Rectangle`	Rectangle shape	Visual feature tasks
`Single-point`	Point or dot	Dot probes, fixation point
`Image`	Visual image	Generic image stimulus
`Movie`	Moving visual media	Naturalistic paradigms
`Questionnaire`	Questionnaire document	Survey steps
`Device`	Generic device	Equipment references
`Measurement-device`	Recording device	EEG cap, scanner, eye tracker

Sound examples

Tag	Meaning	Common use
`Tone`	Tone stimulus	Auditory oddball, ERP tasks
`Click`	Click sound	Auditory probes
`Buzz`	Buzzing sound	Audio warning or stimulus
`Siren`	Siren sound	Salient sound paradigms
`Instrument-sound`	Sound from an instrument	Music cognition
`Vocalized-sound`	Sound from a vocal source	Voice perception
`Crowd-sound`	Environmental crowd noise	Naturalistic listening

5. Property: what characteristics matter?

The Property branch is often the hardest for beginners, because many tags here do not describe a thing directly. They describe attributes, metadata, task roles, or context.

A good way to think about Property is:

If Item tells you what something is, Property often tells you what is true about it.

Major Property families

Property family	What it is for	Example tags from the browser
`Agent-property`	Describes state, role, or traits of an agent	`Alert`, `Asleep`, `Happy`, `Experiment-participant`, `Experimenter`, `Age`, `Gender`, `Handedness`
`Data-property`	Describes aspects of data	`Data-marker`, `Data-value`, `Data-source-type`, `Data-resolution`
`Environmental-property`	Describes the environment	`Indoors`, `Outdoors`, `Urban`, `Rural`, `Virtual-world`, `Real-world`
`Informational-property`	Describes labels, identifiers, metadata, parameters	`Label`, `Description`, `ID`, `Creation-date`, `URL`, `Pathname`, `Parameter`
`Organizational-property`	Describes structure and experiment organization	`Condition-variable`, `Control-variable`, `Event-context`, `Task`, `Recording`, `Experimental-trial`, `Time-block`
`Sensory-property`	Describes presentation and sensory aspects	`Sensory-attribute`, `Sensory-presentation`
`Task-property`	Describes task roles and task meaning	`Appropriate-action`, `Correct-action`, `Target`, `Distractor`, `Instructional`

Agent-property examples

Tag	Meaning	Why it matters
`Alert`	Cognitively alert state	Arousal/state description
`Asleep`	Sleep state	Sleep studies
`Attentive`	Paying attention	Vigilance or attention tasks
`Distracted`	Attention is diverted	Behavioral state annotation
`Happy`	Emotional state	Emotion studies
`Fearful`	Emotional state of fear	Affective annotation
`Experiment-participant`	The acting human is the participant	Clarifies task role
`Experimenter`	The acting human is the experimenter	Clarifies role during setup or instruction
`Age`	Age-related trait	Subject metadata
`Gender`	Gender-related metadata	Participant characterization
`Handedness`	Left/right handedness	Motor and lateralization studies

Informational-property examples

Tag	Meaning	Common use
`Label`	Human-readable label	Label a condition or item
`ID`	Identifier	Link to stimulus code or event code
`Description`	Free descriptive text slot	Human explanation
`Pathname`	File path	Stimulus file paths in BIDS sidecars
`URL`	Web location	External reference
`Parameter`	Parameter-like metadata	Trial parameters, settings
`Creation-date`	Date metadata	Resource provenance

Organizational-property examples

Tag	Meaning	Common use
`Condition-variable`	Experimental condition descriptor	Congruent vs incongruent
`Control-variable`	Controlled factor	Lighting, display settings
`Event-context`	Context active around an event	Trial context, block context
`Task`	Task identity or role	Stroop task, oddball task
`Recording`	Recording context	Acquisition context
`Experimental-trial`	Trial unit	Trial onset/offset definitions
`Time-block`	Block-level structure	Rest block, task block

Task-property examples

Tag	Meaning	Common use
`Appropriate-action`	Action matches task demand	Correct response type
`Correct-action`	Response is correct	Behavioral outcome
`Incorrect-action`	Response is wrong	Error trials
`Imagined-action`	Action was imagined, not executed	Motor imagery
`Target`	Stimulus is task-relevant target	Oddball, visual search
`Distractor`	Stimulus is a distractor	Attention tasks
`Instructional`	Stimulus provides instruction	Task instructions

6. Relation: how are things connected?

The Relation branch expresses relationships between entities, values, or positions.

This branch is often underused by beginners, but it becomes very powerful in richer annotations.

Major Relation families

Relation family	What it expresses	Example tags from the browser
`Comparative-relation`	Comparison of values or qualities	`Equal-to`, `Greater-than`, `Less-than`, `Not-equal-to`
`Connective-relation`	General conceptual or structural relation	`Connected-to`, `Part-of`, `Member-of`, `Includes`, `Described-by`, `Performed-by`, `Performed-using`
`Directional-relation`	Direction or orientation	Spatial or directional structure
`Logical-relation`	Logic-level linkage	`And`, `Or`
`Spatial-relation`	Position in space	`Above`, `Below`, `Adjacent-to`, `Between`, `Behind`, `Around`

Useful Relation examples

Tag	Meaning in plain language	Example use
`Part-of`	One item belongs to a larger whole	Eye is part of Face
`Performed-by`	Action is carried out by an agent	Press performed by participant
`Performed-using`	Action uses a tool/body part	Response made using index finger
`Described-by`	Something is described by metadata	Stimulus described by label
`Connected-to`	Two things are linked	Electrode connected to amplifier
`Above`	Spatially above	Stimulus above fixation
`Below`	Spatially below	Target below center
`Equal-to`	Comparison equality	Parameter equal to threshold

A beginner-friendly recipe for choosing tags