Must a filler be a person?

No; the schema can include objects, organizations, places, or values.

Does finding all fillers guarantee correct event records?

No; their grouping can still be wrong.

AI Glossary / Natural language and speech / Advanced

Role-filler extraction

Definition

Role-filler extraction identifies the text spans or entities that occupy specified roles in an event or situation. In a delivery template, roles might include sender, recipient, item, and destination. The task assigns appropriate information to those roles; recognizing that a delivery is mentioned is a related but distinct task.

What it means

A role describes how something participates in the situation being represented. A filler supplies the particular person, organization, object, place, or other value occupying that role. The same organization could be a sender in one event and a recipient in another.
This goes beyond recognizing entity types. Knowing that Cedar Supplies is an organization does not establish whether it sent or received a package. Interpretation depends on the surrounding language and the template being filled.
The task's exact scope varies. Some evaluations focus on finding role-bearing mentions throughout a document. Constructing complete event records can additionally require linking mentions of the same entity and deciding which fillers belong to which event. A strong role-extraction score does not automatically establish success on these additional tasks.

How it works

Define the event schema and annotation rules first. Specify the roles, permitted filler types, and whether a role can contain several values or remain unfilled. Training examples then identify the relevant spans or entities in representative text.
One approach classifies candidate noun phrases separately for each role. Another labels token sequences to mark filler spans. Neural approaches can use contextual representations that combine information from the local sentence and surrounding document. The appropriate context depends on where the evidence appears.
A complete application also needs rules for duplicates and ambiguity. Two different names may refer to one organization, while two occurrences of the same name may participate in different events. These possibilities require explicit handling rather than simply collecting every nearby noun.
Evaluation should state whether it measures exact spans, normalized entities, or event-linked role assignments. These are different units. A prediction can identify the right organization with a different surface form, or identify the right text span but attach it to the wrong event. Reporting one undifferentiated accuracy figure conceals those distinctions.

Three practical examples

A seminar-calendar assistant extracts speaker, venue, and start time from an announcement, leaving an unstated room number blank.
A delivery-summary tool identifies sender and recipient even when the sentence uses passive wording.
A maintenance-record processor collects equipment, technician, and location mentions across sentences before assembling records for individual visits.

Think of it this way

Think of filling a form from a written account. The headings establish the roles, and the account supplies the entries. Finding a name is only the beginning: it must go in the correct box. When the account describes two transactions, putting all names into one form can create a transaction that never appeared in the text.

Worked example

Consider this fictional passage: “Cedar Supplies sent six lamps to Harbor Studio on Monday. Cedar shipped two desks to Pine Office on Tuesday.” Assume Cedar in the second sentence refers to Cedar Supplies, and the two sentences describe separate delivery events.
Use four roles: sender, recipient, item, and quantity. Dates help distinguish the events but are outside the four scored roles. Normalize Cedar and Cedar Supplies to one sender entity. Retain the item values lamps and desks and represent quantities as integers.
The first event's reference record is sender=Cedar Supplies, recipient=Harbor Studio, item=lamps, quantity=6. The second is sender=Cedar Supplies, recipient=Pine Office, item=desks, quantity=2. There are eight event-role-value assignments in total, including the sender assignment once for each event.
Suppose a system gets every assignment right except that it exchanges the recipients. Its first record names Pine Office and its second names Harbor Studio. Under exact event-role-value matching, six predictions are correct, two predictions are wrong, and two reference assignments are missing. Precision is 6/8=75%, recall is 6/8=75%, and F1 is 75%.
Now evaluate only the document-level set of distinct role-value pairs, discarding event identity. That set contains one sender, two recipients, two items, and two quantities: seven pairs. The system has found all seven correct pairs despite attaching the recipients to the wrong events. Its set-level precision and recall are both 100%.
The second measure tests whether the document's fillers were found; the first additionally tests whether each belongs to the right event. Neither figure should be presented without its scoring unit.
The reference also illustrates why quantity is not the number of event records. Six lamps and two desks give eight objects across two described deliveries, not eight delivery events. Similarly, the two sender mentions can refer to one entity without collapsing the two deliveries into one.
This invented example assumes entity normalization and event separation; it does not evaluate the full pipeline.

Illustrative case study

Fictional example: ParcelNotes turns shipment narratives into internal summaries. Its team initially evaluates whether each document's organization names were found. Reviewers discover that a summary can pass this check while reversing sender and recipient.
The team adds labeled roles, event identifiers, and links back to the supporting sentences. It separately tracks missing fillers and assignments to the wrong event. This fictional scenario illustrates evaluation design.

Common misconceptions and mistakes

Role-filler extraction is not interchangeable with named-entity recognition. Entity type alone does not establish event role. Nor is it identical to detecting an event trigger: finding sent does not identify all the participants or values.
A missing filler should not automatically be invented. The text may omit it, or the model may have missed it; those possibilities should remain distinguishable. Exact-span matching and entity-level matching also need different conventions. Finally, role detection without event tracking may mix information from separate situations.

Business use

Define the records the business actually needs before selecting an evaluation measure. Retain supporting text and document normalization rules. Test multi-event passages, passive sentences, repeated names, and missing information. Review complete records as well as individual fillers, because plausible values can form an incorrect combination.

Check your understanding

How many assignments are scored with event identity?

Answer: Eight.

Why can document-level scoring reach 100%?

Answer: It ignores the swapped event attachments.

Do eight objects imply eight deliveries?

Answer: No.

Related terms

Learn next

Create a small event schema and compare span-level, entity-level, and event-linked scoring on the same predictions. Identify which errors each measure misses.

Sources

Jurafsky and Martin, Speech and Language Processing, chapter 21, template filling and role-filler extraction: https://web.stanford.edu/~jurafsky/slp3/21.pdf Du and Cardie, Document-Level Event Role Filler Extraction using Multi-Granularity Contextualized Encoding, ACL 2020: https://aclanthology.org/2020.acl-main.714/

Previous: Rocchio algorithm Next: Role prompting