Topic
Visual language
About: Visual language is a research topic. Over the lifetime, 1407 publications have been published within this topic receiving 18444 citations.
Papers published on a yearly basis
1 / 2
Papers
Book•
30 Mar 2007
TL;DR: Introduction to Multimodal Analysis is a unique and accessible textbook that clearly and critically explains this groundbreaking approach to visual analysis and outlines the tools for analysis and takes the reader through examples of analysis, providing a model that can be followed.
Abstract: Introduction to Multimodal Analysis is a unique and accessible textbook that clearly and critically explains this groundbreaking approach to visual analysis. Each chapter outlines the tools for analysis and takes the reader through examples of analysis, providing a model that can then be followed. All visual media compositions, such as photographs, advertisements, newspapers and websites, are carefully designed. A photograph of a soldier, an advertisement for a car, a magazine cover or the opening titles to a news programme are thought out to create the appropriate effect. Designers use semiotic tools such as colour, framing, focus, positioning of elements and font style to communicate with the viewer. These choices make up a visual language that we can analyse. Multimodal analysis looks at the separate components of this language to build up a toolkit for analysing the grammar of visual design. The book includes an assessment of the claim that there is a visual grammar and important differences between images and language and the way they create meaning are identified. Including images throughout and a colour plate section, Introduction to Multimodal Analysis is an essential resource for students studying multimodality within visual communication in media and cultural studies, critical discourse analysis, journalism studies or linguistics.
561 citations
483 citations
Posted Content•
TL;DR: It is shown that a baseline model based on recent embodied vision-and-language tasks performs poorly on ALFRED, suggesting that there is significant room for developing innovative grounded visual language understanding models with this benchmark.
Abstract: We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks. ALFRED includes long, compositional tasks with non-reversible state changes to shrink the gap between research benchmarks and real-world applications. ALFRED consists of expert demonstrations in interactive visual environments for 25k natural language directives. These directives contain both high-level goals like "Rinse off a mug and place it in the coffee maker." and low-level language instructions like "Walk to the coffee maker on the right." ALFRED tasks are more complex in terms of sequence length, action space, and language than existing vision-and-language task datasets. We show that a baseline model based on recent embodied vision-and-language tasks performs poorly on ALFRED, suggesting that there is significant room for developing innovative grounded visual language understanding models with this benchmark.
465 citations
Journal Article•
TL;DR: This chapter introduces Visual Language Section 1: Structure of Visual Language, which discusses the structure of visual language across the world and the role of language grammar in this structure.
Abstract: Chapter 1. Introducing Visual Language SECTION 1: STRUCTURE OF VISUAL LANGUAGE Chapter 2. The Visual Lexicon, Part 1: Visual morphology Chapter 3. The Visual Lexicon, Part 2: Panels and Constructions Chapter 4. Visual Language Grammar: Narrative Structure Chapter 5. Navigation of External Compositional Structure Chapter 6. Cognition of Visual Language SECTION 2: VISUAL LANGUAGE ACROSS THE WORLD Chapter 7. American Visual Language Chapter 8. Japanese Visual Language Chapter 9. Central Australian Visual Language Chapter 10. The Principle of Equivalence
456 citations
14 Jun 2020
TL;DR: Action Learning From Realistic Environments and Directives (ALFRED) as mentioned in this paper is a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks.
Abstract: We present ALFRED (Action Learning From Realistic Environments and Directives), a benchmark for learning a mapping from natural language instructions and egocentric vision to sequences of actions for household tasks. ALFRED includes long, compositional tasks with non-reversible state changes to shrink the gap between research benchmarks and real-world applications. ALFRED consists of expert demonstrations in interactive visual environments for 25k natural language directives. These directives contain both high-level goals like “Rinse off a mug and place it in the coffee maker.” and low-level language instructions like “Walk to the coffee maker on the right.” ALFRED tasks are more complex in terms of sequence length, action space, and language than existing vision- and-language task datasets. We show that a baseline model based on recent embodied vision-and-language tasks performs poorly on ALFRED, suggesting that there is significant room for developing innovative grounded visual language understanding models with this benchmark.
408 citations
Related Topics (5)
Performance Metrics
| Year | Papers |
|---|---|
| 2026 | 3 |
| 2025 | 31 |
| 2024 | 41 |
| 2023 | 30 |
| 2022 | 38 |
| 2021 | 41 |