Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

While Autonomous Systems (ASes) are crucial for routing Internet traffic, organizations that own them are little understood. Regional Internet Registries (RIRs) inconsistently collect, release, and update basic AS organization information (e.g., website), and prior work provides only coarse-grained classification. Bootstrapping from RIR WHOIS data, we build ASdb, a system that uses data from established business intelligence databases and machine learning to accurately categorize ASes at scale. ASdb achieves 96% coverage of ASes, and 93% and 75% accuracy on 17 industry categories and 95 sub-categories, respectively. ASdb creates a more rich, accurate, comprehensive, and maintainable dataset cataloging AS-owning organizations. This system, and resulting dataset, will allow researchers to better understand who owns the Internet, and perform new forms of meaningful analysis and interpretation at scale.

ASdb: a system for classifying owners of autonomous systems

We propose InstructDET, a data-centric method for referring object detection (ROD) that localizes target objects based on user instructions. While deriving from referring expressions (REC), the instructions we leverage are greatly diversified to encompass common user intentions related to object detection. For one image, we produce tremendous instructions that refer to every single object and different combinations of multiple objects. Each instruction and its corresponding object bounding boxes (bbxs) constitute one training data pair. In order to encompass common detection expressions, we involve emerging vision-language model (VLM) and large language model (LLM) to generate instructions guided by text prompts and object bbxs, as the generalizations of foundation models are effective to produce human-like expressions (e.g., describing object property, category, and relationship). We name our constructed dataset as InDET. It contains images, bbxs and generalized instructions that are from foundation models. Our InDET is developed from existing REC datasets and object detection datasets, with the expanding potential that any image with object bbxs can be incorporated through using our InstructDET method. By using our InDET dataset, we show that a conventional ROD model surpasses existing methods on standard REC datasets and our InDET test set. Our data-centric method InstructDET, with automatic data expansion by leveraging foundation models, directs a promising field that ROD can be greatly diversified to execute common object detection instructions.

pdf/instructdet-diversifying-referring-object-detection-with-5dg7jkxfad.pdf

InstructDET: Diversifying Referring Object Detection with Generalized Instructions

Narrated “how-to” videos have emerged as a promising data source for a wide range of learning problems, from learning visual representations to training robot poli-cies. However, this data is extremely noisy, as the narrations do not always describe the actions demonstrated in the video. To address this problem we introduce the novel task of visual narration detection , which entails de-termining whether a narration is visually depicted by the actions in the video. We propose What You Say is What You Show (WYS 2 ), a method that leverages multi-modal cues and pseudo-labeling to learn to detect visual narrations with only weakly labeled data. We further generalize our approach to operate on only audio input, learning properties of the narrator’s voice that hint if they are currently doing what they describe. Our model successfully detects visual narrations in in-the-wild videos, outperforming strong baselines, and we demonstrate its impact for state-of-the-art summarization and alignment of instructional video.

What You Say Is What You Show: Visual Narration Detection in Instructional Videos

Artificial Dreams: Surreal Visual Storytelling as Inquiry Into AI 'Hallucination'

35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets
and Benchmarks

/pdf/a-spoken-language-dataset-of-descriptions-for-speech-based-13vnvcnojg.pdf

A Spoken Language Dataset of Descriptions for Speech-Based Grounded Language Learning

Grounded language acquisition -- learning how language-based interactions refer to the world around them -- is amajor area of research in robotics, NLP, and HCI. In practice the data used for learning consists almost entirely of textual descriptions, which tend to be cleaner, clearer, and more grammatical than actual human interactions. In this work, we present the Grounded Language Dataset (GoLD), a multimodal dataset of common household objects described by people using either spoken or written language. We analyze the differences and present an experiment showing how the different modalities affect language learning from human in-put. This will enable researchers studying the intersection of robotics, NLP, and HCI to better investigate how the multiple modalities of image, text, and speech interact, as well as show differences in the vernacular of these modalities impact results.

Presentation and Analysis of a Multimodal Dataset for Grounded LanguageLearning.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Rishabh Sachdeva

Author Tools

Chat about Author

Papers

A Spoken Language Dataset of Descriptions for Speech-Based Grounded Language Learning

Presentation and Analysis of a Multimodal Dataset for Grounded LanguageLearning.