Jiwan Chung
Seoul National University
19 Papers
1 Citations
Jiwan Chung is an academic researcher from Seoul National University. The author has contributed to research in topics: Computer science & Feature learning. The author has an hindex of 1, co-authored 4 publications.
Chat about Author
Papers
Multimodal Knowledge Alignment with Reinforcement Learning
Youngjae Yu,Jiwan Chung,Heeseung Yun,Jack Hessel,J. Park,Ximing Lu,Prithviraj Ammanabrolu,Rowan Zellers,Ronan LeBras,Gunhee Kim,Yejin Choi +10 more
TL;DR: This work proposes ESPER, a novel approach to reinforcement learning which extends language-only zero-shot models to unseen multimodal tasks, like image and audio captioning, and demonstrates that it outperforms baselines and prior work on a variety of zero- shot tasks.
•Posted Content
ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning.
TL;DR: In this article, the authors present an automatic dataset curation approach based on subset optimization where the objective is to maximize the mutual information between audio and visual channels in videos, and demonstrate that their approach finds videos with high audio-visual correspondence and show that self-supervised models trained on their data achieve competitive performances compared to models trained by existing manually curated datasets.
17
Fusing Pre-Trained Language Models with Multimodal Prompts through Reinforcement Learning
Youngjae Yu,Jiwan Chung,Heeseung Yun,Jack Hessel,J. Park,Ximing Lu,Rowan Zellers,Prithviraj Ammanabrolu,Ronan LeBras,Gunhee Kim,Yejin Choi +10 more
- 01 Jun 2023
TL;DR: This work proposes ‡ESPER (Extending Sensory PErception with Reinforcement learning) which enables text-only pretrained models to address multimodal tasks such as visual commonsense reasoning.
13
Character Grounding and Re-identification in Story of Videos and Text Descriptions
Youngjae Yu,Jongseok Kim,Heeseung Yun,Jiwan Chung,Gunhee Kim +4 more
- 23 Aug 2020
TL;DR: The CiSIN model achieves the best performance in the Fill-in the Characters task of LSMDC 2019 challenges and outperforms previous state-of-the-art models in M-VAD Names dataset as a benchmark of multimodal character grounding and re-identification.
9
Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models
Hyungjoo Chae,Yeonghyeon Kim,Seungone Kim,Kai Tzu-iunn Ong,Beong-woo Kwak,Moohyeon Kim,Seonghwan Kim,Taeyoon Kwon,Jiwan Chung,Youngjae Yu,Jinyoung Yeo +10 more
TL;DR: Think-and-Execute framework improves algorithmic reasoning in LLMs by decomposing the reasoning process into task-level logic and instance-specific code execution.
8