Human-like chatbots necessitate the use of commonsense reasoning in order to effectively comprehend and respond to implicit information present within conversations. Achieving such coherence and informativeness in responses, however, is a non-trivial task. Even for large language models (LLMs), the task of identifying and aggregating key evidence within a single hop presents a substantial challenge. This complexity arises because such evidence is scattered across multiple turns in a conversation, thus necessitating integration over multiple hops. Hence, our focus is to facilitate such multi-hop reasoning over a dialogue context, namely dialogue chain-of-thought (CoT) reasoning. To this end, we propose a knowledge distillation framework that leverages LLMs as unreliable teachers and selectively distills consistent and helpful rationales via alignment filters. We further present DOCTOR, a DialOgue Chain-of-ThOught Reasoner that provides reliable CoT rationales for response generation. We conduct extensive experiments to show that enhancing dialogue agents with high-quality rationales from DOCTOR significantly improves the quality of their responses.

pdf/dialogue-chain-of-thought-distillation-for-commonsense-aware-5deb69hc13.pdf

Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents

Knowledge graphs are large-scale semantic networks that considerably impact knowledge representation. Mining hidden knowledge from existing data, including triplet knowledge reasoning, is a primary objective of knowledge graphs. With the development of Neural Network (NN) and Deep Learning (DL), the interpretability of triplet knowledge reasoning gradually decreases; furthermore, what machines learn is not actual reasoning but digital reasoning shortcuts. To solve this problem, more background knowledge needs to be introduced into knowledge graphs: causal graphs can offer valuable causal logic knowledge for reasoning; temporal quadruples can provide essential temporal distribution details; and commonsense graphs can furnish pertinent commonsense understanding to support reasoning. In recent years, many scholars have incorporated additional background knowledge into knowledge graphs to construct more complex reasoning mechanisms. This paper reviews the basic concepts and definitions of knowledge reasoning and the reasoning methods used for knowledge graphs. Specifically, we dissect the reasoning methods into four categories: triplet reasoning, causal inference, temporal inference, and commonsense reasoning. Finally, we discuss the remaining challenges and research opportunities related to knowledge graph reasoning. 

Overview of Knowledge Reasoning for Knowledge Graph

Large Language Models (LLMs) has shown exceptional capabilities in many natual language understanding and generation tasks. However, the personalization issue still remains a much-coveted property, especially when it comes to the multiple sources involved in the dialogue system. To better plan and incorporate the use of multiple sources in generating personalized response, we firstly decompose it into three sub-tasks: Knowledge Source Selection, Knowledge Retrieval, and Response Generation. We then propose a novel Unified Multi-Source Retrieval-Augmented Generation system (UniMS-RAG) Specifically, we unify these three sub-tasks with different formulations into the same sequence-to-sequence paradigm during the training, to adaptively retrieve evidences and evaluate the relevance on-demand using special tokens, called acting tokens and evaluation tokens. Enabling language models to generate acting tokens facilitates interaction with various knowledge sources, allowing them to adapt their behavior to diverse task requirements. Meanwhile, evaluation tokens gauge the relevance score between the dialogue context and the retrieved evidence. In addition, we carefully design a self-refinement mechanism to iteratively refine the generated response considering 1) the consistency scores between the generated response and retrieved evidence; and 2) the relevance scores. Experiments on two personalized datasets (DuLeMon and KBP) show that UniMS-RAG achieves state-of-the-art performance on the knowledge source selection and response generation task with itself as a retriever in a unified manner. Extensive analyses and discussions are provided for shedding some new perspectives for personalized dialogue systems.

UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems

Incorporating external graph knowledge into neural chatbot models has been proven effective for enhancing dialogue generation. However, in conventional graph neural networks (GNNs), message passing on a graph is independent from text, resulting in the graph representation hidden space differing from that of the text. This training regime of existing models therefore leads to a semantic gap between graph knowledge and text. In this study, we propose a novel framework for knowledge graph enhanced dialogue generation. We dynamically construct a multi-hop knowledge graph with pseudo nodes to involve the language model in feature aggregation within the graph at all steps. To avoid the semantic biases caused by learning on vanilla subgraphs, the proposed framework applies hierarchical graph attention to aggregate graph features on pseudo nodes and then attains a global feature. Therefore, the framework can better utilise the heterogeneous features from both the post and external graph knowledge. Extensive experiments demonstrate that our framework outperforms state-of-the-art (SOTA) baselines on dialogue generation. Further analysis also shows that our representation learning framework can fill the semantic gap by coagulating representations of both text and graph knowledge. Moreover, the language model also learns how to better select knowledge triples for a more informative response via exploiting subgraph patterns within our feature aggregation process. Our code and resources are available at https://github.com/tangg555/SaBART.

pdf/enhancing-dialogue-generation-via-dynamic-graph-knowledge-nng4rn09.pdf

Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation

Knowledge-enhanced methods have bridged the gap between human beings and machines in generating dialogue responses. However, most previous works solely seek knowledge from a single source, and thus they often fail to obtain available knowledge because of the insufficient coverage of a single knowledge source. To this end, infusing knowledge from multiple sources becomes a trend. This paper proposes a novel approach Knowledge Source Aware Multi-Head Decoding, KSAM, to infuse multi-source knowledge into dialogue generation more efficiently. Rather than following the traditional single decoder paradigm, KSAM uses multiple independent source-aware decoder heads to alleviate three challenging problems in infusing multi-source knowledge, namely, the diversity among different knowledge sources, the indefinite knowledge alignment issue, and the insufficient flexibility/scalability in knowledge usage. Experiments on a Chinese multi-source knowledge-aligned dataset demonstrate the superior performance of KSAM against various competitive approaches.

KSAM: Infusing Multi-Source Knowledge into Dialogue Generation via Knowledge Source Aware Multi-Head Decoding

Traditional conversational systems can only access the given query during the response generation, leading to meaningless responses. To this end, researchers proposed to enhance dialogue generation by integrating external knowledge. Although such methods have achieved remarkable gains, the use of only single-source knowledge often makes existing knowledge-enhanced methods degenerate into traditional models in real scenarios because of the insufficient knowledge coverage of single-source knowledge. To improve the applicability of knowledge-enhanced methods, we propose two novel frameworks to use heterogeneous knowledge from multiple sources. We first propose an MHKD-Seq2Seq framework, which can use different heterogeneous knowledge by identifying abstract-level knowledge behaviors; meanwhile, a Diffuse-Aggregate scheme is used to process multiple knowledge simultaneously and produce a unified result. The next framework MHKD-ARPLM can leverage the advantages of pretrained language models with Knowledge Linearization techniques. In experiments, we collected dialogues from previously open-released datasets and built a multi-source knowledge-aligned dataset TriKE-Weibo, which involves three knowledge sources: commonsense, texts, and infobox tables. Extensive evaluations demonstrate the performance leadership of our approaches against competitive baseline models.

Improving the Applicability of Knowledge-Enhanced Dialogue Generation Systems by Using Heterogeneous Knowledge from Multiple Sources

In knowledge-grounded dialogue generation, pre-trained language models (PLMs) can be expected to deepen the fusing of dialogue context and knowledge because of their superior ability of semantic understanding. Unlike adopting the plain text knowledge, it is thorny to leverage the structural commonsense knowledge when using PLMs because most PLMs can only operate plain texts. Thus, linearizing commonsense knowledge facts into plan text is a compulsory trick. However, a dialogue is always aligned to a lot of retrieved fact candidates; as a result, the linearized text is always lengthy and then significantly increases the burden of using PLMs. To address this issue, we propose a novel two-stage framework SAKDP. In the first pre-screening stage, we use a ranking network PriorRanking to estimate the relevance of a retrieved knowledge fact. Thus, facts can be clustered into three sections of different priorities. As priority decreases, the relevance decreases, and the number of included facts increases. In the next dialogue generation stage, we use section-aware strategies to encode the linearized knowledge. The powerful but expensive PLM is only used for a few facts in the higher priority sections, reaching the performance-efficiency balance. Both the automatic and human evaluation demonstrate the superior performance of this work.

pdf/section-aware-commonsense-knowledge-grounded-dialogue-2mmzrj32.pdf

Section-Aware Commonsense Knowledge-Grounded Dialogue Generation with Pre-trained Language Model

This paper proposes a new method for measuring thickness data of surface compensation gasket and an algorithm for ellipse fitting and multiple ellipse fitting. First of all, the laser ranging sensor is used to scan the surface to obtain the data types of different specifications and to analyze the advantages and disadvantages of the data types. Secondly, the least square method is used to fit the data and analyze algorithm error. Finally, a new multi-ellipse fitting algorithm is proposed for determining the position of the fastening hole. It will facilitate the application of force sensors.

Profile Detection and Surface Fitting Based on the Key Optical Components of the Seeker

Dialogues systems endow machines with the ability to converse with humans using natural language. Nonetheless, previous Seq2Seq-based generative dialogue systems often generate safe but meaningless responses, such as ‘I don't know’ or ‘I think so’. To this end, researchers proposed to infuse external knowledge into dialogue generation, and such knowledge-enhanced methods have achieved remarkable improvements in the open-domain dialogue systems. External knowledge is an exogenous input, where the estrangement inevitably exists between knowledge and dialogue context. Although previous knowledge-enhanced works can already use commonsense knowledge to generate informative responses, they always use knowledge in a single-channel paradigm, which is hard to accurately handle different data-flows and then tends to generate irrational dialogue responses. Thus, they tend to be confused and generate strange responses when infusing the knowledge into dialogue generation, such as ‘I just ate a basketball,’ dramatically degrading the user experience. To address this problem, this paper proposes a novel Channel-Aware Knowledge Fusing Network (CAKF). Rather than following the traditional single-channel paradigm, CAKF employs three unique channels to handle different data-flows more clearly and rationally: a base channel serves like a vanilla Seq2Seq decoder; a context channel to utilize the contextual information, and a knowledge channel to infuse commonsense knowledge into the dialogue generation. Above such three channels, a Sequential Manager is built to maintain the global sequential decision state, aggregate the local data-flows, and make the final prediction. Experiments on two open-released datasets (a Chinese Weibo and an English Reddit) demonstrated the superior performance of this work against various state-of-the-art approaches.

Generating Rational Commonsense Knowledge-Aware Dialogue Responses With Channel-Aware Knowledge Fusing Network

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Si Wu

Author Tools

Chat about Author