TL;DR: This work proposes a broader hyper-parameter search space when fine-tuning for downstream tasks and releases LEGAL-BERT, a family of BERT models intended to assist legal NLP research, computational law, and legal technology applications.
Abstract: BERT has achieved impressive performance in several NLP tasks. However, there has been limited investigation on its adaptation guidelines in specialised domains. Here we focus on the legal domain, where we explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets. Our findings indicate that the previous guidelines for pre-training and fine-tuning, often blindly followed, do not always generalize well in the legal domain. Thus we propose a systematic investigation of the available strategies when applying BERT in specialised domains. These are: (a) use the original BERT out of the box, (b) adapt BERT by additional pre-training on domain-specific corpora, and (c) pre-train BERT from scratch on domain-specific corpora. We also propose a broader hyper-parameter search space when fine-tuning for downstream tasks and we release LEGAL-BERT, a family of BERT models intended to assist legal NLP research, computational law, and legal technology applications.
TL;DR: In this article, the authors explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets, and propose a broader hyper-parameter search space when fine-tuning for downstream tasks.
Abstract: BERT has achieved impressive performance in several NLP tasks. However, there has been limited investigation on its adaptation guidelines in specialised domains. Here we focus on the legal domain, where we explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets. Our findings indicate that the previous guidelines for pre-training and fine-tuning, often blindly followed, do not always generalize well in the legal domain. Thus we propose a systematic investigation of the available strategies when applying BERT in specialised domains. These are: (a) use the original BERT out of the box, (b) adapt BERT by additional pre-training on domain-specific corpora, and (c) pre-train BERT from scratch on domain-specific corpora. We also propose a broader hyper-parameter search space when fine-tuning for downstream tasks and we release LEGAL-BERT, a family of BERT models intended to assist legal NLP research, computational law, and legal technology applications.
TL;DR: The digitization of legal texts and advances in artificial intelligence, natural language processing, text mining, network analysis, and machine learning have led to new forms of legal analysis by using text mining and network analysis.
Abstract: The digitization of legal texts and advances in artificial intelligence, natural language processing, text mining, network analysis, and machine learning have led to new forms of legal analysis by ...
TL;DR: This paper proposes to apply theoretical and computational developments in the fields of deontology, deontic logic, and Natural Language Processing to the problem of regulatory compliance checking in construction to overcome the limitations of existing compliance checking efforts.
Abstract: Automated compliance checking of construction projects remains to be a challenge. Existing computer-supported compliance checking methods are mainly rule-checking systems (utilizing if-then-else logic statements) that assess building designs based on a set of well-defined criteria. However, laws and regulations are normally complex to interpret and implement; and thus if-then-else rule-checking does not provide the level of knowledge representation and reasoning that is needed to efficiently interpret applicable laws and regulations and check conformance of designs and operations to those interpretations. In this paper, we explore a new approach to automated regulatory compliance checking – we propose to apply theoretical and computational developments in the fields of deontology, deontic logic, and Natural Language Processing (NLP) to the problem of regulatory compliance checking in construction. Deontology is a theory of rights and obligations; and deontic logic is a branch of modal logic that deals with obligations, permissions, etc. The paper starts by discussing the need for automated compliance checking of construction operations and analyzing the limitations of existing compliance checking efforts in this regard. The paper, then, provides an overview of the proposed approach for automated compliance checking; and follows by an introduction of deontology and deontic logic and their applications in other domains (e.g. computational law). Finally, the paper presents the initial deontic modeling efforts towards automated compliance checking.
TL;DR: The roadmap of an iterative refinement-based, model-driven formal design methodology is introduced, not only to validate smart contracts but also to support the whole life cycle of their engineering.
Abstract: A smart contract is the algorithmic description of a contractual transaction protocol that is automatically executed together with the information provided by its parties. It is written in a simplified programming language that is specific to a particular domain. Not only correctness and unambiguity are its essential formal properties, but also conformance to any legislation governing the matter of the transaction. Finally, and importantly, the trustworthiness, safety and security of the platform executing the transactions are its main attributes. An emerging challenge is to define a proper engineering process to meet the demanding requirements while supporting mass production and distribution. This paper proposes the concept of smart contract engineering (SCE) to facilitate the generation of smart legal contracts, which is the combination of software engineering, formal methods and computational law. SCE aims to reduce the potential errors and improve efficiency during the contract development process, meanwhile promote the standardization of contract design methodologies. In this paper, the roadmap of an iterative refinement-based, model-driven formal design methodology is introduced, not only to validate smart contracts but also to support the whole life cycle of their engineering.