A Transformer-Convolutional Neural Network Based Framework for Predicting Ionic Liquid Properties
13 Sep 2022
2
TL;DR: In this paper , molecular fingerprints are derived from the encoder state of a Transformer model pre-trained on the PubChem database, which allows transfer learning from large-scale unlabeled data and significantly improves generalization performance for developing models with small datasets.
read more
Abstract: Of central importance to evaluate the suitability of ionic liquids (ILs) for a process is the accurate estimation of IL properties related to target performances. In this work, a versatile deep learning method for predicting IL properties is developed. Molecular fingerprints are derived from the encoder state of a Transformer model pre-trained on the PubChem database, which allows transfer learning from large-scale unlabeled data and significantly improves generalization performance for developing models with small datasets. Employing the pre-trained molecular fingerprints, convolutional neural network (CNN) models for IL properties prediction are trained and tested on 11 databases. The obtained Transformer-CNN models present superior performance to state-of-the-art models in all cases and enable property prediction of millions of ILs shortly. The application of the proposed models is exemplified by searching CO2 absorbent from a huge database of 8,333,096 synthetically feasible ILs, which is by far the most high-throughput IL screening in literature.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Designing Deep Eutectic Solvents for Efficient CO2 Capture: A Data-driven Screening Approach
TL;DR: Researchers design novel deep eutectic solvents for efficient CO2 capture using a data-driven screening approach, combining machine learning models with thermodynamic calculations to identify 1447 promising candidates with high CO2 absorption ability and low melting temperatures.
23
Predictive modeling of physicochemical properties and ionicity of ionic liquids for virtual screening of novel electrolytes
TL;DR: This study develops three regression models for predicting ionic liquid properties (conductivity, viscosity, density) and a classification model for ionicity, enabling virtual screening of novel electrolytes with desired properties through machine learning and cross-validation protocols.
References
Attention Is All You Need
Ashish Vaswani,Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,Łukasz Kaiser,Illia Polosukhin +7 more
- 01 Jan 2017
Abstract: The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.
51.8K
•Proceedings Article
Sequence to Sequence Learning with Neural Networks
Ilya Sutskever,Oriol Vinyals,Quoc V. Le +2 more
- 08 Dec 2014
TL;DR: The authors used a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector.
Convolutional Neural Networks for Sentence Classification
Yoon Kim
- 25 Aug 2014
TL;DR: The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification, and are proposed to allow for the use of both task-specific and static vectors.
On the Properties of Neural Machine Translation: Encoder--Decoder Approaches
Kyunghyun Cho,Bart van Merriënboer,Dzmitry Bahdanau,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio +5 more
- 03 Sep 2014
TL;DR: In this paper, a gated recursive convolutional neural network (GRNN) was proposed to learn a grammatical structure of a sentence automatically, which performed well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase.
6.9K
SMILES, a chemical language and information system. 1. introduction to methodology and encoding rules
TL;DR: This chapter discusses the construction of Benzenoid and Coronoid Hydrocarbons through the stages of enumeration, classification, and topological properties in a number of computers used for this purpose.
6.7K