Abstract This paper proposes an approach to automate chemistry experiments using robots by translating natural language instructions into robot-executable plans, using large language models together with task and motion planning. Adding natural language interfaces to autonomous chemistry experiment systems lowers the barrier to using complicated robotics systems and increases utility for non-expert users, but translating natural language experiment descriptions from users into low-level robotics languages is nontrivial. Furthermore, while recent advances have used large language models to generate task plans, reliably executing those plans in the real world by an embodied agent remains challenging. To enable autonomous chemistry experiments and alleviate the workload of chemists, robots must interpret natural language commands, perceive the workspace, autonomously plan multi-step actions and motions, consider safety precautions, and interact with various laboratory equipment. Our approach, CLAIRify , combines automatic iterative prompting with program verification to ensure syntactically valid programs in a data-scarce domain-specific language that incorporates environmental constraints. The generated plan is executed through solving a constrained task and motion planning problem using PDDLStream solvers to prevent spillages of liquids as well as collisions in chemistry labs. We demonstrate the effectiveness of our approach in planning chemistry experiments, with plans successfully executed on a real robot using a repertoire of robot skills and lab tools. Specifically, we showcase the utility of our framework in pouring skills for various materials and two fundamental chemical experiments for materials synthesis: solubility and recrystallization. Further details about CLAIRify can be found at https://ac-rad.github.io/clairify/ . 

Large language models for chemistry robotics

ABSTRACT Introduction Interest in nanomedicines has surged in recent years due to the critical role they have played in the COVID-19 pandemic. Nanoformulations can turn promising therapeutic cargo into viable products through improvements in drug safety and efficacy profiles. However, the developmental pathway for such formulations is non-trivial and largely reliant on trial-and-error. Beyond the costly demands on time and resources, this traditional approach may stunt innovation. The emergence of automation, artificial intelligence (AI) and machine learning (ML) tools, which are currently underutilized in pharmaceutical formulation development, offers a promising direction for an improved path in the design of nanomedicines. Areas covered the potential of harnessing experimental automation and AI/ML to drive innovation in nanomedicine development. The discussion centers on the current challenges in drug formulation research and development, and the major advantages afforded through the application of data-driven methods. Expert opinion The development of integrated workflows based on automated experimentation and AI/ML may accelerate nanomedicine development. A crucial step in achieving this is the generation of high-quality, accessible datasets. Future efforts to make full use of these tools can ultimately contribute to the development of more innovative nanomedicines and improved clinical translation of formulations that rely on advanced drug delivery systems.

Re-envisioning the design of nanomedicines: harnessing automation and artificial intelligence

Transparent object perception is a rapidly developing research problem in artificial intelligence. The ability to perceive transparent objects enables robots to achieve higher levels of autonomy, unlocking new applications in various industries such as healthcare, services and manufacturing. Despite numerous datasets and perception methods being proposed in recent years, there is still a lack of in-depth understanding of these methods and the challenges in this field. To address this gap, this article provides a comprehensive survey of the platforms and recent advances for robotic perception of transparent objects. We highlight the main challenges and propose future directions of various transparent object perception tasks, i.e., segmentation, reconstruction, and pose estimation. We also discuss the limitations of existing datasets in diversity and complexity, and the benefits of employing multi-modal sensors, such as RGB-D cameras, thermal cameras, and polarised imaging, for transparent object perception. Furthermore, we identify perception challenges in complex and dynamic environments, as well as for objects with changeable geometries. Finally, we provide an interactive online platform to navigate each reference: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/view/transperception</uri> . 

pdf/robotic-perception-of-transparent-objects-a-review-3tyfi76y10.pdf

Robotic Perception of Transparent Objects: A Review

Sustainable elastomers derived from renewable biobased resources with excellent mechanical properties and varied functions are highly pursued to substitute traditional petroleum-based polymers yet challenging due to their limited macroscopic performance. In this work, we designed a series of wholly biobased cellulose-graft-poly(vanillin acrylate-co-tetrahydrofurfuryl acrylate) (Cell-g-P(VA-co-THFA) copolymer elastomers with cellulose as the rigid backbone, sustainable VA derived from lignin and soft THFA derived from hemicellulose as the hard and soft segments in the rubbery side chains. Moreover, the grafted side chains can be cross-linked to introduce an additional dynamic network structure via Schiff-base chemistry between the aldehyde and amino groups. The mechanical properties of Cell-g-P(VA-co-THFA) copolymer elastomers, including tensile strength, extensibility, elasticity, and toughness can be facilely manipulated by the VA/THFA feed ratio, cellulose content, and cross-linking density. These Cell-g-P(VA-co-THFA) copolymer elastomers are thermally stable and possess outstanding adhesion behavior and prominent UV-shielding performance. Besides dramatically enhanced mechanical properties, the cross-linked Cell-g-P(VA-co-THFA) counterparts exhibit remarkable shape memory behavior. This work provides a robust and convenient strategy for developing strong and versatile sustainable elastomers with different application demands by integrating different biomass feedstocks via elaborate molecular design. 

Wholly sustainable graft copolymers derived from cellulose, lignin, and hemicellulose for high-performance elastomers, adhesives, and UV-blocking materials

It is a challenging problem to infer objects with reasonable shapes and appearance from a single picture. Existing research often pays more attention to the structure of the point cloud generation network, while ignoring the feature extraction of 2D images and reducing the loss in the process of feature propagation in the network. In this paper, a single-stage and single-view 3D point cloud reconstruction network, 3D-SSRecNet, is proposed. The proposed 3D-SSRecNet is a simple single-stage network composed of a 2D image feature extraction network and a point cloud prediction network. The single-stage network structure can reduce the loss of the extracted 2D image features. The 2D image feature extraction network takes DetNet as the backbone. DetNet can extract more details from 2D images. In order to generate point clouds with better shape and appearance, in the point cloud prediction network, the exponential linear unit (ELU) is used as the activation function, and the joint function of chamfer distance (CD) and Earth mover’s distance (EMD) is used as the loss function of 3DSSRecNet. In order to verify the effectiveness of 3D-SSRecNet, we conducted a series of experiments on ShapeNet and Pix3D datasets. The experimental results measured by CD and EMD have shown that 3D-SSRecNet outperforms the state-of-the-art reconstruction methods.

/pdf/a-single-stage-and-single-view-3d-point-cloud-reconstruction-1ukv0q9q.pdf

A Single Stage and Single View 3D Point Cloud Reconstruction Network Based on DetNet

We present TransProteus, a dataset, and methods for predicting the 3D structure and properties of materials inside transparent vessels from a single image. Manipulating materials in containers is essential in...

Predicting 3D shapes, masks, and properties of materials inside transparent containers, using the TransProteus CGI dataset

Transparent object perception is a crucial skill for applications such as robot manipulation in household and laboratory settings. Existing methods utilize RGB-D or stereo inputs to handle a subset of perception tasks including depth and pose estimation. However transparent object perception remains to be an open problem. In this paper, we forgo the unreliable depth map from RGB-D sensors and extend the stereo based method. Our proposed method, MVTrans, is an end-to-end multi-view architecture with multiple perception capabilities, including depth estimation, segmentation, and pose estimation. Additionally, we establish a novel procedural photo-realistic dataset generation pipeline and create a large-scale transparent object detection dataset, Syn-TODD, which is suitable for training networks with all three modalities, RGB-D, stereo and multi-view RGB. https://ac-rad.github.io/MVTrans/

MVTrans: Multi-View Perception of Transparent Objects

Strong adhesives as structural load-bearing materials can provide both adhesive bonding to substrate surfaces and cohesive bonding throughout the bulk material, which are widely used in the cement nail, electronic device, and automotive industries. By a molecular topological regulation, in this work, we developed a series of strong adhesives based on comb-like polymers. By systematically regulating the topological parameters including the side chain length, the spacer between side chains, and the degree of polymerization of the overall backbone, comprehensive studies on linear rheology (the terminal relaxation time and zero viscosity) and non-linear debonding adhesion (the work of adhesion) are performed on these comb-like polymers. It was found that the rheological behaviors depend not only on the structure parameters but also on whether or not the samples exhibit phase separation. Moreover, regardless of the topological structure of comb-like polymers with strong and weak phase separation, the universal work of adhesion is observed to be dependent on the multiplication of terminal relaxation time and the debonding rate, indicating dynamic control over the nonlinear work of adhesion. Furthermore, comb-like polymers are feasible to bind with diverse substrates, including glass, polypropylene and polyethylene terephthalate, wood, and metal. In comparison to existing works using dynamic physical or chemical bonds, molecular topology regulation is more easily fulfilled in strong adhesives without the use of additives and complicated chemical and physical modifications, lowering the barrier for practical applications. 

Topology and Dynamic Regulations of Comb-like Polymers as Strong Adhesives

The basis of many object manipulation algorithms is RGB-D input. Yet,
commodity RGB-D sensors can only provide distorted depth maps for a wide range
of transparent objects due light refraction and absorption. To tackle the
perception challenges posed by transparent objects, we propose TranspareNet, a
joint point cloud and depth completion method, with the ability to complete the
depth of transparent objects in cluttered and complex scenes, even with
partially filled fluid contents within the vessels. To address the shortcomings
of existing transparent object data collection schemes in literature, we also
propose an automated dataset creation workflow that consists of
robot-controlled image collection and vision-based automatic annotation.
Through this automated workflow, we created Toronto Transparent Objects Depth
Dataset (TODD), which consists of nearly 15000 RGB-D images. Our experimental
evaluation demonstrates that TranspareNet outperforms existing state-of-the-art
depth completion methods on multiple datasets, including ClearGrasp, and that
it also handles cluttered scenes when trained on TODD. Code and dataset will be
released at https://www.pair.toronto.edu/TranspareNet/

Seeing Glass: Joint Point Cloud and Depth Completion for Transparent Objects.

We present TransProteus, a dataset, and methods for predicting the 3D structure, masks, and properties of materials, liquids, and objects inside transparent vessels from a single image without prior knowledge of the image source and camera parameters. Manipulating materials in transparent containers is essential in many fields and depends heavily on vision. This work supplies a new procedurally generated dataset consisting of 50k images of liquids and solid objects inside transparent containers. The image annotations include 3D models, material properties (color/transparency/roughness...), and segmentation masks for the vessel and its content. The synthetic (CGI) part of the dataset was procedurally generated using 13k different objects, 500 different environments (HDRI), and 1450 material textures (PBR) combined with simulated liquids and procedurally generated vessels. In addition, we supply 104 real-world images of objects inside transparent vessels with depth maps of both the vessel and its content. We propose a camera agnostic method that predicts 3D models from an image as an XYZ map. This allows the trained net to predict the 3D model as a map with XYZ coordinates per pixel without prior knowledge of the image source. To calculate the training loss, we use the distance between pairs of points inside the 3D model instead of the absolute XYZ coordinates. This makes the loss function translation invariant. We use this to predict 3D models of vessels and their content from a single image. Finally, we demonstrate a net that uses a single image to predict the material properties of the vessel content and surface.

Predicting 3D shapes, masks, and properties of materials, liquids, and objects inside transparent containers, using the TransProteus CGI dataset.

Popular Tasks used by Researchers

How can I help with your research?

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

We encountered an error while loading the panel. Please try refreshing the page.

Something went wrong

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

Anyone with the link can view this chat. You can switch it back to private anytime.

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Yi Ru Wang

Author Tools

Chat about Author