Machine Learning (ML) has already fundamentally changed several businesses. More recently, it has also been profoundly impacting the computational science and engineering domains, like geoscience, climate science, and health science. In these domains, users need to perform comprehensive data analyses combining scientific data and ML models to provide for critical requirements, such as reproducibility, model explainability, and experiment data understanding. However, scientific ML is multidisciplinary, heterogeneous, and affected by the physical constraints of the domain, making such analyses even more challenging. In this work, we leverage workflow provenance techniques to build a holistic view to support the lifecycle of scientific ML. We contribute with (i) characterization of the lifecycle and taxonomy for data analyses; (ii) design principles to build this view, with a W3C PROV compliant data representation and a reference system architecture; and (iii) lessons learned after an evaluation in an Oil & Gas case using an HPC cluster with 393 nodes and 946 GPUs. The experiments show that the principles enable queries that integrate domain semantics with ML models while keeping low overhead (<1%), high scalability, and an order of magnitude of query acceleration under certain workloads against without our representation.

Workflow Provenance in the Lifecycle of Scientific Machine Learning

Context‐awareness techniques generally deal with steps for processing information that can be gathered from perspectives of objects in real‐world scenes in accordance with resolving heterogeneous devices and providing meaningful context information in recent computing domains such as Internet of Things, cloud computing, edge computing, and big data analysis. However, analysis applications are necessary to be improved in a way of cooperative approaches because they are independently executed far away from context‐aware systems. This means that the stages are applied to process the data or the derived results, are isolated. That leads difficulties of management and integration for other software modules. As a result, a computing paradigm called edge computing has been suggested to compromise such isolations. For processing context information, it is necessary to establish knowledge parts with validation processes against the isolations. In this paper, we suggest a method for processing datasets in a view of building knowledge parts. The suggested method uses an extensible data enrichment scheme, which represents the relations for the object. In the experiment section, applications of the proposed method are shown in one of the healthcare community centers, and the data abstraction with the analysis jobs can be processed for each real‐world object.

A method to generate context information sets from analysis results with a unified abstraction model based on an extension of data enrichment scheme

This paper showcases the Cycle Orchestrator, a microservices infrastructure designed to structure and manage workflows related to heterogeneous data, through a knowledge-based perspective. It aims at leveraging reasoning, explainability and collaboration among users over experiments that comprise workflow executions. We briefly discuss about design and implementation aspects to support the lifecycle of workflows (i.e., modeling, configuration, execution, provenance tracking and querying), exploring a holistic representation called Hyperknowledge that is amenable to be consumed and reasoned upon.

/pdf/a-knowledge-based-approach-for-structuring-cyclic-workflows-28t5bsg951.pdf

A Knowledge-Based Approach for Structuring Cyclic Workflows.

MAESTRO: a lightweight ontology-based framework for composing and analyzing script-based scientific experiments

This work introduces the Cycle Orchestrator, a microservices infrastructure to structure and manage workflows related to heterogeneous data from the O&G domain. Through a knowledge-based perspective, it leverages reasoning, explainability and collaboration among stakeholders.

/pdf/cycle-orchestrator-a-knowledge-based-approach-for-4axm9gvh63.pdf

Cycle Orchestrator: A Knowledge-Based Approach for Structuring Cyclic ML Pipelines in the O&G Industry.

Machine Learning Workflows (MLWfs) have become essential and a disruptive approach in problem-solving over several industries. However, the development process of MLWfs may be complicated, hard to achieve, time-consuming, and error-prone. To handle this problem, in this paper, we introduce machine learning workflow management (MLWfM) as a technique to aid the development and reuse of MLWfs and their components through three aspects: representation, execution, and creation. More precisely, we discuss our approach to structure the MLWfs' components and their metadata to aid retrieval and reuse of components in new MLWfs. Also, we consider the execution of these components within a tool. The hybrid knowledge representation, called Hyperknowledge, frames our methodology, supporting the three MLWfM's aspects. To validate our approach, we show a practical use case in the Oil & Gas industry.

Managing Machine Learning Workflow Components

/pdf/evaluating-semantic-queries-for-dataset-engineering-on-the-2uj94bmq3a.pdf

Evaluating Semantic Queries for Dataset Engineering on the Hyperknowledge Platform.

Machine Learning Workflows (MLWfs) have become an essential and disruptive approach in problem-solving over several industries. However, the development process of MLWfs may be complex, time-consum...

In this paper, we discuss the use of semantic technologies in the music industry. Specifically, we describe our vision of how the Hyperknowledge model can leverage the development of AI-based applications in this domain. Hyperknowledge is a multimodal-aware knowledge representation framework tailored for hypermedia information. It allows the modeler to describe and link fragments of multimedia content (e.g., audio, video, text, image, etc.) with high-level semantic structures. Hyperknowledge can also complement existing approaches by providing a glue model that can be used to map among distinct, but related, musical and audio material. We discuss the use of Hyperknowledge as a representation framework for music information and some novel applications that this approach can support.

Towards Leveraging the Music Industry with Hyperknowledge

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Polyana Costa

Author Tools

Chat about Author

Papers

Managing Machine Learning Workflow Components

Managing Machine Learning Workflow Components

Evaluating Semantic Queries for Dataset Engineering on the Hyperknowledge Platform.

Managing Machine Learning Workflow Components

Towards Leveraging the Music Industry with Hyperknowledge