In big data analysis, various zero-inflated problems are occurring. In particular, the problem of inflated zeros has a great influence on text big data analysis. In general, the preprocessed data from text documents are a matrix consisting of the documents and terms for row and column, respectively. Each element of this matrix is an occurred frequency of term in a document. Most elements of the matrix are zeros, because the number of columns is much larger than the rows. This problem is a cause of decreasing model performance in text data analysis. To overcome this problem, we propose a method of zero-inflated text data analysis using generative adversarial networks (GAN) and statistical modeling. In this paper, we solve the zero-inflated problem using synthetic data generated from the original data with zero inflation. The main finding of our study is how to change zero values to the very small numeric values with random noise through the GAN. The generator and discriminator of the GAN learned the zero-inflated text data together and built a model that generates synthetic data that can replace the zero-inflated data. We conducted experiments and showed the results, using real and simulation data sets to verify the improved performance of our proposed method. In our experiments, we used five quantitative measures, prediction sum of squares, R-squared, log-likelihood, Akaike information criterion and Bayesian information criterion to evaluate the model’s performance between original and synthetic data sets. We found that all performances of our proposed method are better than the traditional methods.

Zero-Inflated Text Data Analysis using Generative Adversarial Networks and Statistical Modeling

For text big data analysis, we preprocessed text data and constructed a document–keyword matrix. The elements of this matrix represent the frequencies of keywords occurring in a document. The matrix has a zero-inflation problem because many elements are zero values. Also, in the process of preprocessing, the data size of the document–keyword matrix is reduced. However, various machine learning algorithms require a large amount of data, so to solve the problems of data shortage and zero inflation, we propose the use of generative models based on statistics and machine learning. In our experimental tests, we compared the performance of the models using simulation and practical data sets. Thus, we verified the validity and contribution of our research for keyword data analysis. 

Keyword Data Analysis Using Generative Models Based on Statistics and Machine Learning Algorithms

In contemporary times, science-based technologies are needed for launching innovative products and services in the market. As technology-based management strategies are gaining importance, associated patents need to be comprehensively studied. Previous studies have proposed predictive models based on patent factors. However, technology-based management strategies can influence the growth and decline of firms. Thus, this study aims to estimate uncertainties of the factors that are frequently used in technology-based studies. Furthermore, the importance of the factors may fluctuate over time. Therefore, we propose a Bayesian neural network model based on Flipout and four research hypotheses to evaluate the validity of our method. The proposed method not only estimates the uncertainties of the factors, but also predicts the future value of technologies. Our contribution is to (i) provide a tractable Bayesian neural network applicable to big data, (ii) discover factors that affect the value of technology, and (iii) present empirical evidence for the timeliness and objectivity of technology evaluation. In our experiments, 3781 healthcare-related cases of patents were used, and we found that the proposed hypotheses were all statistically significant. Therefore, we believe that reliable and stable technology-based management strategies can be established through our method.

/pdf/estimation-of-uncertainty-for-technology-evaluation-factors-dovt0xco.pdf

Estimation of Uncertainty for Technology Evaluation Factors via Bayesian Neural Networks

With the rapid development of computer technology, artificial intelligence algorithms have been widely used. Under the background of global low carbon development consensus, construction industry electrification has also become one of the future development trends. Its technical means include not only promoting the substitution and electrification of building electricity, but also promoting the development of building power distribution system to improve its flexibility, safety, reliability and efficiency. Adapt to the high proportion of renewable energy and different demands for energy services in the future. In addition, the building distribution network is no longer a simple user, but is introduced into the urban power grid to provide network support and support services directly beneficial to the power grid. Artificial intelligence is a branch of computer science. Artificial intelligence is of great significance not only in the field of computer, but also in the fields of robots, control systems and simulation systems. The application of artificial intelligence, including computer science, psychology, philosophy, linguistics and other disciplines, plays a very important role in the development of electrification. In the development process of electrification, it involves special contents in machine vision, robot and other related fields, so it is particularly important to introduce artificial intelligence algorithms. 

Research on Optimization of Indoor Optical Storage Technology Based on Artificial Intelligence Algorithm

구간치 퍼지집합은 Gorzalczang (1983)과 Turken(1986)에 의해 처음 제의되었다. 이를 토대로 Wang과 Li 는 구간치 퍼지수에 관한 연산으로 일반화 하여 연구하였다. 최근에 홍(2002)는 왕과 리의 이론을 리만적분에 의해 구간치 퍼지집합상의 거리측도에 관한 연구를 하였다 우리는 일반측도와 관련된 리만적분 대신에 퍼지측도와 관련된 쇼케이적분을 이용한 구간치 퍼지수 상의 쇼케이 거리측도를 연구하였다(2005). 본 논문에서는 퍼지수에서 퍼지수로의 쇼케이 거리측도를 정의하고 이와 관련된 성질들을 조사하였다.

A note on the Choquet distance measures for fuzzy number-valued fuzzy numbers

Cognitive artificial intelligence (CAI) is an intelligent machine that thinks and behaves similar to humans. CAI also has an ability to mimic human emotions. With the development of AI in various fields, the interest and demand for CAI are continuously increasing. Most of the current AI research focuses on the realization of intelligence that can make optimal decisions. Existing AI studies have not conducted in-depth research on human emotions and cognitive perspectives. However, in the future, the demand for the use of AI that can imitate human emotions in various fields, such as healthcare and education, will continue. Therefore, we propose a method to build CAI in this paper. We also use Bayesian inference and computing based on the hybrid Monte Carlo algorithm for CAI development. To show how the proposed method for CAI can be applied to practical problems, we create an experiment using simulation data.

/pdf/cognitive-artificial-intelligence-using-bayesian-computing-2z9817lg.pdf

Cognitive Artificial Intelligence Using Bayesian Computing Based on Hybrid Monte Carlo Algorithm

빅데이터는 다양한 산업군의 혁신을 이끌며, 많은 융복합 기술의 발전을 유도했다. 기업과 연구기관은 이러한 기술 시장에서 생존하기 위해 기술을 활용한 연구 개발 전략을 활용하고 있다. 특허는 기술 시장 분석을 위해 활용되는 빅데이터 중 하나이다. 그러나 특허의 수가 증가함에 따라 효율적인 분석이 어려워지고 있다. 본 연구의 목적은 가치가 높은 특허를 선별하고, 이를 설명할 수 있는 방법을 제안하는 것이다. 이를 통해, 특허 빅데이터 중 가치가 높은 특허를 통해 선행기술 조사나 공백기술 예측 등의 심층 분석이 가능하다. 따라서 우리는 특허의 가치를 예측할 수 있는 모델을 학습한 뒤, 이를 앙상블하여 예측 성능을 향상시키고자 한다. 그리고 특허의 가치 평가에 영향을 주는 변수를 추출하고, 시각화하고자 한다. 실험은 제안된 방법의 실제 적용 가능성을 검토하기 위해 에너지 자원 관련 미국 특허 11,423건을 수집하여 진행했다. 실험 결과, 신재생 에너지와 리튬 이온 배터리 전극 표면 개선을 위한 기술의 가치가 높을 것으로 추론하는 것이 가능했다. 

A Study on Inferable Patent Evaluation Model using Ensemble Method

Self Organizing Map(SOM) is a powerful neural network model for unsupervised learning. In many clustering works with exploratory data analysis, it has been popularly used. But it has a weakness which is the poorly theoretical base. A lot more researches for settling the problem have been published. Also, our paper proposes a method to overcome the drawback of SOM. As compared with the presented researches, our method has a different approach to solve the problem. So, a hybrid SOM is proposed in this paper. Using Monte Carlo computing, a hybrid SOM improves the performance of clustering. We verify the improved performance of a hybrid SOM according to the experimental results using UCI machine learning repository. In addition to, the number of clusters is determined by our hybrid SOM.

Hybrid Self Organizing Map using Monte Carlo Computing

Many parts of big data, such as web documents, online posts, papers, patents, and articles, are in text form. So, the analysis of text data in the big data domain is an important task. Many methods based on statistics or machine learning algorithms have been studied for text data analysis. Most of them were analytical methods based on the generalized linear model (GLM). For the GLM, text data analysis is performed based on the assumption of the error included in the given data and follows the Gaussian distribution. However, the GLM has shown limitations in the analysis of text data, including data sparseness. This is because the preprocessed text data has a zero-inflated problem. To solve this problem, we proposed a text data analysis using the generalized linear mixed model (GLMM) and Bayesian visualization. Therefore, the objective of our study is to propose the use of GLMM to overcome the limitations of the conventional GLM in the analysis of text data with a zero-inflated problem. The GLMM uses various probability distributions as well as Gaussian for error terms and considers the difference between observations by clustering. We also use Bayesian visualization to find meaningful associations between keywords. Lastly, we carried out the analysis of text data searched from real domains and provided the analytical results to show the performance and validity of our proposed method.

/pdf/text-data-analysis-using-generalized-linear-mixed-model-and-23hdwbvq.pdf

Text Data Analysis Using Generalized Linear Mixed Model and Bayesian Visualization

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Sung-Hee Jun

Author Tools

Chat about Author