pdf/an-experimental-survey-on-big-data-frameworks-3e62epcwi0.pdf

An experimental survey on big data frameworks

We present BGPStream, an open-source software framework for the analysis of both historical and real-time Border Gateway Protocol (BGP) measurement data. Although BGP is a crucial operational component of the Internet infrastructure, and is the subject of research in the areas of Internet performance, security, topology, protocols, economics, etc., there is no efficient way of processing large amounts of distributed and/or live BGP measurement data. BGPStream fills this gap, enabling efficient investigation of events, rapid prototyping, and building complex tools and large-scale monitoring applications (e.g., detection of connectivity disruptions or BGP hijacking attacks). We discuss the goals and architecture of BGPStream. We apply the components of the framework to different scenarios, and we describe the development and deployment of complex services for global Internet monitoring that we built on top of it.

/pdf/bgpstream-a-software-framework-for-live-and-historical-bgp-1ma9lv4erx.pdf

BGPStream: A Software Framework for Live and Historical BGP Data Analysis

Administrators of most user-facing systems depend on periodic log data to get an idea of the health and status of production applications. Logs report information, which is crucial to diagnose the root cause of complex problems. In this paper, we present a real-time log analysis system called LogLens that automates the process of anomaly detection from logs with no (or minimal) target system knowledge and user specification. In LogLens, we employ unsupervised machine learning based techniques to discover patterns in application logs, and then leverage these patterns along with the real-time log parsing for designing advanced log analytics applications. Compared to the existing systems which are primarily limited to log indexing and search capabilities, LogLens presents an extensible system for supporting both stateless and stateful log analysis applications. Currently, LogLens is running at the core of a commercial log analysis solution handling millions of logs generated from the large-scale industrial environments and reported up to 12096x man-hours reduction in troubleshooting operational problems compared to the manual approach.

LogLens: A Real-Time Log Analysis System

Data quality is a challenging problem in many real world application domains. While a lot of attention has been given to detect anomalies for data at rest, detecting anomalies for streaming applications still largely remains an open problem. For applications involving several data streams, the challenge of detecting anomalies has become harder over time, as data can dynamically evolve in subtle ways following changes in the underlying infrastructure. In this paper, we describe and empirically evaluate an online anomaly detection pipeline that satisfies two key conditions: generality and scalability. Our technique works on numerical data as well as on categorical data and makes no assumption on the underlying data distributions. We implement two metrics, relative entropy and Pearson correlation, to dynamically detect anomalies. The two metrics we use provide an efficient and effective detection of anomalies over high velocity streams of events. In the following, we describe the design and implementation of our approach in a Big Data scenario using state-of-the-art streaming components. Specifically, we build on Kafka queues and Spark Streaming for realizing our approach while satisfying the generality and scalability requirements given above. We show how a combination of the two metrics we put forward can be applied to detect several types of anomalies — like infrastructure failures, hardware misconfiguration or user-driven anomalies — in large-scale telecommunication networks. We also discuss the merits and limitations of the resulting architecture and empirically evaluate its scalability on a real deployment over live streams capturing events from millions of mobile devices.

Online Anomaly Detection over Big Data Streams.

With user-facing apps adopting serverless computing, good latency performance of serverless platforms has become a strong fundamental requirement. However, it is difficult to achieve this on platforms today due to the design of their underlying control and data planes that are particularly ill-suited to short-lived functions with unpredictable arrival patterns. We present Atoll, a serverless platform, that overcomes the challenges via a ground-up redesign of the control and data planes. In Atoll, each app is associated with a latency deadline. Atoll achieves its per-app request latency goals by: (a) partitioning the cluster into (semi-global scheduler, worker pool) pairs, (b) performing deadline-aware scheduling and proactive sandbox allocation, and (c) using a load balancing layer to do sandbox-aware routing, and automatically scale the semi-global schedulers per app. Our results show that Atoll reduces missed deadlines by ~66x and tail latencies by ~3x compared to state-of-the-art alternatives.

Atoll: A Scalable Low-Latency Serverless Platform

Tomato cultivation forms the backbone of the vegetable production sector in most countries and is a vital constituent of the staple diet of a variety of populations across the globe. Although multiple factors affect production, plant diseases are the foremost among them and need to be tackled to preserve their worth. This is one of research area of agriculture. The authors seek to pinpoint this fundamental aspect affecting the growth of the plant emphasizing that early detection of plant diseases is necessary to confront their root causes at the initial stages thus, they have deployed a deep learning model constituting CNN and SVM which can recognize 7 prevalent diseases namely Bacterial Spot, Spider Mite, Target Spot, Early Blight, Late Blight, Bacterial Wilt plus Fusarium Wilt and categorize the images into 8 classes comprising of respective diseases as well as the healthy one. The model happened to be trained on a dataset containing 8,000 images of the considered classes and is tested on a test set having 2,000 images. A hybrid approach has been proposed via concatenation of Convolutional Neural Network (CNN) and Support Vector Machine (SVM). CNN is used for the extraction of utilitarian features from the input data and these are then classified using an optimized Support Vector Machine classifier and results are recorded in terms of accuracy. An accuracy of 92.6 % has been demonstrated by the suggested methodology and has thus performed proficiently.

Classification of Tomato Diseases using Hybrid Model (CNN-SVM)

Orange is an important citrus fruit grown globally, and its consumption is encouraged by health-conscious individuals due to its nutritional value. Classifying oranges is important for quality control, sorting, and grading in the food industry. For the production of high-quality oranges, farm-based disease prediction is not utilizing technology to its full potential. A hybrid version is proposed in this research paper for the categorization of six common disorders of oranges, namely Penicillium, Scab, Anthracnose, Melanose, Phytophthora, and Citrus Canker, using a blend of the classifier - Support Vector Machine and ANN prototype - Convolutional Neural Network. With CNN being accustomed for feature derivation and SVM being utilized for classification, the suggested model leverages the best aspects of both algorithms. Using a dataset of 4,864 orange photos, the suggested hybrid model’s performance is assessed, and as a result, an accuracy of 88.13734% is achieved. Our sensitivity analysis indicates that the form, size, and texture of the lesions were the most crucial characteristics for categorizing orange-colored illnesses, followed by their texture and color. The effectiveness of utilizing a hybrid model for illness diagnosis in citrus fruits is shown by the postulated hybrid model’s superior performance over existing classification models like SVM, Random Forest, and K-Nearest Neighbor (KNN). The impeccable competence of the proposed hybrid model makes it suitable to be employed in automated disease detection systems to make prompt and well-informed decisions about disease management and prevention, thereby enhancing citrus crop productivity and quality.

Efficient Detection and Classification of Orange Diseases using Hybrid CNN-SVM Model

Set up Apache Kafka clusters and develop custom message producers and consumers using practical, hands-on examples Overview Write custom producers and consumers with message partition techniques Integrate Kafka with Apache Hadoop and Storm for use cases such as processing streaming data Provide an overview of Kafka tools and other contributions that work with Kafka in areas such as logging, packaging, and so on In Detail Message publishing is a mechanism of connecting heterogeneous applications together with messages that are routed between them, for example by using a message broker like Apache Kafka. Such solutions deal with real-time volumes of information and route it to multiple consumers without letting information producers know who the final consumers are. Apache Kafka is a practical, hands-on guide providing you with a series of step-by-step practical implementations, which will help you take advantage of the real power behind Kafka, and give you a strong grounding for using it in your publisher-subscriber based architectures. Apache Kafka takes you through a number of clear, practical implementations that will help you to take advantage of the power of Apache Kafka, quickly and painlessly. You will learn everything you need to know for setting up Kafka clusters. This book explains how Kafka basic blocks like producers, brokers, and consumers actually work and fit together. You will then explore additional settings and configuration changes to achieve ever more complex goals. Finally you will learn how Kafka works with other tools like Hadoop, Storm, and so on. You will learn everything you need to know to work with Apache Kafka in the right format, as well as how to leverage its power of handling hundreds of megabytes of messages per second from multiple clients. What you will learn from this book Download and build Kafka Set up single as well as multi-node Kafka clusters and send messages Learn Kafka design internals and message compression Understand how replication works in Kafka Write Kafka message producers and consumers using the Kafka producer API Get an overview of consumer configurations Integrate Kafka with Apache Hadoop and Storm Use Kafka administration tools Approach The book will follow a step-by-step tutorial approach which will show the readers how to use Apache Kafka for messaging from scratch. Who this book is written for Apache Kafka is for readers with software development experience, but no prior exposure to Apache Kafka or similar technologies is assumed. This book is also for enterprise application developers and big data enthusiasts who have worked with other publisher-subscriber based systems and now want to explore Apache Kafka as a futuristic scalable solution.

Apache Kafka

In many regions around the globe, the agricultural business is heavily dependent on the cultivation and production of lemons. Unfortunately, a number of diseases, including Lemon Scab, Septoria Spot, Sooty Mould, Armillaria, and Huanglongbing can adversely influence the yield and quality of the fruit as well as the growth of lemon trees. In order to stop the spread of these diseases and lessen their effects, there is a huge requirement for precise and effective techniques of identification and classification. In order to classify various lemon disorders, this study suggests a hybrid model that incorporates Support Vector Machines (SVM) and Convolutional Neural Networks (CNN). The suggested model makes use of a dataset that includes pictures of both healthy and infected lemon leaves, including those with Armillaria, Septoria Spot, Sooty Mould, and Huanglongbing. To broaden the dataset's diversity and size, the photos were pre-processed utilizing data augmentation techniques. The suggested model makes the most of both techniques by combining the best features of CNN for feature extraction and SVM for classification. The recommended hybrid model's functionality is assessed using a dataset of 4,821 orange pictures, and as a result, an accuracy of 89.6% is attained. As it offers a precise and effective strategy for analyzing the various lemon diseases, this model is a useful tool for the early detection and control of the spread of various diseases in lemon plants thus, enhancing the yield and quality of lemon fruit along with the sustainability concerning the agricultural sector.

Lemon Diseases Detection and Classification using Hybrid CNN-SVM Model

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Nishant Garg

Author Tools

Chat about Author

Papers

Classification of Tomato Diseases using Hybrid Model (CNN-SVM)

Efficient Detection and Classification of Orange Diseases using Hybrid CNN-SVM Model

Apache Kafka

Lemon Diseases Detection and Classification using Hybrid CNN-SVM Model