https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Deep learning (DL) techniques have obtained remarkable achievements on various tasks, such as image recognition, object detection, and language modeling. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering its wide application. Meanwhile, automated machine learning (AutoML) is a promising solution for building a DL system without human assistance and is being extensively studied. This paper presents a comprehensive and up-to-date review of the state-of-the-art (SOTA) in AutoML. According to the DL pipeline, we introduce AutoML methods – covering data preparation, feature engineering, hyperparameter optimization, and neural architecture search (NAS) – with a particular focus on NAS, as it is currently a hot sub-topic of AutoML. We summarize the representative NAS algorithms’ performance on the CIFAR-10 and ImageNet datasets and further discuss the following subjects of NAS methods: one/two-stage NAS, one-shot NAS, joint hyperparameter and architecture optimization, and resource-aware NAS. Finally, we discuss some open problems related to the existing AutoML methods for future research.

AutoML: A survey of the state-of-the-art

Natural DNA can encode complexity on an enormous scale. Researchers are attempting to achieve the same representational efficiency in computers by implementing developmental encodings, i.e. encodings that map the genotype to the phenotype through a process of growth from a small starting point to a mature form. A major challenge in in this effort is to find the right level of abstraction of biological development to capture its essential properties without introducing unnecessary inefficiencies. In this paper, a novel abstraction of natural development, called Compositional Pattern Producing Networks (CPPNs), is proposed. Unlike currently accepted abstractions such as iterative rewrite systems and cellular growth simulations, CPPNs map to the phenotype without local interaction, that is, each individual component of the phenotype is determined independently of every other component. Results produced with CPPNs through interactive evolution of two-dimensional images show that such an encoding can nevertheless produce structural motifs often attributed to more conventional developmental abstractions, suggesting that local interaction may not be essential to the desirable properties of natural encoding in the way that is usually assumed.

/pdf/compositional-pattern-producing-networks-a-novel-abstraction-49f5ds9t48.pdf

Compositional pattern producing networks: A novel abstraction of development

We present grammatical evolution, an evolutionary algorithm that can evolve complete programs in an arbitrary language using a variable-length binary string. The binary genome determines which production rules in a Backus-Naur form grammar definition are used in a genotype-to-phenotype mapping process to a program. We demonstrate how expressions and programs of arbitrary complexity may be evolved and compare its performance to genetic programming.

Grammatical evolution

In CGP, programs are represented in the form of directed acyclic graphs. These graphs are represented as a two-dimensional grid of computational nodes. The genes that make up the genotype in CGP are integers that represent where a node gets its data, what operations the node performs on the data and where the output data required by the user is to be obtained. When the genotype is decoded, some nodes may be ignored. This happens when node outputs are not used in the calculation of output data. When this happens, we refer to the nodes and their genes as ‘non-coding’. We call the program that results from the decoding of a genotype a phenotype. The genotype in CGP has a fixed length. However, the size of the phenotype (in terms of the number of computational nodes) can be anything from zero nodes to the number of nodes defined in the genotype. A phenotype would have zero nodes if all the program outputs were directly connected to program inputs. A phenotype would have the same number of nodes as defined in the genotype when every node in the graph was required. The genotype–phenotype mapping used in CGP is one of its defining characteristics. The types of computational node functions used in CGP are decided by the user and are listed in a function look-up table. In CGP, each node in the directed graph

Cartesian Genetic Programming.

A review is given of approaches to growing neural networks and electronic circuits. A new method for growing graphs and circuits using a developmental process is discussed. The method is inspired by the view that the cell is the basic unit of biology. Programs that construct circuits are evolved to build a sequence of digital circuits at user specified iterations. The programs can be run for an arbitrary number of iterations so circuits of huge size could be created that could not be evolved. It is shown that the circuit building programs are capable of correctly predicting the next circuit in a sequence of larger even parity functions. The new method however finds building specific circuits more difficult than a non-developmental method.

/pdf/a-developmental-method-for-growing-graphs-and-circuits-28w0frrz74.pdf

A developmental method for growing graphs and circuits

In this paper we present a new chromosome representation for evolving digital circuits. The representation is based very closely on the chip architecture of the Xilinx 6216 FPGA. We examine the effectiveness of evolving circuit functionality by using randomly chosen examples taken from the truth table. We consider the merits of a cell architecture in which functional cells alternate with routing cells and compare this with an architecture in which any cell can implement a function or be merely used for routing signals. It is noteworthy that the presence of elitism significantly improves the Genetic Algorithm performance.

Aspects of Digital Evolution: Geometry and Learning

The use of genetic algorithms for the generation of optimal state assignments for synchronous finite state machines (FSM) is proposed. Results are presented to show that, in all examples attempted, the resulting state assignments are better than or at least as good as those produced by SPECTRAL, NOVA and MUSTANG and also closed partition assignments. On average, the genetic algorithm produced assignments with 33% less logic than the best produced by other algorithms.

State assignment of finite state machines using a genetic algorithm

Tabular techniques are described for the conversion between boolean expressions and Reed-Muller polynomials, and for the derivation of fixed polarities. The techniques are simple, systematic, and can be used manually or programmed on a computer. Further, they can be used for any number of variables and hence overcome map limitations. Computer programs have been developed to implement the algorithms.

Tabular techniques for Reed—Muller logic

This paper describes experiments to determine how the architecture vis-a-vis routing and functional resources affect the ease with which combinational logic designs may be evolved on a field-programmable gate array (FPGA). We compare two chromosome representations with differing levels of connectivity, and show that the amount of routing and functional resource have a marked effect on the success of the evolutionary process.

Aspects of Digital Evolution: Evolvability and Architecture

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

P. Thomson

Author Tools

Chat about Author