Abstract: We give an example of a neural net without hidden layers and with a sigmoid transfer function, together with a training set of binary vectors, for which the sum of the squared errors, regarded as a function of the weights, has a local minimum which is not a global minimum. The example consists of a set of 125 training instances, with four weights and a threshold to be learnt. We do not know if substantially smaller binary examples exist.

Abstract: Culik and Yu suggested a classification of cellular a u­ tomata into four classes based on Wolfram's earlier heuristic classifi­ cation. The purpose of this note is to determine the position of these classes within th e arithmetical hierarchy. We will show t hat CLASS ONE and C LASS T wo are rrg-complete whereas C LASS THREE is ~g­ complete . C LASS FOUR is tri vial.

Abstract: We exhibit a class of symmetric neural networks which synchronous iteration possesses an exponen tial transient length. In fact if {I , ... , n} is the set of nodes we prove the transient length satisfies T 2: 2 n / 3 . For sequential up dating we get the bound f 2: 2 n / 6 • Thi s behavior shows t hat the dynami cs of these class of network s is complex while the steady states are simple: only fixed points or orbits of period 2.

Abstract: We reformulate the problem of supervised learning in neu­ ral nets to include the search for a network with minimal resources . The information processing in feedforward networks is described in geometrical terms as the partitioning of the space of possible input configurations by hyperplanes corresponding to hidden units. Regu­ lar partitionings introduced here are a special class of partitionings. Corresponding architectures can represent any Boolean function using a single layer of hidden units whose number depends on the specific symmetries of the function. Accordin gly, a new class of pla ne-cutting algorithms is proposed that const ruct in polynomial time a "custom­ made" architecture implementing the desired set of inputj'ouput ex­ amples . We report the results of our experiments on the storage and rule-extraction abilities of three-layer perceptrons synthetized by a simple greedy algorithm. As expected, simple neuronal structures with good generalization properties emerge only for a strongly corre ­ lated set of examples.

Abstract: A bst ract. A one-dimensional cellular automaton rule with specified boundary conditions can be considered as acting simultaneously on all finite lattices, which gives a mapping between formal languages. Regular lang uages are always mapped to regular langu ages, contex t-free to context-free, context-sensitive to context-sensitive, and recursive sets to recursive sets . In particular, the finite time sets on finite lattices are regular languages. The limit set on finite lattices (t he periodic set) is shown to be neither a regular nor an unambiguous context-free language for certain additive rules with chaotic behavior, and for rules that can simulate one of these additive rules t hrough a finit e blocking transformation. The relation between cellular automata on finite and infinite lattices is discussed .

Abstract: With the emergence of neural network architectures, combinatorial optimization problems and NP-complete problems may be tackled with a new attention combining biology, physics and data processing. This paper deals with one of these problems: the graph K-partitioning. After a brief critical review of the conventional methods, we show how a particular vectorial encoding associated with this problem produces original neural network methods. Through different graph families, a comparative analysis of our approaches with one of the best conventional algorithms is developed.

Abstract: The Hedlund-Richardsod Theorem states t ha t a global mapping from configuration space to itself can be realized by a Eu­ clidean cellular automaton if and only if it takes the quiescent configu­ ration to itself , commutes with shifts, and is continuous in the product topology. An analogous theorem characte rizing t he realizability of self­ mappings of finite or infinit e configuration space via neural networks is established. It follows that, under natural hypotheses, a uniform limit of global dynamics is a global dynamics . We also give sufficient conditions for the global dynamics of a neural network to be realized by a cellular automaton.

Abstract: A simple method for finding conformational substates of proteins is presented and realized through computer simulations. It is based on a procedure in which the amino acids in a protein take the places of the cities in the three-dimensional travelling salesman prob lem. Optimization by simulated annealing was employed in the computer simulations to obtain conformational substates originating from a given three-dimensional structure of the protein backbone. Two polypeptides, Avian Pancreatic Polypeptide and Leucin-Enkephalin, were modelled and compared with available x-ray diffraction data. The method gives an interesting spinoff: the possibility of assigning a measure of complexity to real protein st ructures, due to the fact that a metric on the set of interactions employed by the protein can be defined naturally.

Abstract: A cellular automaton model for the simulation of fluid flow in porous media is presented. A lattice and a set of rules are int roduced, such th at the flow equations in the continuum limit are formally the same as the equations for one-phase liquid flow in porous media. The model is valid in two as well as th ree dimensions. Numerical calculatio ns of some simple problems are presented and compared with known analytical results. Agreement is within estim ated error s. 1. Introdu ction It is well known that the Navier-Stokes equation can be deri ved by two alternative methods, which may be called macroscopic and microscopic. The macroscopic me thod builds on the hypothesis that fluid s are structureless continua and uses conservation laws of general validity [6]. The viscosity coefficients are introduced as constants to be determined by experiments. T he microscopic approach, on the other hand, is based on the molecular structure of fluids and uses the framework of statistical physics [8]. The viscosity is here calcu lable in terms of the intermolecular potent ial. T here is an analogous division in the theory of flow in porous media and the Darcy equation . The macroscopic point of view st arts with Darcy 's law, which states that the rate of one-dimensional flow is proportional to the pressure gradient, and generalizes this law by introducing a permeabili ty te nsor. T he components of this tensor are to be determined by experiments. T he microscopic me thod is represented by an extensive literature (see for example [9] and refere nces given there) where it is shown that the Darcy equat ion ari ses from an averaging ou t of Navier-Stokes flow by the pores and that the permeability te nsor is in principle deducible from the pore geome try. The length scale is here microscopic only as far as the porous me dium is concerned: the pores are visible but the fluid is struct ureless . The recent paper by Rothman [7] on the cellular automaton simulation of flow in po rous media is in t he microscopic tradition. Using the FHP [3] triangular-lattice gas it shows how to estimate the permeability of almost © 1989 Comp lex Systems Publications , Inc.

Abstract: A formal mathematical presentation of various algebraic properties of rule 90 elementary one-dimensional cellular automata (CA) with null boundary conditions is given. The CA global rule transition matrix is given and its characteristic polynomial is formally obtained. Mathematical relationships between the CA register lengths and the orders of the corresponding group or semigroup algebraic structures are derived.

Abstract: New algorithms for deciding the injectivit y of the global update funct ion associat ed with a cellular automaton C A of dimension one are presented. This is done by interpreting each ordered pair dete rmined by the local update function as th e edge of a labeled directed graph GR which has the prope rty that each bi-infinite sequence of states of the cellular automaton is th e sequence of input labels of one and only one bi-infinite path in the graph . For an appropriate conversion of GR into a finite automaton, injectivity of the global update function of CA on the set of pseudofinit e sequences is equivalent to the unambiguity of this automaton. For appropria te conversions of GR into a finite set of finite automata, the injectivity of the global update function on all sequences is equivalent to the condition that every automaton in the finit e set be unambiguous.

Abstract: The original lattice gas automaton model requires a density-dependent rescaling of time, viscosity, and pressure in order to obtain the Navier-Stokes equation. Also, the corresponding equationof-state contains an unphysical velocity dependence. We show that an extension of this model which includ es six additional par ticles with a new speed overcomes both problems to a large extent. The new model considerably extends the ran ge of allowed Reynolds numb er.

Abstract: Powerful new training algori thms developed for ar tificial neural networks hold th e promise of identifying regulari ties in the t raining data and generalizing over the test data. T he backprop agation algorithm is one such training algorithm t ha t, with th e use of hidden units, can learn functions such as exclusive-or. These fun ctions can be learned by statis tical techniques such as multiple-linear regression only by introducing additional parameters . We report experimental comparisons of th e performance of backpropagation, multiplelinear regression, and an expert system. We conclude that, for the data studied here, backpropagation is unsuitable for discovering st atist ical relationships. It may be possible to customize neural-net algorithms for niche applicatio ns in discovery syst ems.

Abstract: We study the dynamical behavior of automata networks defined by x(t + 1) = x(t) + f(Ax(t) + b); where A is a symmetric n x n matrix, b is a real n-vector and f is the subgradient of a con­ vex function . More precisely we prove, by using Lyapunov operators associated to the network, that the steady state behavior of these au ­ tomata is simple: fixed points or two-cycles . We also give bounds for the transient time needed to reach the steady state. These networks appear in applications such as image restauration or phase unwrap­ ping (6). For this last application, we give bounds for the transient length.

Abstract: A new learning algorithm, lear ning by choice of int ern al repr esentations (CHIR), was recently int roduc ed. Th e basic version of thi s algorit hs was developed for a two-layer, single-out put, feed­ forward network of binary neurons . This paper presents a gener alized version of t he CHIR algorithm th at is capable of training mult iple­ output net works. A way to ada pt the algorit hm to mult ilayered feed­ forward networks is also presented. 'v Ve test t he new version on two typical learnin g tas ks: t he combined parity- symm et ry problem an d t he rand om problem (random associat ions). The dependence of the algori th m performance on the network size and on t he learning pa­ ramet ers is st udied.

Abstract: A simple method based on the lattice Boltzmann equatio n is presented for the evaluati on of the velocity profile of fluid flows near walls or in the vicinity of the interface between two fluids. The met hod is applied to fluid flow near a wall, to channel flow, and to the transition zone between two fluids flowing parallel to each other in opposite direct ions. The results show good agreement with micrody namical lattice gas simulati ons and with classical fluid dynamics. 1. The lattice boundary layer p roblem Since the pioneer ing work by Hardy, Pomeau, and de P azzis in 1973 [1,2]' Wolfr am in 1983 [3J, and mostly since the recent int rodu ction of the hexagonal lattice gas by Frisch, Hass lacher, and Pomeau [4], lat t ice gas methods have evolved bo th in efficiency and complexity (an extensive introduction to the subject can be found in [5]). T he theor eti cal and computational developme nt of the field has been so exte nsive in the last coup le of years that it has given rise to applications in var ious areas of physics [6J . Lat ti ce gases share common operational features with cellular automata and so ar e mos t easi ly implemented on parallel machines, in particular for fluid dynamical pro blems at large Reynolds nu mb ers [7J which require high computat ional performances. On the ot her hand, there exists a variety of operationally simple problems of valuable physica l interest that can be solved wit h modest computational means for which small computers provid e sufficient power . For the class of pr oblems considered here, t he lat t ice gas flow description can be reduced to a "one-dimens ional" formula t ion; therefore such problems can be solved wit h low power computational techniques. @ 1989 Complex Systems Publications, Inc. 318 Lattice Boltzmann Equation for Laminar Boundary Flow The formation and growth of boundary layers is of cru cial importance in fluid dynamical flows, in particular as t heir occurrence triggers the development of turbulence at high Reynolds numbers . For viscous flow (at low Rey nolds number) , boundary layer pr oblems can be solved within the limi ts of reasonable approximations . Such problems so appear as an interesting test for the validity of the lat tice gas method and their solutions are a prerequisite to the understanding of mor e complex flows and of the three-dimensionalization in the transitio n to tur bulence. The pur pose of the present work is to show that laminar boundary flow can be treated efficient ly, that is, simply and economically, by the lattice gas method . The basic idea is the following: consider that a lattice gas, initially in homogeneous un idirectional motion, is suddenly put in contact with a wall. All lattice gas nodes in any layer parallel to the flow direct ion have the same particle distribution and, the system being translationally invariant, it suffices to perform one-dimensional computat ion to evaluate the velocity profil e. T he wall effects on the flow velocity are propagat ed by the particl es at the microscopic "t hermal velocity," whereas t he flow profil e modifi cations pr opagate via particle interactions, i.e. , at much lower speed. Interactions with the wall will first be felt on the first layer of the gas (i.e., the layer adjacent to the wall); at the next time st ep, they will be felt on the first and second layers , and progressively the successive lattice layers will be interactively involved. More pr ecisely, we consider a gas (density d) flowing parallel to a wall wit h free flow velocity Uo. Momentum is first exchanged between the wall and the first layer: the flow is slowed down in that layer due to velocity reversal of the particles colliding with the wall. T he first layer will come to a state of local equilibrium acquiring velocity U parallel to the wall with U jp,\ ---...'* Trip le collis ion I ~-< H\

Abstract: It is found numerically that a two-dimensional left-turning particle-scatterer system does not diffusefor scatterer densities slightly above one-half, or greater. For smaller densities, the diffusion coefficient is much lower than what the Boltzmann approximation predicts; this is caused by orbiting events. An isotropic-scattering model with reflective impurities also shows deviations from the diffusion equation for various densities of isotropic and reflective scatterers, caused by retracing events.

Abstract: One way to evolve new artificial lifeforms is to fuse existing forms in var ious ways . If one wishes for a new lifeform 's program to be the same size as each of it s parent pr ogram s, then one might take half of each parent program and pat ch the two halves together . T his method is typical of sexual repr oduction, and of the genet ic programming approach . A different meth od of crossbreeding two of three lifeforms is to write a double or t riple length program which includes the full code of the original programs and which provides an arena in which the programs can interact . This might be called a symbiot ic programming approach . In this pap er I will descri be som e new and lively cellular au tomaton rul es whi ch I have found through symbiot ic programming on the CAM-6 cellul ar automaton m achine [1,4].

Abstract: We study the learning and generalization capacity of layered feed-forward neural networks in the context of mappings of arbitrarily long bit strings into one of three ordered outputs. Many signal-processing applications reduce to this problem. We use the back-propagation learning algorithm to train the network. We describe these mappings in terms of collections of linear partitions of the input space and the state spaces of the hidden layers of neurons. This description accounts for the properties of the mappings and suggests that learning and generalization are enhanced by training with boundary points of the input space. Several examples are included. We close with implications for layered feed-forward networks in general.

Abstract: We study properties of operators defined on the space E+(p) of right half-infinite sequences with entries chosen from Zp where p is prime. The operators in question allow solution of the problem of finding predecessor states for certain cellular automata evolutions and they can be thought of as discrete integration with respect to sequence index. These operators are self-accumulating, not solipsistic, and have no dense orbits. In addition, they exhibit a period-multiplying property. Many of these results are derived from properties of Pascal's triangle modulo p which are presented in an appendix.

Abstract: The Boltzmann value of the diffusion coefficient in Gates's latti ce wind-tree model reported in reference [1] is corrected. The new expression is in agreement with low-density simulations. The low-density value of the diffusion coefficient calcula ted in reference [1] for a model of Gates [2J is incorrect . Here we obtain the correct expres sion, which agrees well in the relevant low-den sity regime with the molecular dynamics measurement of reference [IJ and wit h the new simulation data reported in table 1. T he model under con sid eration is a la ttice vers ion of t he Ehre nfest windt ree model, in which t rees are leftturning scatterers placed randomly at a fraction c of the sites of a square lat ti ce [2J. Let pi(n , t) represent the probabilit y of finding the wind particle a t time t at site n = {nx,ny} with arrival ( \"precollisional\") velocity e. , i = 1, 2, 3, 4 c D (Boltzmann) D (t = 128) D (t = 512) 0.1 2.25 2.24 2.22 0.2 1.00 1.00 0.94 0.3 0.58 0.58 0.55 0.4 0.38 0.35 0.32 0.5 0.25 0.21 0.18 Table 1: Scat terer concentration, D~x from equat ion (4) and diffusion coefficient (slope of th e meansquared displacement versus time) measured at 128 and 512 time steps. © 1989 Complex Syst ems P ublicat ions, Inc. 314 B. Th. Bouwens, M.H. Ernst, and P.M. Binder (mod 4) , equal to one of the nearest-neighbor lattice vectors. A configuration of scatterers is den ot ed by th e set of quenched random variables {cn }, where Cn takes the valu es 0 or 1 if site n is empty or occupi ed by a tree, resp ecti vely. T he Liouville equ at ion for the deterministic lattice Lorentz gas , intro du ced in reference [1J, is Pi(n + ei, t + 1) = (1 Cn)Pi(n ,t) +CnPi-l(n, t) (1.1) In the Boltzmann ap pro ximation all collisions in which the moving particle returns to a previously visited scatterer are neglect ed. In this approximat ion the average over the configurations of scatterers {Cn} can be direct ly performed by replacing (cn ) by its averag e value c. T his yields the model's Boltzmann equat ion. It reads, in the notation of reference [3], Pi(n + e., t + 1) = [(1 + cT)p(n , t)Ji (1.2 ) where Tij is a 4 x 4 collision matrix, formally written as T = b 1, with bpi = Pi-I . We note that , contrary to th e statement in reference [1] the giral scattering rules do not possess the full symmetry of the squ ar e lattice. T he diffusion tensor Det{3> where 0 , j3 denote cartesian components (x ,y) , is given by the long-time behavior of ~llt(ret(t)r,6(t )), where llta(t) = a(t + 1) a(t) is a forward t ime difference. It is therefore symmetric in 0 and j3 (Onsager symmetry) , and given by the Green-Kubo formula, 1 00 Det,6 = 2' L [cpet,6(T ) + CP,6et(T) CPet,6(O) ] 7=0 (1.3) where CP et,6 = (vet(t)v,6(t)) is the velocity correlation function . According to reference [3] it is given in the Boltzmann approximat ion by the symmetric part of ~ \"e (cT):-le ',6 ~8 ,6 40 r o 'J J 4 c t (1.4) T he term involving ~CPet,6(O) = ~8et,6, is the \"propagation diffusion\" resulting from the discret e structure of space and time [4]. As it is of re lative order c, it was neglected in the low density limit of reference [3J. To evaluate equation (1.4) we calculate the rele vant eigenvectors and eigenvalues of Iij , defined through cTv = AV . The eigenvectors are vJ = ejx ± iejy, with corresponding eigenvalues A± = c(l ±i). Inserting these results in equ at ion (1.4) yields for the diffus ion tensor in Gates's model

Abstract: Time is at the heart of many pattern recognition tasks (e.g., speech recognition). However, connectionist learning algorithms to date are not well-suited for dealing with time-varying input patterns. This chapter introduces a specialized connectionist architecture and corresponding specialization of the back-propagation learning algorithm that operates efficiently, both in computational time and space requirements, on temporal sequences. The key feature of the architecture is a layer of selfconnected hidden units that integrate their current value with the new input at each time step to construct a static representation of the temporal input sequence. This architecture avoids two deficiencies found in the back-propagation unfolding-intime procedure (Rumelhart, Hinton, & Williams, 1986) for handing sequence recognition tasks: first, it reduces the difficulty of temporal credit assignment by focusing the back-propagated error signal; second, it eliminates the need for a buffer to hold the input sequence and/or intermediate activity levels. The latter property is due to the fact that during the forward (activation) phase, incremental activity traces can be locally computed that hold all information necessary for back propagation in time. It is argued that this architecture should scale better than conventional recurrent architectures with respect to sequence length. The architecture has been used to implement a temporal version of Rumelhart and McClelland's (1986) verb past-tense model. The hidden units learn to behave something like Rumelhart and McClelland's \"Wickelphones,\" a rich and flexible representation of temporal information.

Abstract: This paper investigates the application of Walsh functions to the analysis of genetic algorithms operating on different codingfunct ion combinations. Although these analysis tools have been in existence for some time, they have not been widely used. To promote their understanding and use, this paper introduces Bethke's Walshschema transform through the Walsh polynomials. This form of the method provides an intuitive basis for visualizing the nonlinearities being considered, thereby permitting the consideration of a number of useful extensions to the theory in Part II.

Abstract: Part I considered the application of Walsh functions to the analysis of genetic algorithms operating on different coding-function combinations. In this paper, those meth ods are extended to permit rigorous analysis of deception by considering the expect ed disruption to schema processing caused by different genetic operators. Algebraic extensions of these metho ds are considered, and a sensitivit y analysis is described.

Abstract: We prove that the problem of counting the number of stable states in a given Hopfield net is #P-complete and the problem of computing the size of the attraction domain of a given stable state is NP-hard.