1. How does machine learning approximate natural phenomena?
Machine learning approximates natural phenomena by using multi-unit multilayer neural networks, as stated in the universal approximation theorem. The activation function, such as step function or ReLU, can be identified with approximation by a piecewise constant or piecewise linear function. This approach aligns with the ancient Greek method of approximating rotationally symmetric objects like circles and spheres with piecewise linear functions. By rediscovering discrete geometry, researchers can develop methods for discretizing smooth surfaces, which have applications in computer graphics and quantum physics. In this study, a sphere is approximated by a neural network function, generating polygons and polytopes when using ReLU activation function. Other activation functions can lead to infinite families of generalization of polytopes, termed neural polytopes.
read more
2. How is a plane in Euclidean d-dimensional space parameterized?
A plane in the Euclidean d-dimensional space is parameterized as EQUATION, where a_i (i = 1, ..., d) are real constant parameters. This equation represents the plane in the d-dimensional space spanned by the coordinate (x_1, ..., x_d). Polyhedra are a generalization of this equation to a piecewise linear function. It is important to note that the right-hand side of the equation needs to be fixed to unity, otherwise, an affine quotient is required. Deep neural networks with ReLU activation function without bias can be represented in the form of the left-hand side of the equation. A deep neural network architecture with N intermediary fully-connected layers and ReLU activation function is used. The input layer consists of d units, and the output layer is a summation layer that sums the values of the n_N units at the last intermediate layer. The training data is prepared by generating random points on the (d-1)-sphere in Cartesian coordinates. The activation function ph(x) = |x|^p, with a positive real constant p, gives geometrically symmetric neural network functions. The paper focuses on the results with p = 2 for a better symmetric approximation of spheres. The training involves producing roughly 10000 random points on the sphere, using the ADAM optimizer with a batch size of 1000, and 1000 epochs. The cross section defined by EQUATION, where f(x_i) is the trained neural network function, is called 'neural polytopes.' These polytopes are named as d-polytopes of type (n_1, ..., n_N; p_1, ..., p_N), where d is the spatial dimension of the minimal Euclidean space in which the polytope is embedded.
read more
3. How do neural networks approximate regular polygons?
Neural networks can approximate regular polygons by establishing a map between the network architecture and the polygons. In the provided section, it is mentioned that 2n-sided regular polygons, known for thousands of years, are reproduced beautifully by neural networks. This demonstrates the ability of neural networks to accurately represent and generate regular polygons, showcasing the connection between the network architecture and the geometric properties of these polygons. The results in Fig. 2 illustrate the successful approximation of 2-polytopes of type (n; 1) by neural polygons, highlighting the potential of neural networks in capturing the essence of regular polygons.
read more
4. What are neural polytopes with p=1?
Neural polytopes with p=1 are spiky or round generalizations of ordinary polytopes. In the limit p-1, they become ordinary polytopes different from those for p=1. At p=2, neural polytopes are spheres. Neural polygons of type (2; 1) with p=0.8, 1.0, 1.2, 1.5, 2.0, 3.0, 5.0, and 10.0 show edge vertex rounded as p increases. The edge shape of neural polygons is identical to the shape of the activation function |x|p. Neural polyhedra of type (3; p) have a duality among polytopes, with p=1 being an octahedron and p=2 being a cube. This duality is natural due to the kink at x=0 in the activation function |x|p.
read more