Bluehive - A field-programable custom computing machine for extreme-scale real-time neural network simulation

Question

1. How do the authors keep the network light-loaded?

2. How many GB/s of bandwidth can the authors manage?

3. How many times did it take to simulate a single thread?

4. What is the advantage of storing parameters in external memory?

Accepted Answer

The authors aim to keep the network lightly loaded to reduce the risk of congestion, and use low-latency routing, so communication latency across a large FPGA fabric is not a problem.

Accepted Answer

Their 12 SATA links per FPGA give us 72Gbit/s or 9GB/s of raw bandwidth so, even with protocol overheads, the authors can manage 6GB/s of FPGA-to-FPGA bandwidth.

Accepted Answer

Their single-threaded neural network simulator written in C required 48.8s to calculate 300ms of simulation time on a single thread of a 16-thread, 4-core Xeon X5560 2.80GHz server with 48GB RAM.

Accepted Answer

there are two advantages of storing parameters in external memory:1) The external memory is effectively being used as a giant switch, mapping neuron firings to lists of firing events.

Accepted Answer

A small NIOS processor system on the DE2-115 board facilitates communication with the programming PC allowing the multicast configuration to be selected.

Accepted Answer

For every tuple, the Accumulator burst-reads a stream of (neuron number,weight) pairs and performs the I-value updates:I[neuron number]+=weightHowever, for good performance, the authors wish to burst-read the update pairs four at a time per clock cycle and so four accumulations need to be performed in parallel.

Accepted Answer

So if every neuron is evaluated once every 1ms, the authors can easily evaluate 105 neurons using just one copy of the evaluation pipeline, leaving plenty of space on the FPGA.

Accepted Answer

Input: a stream of tuples: (delay,pointer to neuron updates,length) burst-read FIFO data previously spilled to external memory Function: sort work into FIFO delay buckets Output: a stream of tuples for the current time period:(pointer to neuron updates,length) burst-write FIFO data spilled to external memory

Accepted Answer

Whilst PyNN can produce a range of complete neural netlists, it appears not to scale much beyond 8k neuron, which is insufficient to demonstrate a 4-FPGA system with 64k neurons per FPGA.

Accepted Answer

It has been shown that interconnect in mammalian brains can be analysed using a variant of Rent’s rule which is often used to analyse communication requirements in VLSI chips [8].

Accepted Answer

FPGAs pay a significant area and performance penalty for being reconfigurable, however they can be produced cost-effectively using small feature-size processes (40nm for Stratix IV parts), which allows integrated high-speed memory interfaces and serial transceivers not possible using older implementation technologies.

Accepted Answer

The logic to do this is written in BSV and relies on the BSV compiled Verilog going through the Altera Quartus II synthesis system to map multiplication and addition functions onto the embedded DSP blocks (multipliers, etc.).

Accepted Answer

The parallel programming board can be directly used to program other Alterabased FPGA boards and with some adaptation would work with Xilinx-based boards.

Accepted Answer

However Altera’s jtagd is only capable of programming boards sequentially and, at 22s per board, this becomes inconvenient for large arrays.

Accepted Answer

The authors also use the PCIe connector with eight bidirectional serial links (see Figure 2), giving us 12 in total, thus 12×6= 72Gbit/s of bidirectional communication bandwidth per FPGA board.

Bluehive - A field-programable custom computing machine for extreme-scale real-time neural network simulation

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. How do the authors keep the network light-loaded?

2. How many GB/s of bandwidth can the authors manage?

3. How many times did it take to simulate a single thread?

4. What is the advantage of storing parameters in external memory?

5. What is the purpose of the DE2-115 board?

6. how many tuples do the accumulator read?

7. How many neurons can be evaluated using the Izhikevich algorithm?

8. What is the function that reads a stream of tuples?

9. How many neurons can be generated using PyNN?

10. What is the simplest way to analyse interconnect in mammalian brains?

11. How many times can a fPGA be reconfigurable?

12. What is the logic to do this?

13. What is the alternative to the Altera-based FPGA?

14. How many boards are able to program?

15. How many serial links do the authors use?

Figures

Citations

The SpiNNaker Project

The SpiNNaker project

A Survey of Neuromorphic Computing and Neural Networks in Hardware.

Brian 2, an intuitive and efficient neural simulator

SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation

References

Simple model of spiking neurons

Which model to use for cortical spiking neurons

Neuromorphic electronic systems

PyNN: A Common Interface for Neuronal Network Simulators.

Efficient physical embedding of topologically complex information processing networks in brains and computer circuits.

Related Papers (5)

A million spiking-neuron integrated circuit with a scalable communication network and interface

Simple model of spiking neurons

A quantitative description of membrane current and its application to conduction and excitation in nerve

The SpiNNaker Project

Which model to use for cortical spiking neurons