|
A Beowulf cluster is a bunch of plain personal computers connected with a fast network and dedicated to performing number crunching. Artificial Neural Nets (ANN) are computational models inspired by the human brain, because many similar processing units, called neurons, work together in parallel to perform some global tasks. Even though ANN are rather unlike real neurons, let alone brains, they can perform useful tasks, e.g. recognizing handwritten digits. On one computer with a single processor only one neuron can be active at a time. An obvious idea is to run such a program on a parallel system, like a Beowulf cluster.
Parallelizing a sequential program takes special care and various problems come up. We showed, that parallelizing feedforward ANN is not trivial and in contrast to common belief will not neccessarily run faster on Beowulf clusters. Based on our analysis it is possible to estimate the possible speedup. We also proposed a method that tries to alleviate the main problem by reducing the amount of communication neede between the different machines.
This research was performed under the supervision of Prof. Dr. Bruce Wade at the University of Milwaukee, Wisconsin.
|
The picture shows the speedup as a function of the block size, i.e. the number of training rounds that can be done in parallel before communication is neccessary. The different curves correspond to different cluster sizes.
|
)
