Accelerating Integer Neural Networks On Low Cost DSPs

In this paper, low end Digital Signal Processors (DSPs) are applied to accelerate integer neural networks. The use of DSPs to accelerate neural networks has been a topic of study for some time, and has demonstrated significant performance improvements. Recently, work has been done on integer only neural networks, which greatly reduces hardware requirements, and thus allows for cheaper hardware implementation. DSPs with Arithmetic Logic Units (ALUs) that support floating or fixed point arithmetic are generally more expensive than their integer only counterparts due to increased circuit complexity. However if the need for floating or fixed point math operation can be removed, then simpler, lower cost DSPs can be used. To achieve this, an integer only neural network is created in this paper, which is then accelerated by using DSP instructions to improve performance.

Authors:

• Thomas Behan
• Zaiyi Liao
• Lian Zhao
• Chunting Yang

Keywords:

• Digital Signal Processor (DSP)
• Integer Neural Network(INN)
• Low Cost Neural Network
• Integer Neural Network DSPImplementation.

References:

[1] A. H. Khan and E. L. Hines, "Integer-weight neural nets," Electronics
Letters, vol. 30, no. 15, pp. 1237-1238, 1994.
[2] V. P. Plagianakos and M. N. Vrahatis, "Neural network training with
constrained integer weights," in Evolutionary Computation, Proceedings
of the 1999 Congress on, vol. 3, 1999, p. 2013.
[3] S. Draghici, "Some new results on the capabilities of integer weights
neural networks in classification problems," in Neural Networks, 1999.
IJCNN -99. International Joint Conference on, vol. 1, 1999, pp. 519-524.
[4] J. Onuki, "Ann accelerator by parallel processor based on DSP," in Neural
Networks, 1993. IJCNN -93-Nagoya. Proceedings of 1993 International
Joint Conference on, vol. 2, 1993, pp. 1913-1916.
[5] M. Mohamadian, E. Nowicki, F. Ashrafzadeh, A. Chu, R. Sachdeva,
and E. Evanik, "A novel neural network controller and its efficient DSP
implementation for vector-controlled induction motor drives," Industry
Applications, IEEE Transactions on, vol. 39, no. 6, pp. 1622-1629, 2003.
[6] S.-C. Chen, C.-C. Hsu, and W.-Y. Wang, "DSP-based fuzzy neural
network and its application in speech recognition," in Systems, Man, and
Cybernetics, 1999 IEEE International Conference on, vol. 6, 1999, pp.
110-114.
[7] J. Tang, M. R. Varley, and M. S. Peak, "Hardware implementations
of multi-layer feedforward neural networks and error backpropagation
using 8-bit pic microcontrollers," in Neural and Fuzzy Systems: Design,
Hardware and Applications (Digest No: 1997/133), IEE Colloquium on,
1997, pp. 2/1-2/5.
[8] H. Y. Xu, G. Z. Wang, and C. B. Baird, "A fuzzy neural networks
technique with fast backpropagation learning," in Neural Networks,
International Joint Conference on, vol. 1, 1992, pp. 214-219.