Binary Quantization Analysis of Neural Networks Weights on MNIST Dataset

Zoran H. Peric; Bojan D. Denic; Milan S. Savic; Nikola J. Vucic; Nikola B. Simic

doi:10.5755/j02.eie.28881

Binary Quantization Analysis of Neural Networks Weights on MNIST Dataset

Authors

Zoran H. Peric Faculty of Electronic Engineering, University of Nis, Serbia https://orcid.org/0000-0002-8267-9541
Bojan D. Denic Faculty of Electronic Engineering, University of Nis, Serbia https://orcid.org/0000-0002-9003-2545
Milan S. Savic Faculty of Sciences and Mathematics, University of Pristina, Serbia https://orcid.org/0000-0002-7131-132X
Nikola J. Vucic Faculty of Electronic Engineering, University of Nis, Serbia https://orcid.org/0000-0003-4781-8566
Nikola B. Simic Faculty of Technical Sciences, University of Novi Sad, Serbia https://orcid.org/0000-0002-0748-4672

DOI:

https://doi.org/10.5755/j02.eie.28881

Keywords:

Image classification, Multilayer perceptron, Neural network, Quantization, Source coding

Abstract

This paper considers the design of a binary scalar quantizer of Laplacian source and its application in compressed neural networks. The quantizer performance is investigated in a wide dynamic range of data variances, and for that purpose, we derive novel closed-form expressions. Moreover, we propose two selection criteria for the variance range of interest. Binary quantizers are further implemented for compressing neural network weights and its performance is analysed for a simple classification task. Good matching between theory and experiment is observed and a great possibility for implementation is indicated.

Downloads

Published

2021-08-23

How to Cite

Peric, Z. H., Denic, B. D., Savic, M. S., Vucic, N. J., & Simic, N. B. (2021). Binary Quantization Analysis of Neural Networks Weights on MNIST Dataset. Elektronika Ir Elektrotechnika, 27(4), 55-61. https://doi.org/10.5755/j02.eie.28881

Download Citation

Issue

Vol. 27 No. 4 (2021)

Section

SIGNAL TECHNOLOGY

License

The copyright for the paper in this journal is retained by the author(s) with the first publication right granted to the journal. The authors agree to the Creative Commons Attribution 4.0 (CC BY 4.0) agreement under which the paper in the Journal is licensed.

By virtue of their appearance in this open access journal, papers are free to use with proper attribution in educational and other non-commercial settings with an acknowledgement of the initial publication in the journal.