Publication: Stochastic dividers for low latency neural networks
dc.affiliation.dpto | UC3M. Departamento de Ingeniería Telemática | es |
dc.affiliation.grupoinv | UC3M. Grupo de Investigación: Network Technologies | es |
dc.contributor.author | Liu, Shanshan | |
dc.contributor.author | Tang, Xiaochen | |
dc.contributor.author | Niknia, Farzad | |
dc.contributor.author | Reviriego Vasallo, Pedro | |
dc.contributor.author | Liu, Weiqiang | |
dc.contributor.author | Louri, Ahmed | |
dc.contributor.author | Lombardi, Fabrizio | |
dc.contributor.funder | Comunidad de Madrid | es |
dc.contributor.funder | Ministerio de Ciencia e Innovación (España) | es |
dc.date.accessioned | 2021-09-29T11:23:27Z | |
dc.date.available | 2021-09-29T11:23:27Z | |
dc.date.issued | 2021-10 | |
dc.description.abstract | Due to the low complexity in arithmetic unit design, stochastic computing (SC) has attracted considerable interest to implement Artificial Neural Networks (ANNs) for resources-limited applications, because ANNs must usually perform a large number of arithmetic operations. To attain a high computation accuracy in an SC-based ANN, extended stochastic logic is utilized together with standard SC units and thus, a stochastic divider is required to perform the conversion between these logic representations. However, the conventional divider incurs in a large computation latency, so limits an SC implementation for ANNs used in applications needing high performance. Therefore, there is a need to design fast stochastic dividers for SC-based ANNs. Recent works (e.g., a binary searching and triple modular redundancy (BS-TMR) based stochastic divider) are targeting a reduction in computation latency, while keeping the same accuracy compared with the traditional design. However, this divider still requires $N$ iterations to deal with $2^{N}$ -bit stochastic sequences, and thus the latency increases in proportion to the sequence length. In this paper, a decimal searching and TMR (DS-TMR) based stochastic divider is initially proposed to further reduce the computation latency; it only requires two iterations to calculate the quotient, so regardless of the sequence length. Moreover, a trade-off design between accuracy and hardware is also presented. An SC-based Multi-Layer Perceptron (MLP) is then considered to show the effectiveness of the proposed dividers over current designs. Results show that when utilizing the proposed dividers, the MLP achieves the lowest computation latency while keeping the same classification accuracy; although incurring in an area increase, the overhead due to the proposed dividers is low over the entire MLP. When using as combined metric for both hardware design and computation complexity the product of the implementation area, latency, power and number of clock cycles, the proposed designs are also shown to be superior to the SC-based MLPs (at the same level of accuracy) employing other dividers found in the technical literature as well as the commonly used 32-bit floating point implementation. | en |
dc.description.sponsorship | The work of Shanshan Liu, Farzad Niknia, and Fabrizio Lombardi was supported by the NSF Grant CCF-1953961 and Grant 1812467. The work of Pedro Reviriego was supported in part by the Spanish Ministry of Science and Innovation under project ACHILLES (Grant PID2019-104207RB-I00) and the Go2Edge Network (Grant RED2018-102585-T), and in part by the Madrid Community Research Agency under Grant TAPIR-CM P2018/TCS-4496. The work of Weiqiang Liu was supported by the NSFC under Grant 62022041 and Grant 61871216. The work of Ahmed Louri was supported by the NSF Grant CCF-1812495 and Grant 1953980. | en |
dc.format.extent | 14 | |
dc.identifier.bibliographicCitation | Liu, S., Tang, X., Niknia, F., Reviriego, P., Liu, W., Louri, A. & Lombardi, F. (2021). Stochastic Dividers for Low Latency Neural Networks. IEEE Transactions on Circuits and Systems I: Regular Papers, 68(10), 4102–4115. | en |
dc.identifier.doi | https://doi.org/10.1088/1741-4326/ab9dd1 | |
dc.identifier.issn | 1549-8328 | |
dc.identifier.publicationfirstpage | 4102 | |
dc.identifier.publicationissue | 10 | |
dc.identifier.publicationlastpage | 4115 | |
dc.identifier.publicationtitle | IEEE Transactions on Circuits and Systems I: Regular Papers | en |
dc.identifier.publicationvolume | 68 | |
dc.identifier.uri | https://hdl.handle.net/10016/33339 | |
dc.identifier.uxxi | AR/0000028252 | |
dc.language.iso | eng | |
dc.publisher | IEEE | |
dc.relation.projectID | Gobierno de España. PID2019-104207RB-I00 | es |
dc.relation.projectID | Comunidad de Madrid. P2018/TCS-4496 | es |
dc.rights | © 2021, IEEE | |
dc.rights.accessRights | open access | en |
dc.subject.eciencia | Telecomunicaciones | es |
dc.subject.other | Divider | en |
dc.subject.other | Stochastic computing | en |
dc.subject.other | Decimal searching | en |
dc.subject.other | Artificial neural network | en |
dc.title | Stochastic dividers for low latency neural networks | en |
dc.type | research article | * |
dc.type.hasVersion | AM | * |
dspace.entity.type | Publication |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Stochastic_IEEE-TCSI_2021_ps.pdf
- Size:
- 2.9 MB
- Format:
- Adobe Portable Document Format