TY  - CONF
AU  - Dube, Aradhana
AU  - Manea, Paul
AU  - Gibertini, Paolo
AU  - Covi, Erika
AU  - Strachan, John Paul
TI  - Analog Softmax with Wide Input Current Range for In-Memory Computing
PB  - IEEE
M1  - FZJ-2026-00226
SP  - 1-5
PY  - 2025
AB  - The Softmax activation function plays a pivotalrole in both the attention mechanism of Transformers andin the final layer of neural networks performing classification.The Softmax function outputs probabilities by normalizing theinput values, emphasizing differences among them to highlightthe largest values. In digital implementations, the complexityof softmax grows linearly with the number of inputs. Incontrast, analog implementations enable parallel computationswith lower latency. In this work, we demonstrate that thisapproach achieves a more efficient linear scaling of latencyas vector size increases logarithmically. This analog softmaxcircuits are implemented in TSMC 28 nm PDK technology,capable of driving up to 128 inputs and producing an ana-log current output spanning three orders of magnitude. Thestudy examines the circuit’s power consumption, latency, anderror, emphasizing its efficiency compared to the alternativeapproach of converting outputs to digital signals via ADCsand performing the softmax calculation digitally. By reducingreliance on these power-intensive operations, this work aims tosignificantly enhance energy efficiency in in-memory computingsystems.
T2  - 2025 IEEE International Symposium on Circuits and Systems (ISCAS)
CY  - 25 May 2025 - 28 May 2025, London (United Kingdom)
Y2  - 25 May 2025 - 28 May 2025
M2  - London, United Kingdom
LB  - PUB:(DE-HGF)8 ; PUB:(DE-HGF)7
DO  - DOI:10.1109/ISCAS56072.2025.11043251
UR  - https://juser.fz-juelich.de/record/1050456
ER  -