Analog AI     

links

Analog AI - overview


In the Analog AI team -- based at IBM Research-Almaden in San Jose, California -- we work on innovations that could enable more capable and energy-efficient Deep Neural Network systems.  We are exploring the use of on-chip learning to accelerate training, using new reconfigurable architectures and circuit designs that can replace slow and inefficient data transfers with massively parallel reads and writes to arrays of analog memory devices, so that essential computations are performed at the data.

This team began as a member project of the IBM Research Frontiers Institute, and we are now a proud part of the IBM AI Hardware Center.

Our project is also related to the "Physics of AI" and "AI hardware" themes within IBM Research AI.

Major publications

  1. "Fully on-chip MAC at 14nm enabled by accurate row-wise programming of PCM-based weights and parallel vector-transport in duration-format," P. Narayanan, S. Ambrogio, A. Okazaki, K. Hosokawa, H. Tsai, A. Nomura, T. Yasuda, C. Mackin, S. C. Lewis, A. Friz, M. Ishii, Y. Kohda, H. Mori, K. Spoon, R. Khaddam-Aljameh, N. Saulnier, M. Bergendahl, J. Demarest, K. W. Brew, V. Chan, S. Choi, I. Ok, I. Ahsan, F. L. Lie, W. Haensch, V. Narayanan, G. W. Burr, 2021 Symposium on VLSI Technology, pp. T1-T2, June 2021.
  2. "Toward Software-Equivalent Accuracy on Transformer-Based Deep Neural Networks With Analog Memory Devices," K. Spoon, H. Tsai, A. Chen, M. J. Rasch, S. Ambrogio, C. Mackin, A. Fasoli, A. M. Friz, P. Narayanan, M. Stanisavljevic, and G. W. Burr, Frontiers in Computational Neuroscience, 53, 2021. 
  3. "Reducing the Impact of Phase-Change Memory Conductance Drift on the Inference of large-scale Hardware Neural Networks," S. Ambrogio, M. Gallot, K. Spoon, H. Tsai, C. Mackin, M. Wesson, S. Kariyappa, P. Narayanan, C.-C. Liu, A. Kumar, A. Chen, and G. W Burr, 2019 IEEE International Electron Devices Meeting (IEDM 2019), Dec 2019.
  4. "AI hardware acceleration with analog memory: micro-architectures for low energy at high speed," H.-Y. Chang, P. Narayanan, S. C. Lewis, N. C. P. Farinha, K. Hosokawa, C. Mackin, H. Tsai, S. Ambrogio, An Chen, and G. W. Burr, IBM Journal of Research and Development, 63(6), 8:1-8:14, 2019.
  5. "Inference of Long-Short Term Memory networks at software-equivalent accuracy using 2.5M analog Phase Change Memory devices," H. Tsai, S. Ambrogio, C. Mackin, P. Narayanan, R. M. Shelby, K. Rocki, A. Chen and G. W. Burr, 2019 VLSI Technology Symposium, T8.1, June 2019.
  6. "Parallelized Weight Programming in DNN Analog Hardware Accelerators," C. Mackin, H. Tsai, S. Ambrogio, P. Narayanan, A. Chen, and G. W. Burr, Adv. Electronic Materials, invited paper, 5(9), 1900026 (2019).
  7. "Perspective on Training Fully Connected Networks with Resistive Memories: Device Requirements for Multiple Conductances of Varying Significance," G. Cristiano, M. Giordano, S. Ambrogio, L. P. Romero, C. Cheng, P. Narayanan, H. Tsai, R. M. Shelby, and G. W. Burr, Journal of Applied Physics, 124(15), 151901 (2018).
  8. "Equivalent-accuracy acceleration of Neural Network Training using Analog Memory," S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. P. Farinha, B. Killeen, C. Cheng, Y. Jaoudi, and G. W. Burr, Nature, 558 (7708) 60-67 (2018).  See also our June 2018 blog post.
  9. "Towards on-chip acceleration of the backpropagation algorithm using non-volatile memory," P. Narayanan, A. Fumarola, L. Sanches, S. Lewis, K. Hosokawa, R. M. Shelby, and G. W. Burr, IBM Journal of Research and Development, special issue on "Deep Learning," 61(4/5), 1-11 (2017).
  10. "Large-scale neural networks implemented with nonvolatile memory as the synaptic weight element: comparative performance analysis (accuracy, speed, and power)," G. W. Burr, P.Narayanan, R.M.Shelby, S. Sidler, I.Boybat, C.di Nolfo, and Y.Leblebici, 2015 IEEE International Electron Devices Meeting (IEDM 2015), invited talk, 4.4, December 2015.
  11. "Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element," G. W. Burr, R. M. Shelby, C. di Nolfo, J. W. Jang, I. Boybat, R. S. Shenoy, P. Narayanan, K. Virwani, E. U. Giacometti, B. Kurdi, and H. Hwang, IEEE Transactions on Electron Devices, special issue commemorating the 60th anniversary of IEDM, 62(11), 3498-3507 (2015).
  12. "Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element," G. W. Burr, R. M. Shelby, C. di Nolfo, J. W. Jang, R. S. Shenoy, P. Narayanan, K. Virwani, E. U. Giacometti, B. Kurdi, and H. Hwang, 2014 IEEE International Electron Devices Meeting (IEDM 2014), 29.5, December 2014.

Review articles

  1. "Brain-inspired computing using phase-change memory devices," A. Sebastian, M. Le Gallo, G. W. Burr, S. Kim, M. BrightSky, and E. Eleftheriou, J. Appl. Phys., invited Tutorial for special issue on "New Physics and Materials for Neuromorphic Computation," 124(11), 111101, 2018.
  2. "Recent progress in analog memory-based accelerators for Deep Learning," H. Tsai, S. Ambrogio, P. Narayanan, R. M. Shelby, and G. W. Burr, J. Phys. D, special issue on "Brain-inspired pervasive computing: from materials engineering to neuromorphic architectures," accepted version on-line (2018).
  3. "Neuromorphic computing using non-volatile memory," G. W. Burr, R. M. Shelby, A. Sebastian, S. Kim, S. Kim, S. Sidler, K. Virwani, M. Ishii, P. Narayanan, A. Fumarola, L. L. Sanches, I. Boybat, M. Le Gallo, K. Moon, J. Woo, H. Hwang, and Y. Leblebici, Advances in Physics X, 2(1), 89-124 (2017).

Our "blog" posts

  1. "Redefining AI accelerators: A new chip packaging tech & a first in analog compute," July 21, 2021.
  2. "AI Hardware Composer launches on two-year anniversary of IBM AI Hardware Center," April 14, 2021.
  3. "The path to the “perfect” analog material and system: IBM at IEDM and NeurIPS," December 11, 2019
  4. "Unveiling Analog Memory-based Technologies to Advance AI at VLSI," June 10, 2019
  5. "Steering Material Scientists to Better Memory Devices," October 9, 2018
  6. "The Future of AI Needs Better Compute: Hardware Accelerators Based on Analog Memory Devices," June 6, 2018

Conference and other presentations

  1. "Confined PCM-based Analog Synaptic Devices offering Low Resistance-drift and 1000 Programmable States for Deep Learning," W. Kim, R. L. Bruce, T. Masuda, G. W. Fraczak, N. Gong, P. Adusumilli, S. Ambrogio, H. Tsai, J. Bruley, J.-P. Han, M. Longstreet, F. Carta, K. Suu and M. BrightSky, 2019 VLSI Technology Symposium, T6.4, June 2019.
  2. "Training Fully Connected Networks with Resistive Memories: Impact of Device Failures," L. P. Romero, S. Ambrogio, M. Giordano, G. Cristiano, M. Bodini, P. Narayanan, H. Tsai, R. M. Shelby, and G. W. Burr, Faraday Discussions, 2018.
  3. "Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr0.7Ca0.3MnO3 Device Characteristics on Neural Network Training Accuracy," A. Fumarola, S. Sidler, K. Moon, J. Jang, R. M. Shelby, P. Narayanan, Y. Leblebici, H. Hwang, and G. W. Burr, J. Electr. Dev. Soc. 6(1), 169-178, December 2017.
  4. "Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part I: Al/Mo/Pr0.7Ca0.3MnO3 Material Improvements and Device Measurements," K. Moon, A. Fumarola, S. Sidler, J. Jang, P. Narayanan, R. M. Shelby, G. W. Burr and H. Hwang, J. Electr. Dev. Soc. 6(1), 146-155, December 2017.
  5. "Improved Deep Neural Network hardware-accelerators based on Non-Volatile-Memory: the Local Gains technique," I. Boybat, C. di Nolfo, S. Ambrogio, M. Bodini, N. C. P. Farinha, R. M. Shelby, P. Narayanan, S. Sidler, H. Tsai, Y. Leblebici, and G. W. Burr, ICRC - International Conference on Rebooting Computing, November 2017.
  6. "Reducing Circuit Design Complexity for Neuromorphic Machine Learning Systems Based on Non-Volatile Memory Arrays," P. Narayanan, L. L. Sanches, A. Fumarola, R. M. Shelby, S. Ambrogio, J. Jang, H. Hwang, Y. Leblebici, and G. W. Burr, ISCAS - International Symposium on Circuits and Systems, May 2017.
  7. "Accelerating Machine Learning with Non-Volatile Memory: exploring device and circuit tradeoffs," A. Fumarola, P. Narayanan, L. L. Sanches, S. Sidler, J. Jang, , K. Moon, R. M. Shelby, H. Hwang, and G. W. Burr, ICRC - International Conference on Rebooting Computing, October 2016.
  8. " Large-scale neural networks implemented with non-volatile memory as the synaptic weight element: impact of conductance response," S. Sidler, I. Boybat, R. M. Shelby, P. Narayanan, J. Jang, A. Fumarola, K. Moon, Y. Leblebici, H. Hwang, and G. W. Burr, ESSDERC 2016,September 2016.
  9. "Optimization of Conductance Change in Pr1-xCaxMnO3-based Synaptic Devices for Neuromorphic Systems," J.-W. Jang, S. Park, G. W. Burr, H. Hwang, and Y.-H. Jeong, IEEE Electron Device Letters, 36(5), 457-459 (2015).