High-Performance and Reliable Processing-in-Memory Accelerators for Graph-based Machine Learning by Chukwufumnanya Ogbogu – Preliminary Exam

About the event

Student: Chukwufumnanya Ogbogu

Advisor: Dr. Patha Pande

Degree: Electrical and Computer Engineering, Ph.D.

Dissertation Title: High-Performance and Reliable Processing-in-Memory Accelerators for Graph-based Machine Learning

Abstract: The saturating scaling trends of CMOS technology have motivated the exploration of emerging Process-in-Memory (PIM) architectures as a promising alternative for accelerating data intensive Machine Learning (ML) workloads. To that effect, researchers have explored special-purpose accelerators based on Non-volatile Memory (NVM) crossbar primitives such as Resistive Random Access Memory (ReRAM)-based PIM accelerators. Graph neural networks (GNNs) are a special class of ML workload that have become increasingly popular in real-world domain applications. This is because of their ability to perform predictive analytics on graph-structured data, and they have become very popular in diverse real-world applications. However, GNN training on ReRAM-based architectures is both compute- and data-intensive in nature. This is because GNN training/inferencing on ReRAM-based manycore architectures gives rise to both computation and on-chip communication challenges. Hence, it is necessary to explore techniques that can tackle these bottlenecks associated with GNN training on PIM architectures. So far, we have explored model and data pruning as well as quantization techniques to accelerate GNN training on ReRAM-based platforms. We have demonstrated that combining model pruning and data pruning can speed-up GNN training by up to 4.5× while consuming 7× less energy compared to their unpruned counterparts. In addition, another key challenge in GNN training on ReRAM-based architectures arises from the low write endurance problem of ReRAM devices, which is exacerbated with massive input graphs. This restricts the number of write operations to the ReRAM crossbar cells due to weight updates and storage of activations during training. As a result, this in turn limits the overall lifetime of ReRAM-based platforms for energy-efficient GNN training. In this work, we have also demonstrated that data pruning can also enhance both the lifetime and performance of ReRAM-based systems for GNN training. Our results demonstrate that data pruning achieves up to 5× improvement in lifetime and 10× speedup in performance compared to state-of-the-art ReRAM-based PIM accelerators for GNN training.