Publications

2025

1. MICRO

   RayN: Ray Tracing Acceleration with Near-memory Computing

   Mohammadreza Saed, Prashant J. Nair, and Tor M. Aamodt

   In Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture (MICRO), Seoul, Republic of Korea, 2025

   Abs DOI Bib HTML PDF Code Slides

   

   Ray tracing is a rendering technique for producing photorealistic images by simulating light transport. While hardware-accelerated ray tracing has become increasingly prevalent in modern GPUs, the memory-intensive nature of ray tracing workloads creates significant performance bottlenecks due to the irregular memory access patterns inherent in bounding volume hierarchy (BVH) traversal and ray-primitive intersection tests. This paper proposes RayN, a near-memory computing architecture designed to accelerate ray tracing by bringing computation closer to where data resides. RayN addresses the memory bandwidth limitations that constrain current ray tracing accelerators by enabling efficient processing of BVH traversal and ray-primitive intersection tests near memory. Our evaluation demonstrates that RayN achieves significant performance improvements over baseline GPU ray tracing implementations while reducing memory traffic and energy consumption.

   @inproceedings{saed2025rayn,
     author = {Saed, Mohammadreza and Nair, Prashant J. and Aamodt, Tor M.},
     booktitle = {Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1145/3725843.3756067},
     latest = {true},
     location = {Seoul, Republic of Korea},
     pages = {277-291},
     series = {MICRO 2025},
     title = {RayN: Ray Tracing Acceleration with Near-memory Computing},
     year = {2025}
   }

2. ISCA

   Avalanche: Optimizing Cache Utilization via Matrix Reordering for Sparse Matrix Multiplication Accelerator

   Gwangeun Byeon, Seongwook Kim, Hyungjin Kim, Sukhyun Han, Jinkwon Kim, Prashant J. Nair, Taewook Kang, and Seokin Hong

   In Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA), Tokyo, Japan, 2025

   Abs DOI Bib HTML PDF Code Slides

   

   Sparse Matrix Multiplication (SpMM) is essential in various scientific and engineering applications but poses significant challenges due to irregular memory access patterns. Many hardware accelerators have been proposed to accelerate SpMM. However, they have yet to focus on on-chip memory utilization. In this paper, we highlight the underutilization of the on-chip memory in the SpMM accelerators. Then we propose Avalanche, a novel hardware accelerator that optimally utilizes the on-chip memory to efficiently cache both matrices B and C. Avalanche incorporates three key techniques: Matrix Reordering (Mat-Reorder), Dead-Product Early Eviction (DP-Evict), and Reuse Distance-Aware Matrix Caching (RM-Caching). Mat-Reorder enhances data locality by reordering the columns of matrix, ensuring early completion of computations for matrix. DP-Evict optimizes on-chip memory usage by promptly evicting fully computed (dead) products from on-chip memory. RM-Caching maximizes data reuse by caching frequently accessed elements of matrix based on their reuse distance. Experimental results demonstrate that Avalanche achieves an average performance improvement of 1.97x compared to the state-of-the-art SpMM accelerator, with a chip area of 6.15 mm2.

   @inproceedings{byeon2025avalanche,
     author = {Byeon, Gwangeun and Kim, Seongwook and Kim, Hyungjin and Han, Sukhyun and Kim, Jinkwon and Nair, Prashant J. and Kang, Taewook and Hong, Seokin},
     booktitle = {Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA)},
     doi = {10.1145/3695053.3730990},
     latest = {true},
     location = {Tokyo, Japan},
     pages = {1746-1759},
     series = {ISCA 2025},
     title = {Avalanche: Optimizing Cache Utilization via Matrix Reordering for Sparse Matrix Multiplication Accelerator},
     year = {2025}
   }

3. ISCA

   Accelerating Simulation of Quantum Circuits under Noise via Computational Reuse

   Meng Wang, Swamit Tannu, and Prashant J. Nair

   In Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA), Tokyo, Japan, 2025

   Abs DOI Bib HTML PDF Code Slides

   

   To realize the full potential of quantum computers, we must mitigate qubit errors by developing noise-aware algorithms, compilers, and architectures. Thus, simulating quantum programs on high-performance computing (HPC) systems with different noise models is a de facto tool researchers use. Unfortunately, noisy simulators iteratively execute a similar circuit for thousands of trials, thereby incurring significant performance overheads. To address this, we propose a noisy simulation technique called Tree-Based Quantum Circuit Simulation (TQSim). TQSim exploits the reusability of intermediate results during the noisy simulation, reducing computation. TQSim dynamically partitions a circuit into several subcircuits. It then reuses the intermediate results from these subcircuits during computation. Compared to a noisy Qulacs-based baseline simulator, TQSim achieves a speedup of up to 3.89x for noisy simulations.

   @inproceedings{wang2025tqsim,
     author = {Wang, Meng and Tannu, Swamit and Nair, Prashant J.},
     booktitle = {Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA)},
     doi = {10.1145/3695053.3730992},
     latest = {true},
     location = {Tokyo, Japan},
     pages = {1539-1553},
     series = {ISCA 2025},
     title = {Accelerating Simulation of Quantum Circuits under Noise via Computational Reuse},
     year = {2025}
   }

4. ISCA

   When Mitigations Backfire: Timing Channel Attacks and Defense for PRAC-Based RowHammer Mitigations

   Jeonghyun Woo, Joyce Qu, Gururaj Saileshwar, and Prashant Jayaprakash Nair

   In Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA), Tokyo, Japan, 2025

   Abs DOI Bib HTML PDF Code Slides

   

   RowHammer vulnerabilities pose a significant threat to modern DRAM-based systems, where rapid activation of DRAM rows can induce bit-flips in neighboring rows. JEDEC has introduced the Per Row Activation Counting (PRAC) framework for DDR5 and future DRAMs to enable precise counting of DRAM row activations. While PRAC enables holistic mitigation of Rowhammer attacks, we identify a new vulnerability: PRAC-based mitigations can leak timing information that enables covert and side-channel attacks. We present PRACLeak, which exploits the timing variations introduced by PRAC’s Alert Back-Off (ABO) protocol and Refresh Management (RFM) operations to create reliable timing channels. PRACLeak achieves covert channel bandwidth of several hundred bits per second with high accuracy. To address this vulnerability, we propose TPRAC (Timing-Safe PRAC), a defense mechanism that eliminates the timing channel by introducing constant-time mitigation operations and randomized scheduling. Our evaluation shows that TPRAC maintains the security guarantees of PRAC while eliminating timing channel vulnerabilities with minimal performance overhead.

   @inproceedings{woo2025practimingchannel,
     address = {New York, NY, USA},
     author = {Woo, Jeonghyun and Qu, Joyce and Saileshwar, Gururaj and Nair, Prashant Jayaprakash},
     booktitle = {Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA)},
     doi = {10.1145/3695053.3731007},
     isbn = {9798400712616},
     keywords = {DRAM, RowHammer, Timing Channel, Security, PRAC, RFM},
     latest = {true},
     location = {Tokyo, Japan},
     numpages = {18},
     pages = {739-756},
     publisher = {Association for Computing Machinery},
     series = {ISCA 2025},
     title = {When Mitigations Backfire: Timing Channel Attacks and Defense for PRAC-Based RowHammer Mitigations},
     year = {2025}
   }

5. ISCA

   Magellan: A High-Performance Loop-Guided Prefetcher for Indirect Memory Access

   Gelin Fu, Tian Xia, Mingzhuo Yin, Prashant J. Nair, Mieszko Lis, and Pengju Ren

   In Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA), Tokyo, Japan, 2025

   Abs DOI Bib HTML PDF Code Slides

   

   Graph analytics and sparse linear algebra applications heavily rely on indirect memory access (IMA). IMAs are characterized by poor temporal and spatial locality, which causes frequent high-latency DRAM accesses. While dedicated hardware prefetchers for IMA have been explored, they target narrow access patterns and tend to introduce significant hardware complexity. Software prefetching offers a promising alternative, leveraging compiler analysis to prefetch indirection patterns. However, existing software prefetchers struggle with sparse applications due to limited loop iterations and complex IMA patterns across nested loops. We propose Magellan, a novel loop-guided software prefetcher designed to detect and schedule IMA prefetches efficiently. Magellan introduces two key innovations: (1) extracting dependence graphs across loop levels to detect complex IMA patterns and (2) capturing inner-outer loop semantics to prefetch for both current and future iterations. We evaluate Magellan on 14 memory-intensive benchmarks using real-world datasets from social networks and web graphs. Compared to the best existing IMA software prefetcher, Magellan reduces cache misses by 25% and dynamic instruction counts by 14% on average. This results in a 1.14x average speedup, with performance gains of up to 1.41x.

   @inproceedings{fu2025magellan,
     author = {Fu, Gelin and Xia, Tian and Yin, Mingzhuo and Nair, Prashant J. and Lis, Mieszko and Ren, Pengju},
     booktitle = {Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA)},
     doi = {10.1145/3695053.3731054},
     latest = {true},
     location = {Tokyo, Japan},
     pages = {601-615},
     series = {ISCA 2025},
     title = {Magellan: A High-Performance Loop-Guided Prefetcher for Indirect Memory Access},
     year = {2025}
   }

6. HPCA

   QPRAC: Towards Secure and Practical PRAC-based Rowhammer Mitigation using Priority Queues

   Jeonghyun Woo, Shaopeng Chris Lin, Prashant J. Nair, Aamer Jaleel, and Gururaj Saileshwar

   In Proceedings of the 31st International Symposium on High Performance Computer Architecture (HPCA), Las Vegas, NV, USA, 2025

   Distinguished Artifact Award Abs DOI Bib HTML PDF Code Slides

   

   One of the two Distinguished Artifact Awards in HPCA 2025

   JEDEC has introduced the Per Row Activation Counting (PRAC) framework for DDR5 and future DRAMs to enable precise counting of DRAM row activations. PRAC enables a holistic mitigation of Rowhammer attacks even at ultra-low Rowhammer thresholds. PRAC uses an Alert Back-Off (ABO) protocol to request the memory controller to issue Rowhammer mitigation requests. However, recent PRAC implementations are either insecure or impractical. For example, Panopticon, the inspiration for PRAC, is rendered insecure if implemented per JEDEC’s PRAC specification. On the other hand, the recent UPRAC proposal is impractical since it needs oracular knowledge of the ’top-N’ activated DRAM rows that require mitigation. This paper provides the first secure, scalable, and practical RowHammer solution using the PRAC framework. The crux of our proposal is the design of a priority-based service queue (PSQ) for mitigations that prioritizes pending mitigations based on activation counts to avoid the security risks of prior solutions. This provides principled security using the reactive ABO protocol. Furthermore, we co-design our PSQ, with opportunistic mitigation on Refresh Management (RFM) operations and proactive mitigation during refresh (REF), to limit the performance impact of ABO-based mitigations. QPRAC provides secure and practical RowHammer mitigation that scales to Rowhammer thresholds as low as 71 while incurring a 0.8% slowdown for benign workloads, which further reduces to 0% with proactive mitigations.

   @inproceedings{woo2025qprac,
     author = {Woo, Jeonghyun and Lin, Shaopeng Chris and Nair, Prashant J. and Jaleel, Aamer and Saileshwar, Gururaj},
     booktitle = {Proceedings of the 31st International Symposium on High Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA61900.2025.00080},
     keywords = {dram, reliability, security, rowhammer, per row activation counting},
     latest = {true},
     location = {Las Vegas, NV, USA},
     pages = {1021-1037},
     series = {HPCA 2025},
     title = {QPRAC: Towards Secure and Practical PRAC-based Rowhammer Mitigation using Priority Queues},
     year = {2025}
   }

7. HPCA

   DAPPER: A Performance-Attack-Resilient Tracker for RowHammer Defense

   Jeonghyun Woo and Prashant J. Nair

   In Proceedings of the 31st International Symposium on High Performance Computer Architecture (HPCA), Las Vegas, NV, USA, 2025

   Abs DOI Bib HTML PDF Code Slides

   

   RowHammer vulnerabilities pose a significant threat to modern DRAM-based systems, where rapid activation of DRAM rows can induce bit-flips in neighboring rows. Prior RowHammer mitigations rely on trackers that identify frequently activated (aggressor) rows and issue mitigative actions. However, these trackers are vulnerable to performance attacks where adversaries craft access patterns that maximize tracker overhead while avoiding actual RowHammer thresholds. Such performance attacks can cause significant slowdowns for benign applications sharing the system. This paper proposes DAPPER, a performance-attack-resilient tracker for RowHammer defense. DAPPER introduces randomization into the tracking mechanism to prevent adversaries from predicting and exploiting tracker behavior. Our design ensures that the worst-case tracker overhead remains bounded regardless of the access pattern, while maintaining security against RowHammer attacks. DAPPER achieves robust RowHammer mitigation with bounded performance overhead, making it practical for deployment in systems where both security and performance guarantees are critical.

   @inproceedings{woo2025dapper,
     author = {Woo, Jeonghyun and Nair, Prashant J.},
     booktitle = {Proceedings of the 31st International Symposium on High Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA61900.2025.00079},
     keywords = {security, reliability, dram, rowhammer, randomization, dos, performance attacks},
     latest = {true},
     location = {Las Vegas, NV, USA},
     pages = {1005-1020},
     series = {HPCA 2025},
     title = {DAPPER: A Performance-Attack-Resilient Tracker for RowHammer Defense},
     year = {2025}
   }

8. DATE

   Slipstream: Semantic-Based Training Acceleration for Recommendation Models

   Yassaman Ebrahimzadeh Maboud, Muhammad Adnan, Divya Mahajan, and Prashant J. Nair

   In Proceedings of the 2025 Design, Automation and Test in Europe Conference (DATE), Lyon, France, 2025

   Abs DOI Bib HTML PDF Code Slides

   

   Training recommendation models pose significant challenges regarding resource utilization and performance. Recommender models use massive embedding tables that are memory intensive and employ neural networks that are compute intensive. Due to the popularity of certain inputs, the accesses to the embeddings are highly skewed. During training, some embeddings become fully trained and their values stabilize early. Consequently, updates to these embeddings become redundant. This paper presents Slipstream, a software framework that identifies stale embeddings on the fly and skips their updates to enhance performance. Slipstream utilizes the variations in embedding values to identify and skip fully trained embeddings at runtime. Slipstream helps reduce communication and computational overheads during training. Additionally, Slipstream uses a feature normalization process to attain (and significantly exceed) baseline accuracy. Slipstream achieves a training time reduction of approximately 2x compared to commercial XDL, Intel-Optimized DLRM, FAE, and Hotline baselines, while also gaining 0.06 testing accuracy across these models and datasets.

   @inproceedings{maboud2025slipstream,
     author = {Maboud, Yassaman Ebrahimzadeh and Adnan, Muhammad and Mahajan, Divya and Nair, Prashant J.},
     booktitle = {Proceedings of the 2025 Design, Automation and Test in Europe Conference (DATE)},
     doi = {10.23919/DATE64628.2025.10992746},
     latest = {true},
     location = {Lyon, France},
     pages = {1-7},
     series = {DATE 2025},
     title = {Slipstream: Semantic-Based Training Acceleration for Recommendation Models},
     year = {2025}
   }

9. DRAMSec

   Counterpoint: One-Hot Counting for PRAC-Based RowHammer Mitigation

   Shih-Lien Lu, Jeonghyun Woo, and Prashant J. Nair

   DRAMSec Workshop, 2025

   Abs Bib HTML PDF

   

   JEDEC has introduced the Per Row Activation Counting (PRAC) framework for DDR5 and future DRAMs to enable precise counting of DRAM row activations. While PRAC provides a foundation for RowHammer mitigation, the implementation of efficient per-row counters remains challenging. This paper proposes Counterpoint, a novel one-hot counting scheme for PRAC-based RowHammer mitigation. Counterpoint uses a one-hot encoding to represent activation counts, which enables efficient counter updates and threshold detection with minimal circuitry. Our design reduces the complexity of per-row counters while maintaining the security guarantees required for effective RowHammer mitigation.

   @article{lu2025counterpoint,
     author = {Lu, Shih-Lien and Woo, Jeonghyun and Nair, Prashant J.},
     journal = {DRAMSec Workshop},
     latest = {true},
     title = {Counterpoint: One-Hot Counting for PRAC-Based RowHammer Mitigation},
     year = {2025}
   }

10. ICML

    Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMs

    Ravi Ghadia, Avinash Kumar, Gaurav Jain, Prashant J. Nair, and Poulami Das

    2025

    Abs Bib HTML PDF

    

    Large Language Models (LLMs) have achieved remarkable success across various natural language tasks. However, the Key-Value (KV) cache, which stores intermediate representations during autoregressive generation, grows linearly with sequence length, creating memory bottlenecks for extended responses. This paper introduces Dialogue Without Limits, a technique that maintains constant-sized KV caches while enabling LLMs to generate extended responses. Our approach dynamically manages the KV cache by identifying and retaining the most important tokens for continued generation, allowing models to produce coherent long-form outputs without the memory overhead of unbounded cache growth.

    @article{ghadia2025dialogue,
      author = {Ghadia, Ravi and Kumar, Avinash and Jain, Gaurav and Nair, Prashant J. and Das, Poulami},
      booktitle = {Forty-second International Conference on Machine Learning},
      title = {Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMs},
      year = {2025}
    }

11. arXiv

    Context Switching for Secure Multi-programming of Near-Term Quantum Computers

    Avinash Kumar, Meng Wang, Chenxu Liu, Ang Li, Prashant J. Nair, and Poulami Das

    arXiv preprint arXiv:2504.07048, 2025

    Abs Bib HTML PDF

    

    Near-term quantum computers operate in the Noisy Intermediate-Scale Quantum (NISQ) regime, where resources are limited and errors are prevalent. Multi-programming, which executes multiple quantum programs concurrently on a single device, can improve resource utilization. However, secure multi-programming requires isolation between co-running programs to prevent information leakage. This paper proposes context switching mechanisms for secure multi-programming of near-term quantum computers. Our approach enables efficient program switching while maintaining isolation guarantees, allowing quantum computers to serve multiple users securely and efficiently.

    @article{kumar2025context,
      author = {Kumar, Avinash and Wang, Meng and Liu, Chenxu and Li, Ang and Nair, Prashant J. and Das, Poulami},
      journal = {arXiv preprint arXiv:2504.07048},
      title = {Context Switching for Secure Multi-programming of Near-Term Quantum Computers},
      year = {2025}
    }

12. NeurIPS

    Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generation

    Muhammad Adnan, Nithesh Kurella, Akhil Arunkumar, and Prashant J. Nair

    In Proceedings of the 39th International Conference on Neural Information Processing Systems (NeurIPS), San Diego, CA, USA, 2025

    Abs Bib HTML PDF Code Slides

    

    Diffusion Transformers (DiTs) achieve state-of-the-art results in text-to-image, text-to-video generation, and editing. However, their large model size and the quadratic cost of spatial-temporal attention over multiple denoising steps make video generation computationally expensive. Static caching mitigates this by reusing features across fixed steps but fails to adapt to generation dynamics, leading to suboptimal trade-offs between speed and quality. We propose Foresight, an adaptive layer-reuse technique that reduces computational redundancy across denoising steps while preserving baseline performance. Foresight dynamically identifies and reuses DiT block outputs for all layers across steps, adapting to generation parameters such as resolution and denoising schedules to optimize efficiency. Applied to OpenSora, Latte, and CogVideoX, Foresight achieves up to 1.63x end-to-end speedup, while maintaining video quality.

    @inproceedings{adnan2025foresight,
      author = {Adnan, Muhammad and Kurella, Nithesh and Arunkumar, Akhil and Nair, Prashant J.},
      booktitle = {Proceedings of the 39th International Conference on Neural Information Processing Systems (NeurIPS)},
      latest = {true},
      location = {San Diego, CA, USA},
      series = {NeurIPS 2025},
      title = {Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generation},
      year = {2025}
    }

13. arXiv

    Taming Wild Branches: Overcoming Hard-to-Predict Branches using the Bullseye Predictor

    Emet Behrendt, Shing Wai Pun, and Prashant J. Nair

    arXiv preprint arXiv:2506.06773, 2025

    Abs Bib HTML PDF

    

    Branch prediction remains a critical component of modern processor performance. While most branches are well-predicted by existing predictors, a small fraction of hard-to-predict branches cause significant performance degradation. This paper presents the Bullseye Predictor, a technique designed to overcome hard-to-predict branches. Bullseye uses targeted prediction mechanisms that focus computational resources on the most challenging branches, achieving higher accuracy where it matters most while maintaining efficiency for well-behaved branches.

    @article{behrendt2025bullseye,
      author = {Behrendt, Emet and Pun, Shing Wai and Nair, Prashant J.},
      journal = {arXiv preprint arXiv:2506.06773},
      title = {Taming Wild Branches: Overcoming Hard-to-Predict Branches using the Bullseye Predictor},
      year = {2025}
    }

14. arXiv

    CnC-PRAC: Coalesce, not Cache, Per Row Activation Counts for an Efficient in-DRAM Rowhammer Mitigation

    Chris S. Lin, Jeonghyun Woo, Prashant J. Nair, and Gururaj Saileshwar

    arXiv preprint arXiv:2506.11970, 2025

    Abs Bib HTML PDF

    

    JEDEC has introduced the Per Row Activation Counting (PRAC) framework for DDR5 and future DRAMs to enable precise counting of DRAM row activations using per-row activation counts. While recent PRAC implementations enable holistic mitigation of Rowhammer attacks, they impose slowdowns of up to 10% due to the increased DRAM timings for performing row activations with counter updates. This paper proposes CnC-PRAC (Coalesce, not Cache), an efficient in-DRAM Rowhammer mitigation that reduces the performance overhead of PRAC by coalescing multiple activation count updates. CnC-PRAC observes that consecutive activations to nearby rows can share counter update overhead, enabling more efficient counting without sacrificing security.

    @article{lin2025cncprac,
      author = {Lin, Chris S. and Woo, Jeonghyun and Nair, Prashant J. and Saileshwar, Gururaj},
      journal = {arXiv preprint arXiv:2506.11970},
      title = {CnC-PRAC: Coalesce, not Cache, Per Row Activation Counts for an Efficient in-DRAM Rowhammer Mitigation},
      year = {2025}
    }

15. arXiv

    Tableau-Based Framework for Efficient Logical Quantum Compilation

    Meng Wang, Chenxu Liu, Sean Garner, Samuel Stein, Yufei Ding, Prashant J. Nair, and Ang Li

    arXiv preprint arXiv:2509.02721, 2025

    Abs Bib HTML PDF

    

    Fault-tolerant quantum computing (FTQC) requires efficient compilation of logical quantum circuits into physical operations that can be executed on quantum hardware with error correction. This paper presents a Tableau-based framework for efficient logical quantum compilation. Our approach uses tableau representations to optimize the compilation process, reducing the overhead of fault-tolerant operations while maintaining the error correction guarantees required for reliable quantum computation.

    @article{wang2025tableau,
      author = {Wang, Meng and Liu, Chenxu and Garner, Sean and Stein, Samuel and Ding, Yufei and Nair, Prashant J. and Li, Ang},
      journal = {arXiv preprint arXiv:2509.02721},
      title = {Tableau-Based Framework for Efficient Logical Quantum Compilation},
      year = {2025}
    }

2024

1. QCE

   CircInspect: Integrating Visual Circuit Analysis, Abstraction, and Real-Time Development in Quantum Debugging

   Mushahid Khan, Prashant J. Nair, and Olivia Di Matteo

   In Proceedings of the 2024 IEEE International Conference on Quantum Computing and Engineering (QCE), Montreal, QC, Canada, 2024

   Abs DOI Bib HTML PDF Code Slides

   

   Software bugs typically result from errors in specifications or code translation. While classical software engineering has evolved with various tools and methodologies to tackle such bugs, the emergence of quantum computing presents unique challenges. Quantum software development introduces complexities due to the probabilistic nature of quantum computing, distinct algorithmic primitives, and potential hardware noise. In this paper, we introduce CircInspect, an interactive tool tailored for debugging quantum programs in Python and PennyLane. By leveraging breakpoints and real-time software development features, CircInspect empowers users to analyze isolated quantum circuit components, monitor program output, visualize structural changes, and abstract information to enhance comprehension. CircInspect has two modes: debugger mode and real-time development mode. In debugger mode, users can debug quantum programs using a visual circuit and a tree structure of functions. On the other hand, real-time development mode allows users to observe their quantum program’s circuit and output in real time.

   @inproceedings{khan2024circinspect,
     author = {Khan, Mushahid and Nair, Prashant J. and Di Matteo, Olivia},
     booktitle = {Proceedings of the 2024 IEEE International Conference on Quantum Computing and Engineering (QCE)},
     doi = {10.1109/QCE60285.2024.00124},
     location = {Montreal, QC, Canada},
     pages = {1000-1006},
     series = {QCE 2024},
     title = {CircInspect: Integrating Visual Circuit Analysis, Abstraction, and Real-Time Development in Quantum Debugging},
     volume = {1},
     year = {2024}
   }

2. arXiv

   PrisonBreak: Jailbreaking Large Language Models with Fewer Than Twenty-Five Targeted Bit-flips

   Zachary Coalson, Jeonghyun Woo, Shiyang Chen, Yu Sun, Lishan Yang, Prashant J. Nair, Bo Fang, and Sanghyun Hong

   arXiv preprint arXiv:2412.07192, 2024

   Abs Bib HTML PDF

   

   Large Language Models (LLMs) are increasingly deployed with safety guardrails to prevent harmful outputs. However, these safety mechanisms can potentially be circumvented through adversarial attacks. This paper presents PrisonBreak, a technique that demonstrates how targeted bit-flips in model parameters can jailbreak LLMs. We show that fewer than twenty-five carefully selected bit-flips can compromise safety guardrails while maintaining the model’s general capabilities. Our findings highlight the importance of hardware-level security for deployed LLMs and motivate the development of robust defenses against such attacks.

   @article{coalson2024prisonbreak,
     author = {Coalson, Zachary and Woo, Jeonghyun and Chen, Shiyang and Sun, Yu and Yang, Lishan and Nair, Prashant J. and Fang, Bo and Hong, Sanghyun},
     journal = {arXiv preprint arXiv:2412.07192},
     title = {PrisonBreak: Jailbreaking Large Language Models with Fewer Than Twenty-Five Targeted Bit-flips},
     year = {2024}
   }

3. arXiv

   Optimizing FTQC Programs through QEC Transpiler and Architecture Codesign

   Meng Wang, Chenxu Liu, Samuel Stein, Yufei Ding, Poulami Das, Prashant J. Nair, and Ang Li

   arXiv preprint arXiv:2412.15434, 2024

   Abs Bib HTML PDF

   

   Fault-tolerant quantum computing (FTQC) requires compiling quantum programs into sequences of fault-tolerant operations that can be executed with quantum error correction. The efficiency of this compilation significantly impacts the practical runtime of quantum algorithms. This paper explores the co-design of quantum error correction (QEC) transpilers and quantum architectures to optimize FTQC program execution. We identify opportunities for optimization at the interface between the transpiler and the underlying architecture, enabling more efficient fault-tolerant quantum computation.

   @article{wang2024optimizingftqc,
     author = {Wang, Meng and Liu, Chenxu and Stein, Samuel and Ding, Yufei and Das, Poulami and Nair, Prashant J. and Li, Ang},
     journal = {arXiv preprint arXiv:2412.15434},
     title = {Optimizing FTQC Programs through QEC Transpiler and Architecture Codesign},
     year = {2024}
   }

4. arXiv

   Workload-Aware Hardware Accelerator Mining for Distributed Deep Learning Training

   Muhammad Adnan, Amar Phanishayee, Janardhan Kulkarni, Prashant J. Nair, and Divya Mahajan

   arXiv preprint arXiv:2404.14632, 2024

   Abs Bib HTML PDF

   

   Distributed deep learning training relies on hardware accelerators to achieve practical training times. The design of these accelerators significantly impacts training efficiency and cost. This paper presents a workload-aware approach to hardware accelerator mining for distributed deep learning training. We analyze the characteristics of modern deep learning workloads and use this analysis to guide the design of hardware accelerators that are optimized for the specific computational patterns found in distributed training scenarios.

   @article{adnan2024workload,
     author = {Adnan, Muhammad and Phanishayee, Amar and Kulkarni, Janardhan and Nair, Prashant J. and Mahajan, Divya},
     journal = {arXiv preprint arXiv:2404.14632},
     title = {Workload-Aware Hardware Accelerator Mining for Distributed Deep Learning Training},
     year = {2024}
   }

5. MICRO

   A Case for Speculative Address Translation with Rapid Validation for GPUs

   Junhyeok Park, Osang Kwon, Yongho Lee, Seongwook Kim, Gwangeun Byeon, Jihun Yoon, Prashant J. Nair, and Seokin Hong

   In Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture (MICRO), Austin, USA, 2024

   MICRO Top Picks (Honorable Mention) Abs Bib HTML PDF Code Slides

   

   An honorable mention to be among the top papers in computer architecture in that year.

   A unified address space is vital for heterogeneous systems as it enables efficient data sharing between CPUs and GPUs. However, GPU address translation faces challenges due to high TLB pressure, particularly with irregular and memory-intensive applications. We observe that, compared to an ideal scenario, address translation overheads cause a slowdown of up to 34.5% in modern heterogeneous systems. This paper introduces Avatar, a novel framework to accelerate address translation in GPUs. Avatar comprises two key components: Contiguity-Aware Speculative Translation (CAST) and In-Cache Validation (CAVA) mechanisms. Avatar identifies the potential for predicting virtual-to-physical address mapping by monitoring contiguous pages that lie in both virtual and physical address spaces. Leveraging this insight, CAST speculatively translates virtual addresses into physical addresses. This enables immediate data fetching into GPUs while addressing translation occurs in the background, reducing TLB-miss overhead. Unfortunately, modern GPUs lack support for speculative execution, which limits CAST’s performance gain. Data fetched from speculated physical addresses is unusable until validation. CAVA addresses this limitation by quickly validating speculated physical addresses. To this end, CAVA embeds page mapping information into each 32B sector of 128B cache lines. Thus, CAVA enables fetching a sector block from memory for a speculated address and rapidly validating the speculative translation using the embedded mapping information. Our experiments show that Avatar achieves a high speculation accuracy of 90.3% and improves GPU performance by 37.2% (on average).

   @inproceedings{park2024avatar,
     author = {Park, Junhyeok and Kwon, Osang and Lee, Yongho and Kim, Seongwook and Byeon, Gwangeun and Yoon, Jihun and Nair, Prashant J. and Hong, Seokin},
     booktitle = {Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     latest = {true},
     location = {Austin, USA},
     series = {MICRO 2024},
     title = {A Case for Speculative Address Translation with Rapid Validation for GPUs},
     year = {2024}
   }

6. MICRO

   Qoncord: A Multi-Device Job Scheduling Framework for Variational Quantum Algorithms

   Meng Wang, Poulami Das, and Prashant J. Nair

   In Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture (MICRO), Austin, USA, 2024

   Abs Bib HTML PDF Code Slides

   

   Quantum computers face challenges due to limited resources, particularly in cloud environments. Despite these obstacles, Variational Quantum Algorithms (VQAs) are considered promising applications for present-day Noisy Intermediate-Scale Quantum (NISQ) systems. VQAs require multiple optimization iterations to converge on a globally optimal solution. Moreover, these optimizations, known as restarts, need to be repeated from different points to mitigate the impact of noise. Unfortunately, the job scheduling policies for each VQA task in the cloud are heavily unoptimized. Notably, each VQA execution instance is typically scheduled on a single NISQ device. Given the variety of devices in the cloud, users often prefer higher-fidelity devices to ensure higher-quality solutions. However, this preference leads to increased queueing delays and unbalanced resource utilization. We propose Qoncord, an automated job scheduling framework to address these cloud-centric challenges for VQAs. Qoncordleverages the insight that not all training iterations and restarts are equal, Qoncord strategically divides the training process into exploratory and fine-tuning phases. Early exploratory iterations, more resilient to noise, are executed on less busy machines, while fine-tuning occurs on high-fidelity machines. This adaptive approach mitigates the impact of noise and optimizes resource usage and queuing delays in cloud environments. Qoncord also significantly reduces execution time and minimizes restart overheads by eliminating low-performance iterations. Thus, Qoncord offers similar solutions 17.4× faster. Similarly, it can offer 13.3% better solutions for the same time budget as the baseline.

   @inproceedings{wang2024vqacloud,
     author = {Wang, Meng and Das, Poulami and Nair, Prashant J.},
     booktitle = {Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     latest = {true},
     location = {Austin, USA},
     series = {MICRO 2024},
     title = {Qoncord: A Multi-Device Job Scheduling Framework for Variational Quantum Algorithms},
     year = {2024}
   }

7. CLUSTER

   Understanding Mixed Precision GEMM with MPGemmFI: Insights into Fault Resilience

   Bo Fang, Xinyi Li, Harvey Dam, Cheng Tan, Siva Kumar Sastry Hari, Timothy Tsai, Ignacio Laguna, Dingwen Tao, Ganesh Gopalakrishnan, Prashant J. Nair, Kevin Barker, and Ang Li

   In Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER), Kobe, Japan, 2024

   Abs Bib HTML PDF Code Slides

   

   Emerging deep learning workloads urgently need fast general matrix multiplication (GEMM). Thus, one of the critical features of machine-learning-speciBc accelerators such as NVIDIA Tensor Cores, AMD Matrix Cores, and Google TPUs is the support of mixed-precision enabled GEMM. For DNN models, lower-precision FP data formats and computation offer acceptable correctness but significant performance, area, and memory footprint improvement. While promising, the mixed-precision computation on error resilience remains unexplored. To this end, we develop a fault injection framework that systematically injects fault into the mixed-precision computation results. We investigate how the faults adect the accuracy of machine learning applications. Based on the characteristics of error resilience, we offer lightweight error detection and correction solutions that significantly improve the overall model accuracy by 75% if the models experience hardware faults. The solutions can be efficiently integrated into the accelerator’s pipelines.

   @inproceedings{fang2024mpgemmfi,
     author = {Fang, Bo and Li, Xinyi and Dam, Harvey and Tan, Cheng and Hari, Siva Kumar Sastry and Tsai, Timothy and Laguna, Ignacio and Tao, Dingwen and Gopalakrishnan, Ganesh and Nair, Prashant J. and Barker, Kevin and Li, Ang},
     booktitle = {Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER)},
     location = {Kobe, Japan},
     series = {CLUSTER 2024},
     title = {Understanding Mixed Precision GEMM with MPGemmFI: Insights into Fault Resilience},
     year = {2024}
   }

8. ISCA

   Heterogeneous Acceleration Pipeline for Recommendation System Training

   Muhammad Adnan, Yassaman Ebrahimzadeh Maboud, Divya Mahajan, and Prashant J. Nair

   In 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA), Buenos Aires, Argentina, 2024

   Abs DOI Bib HTML PDF Slides

   

   Recommendation models utilize deep learning networks and massive embedding tables, making them compute- and memory-intensive. Typically, these models are trained in hybrid CPU-GPU or GPU-only mode. The hybrid mode uses GPU to accelerate the neural network while employing the CPUs to store and supply the memory-intensive embedding tables. However, this approach can result in significant CPU-to-GPU transfer time. In contrast, the GPU-only mode stores embedding tables in the High Bandwidth Memory (HBM) across multiple GPUs. However, this approach requires GPU-to-GPU backend communication. This increases communication overheads as the number of GPUs and the size of embedding entries increases. This paper introduces a heterogeneous acceleration pipeline called Hotline to address these concerns. The Hotline accelerator uses the insight that only a few embedding entries are frequently accessed (popular) and develops a data-aware and model-aware scheduling pipeline. This pipeline utilizes CPU main memory for non-popular embeddings and GPUs’ HBM for popular embeddings. The Hotline accelerator fragments a single mini-batch into popular and non-popular micro-batches (μ-batches). The system then gathers the required working parameters for the non-popular μ-batches from the CPU while allowing GPUs to execute popular μ-batches. During training, the hardware accelerator dynamically stitches the execution of popular embeddings on GPUs and non-popular embeddings from the CPU’s main memory. Our real-world datasets and models studies show that Hotline reduces average training time by 2.2x compared to Intel-optimized CPU-GPU DLRM baseline.

   @inproceedings{adnan2024hotline,
     author = {Adnan, Muhammad and Maboud, Yassaman Ebrahimzadeh and Mahajan, Divya and Nair, Prashant J.},
     booktitle = {2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)},
     doi = {10.1109/ISCA59077.2024.00081},
     keywords = {Training;Deep learning;Computational modeling;Pipelines;Neural networks;Memory management;Graphics processing units;Recommender Systems;Multi-Node Distributed Training;Accelerators},
     latest = {true},
     location = {Buenos Aires, Argentina},
     pages = {1063-1079},
     series = {ISCA 2024},
     title = {Heterogeneous Acceleration Pipeline for Recommendation System Training},
     year = {2024}
   }

9. MLSys

   Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference

   Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant J. Nair, Ilya Soloveychik, and Purushotham Kamath

   In Proceedings of the 7th Annual Conference on Machine Learning and Systems (MLSys), Santa Clara, CA, USA, 2024

   Abs Bib HTML PDF Code Slides

   

   Transformers have emerged as the underpinning architecture for Large Language Models (LLMs). In generative language models, the inference process involves two primary phases: prompt processing and token generation. Token generation, which constitutes the majority of the computational workload, primarily entails vector-matrix multiplications and interactions with the Key-Value (KV) Cache. This phase is constrained by memory bandwidth due to the overhead of transferring weights and KV cache values from the memory system to the computing units. This memory bottleneck becomes particularly pronounced in applications that require long-context and extensive text generation, both of which are increasingly crucial for LLMs. This paper introduces “Keyformer”, an innovative inference-time approach, to mitigate the challenges associated with KV cache size and memory bandwidth utilization. Keyformer leverages the observation that approximately 90% of the attention weight in generative inference focuses on a specific subset of tokens, referred to as “key” tokens. Keyformer retains only the key tokens in the KV cache by identifying these crucial tokens using a novel score function. This approach effectively reduces both the KV cache size and memory bandwidth usage without compromising model accuracy. We evaluate Keyformer’s performance across three foundational models: GPT-J, Cerebras-GPT, and MPT, which employ various positional embedding algorithms. Our assessment encompasses a variety of tasks, with a particular emphasis on summarization and conversation tasks involving extended contexts. Keyformer’s reduction of KV cache reduces inference latency by 2.1x and improves token generation throughput by 2.4x, while preserving the model’s accuracy.

   @inproceedings{adnan2024keyformer,
     author = {Adnan, Muhammad and Arunkumar, Akhil and Jain, Gaurav and Nair, Prashant J. and Soloveychik, Ilya and Kamath, Purushotham},
     booktitle = {Proceedings of the 7th Annual Conference on Machine Learning and Systems (MLSys)},
     location = {Santa Clara, CA, USA},
     series = {MLSys 2024},
     title = {Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative
     Inference},
     year = {2024}
   }

10. ASPLOS

    Red-QAOA: Efficient Variational Optimization through Circuit Reduction

    Meng Wang, Bo Fang, Ang Li, and Prashant J. Nair

    In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, La Jolla, CA, USA, 2024

    Abs DOI Bib HTML PDF Code Slides

    

    The Quantum Approximate Optimization Algorithm (QAOA) provides a quantum solution for combinatorial optimization problems. However, the optimal parameter searching process of QAOA is greatly affected by noise, leading to non-optimal solutions. This paper introduces a novel approach to optimize QAOA by exploiting the energy landscape concentration of similar instances via graph reduction, thus addressing the effect of noise. We formalize the notion of similar instances in QAOA and develop a Simulated Annealing-based graph reduction algorithm, called Red-QAOA, to identify the most similar subgraph for efficient parameter optimization. Red-QAOA outperforms state-of-the-art Graph Neural Network (GNN) based graph pooling techniques in performance and demonstrates effectiveness on a diverse set of real-world optimization problems encompassing 3200 graphs. Red-QAOA reduced the node counts and edge counts by 28% and 37%, respectively, while maintaining a low mean square error of 2%. These enable the identification of an optimal parameter set that is closer to the ideal true optimal solution in the presence of noise. By substantially streamlining the search for QAOA parameters, our approach sets the stage for the practical application of quantum algorithms in solving complex optimization problems.

    @inproceedings{wang2024redqaoa,
      address = {New York, NY, USA},
      author = {Wang, Meng and Fang, Bo and Li, Ang and Nair, Prashant J.},
      booktitle = {Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
      doi = {10.1145/3620665.3640363},
      isbn = {9798400703850},
      latest = {true},
      location = {La Jolla, CA, USA},
      numpages = {19},
      pages = {980–998},
      publisher = {Association for Computing Machinery},
      series = {ASPLOS '24},
      title = {Red-QAOA: Efficient Variational Optimization through Circuit Reduction},
      url = {https://doi.org/10.1145/3620665.3640363},
      year = {2024}
    }

2023

1. PACT

   SparseFT: Sparsity-aware Fault Tolerance for Reliable CNN Inference on GPUs

   Gwangeun Byeon, Seungtae Lee, Seongwook Kim, Yongjun Kim, Prashant J. Nair, and Seokin Hong

   In Proceedings of the 32nd International Conference on Parallel Architectures and Compilation Techniques (PACT), Vienna, Austria, Oct 2023

   Abs DOI Bib HTML PDF

   

   Graphics Processing Units (GPUs), while offering exceptional performance for CNN inference tasks, are susceptible to both transient and permanent hardware faults due to the integration of numerous processing elements and advancements in technology scaling. This paper proposes a novel and cost-effective fault mitigation technique, called Sparsity-aware Fault Tolerance (SparseFT), to ensure reliable CNN inference on GPUs. SparseFT leverages inherent sparsity in the activation maps to detect and correct errors on the processing elements without hardware redundancy. By exploiting the characteristic of dot-products, where multiplications with zero operands are ineffectual, SparseFT dynamically duplicates an effectual computation (i.e., a multiplication with non-zero operands) to the processing element initially assigned to the ineffectual one. It then compares the duplicated computation results to detect errors. Experimental results demonstrate that SparseFT achieves more than 97% error detection coverage with less than 1% performance overhead for the state-of-the-art CNN models.

   @inproceedings{byeon2023sparseft,
     author = {Byeon, Gwangeun and Lee, Seungtae and Kim, Seongwook and Kim, Yongjun and Nair, Prashant J. and Hong, Seokin},
     booktitle = {Proceedings of the 32nd International Conference on Parallel Architectures and Compilation Techniques (PACT)},
     doi = {10.1109/PACT58117.2023.00041},
     location = {Vienna, Austria},
     month = oct,
     numpages = {337-338},
     publisher = {IEEE},
     series = {PACT '23},
     title = {SparseFT: Sparsity-aware Fault Tolerance for Reliable CNN Inference on GPUs},
     url = {https://ieeexplore.ieee.org/abstract/document/10364579},
     year = {2023}
   }

2. EIR

   The Dirty Secret of SSDs: Embodied Carbon

   Swamit Tannu and Prashant J. Nair

   In SIGENERGY Energy Informatics Review (EIR), Oct 2023

   Abs DOI Bib HTML PDF

   

   Scalable Solid-State Drives (SSDs) have ushered in a transformative era in data storage and accessibility, spanning both data centers and portable devices. However, the strides made in scaling this technology can bear significant environmental consequences. On a global scale, a notable portion of semiconductor manufacturing relies on electricity derived from coal and natural gas sources. A striking example of this is the manufacturing process for a single Gigabyte of Flash memory, which emits approximately 0.16 Kg of CO2 - a considerable fraction of the total carbon emissions attributed to the system. Remarkably, the manufacturing of storage devices alone contributed to an estimated 20 million metric tonnes of CO2 emissions in the year 2021.In light of these environmental concerns, this paper delves into an analysis of the sustainability trade-offs inherent in Solid-State Drives (SSDs) when compared to traditional Hard Disk Drives (HDDs). Moreover, this study proposes methodologies to gauge the embodied carbon costs associated with storage systems effectively. The research encompasses four key strategies to enhance the sustainability of storage systems.Firstly, the paper offers insightful guidance for selecting the most suitable storage medium, be it SSDs or HDDs, considering the broader ecological impact. Secondly, the paper advocates for implementing techniques that extend the lifespan of SSDs, thereby mitigating premature replacements and their attendant environmental toll. Thirdly, the paper emphasizes the need for efficient recycling and reuse of high-density multi-level cell-based SSDs, underscoring the significance of minimizing electronic waste.Lastly, for handheld devices, the paper underscores the potential of harnessing the elasticity offered by cloud storage solutions as a means to curtail the ecological repercussions of localized data storage. In summation, this study critically addresses the embodied carbon issues associated with SSDs, comparing them with HDDs, and proposes a comprehensive framework of strategies to enhance the sustainability of storage systems.

   @inproceedings{tannu2023dirtyssd,
     address = {New York, NY, USA},
     author = {Tannu, Swamit and Nair, Prashant J.},
     booktitle = {SIGENERGY Energy Informatics Review (EIR)},
     doi = {10.1145/3630614.3630616},
     issue_date = {October 2023},
     keywords = {solid state drives, embodied carbon, hard disk drive, sustainability},
     month = oct,
     number = {3},
     numpages = {6},
     pages = {4–9},
     publisher = {Association for Computing Machinery},
     series = {EIR},
     title = {The Dirty Secret of SSDs: Embodied Carbon},
     volume = {3},
     year = {2023}
   }

3. NeurIPS

   FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout

   Irene Wang, Prashant J. Nair, and Divya Mahajan

   In Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 2023

   Abs Bib HTML PDF Code Slides

   

   Federated Learning (FL) allows machine learning models to train locally on individual mobile devices, synchronizing model updates via a shared server. This approach safeguards user privacy; however, it also generates a heterogeneous training environment due to the varying performance capabilities across devices. As a result, “straggler” devices with lower performance often dictate the overall training time in FL. In this work, we aim to alleviate this performance bottleneck due to stragglers by dynamically balancing the training load across the system. We introduce Invariant Dropout, a method that extracts a sub-model based on the weight update threshold, thereby minimizing potential impacts on accuracy. Building on this dropout technique, we develop an adaptive training framework, Federated Learning using Invariant Dropout (FLuID). FLuID offers a lightweight sub-model extraction to regulate computational intensity, thereby reducing the load on straggler devices without affecting model quality. Our method leverages neuron updates from non-straggler devices to construct a tailored sub-model for each straggler based on client performance profiling. Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime conditions shift. We evaluate FLuID using five real-world mobile clients. The evaluations show that Invariant Dropout maintains baseline model efficiency while alleviating the performance bottleneck of stragglers through a dynamic, runtime approach.

   @inproceedings{wang2023fluid,
     author = {Wang, Irene and Nair, Prashant J. and Mahajan, Divya},
     booktitle = {Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS)},
     location = {New Orleans, LA, USA},
     series = {NeurIPS 2023},
     title = {{FL}u{ID}: Mitigating Stragglers in Federated Learning using Invariant Dropout},
     year = {2023}
   }

4. QCCC

   Efficient QAOA Optimization Using Directed Restarts and Graph Lookup

   Meng Wang, Bo Fang, Ang Li, and Prashant J. Nair

   In Proceedings of the 2023 International Workshop on Quantum Classical Cooperative, Orlando, FL, USA, 2023

   Abs DOI Bib HTML PDF Slides

   

   Variational Quantum Algorithms (VQA) aim to enhance the capabilities of Noisy Intermediate-Scale Quantum (NISQ) devices. These algorithms utilize parameterized circuits and classical optimizers to iteratively execute circuits with varying parameters. However, VQA faces computational overheads due to repeated iterations and random restarts. Prior work suggests using basic sub-graphs to transfer parameters for the input graph, reducing optimizer overheads but limiting applicability to structured regular graphs. In real-world applications, random irregular graphs are common, and existing methods are not scalable or practical for such graphs. This paper presents a framework that aims to improve random irregular graphs in VQA. The framework uses graph similarity and important features like total edge counts, average edge counts, and variance. It follows an iterative process to choose basis sub-graphs from a small database and adjust parameters accordingly. Classical optimizers then utilize these parameters to determine when to restart and perform gradient descent. This approach increases the chances of reaching global maximum points.

   @inproceedings{wang2023graphlookup,
     address = {New York, NY, USA},
     author = {Wang, Meng and Fang, Bo and Li, Ang and Nair, Prashant J.},
     booktitle = {Proceedings of the 2023 International Workshop on Quantum Classical Cooperative},
     doi = {10.1145/3588983.3596680},
     isbn = {9798400701627},
     keywords = {quantum variational algorithms, QAOA parameter optimization, QAOA parameter transfer, parameterized quantum circuits},
     location = {Orlando, FL, USA},
     numpages = {4},
     pages = {5–8},
     publisher = {Association for Computing Machinery},
     series = {QCCC '23},
     title = {Efficient QAOA Optimization Using Directed Restarts and Graph Lookup},
     url = {https://doi.org/10.1145/3588983.3596680},
     year = {2023}
   }

5. SC

   Structural Coding: A Low-Cost Scheme to Protect CNNs from Large-Granularity Memory Faults

   Ali Asgari Khoshouyeh, Florian Geissler, Syed Qutub, Michael Paulitsch, Prashant J. Nair, and Karthik Pattabiraman

   In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), Denver, CO, USA, 2023

   Abs DOI Bib HTML PDF Code Slides

   

   The advent of High-Performance Computing has led to the adoption of Convolutional Neural Networks (CNNs) in safety-critical applications such as autonomous vehicles. However, CNNs are vulnerable to DRAM errors corrupting their parameters, thereby degrading their accuracy. Existing techniques for protecting CNNs from DRAM errors are either expensive or fail to protect from large-granularity, multi-bit errors, which occur commonly in DRAMs.We propose a software-implemented coding scheme, Structural Coding (SC) for protecting CNNs from large-granularity memory errors. SC achieves three orders of magnitude reduction in Silent Data Corruption (SDC) rates of CNNs compared to no protection. Its average error correction coverage is also significantly higher than other software techniques to protect CNNs from faults in the memory. Further, its average performance, memory, and energy overheads are respectively 3%, 15.71%, and 4.38%. These overheads are much lower than other software protection techniques.

   @inproceedings{asgari2023structuralcoding,
     address = {New York, NY, USA},
     articleno = {85},
     author = {Asgari Khoshouyeh, Ali and Geissler, Florian and Qutub, Syed and Paulitsch, Michael and Nair, Prashant J. and Pattabiraman, Karthik},
     booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC)},
     doi = {10.1145/3581784.3607084},
     isbn = {9798400701092},
     keywords = {error correction, deep neural networks, memory faults},
     location = {Denver, CO, USA},
     numpages = {17},
     publisher = {Association for Computing Machinery},
     series = {SC '23},
     title = {Structural Coding: A Low-Cost Scheme to Protect CNNs from Large-Granularity Memory Faults},
     url = {https://doi.org/10.1145/3581784.3607084},
     year = {2023}
   }

6. SC-W

   Enabling Scalable VQE Simulation on Leading HPC Systems

   Meng Wang, Fei Hua, Chenxu Liu, Nicholas Bauman, Karol Kowalski, Daniel Claudino, Travis Humble, Prashant J. Nair, and Ang Li

   In Proceedings of the SC ’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, Denver, CO, USA, , 2023

   Abs DOI Bib HTML PDF

   

   Large-scale simulations of quantum circuits pose significant challenges, especially in quantum chemistry, due to the number of qubits, circuit depth, and the number of circuits needed per problem. High-performance computing (HPC) systems offer massive computational capabilities that could help overcome these obstacles. We developed a high-performance quantum circuit simulator called NWQ-Sim, and demonstrated its capability to simulate large quantum chemistry problems on NERSC’s Perlmutter supercomputer. Integrating NWQ-Sim with XACC, an open-source programming framework for quantum-classical applications, we have executed quantum phase estimation (QPE) and variational quantum eigensolver (VQE) algorithms for downfolded quantum chemistry systems at unprecedented scales. Our work demonstrates the potential of leveraging HPC resources and optimized simulators to advance quantum chemistry and other applications of near-term quantum devices. By scaling to larger qubit counts and circuit depths, high-performance simulators like NWQ-Sim will be critical for characterizing and validating quantum algorithms before their deployment on actual quantum hardware.

   @inproceedings{wang2023vqesim,
     address = {New York, NY, USA},
     author = {Wang, Meng and Hua, Fei and Liu, Chenxu and Bauman, Nicholas and Kowalski, Karol and Claudino, Daniel and Humble, Travis and Nair, Prashant J. and Li, Ang},
     booktitle = {Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis},
     doi = {10.1145/3624062.3624221},
     isbn = {9798400707858},
     keywords = {quantum algorithms, Quantum computing, quantum simulation, hybrid quantum-classical computing, quantum chemistry, high-performance computing, variational algorithms},
     location = {Denver, CO, USA, },
     numpages = {8},
     pages = {1460–1467},
     publisher = {Association for Computing Machinery},
     series = {SC-W '23},
     title = {Enabling Scalable VQE Simulation on Leading HPC Systems},
     url = {https://doi.org/10.1145/3624062.3624221},
     year = {2023}
   }

7. HPCA

   Scalable and Secure Row-Swap: Efficient and Safe Row Hammer Mitigation in Memory Systems

   Jeonghyun Woo, Gururaj Saileshwar, and Prashant J. Nair

   In Proceedings of the 29th International Symposium on High Performance Computer Architecture (HPCA), Montreal, QC, Canada, 2023

   Best Paper Award Abs DOI Bib HTML PDF Code Slides

   

   Only two papers out of the 360 submitted papers were chosen as the HPCA 2023 Best Papers.

   As Dynamic Random Access Memories (DRAM) scale, they are becoming increasingly susceptible to Row Hammer. By rapidly activating rows of DRAM cells (aggressor rows), attackers can exploit inter-cell interference through Row Hammer to flip bits in neighboring rows (victim rows). A recent work, called Randomized Row-Swap (RRS), proposed proactively swapping aggressor rows with randomly selected rows before an aggressor row can cause Row Hammer. RRS promises several years of protection against Row Hammer attacks. Our paper observes that RRS is neither secure nor scalable. We first propose the ’Juggernaut attack pattern’ that breaks RRS in under 1 day. Juggernaut exploits the fact that the mitigative action of RRS, a swap operation, can itself induce additional target row activations, defeating such a defense. Second, this paper proposes a new defense Secure Row-Swap mechanism that avoids the additional activations from swap (and unswap) operations and protects against Juggernaut. Furthermore, this paper extends Secure Row-Swap with attack detection to defend against even future attacks. While this provides better security, it also allows reducing the frequency of swaps securely, thereby enabling Scalable and Secure Row-Swaps. Our analysis shows that the Scalable and Secure Row-Swap mechanism provides years of Row Hammer protection with 3.3x lower storage overheads as compared to the RRS design. Additionally, it incurs only a 0.7% slowdown as compared to a not-secure baseline for a Row Hammer threshold of 1.2K.

   @inproceedings{woo2023rowswap,
     author = {Woo, Jeonghyun and Saileshwar, Gururaj and Nair, Prashant J.},
     booktitle = {Proceedings of the 29th International Symposium on High Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA56546.2023.10070999},
     keywords = {Correlation;Memory management;Random access memory;System-on-chip;Security;Intercell interference},
     location = {Montreal, QC, Canada},
     number = {},
     pages = {374-389},
     series = {HPCA 2023},
     title = {Scalable and Secure Row-Swap: Efficient and Safe Row Hammer Mitigation in Memory Systems},
     volume = {},
     year = {2023}
   }

8. ASPLOS

   HuffDuff: Stealing Pruned DNNs from Sparse Accelerators

   Dingqing Yang, Prashant J. Nair, and Mieszko Lis

   In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Volume 2, Vancouver, BC, Canada, 2023

   Abs DOI Bib HTML PDF Slides

   

   Deep learning models are a valuable “secret sauce” that confers a significant competitive advantage. Many models are never visible to the user and even publicly known state-of-the-art models are either completely proprietary or only accessible via access-controlled APIs. Increasingly, these models run directly on the edge, often using a low-power DNN accelerator. This makes models particularly vulnerable, as an attacker with physical access can exploit side channels like off-chip memory access volumes. Indeed, prior work has shown that this channel can be used to steal dense DNNs from edge devices by correlating data transfer volumes with layer geometry. Unfortunately, prior techniques become intractable when the model is sparse in either weights or activations because off-chip transfers no longer correspond exactly to layer dimensions. Could it be that the many mobile-class sparse accelerators are inherently safe from this style of attack? In this paper, we show that it is feasible to steal a pruned DNN model architecture from a mobile-class sparse accelerator using the DRAM access volume channel. We describe HuffDuff, an attack scheme with two novel techniques that leverage (i) ‍the boundary effect present in CONV layers, and (ii) ‍the timing side channel of on-the-fly activation compression. Together, these techniques dramatically reduce the space of possible model architectures up to 94 orders of magnitude, resulting in fewer than 100 candidate models — a number that can be feasibly tested. Finally, we sample network instances from our solution space and show that (i) ‍our solutions reach the victim accuracy under the iso-footprint constraint, and (ii) ‍significantly improve black-box targeted attack success rates.

   @inproceedings{yang2023huffduff,
     address = {New York, NY, USA},
     author = {Yang, Dingqing and Nair, Prashant J. and Lis, Mieszko},
     booktitle = {Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Volume 2},
     doi = {10.1145/3575693.3575738},
     isbn = {9781450399166},
     keywords = {Sparse DNN accelerators, Side-channel attacks},
     location = {Vancouver, BC, Canada},
     numpages = {15},
     pages = {385–399},
     publisher = {Association for Computing Machinery},
     series = {ASPLOS 2023},
     title = {HuffDuff: Stealing Pruned DNNs from Sparse Accelerators},
     url = {https://doi.org/10.1145/3575693.3575738},
     year = {2023}
   }

2022

1. MICRO

   AQUA: Scalable Rowhammer Mitigation by Quarantining Aggressor Rows at Runtime

   Anish Saxena, Gururaj Saileshwar, Prashant J. Nair, and Moinuddin K. Qureshi

   In Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2022

   Abs DOI Bib HTML PDF Code Slides

   

   Rowhammer allows an attacker to induce bit flips in a row by rapidly accessing neighboring rows. Rowhammer is a severe security threat as it can be used to escalate privilege or break confidentiality. Moreover, the threshold of activations needed to induce Rowhammer continues to reduce and new attacks like Half-Double break existing solutions that refresh victim rows. The recently proposed Randomized Row-Swap (RRS) scheme is resilient to Half-Double as it provides mitigation by swapping an aggressor row with a random row. However, to ensure security, the threshold for triggering a row-swap must be set much lower than the Rowhammer threshold, leading to a significant performance loss of 20% on average, at a Rowhammer threshold of 1K. Furthermore, the SRAM overhead for storing the indirection table of RRS becomes prohibitively large – 2.4MB per rank at a Rowhammer threshold of 1K. Our goal is to develop a scalable Rowhammer mitigation that incurs negligible performance and storage overheads. To this end, we propose AQUA, a Rowhammer mitigation that breaks the spatial correlation between aggressor and victim rows by dynamically quarantining the aggressor row in a dedicated region of memory. AQUA allows for an effective row migration threshold much higher than in RRS, leading to an order of magnitude less slowdown and SRAM. As the security of AQUA is not reliant on keeping the destination row a secret, we further reduce the SRAM overheads of the indirection table by storing it in DRAM, and accessing it on-demand. We derive the size of the quarantine region required to ensure security for AQUA and show that reserving about 1% of DRAM is sufficient to mitigate Rowhammer at a threshold of 1K. Our evaluations show that AQUA incurs an average slowdown of 2% and an SRAM overhead (for mapping and migration) of only 41KB per rank at a Rowhammer threshold of 1K.

   @inproceedings{saxena2022aqua,
     author = {Saxena, Anish and Saileshwar, Gururaj and Nair, Prashant J. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1109/MICRO56248.2022.00022},
     number = {},
     pages = {108-123},
     title = {AQUA: Scalable Rowhammer Mitigation by Quarantining Aggressor Rows at Runtime},
     volume = {},
     year = {2022}
   }

2. ISCA

   Hydra: Enabling Low-Overhead Mitigation of Row-Hammer at Ultra-Low Thresholds via Hybrid Tracking

   Moinuddin K. Qureshi, Aditya Rohan, Gururaj Saileshwar, and Prashant J. Nair

   In Proceedings of the 49th International Symposium on Computer Architecture (ISCA), New York, New York, 2022

   Abs DOI Bib HTML PDF Code Slides

   

   DRAM systems continue to be plagued by the Row-Hammer (RH) security vulnerability. The threshold number of row activations (TRH) required to induce RH has reduced rapidly from 139K in 2014 to 4.8K in 2020, and TRH is expected to reduce further, making RH even more severe for future DRAM. Therefore, solutions for mitigating RH should be effective not only at current TRH but also at future TRH . In this paper, we investigate the mitigation of RH at ultra-low thresholds (500 and below). At such thresholds, state-of-the-art solutions, which rely on SRAM or CAM for tracking row activations, incur impractical storage overheads (340KB or more per rank at TRH of 500), making such solutions unappealing for commercial adoption. Alternative solutions, which store per-row metadata in the addressable DRAM space, incur significant slowdown (25% on average) due to extra memory accesses, even in the presence of metadata caches. Our goal is to develop scalable RH mitigation while incurring low SRAM and performance overheads. To that end, this paper proposes Hydra, a Hybrid Tracker for RH mitigation, which combines the best of both SRAM and DRAM to enable low-cost mitigation of RH at ultra-low thresholds. Hydra consists of two structures. First, an SRAM-based structure that tracks aggregated counts at the granularity of a group of rows, and is sufficient for the vast majority of rows that receive only a few activations. Second, a per-row tracker stored in the DRAM-array, which can track an arbitrary number of rows, however, to limit performance overheads, this tracker is used only for the small number of rows that exceed the tracking capability of the SRAM-based structure. We provide a security analysis of Hydra to show that Hydra can reliably issue a mitigation within the specified threshold. Our evaluations show that Hydra enables robust mitigation of RH, while incurring an SRAM overhead of only 28 KB per-rank and an average slowdown of only 0.7% (at TRH of 500).

   @inproceedings{qureshi2022hydra,
     address = {New York, NY, USA},
     author = {Qureshi, Moinuddin K. and Rohan, Aditya and Saileshwar, Gururaj and Nair, Prashant J.},
     booktitle = {Proceedings of the 49th International Symposium on Computer Architecture (ISCA)},
     doi = {10.1145/3470496.3527421},
     isbn = {9781450386104},
     keywords = {security, memory system, row-hammer, DRAM, reliability},
     location = {New York, New York},
     numpages = {12},
     pages = {699–710},
     publisher = {Association for Computing Machinery},
     series = {ISCA '22},
     title = {Hydra: Enabling Low-Overhead Mitigation of Row-Hammer at Ultra-Low Thresholds via Hybrid Tracking},
     url = {https://doi.org/10.1145/3470496.3527421},
     year = {2022}
   }

3. VLDB

   Accelerating Recommendation System Training by Leveraging Popular Choices

   Muhammad Adnan, Yassaman Maboud, Divya Mahajan, and Prashant J. Nair

   In Proceedings of the 48th International Conference on Very Large Data Bases (VLDB), 2022

   Abs DOI Bib HTML PDF Code Slides

   

   Recommender models are commonly used to suggest relevant items to a user for e-commerce and online advertisement-based applications. These models use massive embedding tables to store numerical representation of items’ and users’ categorical variables (memory intensive) and employ neural networks (compute intensive) to generate final recommendations. Training these large-scale recommendation models is evolving to require increasing data and compute resources. The highly parallel neural networks portion of these models can benefit from GPU acceleration however, large embedding tables often cannot fit in the limited-capacity GPU device memory. Hence, this paper deep dives into the semantics of training data and obtains insights about the feature access, transfer, and usage patterns of these models. We observe that, due to the popularity of certain inputs, the accesses to the embeddings are highly skewed with a few embedding entries being accessed up to 10000x more. This paper leverages this asymmetrical access pattern to offer a framework, called FAE, and proposes a hot-embedding aware data layout for training recommender models. This layout utilizes the scarce GPU memory for storing the highly accessed embeddings, thus reduces the data transfers from CPU to GPU. At the same time, FAE engages the GPU to accelerate the executions of these hot embedding entries. Experiments on production-scale recommendation models with real datasets show that FAE reduces the overall training time by 2.3x and 1.52x in comparison to XDL CPU-only and XDL CPU-GPU execution while maintaining baseline accuracy.

   @inproceedings{adnan2022hotembeddings,
     author = {Adnan, Muhammad and Maboud, Yassaman and Mahajan, Divya and Nair, Prashant J.},
     booktitle = {Proceedings of the 48th International Conference on Very Large Data Bases (VLDB)},
     doi = {10.14778/3485450.3485462},
     issn = {2150-8097},
     title = {Accelerating Recommendation System Training by Leveraging Popular Choices},
     year = {2022}
   }

4. ASPLOS

   Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation Between Aggressor and Victim Rows

   Gururaj Saileshwar, Bolin Wang, Moinuddin K. Qureshi, and Prashant J. Nair

   In Proceedings of the 27th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022

   Abs DOI Bib HTML PDF Code Slides

   

   Row-Hammer (RH) is a fault-injection attack that occurs when a DRAM row is accessed frequently causing bit-flips in nearby rows. RH is a serious security threat as attackers can cause bit-flips in page-tables to achieve privilege escalation. Several defenses have been recently proposed that track attacker-controlled aggressor-rows and apply mitigating action on immediate neighboring victim rows by refreshing them. However, all such proposals using victim-focused mitigation preserve the spatial connection between victim and aggressor rows, and are hence susceptible to access patterns causing bit-flips in rows beyond the immediate neighbor. For example, the recently disclosed Half-Double attack causes bit-flips in the presence of existing victim-focused mitigation. We propose Randomized Row-Swap (RRS), a novel mitigation action that breaks the spatial connection between the aggressor and victim rows, thus providing robust defense against even complex RH access patterns. RRS is an aggressor-focused mitigation that periodically swaps aggressor-rows with other randomly selected rows in memory, to limit the possible damage in any one locality. While RRS can be used with any tracking mechanism, we implement it with the recently proposed Misra-Gries tracker and target a RH Threshold of 4K activations (lower than recent attacks). Our evaluations show that RRS has negligible performance and power overheads (less than 0.5% on average) and provides strong security guarantees for avoiding RH bit flips throughout system lifetime.

   @inproceedings{saileshwar2022rrs,
     address = {New York, NY, USA},
     author = {Saileshwar, Gururaj and Wang, Bolin and Qureshi, Moinuddin K. and Nair, Prashant J.},
     booktitle = {Proceedings of the 27th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)},
     doi = {10.1145/3503222.3507716},
     isbn = {978145039205},
     numpages = {14},
     publisher = {Association for Computing Machinery},
     title = {Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation Between Aggressor and Victim Rows},
     url = {https://doi.org/10.1145/3503222.3507716},
     year = {2022}
   }

5. HPCA

   SafeGuard: Reducing the Security Risk from Row-Hammer via Low-Cost Integrity Protection

   Ali Fakhrzadehgan, Yale N. Patt, Prashant J. Nair, and Moinuddin K. Qureshi

   In Proceedings of the 28th International Symposium on High-Performance Computer Architecture (HPCA), Apr 2022

   Abs DOI Bib

   

   Row-Hammer (RH) is a DRAM data-disturbance failure that occurs when a row is activated frequently, which causes bit-flips in nearby rows. Row-Hammer is a significant security threat as an attacker can exploit the bit-flips to do privilege escalation and leak confidential data. While several solutions aim to mitigate RH, such solutions depend on the RH threshold and adversarial access patterns. Solutions developed for a given threshold become ineffective for newer devices with lower thresholds, and new attack patterns continue to break existing mitigations. Currently, there is no guaranteed solution for RH, which means that the system remains vulnerable to security threats even in the presence of RH mitigation. In this paper, we contend that simply relying on RH mitigation is insufficient to provide security in the presence of reducing threshold and motivated attackers. We propose SafeGuard, which equips the system with low-cost integrity protection as a defense against potential attacks that break the RH mitigation. As SafeGuard can detect arbitrary failures, it converts the problem of RH bit-flips from a security threat (silent consumption of corrupted data) to a reliability concern (detectable uncorrectable errors caused by integrity violation). We develop SafeGuard for systems that employ ECC modules and show that SafeGuard can provide strong detection to both SECDED (46-bit MAC per cache-line) and Chipkill (32-bit MAC per cache-line) while retaining the correction capability of conventional designs. SafeGuard avoids incurring any storage overheads in DRAM by simply reorganizing the ECC code to operate at a cache-line granularity (64 bytes) instead of a word granularity (8 bytes). Our evaluations show that SafeGuard has a negligible impact on both the system performance (0.7%) and the system reliability due to naturally occurring errors while still providing a strong defense against the security risk of RH by detecting arbitrary bit-flips.

   @inproceedings{fakhrzadehgan2022safeguard,
     author = {Fakhrzadehgan, Ali and Patt, Yale N. and Nair, Prashant J. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 28th International Symposium on High-Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA53966.2022.00035},
     issn = {2378-203X},
     keywords = {Computational modeling;System performance;Random access memory;Computer architecture;Error correction codes;Security;Reliability;Row-Hammer;Reliability;Security;Integrity},
     month = apr,
     number = {},
     pages = {373-386},
     title = {SafeGuard: Reducing the Security Risk from Row-Hammer via Low-Cost Integrity Protection},
     volume = {},
     year = {2022}
   }

2021

1. DATE

   A Case for Emerging Memories in DNN Accelerators

   Avilash Mukherjee, Kumar Saurav, Prashant J. Nair, Sudip Shekhar, and Mieszko Lis

   In Proceedings of the 24th Conference on Design, Automation and Test in Europe (DATE), Feb 2021

   Abs DOI Bib HTML PDF Slides

   

   The popularity of Deep Neural Networks (DNNs) has led to many DNN accelerator architectures, which typically focus on the on-chip storage and computation costs. However, much of the energy is spent on accesses to off-chip DRAM memory. While emerging resistive memory technologies such as MRAM, PCM, and RRAM can potentially reduce this energy component, they suffer from drawbacks such as low endurance that prevent them from being a DRAM replacement in DNN applications. In this paper, we examine how DNN accelerators can be designed to overcome these limitations and how emerging memories can be used for off-chip storage. We demonstrate that through (a) careful mapping of DNN computation to the accelerator and (b) a hybrid setup (both DRAM and an emerging memory), we can reduce inference energy over a DRAM-only design by a factor ranging from 1.12× on EfficientNetB7 to 6.3× on ResNet-50, while also increasing the endurance from 2 weeks to over a decade. As the energy benefits vary dramatically across DNN models, we also develop a simple analytical heuristic solely based on DNN model parameters that predicts the suitability of a given DNN for emerging-memory-based accelerators.

   @inproceedings{mukherjee2021emergingmemdnn,
     author = {Mukherjee, Avilash and Saurav, Kumar and Nair, Prashant J. and Shekhar, Sudip and Lis, Mieszko},
     booktitle = {Proceedings of the 24th Conference on Design, Automation and Test in Europe (DATE)},
     doi = {10.23919/DATE51398.2021.9474252},
     issn = {1558-1101},
     keywords = {},
     month = feb,
     number = {},
     pages = {938-941},
     title = {A Case for Emerging Memories in DNN Accelerators},
     volume = {},
     year = {2021}
   }

2020

1. ASPLOS

   Thesaurus: Efficient Cache Compression via Dynamic Clustering

   Amin Ghasemazar, Prashant J. Nair, and Mieszko Lis

   In Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Lausanne, Switzerland, 2020

   Abs DOI Bib HTML PDF Code Slides

   

   In this paper, we identify a previously untapped source of compressibility in cache working sets: clusters of cachelines that are similar, but not identical, to one another. To compress the cache, we can then store the "clusteroid" of each cluster together with the (much smaller) "diffs" needed to reconstruct the rest of the cluster. To exploit this opportunity, we propose a hardware-level on-line cacheline clustering mechanism based on locality-sensitive hashing. Our method dynamically forms clusters as they appear in the data access stream and retires them as they disappear from the cache. Our evaluations show that we achieve 2.25x compression on average (and up to 9.9x) on SPEC-CPU 2017 suite and is significantly higher than prior proposals scaled to an iso-silicon budget.

   @inproceedings{ghasemazar2020thesaurus,
     address = {New York, NY, USA},
     author = {Ghasemazar, Amin and Nair, Prashant J. and Lis, Mieszko},
     booktitle = {Proceedings of the 25th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)},
     doi = {10.1145/3373376.3378518},
     isbn = {9781450371025},
     keywords = {cache compression, dynamic clustering, memory hierarchy, lsh},
     location = {Lausanne, Switzerland},
     numpages = {14},
     pages = {527–540},
     publisher = {Association for Computing Machinery},
     series = {ASPLOS '20},
     title = {Thesaurus: Efficient Cache Compression via Dynamic Clustering},
     url = {https://doi.org/10.1145/3373376.3378518},
     year = {2020}
   }

2. DATE

   2DCC: Cache Compression in Two Dimensions

   Amin Ghasemazar, Mohammad Ewais, Prashant J. Nair, and Mieszko Lis

   In Proceedings of the 23rd Conference on Design, Automation and Test in Europe (DATE), Grenoble, France, 2020

   Abs Bib HTML PDF Code Slides

   

   The importance of caches for performance, and their high silicon area cost, have motivated hardware solutions that transparently compress the cached data to increase effective capacity without sacrificing silicon area. To this end, prior work has taken one of two approaches: either (a) deduplicating identical cache blocks across the cache to take advantage of inter-block redundancy or (b) compressing common patterns within each cache block to take advantage of intra-block redundancy.In this paper, we demonstrate that leveraging only one of these redundancy types leads to a significant loss in compression opportunities for several applications: some workloads exhibit either inter-block or intra-block redundancy, while others exhibit both. We propose 2DCC (Two Dimensional Cache Compression), a simple technique that takes advantage of both types of redundancy. Across the SPEC and Parsec benchmark suites, 2DCC results in a 2.12x compression factor (geomean) compared to 1.44–1.49x for best prior techniques on an iso-silicon basis. For the cache-sensitive subset of these benchmarks run in isolation, 2DCC also achieves a 11.7% speedup (geomean).

   @inproceedings{ghasemazar20202dcc,
     address = {San Jose, CA, USA},
     author = {Ghasemazar, Amin and Ewais, Mohammad and Nair, Prashant J. and Lis, Mieszko},
     booktitle = {Proceedings of the 23rd Conference on Design, Automation and Test in Europe (DATE)},
     isbn = {9783981926347},
     location = {Grenoble, France},
     numpages = {6},
     pages = {756–761},
     publisher = {EDA Consortium},
     series = {DATE '20},
     title = {2DCC: Cache Compression in Two Dimensions},
     year = {2020}
   }

3. ICCD

   ADAM: Adaptive Block Placement with Metadata Embedding for Hybrid Caches

   Beomjun Kim, Prashant J. Nair, and Seokin Hong

   In Proceedings of the 38th International Conference on Computer Design (ICCD), Oct 2020

   Abs DOI Bib HTML PDF Slides

   

   Spin-Transfer Torque Random Access Memory (STT-RAM) is a potential alternative for SRAM-based on-chip caches. STT-RAM offers high density and low leakage power, thereby can be used to build a large capacity last-level caches (LLC). Unfortunately, the write latency of the STT-RAM is significantly longer, and its write energy is considerably higher compared to SRAM. To mitigate these concerns, researchers have proposed hybrid caches that are comprised of SRAM and STT-RAM regions. In such hybrid caches, an intelligent block placement policy is necessary to store as many write-intensive blocks in the SRAM region. This paper proposes an adaptive block placement framework with metadata embedding (ADAM) for hybrid caches. ADAM embeds metadata (i.e., write-intensity) into a cache block when it is evicted from LLC. When a cache block is brought from the main memory, metadata embedded in the block is extracted and used to determine the write-intensity of the block. Our evaluation shows that ADAM can improve performance by 26 % (on average) over a baseline block placement scheme.

   @inproceedings{kim2020adam,
     author = {Kim, Beomjun and Nair, Prashant J. and Hong, Seokin},
     booktitle = {Proceedings of the 38th International Conference on Computer Design (ICCD)},
     doi = {10.1109/ICCD50377.2020.00077},
     issn = {2576-6996},
     keywords = {},
     month = oct,
     number = {},
     pages = {421-424},
     publisher = {IEEE},
     title = {ADAM: Adaptive Block Placement with Metadata Embedding for Hybrid Caches},
     volume = {},
     year = {2020}
   }

2019

1. MICRO

   Touché: Towards Ideal and Efficient Cache Compression By Mitigating Tag Area Overheads

   Seokin Hong, Bulent Abali, Alper Buyuktosunoglu, Michael B. Healy, and Prashant J. Nair

   In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Columbus, OH, USA, 2019

   Abs DOI Bib HTML PDF Slides

   

   Compression is seen as a simple technique to increase the effective cache capacity. Unfortunately, compression techniques either incur tag area overheads or restrict cache block placement to only include neighboring addresses. Ideally, we should be able to place compressed cache blocks without any restrictions or overheads.This paper proposes Touché, a framework for storing multiple compressed blocks from arbitrary addresses within a cacheline without tag area overheads. The Touché framework consists of three components. The first component, called the "Signature" (SIGN) engine, creates shortened signatures from the tag addresses of compressed blocks. Due to this, the SIGN engine can store multiple signatures in each tag entry. On a cache access, the physical cacheline is accessed only if there is a signature match (which has a negligible probability of false positive). The second component, called the "Tag Appended Data" (TADA) mechanism, stores the full tag addresses with data. TADA enables Touché to detect false positive signature matches by providing the full tag address. The third component, called the "Superblock Marker" (SMARK) mechanism, uses a unique marker in the tag entry to indicate compressed cache blocks from neighboring physical addresses in the same cacheline. Touché is hardware-based and achieves an average speedup of 12% (ideal 13%) when compared to an uncompressed baseline.

   @inproceedings{hong2019touche,
     address = {New York, NY, USA},
     author = {Hong, Seokin and Abali, Bulent and Buyuktosunoglu, Alper and Healy, Michael B. and Nair, Prashant J.},
     booktitle = {Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1145/3352460.3358281},
     isbn = {9781450369381},
     keywords = {Hashing, Data Array, Compression, Tag Array, Caches},
     location = {Columbus, OH, USA},
     numpages = {13},
     pages = {453–465},
     publisher = {Association for Computing Machinery},
     series = {MICRO '52},
     title = {Touch\'{e}: Towards Ideal and Efficient Cache Compression By Mitigating Tag Area Overheads},
     url = {https://doi.org/10.1145/3352460.3358281},
     year = {2019}
   }

2. MICRO

   A Case for Multi-Programming Quantum Computers

   Poulami Das, Swamit S. Tannu, Prashant J. Nair, and Moinuddin Qureshi

   In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Columbus, OH, USA, 2019

   Abs DOI Bib HTML PDF Slides

   

   Existing and near-term quantum computers face significant reliability challenges because of high error rates caused by noise. Such machines are operated in the Noisy Intermediate Scale Quantum (NISQ) model of computing. As NISQ machines exhibit high error-rates, only programs that require a few qubits can be executed reliably. Therefore, NISQ machines tend to underutilize its resources. In this paper, we propose to improve the throughput and utilization of NISQ machines by using multi-programming and enabling the NISQ machine to concurrently execute multiple workloads.Multi-programming a NISQ machine is non-trivial. This is because, a multi-programmed NISQ machine can have an adverse impact on the reliability of the individual workloads. To enable multi-programming in a robust manner, we propose three solutions. First, we develop methods to partition the qubits into multiple reliable regions using error information from machine calibration so that each program can have a fair allocation of reliable qubits. Second, we observe that when two programs are of unequal lengths, measurement operations can impact the reliability of the co-running program. To reduce this interference, we propose a Delayed Instruction Scheduling (DIS) policy that delays the start of the shorter program so that all the measurement operations can be performed at the end. Third, we develop an Adaptive Multi-Programming (AMP) design that monitors the reliability at runtime and reverts to single program mode if the reliability impact of multi-programming is greater than a predefined threshold. Our evaluations with IBM-Q16 show that our proposals can improve resource utilization and throughput by up to 2x, while limiting the impact on reliability.

   @inproceedings{das2019multiprogrammingquantum,
     address = {New York, NY, USA},
     author = {Das, Poulami and Tannu, Swamit S. and Nair, Prashant J. and Qureshi, Moinuddin},
     booktitle = {Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1145/3352460.3358287},
     isbn = {9781450369381},
     keywords = {NISQ, Quantum Computer, Multi-programming, Reliability},
     location = {Columbus, OH, USA},
     numpages = {13},
     pages = {291–303},
     publisher = {Association for Computing Machinery},
     series = {MICRO '52},
     title = {A Case for Multi-Programming Quantum Computers},
     url = {https://doi.org/10.1145/3352460.3358287},
     year = {2019}
   }

3. DSN

   SuDoku: Tolerating High-Rate of Transient Failures for Enabling Scalable STTRAM

   Prashant J. Nair, Bahar Asgari, and Moinuddin K. Qureshi

   In Proceedings of the 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Jun 2019

   Abs DOI Bib HTML PDF Slides

   

   Conventionally, systems have relied on technology scaling to provide smaller cells, which helps in increasing the capacity of on-chip and off-chip structures. Unfortunately, scaling technology to smaller nodes causes increased susceptibility to faults. We study the problem of efficiently tolerating transient failures using scalable Spin-Transfer Torque RAM (STTRAM) as an example. At smaller feature sizes, the energy required to flip a STTRAM cell reduces, which makes these cells more susceptible to random failures caused by thermal noise. Such failures can be tolerated by periodic scrubbing and provisioning each line with Error Correction Code (ECC). However, to tolerate the desired bit-error-rate, the cache needs ECC-6 (six bit error correction) per line, incurring impractical storage overheads. Ideally, we want to tolerate these faults without relying on multi-bit ECC. We propose SuDoku, a design that provisions each line with ECC-1 and a strong error detection code, and relies on a region-based RAID-4 to perform correction of multi-bit errors. Unfortunately, simply having such a RAID-4 based architecture is ineffective at tolerating a high-rate of transient faults and provides an MTTF in the order of only a few seconds. We describe a novel data resurrection scheme that can repair multiple faulty lines in a RAID-4 region to increase the MTTF to several hours. We propose an extension of SuDoku, which hashes a given line into two regions of RAID-4 to significantly enhance reliability and increase the MTTF to trillions of hours. Our evaluations show that SuDoku provides 874x higher reliability than ECC-6, incurs 30% less storage than ECC-6, and performs within 0.1% of an ideal fault-free baseline.

   @inproceedings{nair2019sudoku,
     author = {Nair, Prashant J. and Asgari, Bahar and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)},
     doi = {10.1109/DSN.2019.00048},
     issn = {1530-0889},
     keywords = {},
     month = jun,
     number = {},
     pages = {388-400},
     title = {SuDoku: Tolerating High-Rate of Transient Failures for Enabling Scalable STTRAM},
     volume = {},
     year = {2019}
   }

2018

1. MICRO

   Attaché: Towards Ideal Memory Compression by Mitigating Metadata Bandwidth Overheads

   Seokin Hong, Prashant J. Nair, Bulent Abali, Alper Buyuktosunoglu, Kyu-Hyoun Kim, and Michael B. Healy

   In Proceedings of the 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Fukuoka, Japan, 2018

   Abs DOI Bib HTML PDF Slides

   

   Memory systems are becoming bandwidth constrained and data compression is seen as a simple technique to increase their effective bandwidth. However, data compression requires accessing Metadata which incurs additional bandwidth overheads. Even after using a Metadata-Cache, the bandwidth overheads of Metadata can reduce the benefits of compression.This paper proposes Attaché, a framework that reduces the overheads of Metadata accesses. The Attaché framework consists of two components. The first component, called the Blended Metadata Engine (BLEM), enables data and its Metadata to be accessed together. BLEM incurs additional Metadata accesses only 0.003% times and removes almost all Metadata bandwidth overheads. The second component, called the Compression Predictor (COPR), predicts if the memory block is compressed. The COPR predictor uses a fine-grained line-level predictor, a coarse-grained page-level predictor, and a global indicator. This enables Attaché to predict the compressibility of the memory block before sending a memory read request. We implement Attaché on a memory system that uses Sub-Ranking. On average, Attaché achieves 15.3% speedup (ideal 17%) and saves 22% energy consumption (ideal 23%) when compared to a baseline system that does not employ data compression. Attaché is completely hardware-based and uses only 368KB of SRAM.

   @inproceedings{hong2018attache,
     author = {Hong, Seokin and Nair, Prashant J. and Abali, Bulent and Buyuktosunoglu, Alper and Kim, Kyu-Hyoun and Healy, Michael B.},
     booktitle = {Proceedings of the 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1109/MICRO.2018.00034},
     isbn = {9781538662403},
     keywords = {memory systems, metadata, sub-ranking, data compression, bandwidth},
     location = {Fukuoka, Japan},
     numpages = {13},
     pages = {326–338},
     publisher = {IEEE Press},
     series = {MICRO-51},
     title = {Attach\'{e}: Towards Ideal Memory Compression by Mitigating Metadata Bandwidth Overheads},
     url = {https://doi.org/10.1109/MICRO.2018.00034},
     year = {2018}
   }

2. MICRO

   Morphable Counters: Enabling Compact Integrity Trees For Low-Overhead Secure Memories

   Gururaj Saileshwar, Prashant J. Nair, Prakash Ramrakhyani, Wendy Elsasser, Jose A. Joao, and Moinuddin K. Qureshi

   In Proceedings of the 51th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Oct 2018

   Abs DOI Bib HTML PDF Code Slides

   

   Securing off-chip main memory is essential for protection from adversaries with physical access to systems. However, current secure-memory designs incur considerable performance overheads - a major cause being the multiple memory accesses required for traversing an integrity-tree, that provides protection against man-in-the-middle attacks or replay attacks. In this paper, we provide a scalable solution to this problem by proposing a compact integrity tree design that requires fewer memory accesses for its traversal. We enable this by proposing new storage-efficient representations for the counters used for encryption and integrity-tree in secure memories. Our Morphable Counters are more cacheable on-chip, as they provide more counters per cacheline than existing split counters. Additionally, they incur lower overheads due to counter-overflows, by dynamically switching between counter representations based on usage pattern. We show that using Morphable Counters enables a 128-ary integrity-tree, that can improve performance by 6.3% on average (up to 28.3%) and reduce system energy-delay product by 8.8% on average, compared to an aggressive baseline using split counters with a 64-ary integrity-tree. These benefits come without any additional storage or reduction in security and are derived from our compact counter representation, that reduces the integrity-tree size for a 16GB memory from 4MB in the baseline to 1MB. Compared to recently proposed VAULT, our design provides a speedup of 13.5% on average (up to 47.4%).

   @inproceedings{saileshwar2018morphablecounters,
     author = {Saileshwar, Gururaj and Nair, Prashant J. and Ramrakhyani, Prakash and Elsasser, Wendy and Joao, Jose A. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 51th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1109/MICRO.2018.00041},
     issn = {},
     keywords = {},
     month = oct,
     number = {},
     pages = {416-427},
     title = {Morphable Counters: Enabling Compact Integrity Trees For Low-Overhead Secure Memories},
     volume = {},
     year = {2018}
   }

3. HPCA

   SYNERGY: Rethinking Secure-Memory Design for Error-Correcting Memories

   Gururaj Saileshwar, Prashant J. Nair, Prakash Ramrakhyani, Wendy Elsasser, and Moinuddin K. Qureshi

   In Proceedings of the 24th International Symposium on High Performance Computer Architecture (HPCA), Feb 2018

   MICRO Top Picks (Honorable Mention) Abs DOI Bib HTML PDF Code Slides

   

   An honorable mention to be among the top papers in computer architecture in that year.

   Building trusted data-centers requires resilient memories which are protected from both adversarial attacks and errors. Unfortunately, the state-of-the-art memory security solutions incur considerable performance overheads due to accesses for security metadata like Message Authentication Codes (MACs). At the same time, commercial secure memory solutions tend to be designed oblivious to the presence of memory reliability mechanisms (such as ECC-DIMMs), that provide tolerance to memory errors. Fortunately, ECC-DIMMs possess an additional chip for providing error correction codes (ECC), that is accessed in parallel with data, which can be harnessed for security optimizations. If we can re-purpose the ECC-chip to store some metadata useful for security and reliability, it can prove beneficial to both. To this end, this paper proposes Synergy, a reliability-security co-design that improves performance of secure execution while providing strong reliability for systems with 9-chip ECC-DIMMs. Synergy uses the insight that MACs being capable of detecting data tampering are also useful for detecting memory errors. Therefore, MACs are best suited for being placed inside the ECC chip, to be accessed in parallel with each data access. By co-locating MAC and Data, Synergy is able to avoid a separate memory access for MAC and thereby reduce the overall memory traffic for secure memory systems. Furthermore, Synergy is able to tolerate 1 chip failure out of 9 chips by using a parity that is constructed over 9 chips (8 Data and 1 MAC), which is used for reconstructing the data of the failed chip. For memory intensive workloads, Synergy provides a speedup of 20% and reduces system Energy Delay Product by 31% compared to a secure memory baseline with ECC-DIMMs. At the same time, Synergy increases reliability by 185x compared to ECC-DIMMs that provide Single-Error Correction, Double-Error Detection (SECDED) capability. Synergy uses commercial ECC-DIMMs and does not incur any additional hardware overheads or reduction of security.

   @inproceedings{saileshwar2018synergy,
     author = {Saileshwar, Gururaj and Nair, Prashant J. and Ramrakhyani, Prakash and Elsasser, Wendy and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 24th International Symposium on High Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA.2018.00046},
     issn = {2378-203X},
     keywords = {Encryption;Metadata;Memory management;Reliability engineering;Error correction codes;Memory-Security;Reliability;ECC-DIMM},
     month = feb,
     number = {},
     pages = {454-465},
     title = {SYNERGY: Rethinking Secure-Memory Design for Error-Correcting Memories},
     volume = {},
     year = {2018}
   }

4. arXiv

   LISA: Increasing Internal Connectivity in DRAM for Fast Data Movement and Low Latency

   Kevin K. Chang, Prashant J. Nair, Saugata Ghose, Donghyuk Lee, Moinuddin K. Qureshi, and Onur Mutlu

   arXiv preprint (arXiv:1805.03184), 2018

   Abs Bib HTML PDF Slides

   This paper summarizes the idea of Low-Cost Interlinked Subarrays (LISA), which was published in HPCA 2016, and examines the work’s significance and future potential. Contemporary systems perform bulk data movement movement inefficiently, by transferring data from DRAM to the processor, and then back to DRAM, across a narrow off-chip channel. The use of this narrow channel results in high latency and energy consumption. Prior work proposes to avoid these high costs by exploiting the existing wide internal DRAM bandwidth for bulk data movement, but the limited connectivity of wires within DRAM allows fast data movement within only a single DRAM subarray. Each subarray is only a few megabytes in size, greatly restricting the range over which fast bulk data movement can happen within DRAM. Our HPCA 2016 paper proposes a new DRAM substrate, Low-Cost Inter-Linked Subarrays (LISA), whose goal is to enable fast and efficient data movement across a large range of memory at low cost. LISA adds low-cost connections between adjacent subarrays. By using these connections to interconnect the existing internal wires (bitlines) of adjacent subarrays, LISA enables wide-bandwidth data transfer across multiple subarrays with little (only 0.8%) DRAM area overhead. As a DRAM substrate, LISA is versatile, enabling a variety of new applications. We describe and evaluate three such applications in detail: (1) fast inter-subarray bulk data copy, (2) in-DRAM caching using a DRAM architecture whose rows have heterogeneous access latencies, and (3) accelerated bitline precharging by linking multiple precharge units together. Our extensive evaluations show that each of LISA’s three applications significantly improves performance and memory energy efficiency on a variety of workloads and system configurations.

   @article{chang2018lisaarxiv,
     author = {Chang, Kevin K. and Nair, Prashant J. and Ghose, Saugata and Lee, Donghyuk and Qureshi, Moinuddin K. and Mutlu, Onur},
     journal = {arXiv preprint (arXiv:1805.03184)},
     title = {LISA: Increasing Internal Connectivity in DRAM for Fast Data Movement and Low Latency},
     year = {2018}
   }

2017

1. Thesis

   Architectural Techniques to Enable Reliable and Scalable Memory Systems

   Prashant J. Nair

   Ph.D. Thesis (arXiv preprint:1704.03991), 2017

   Abs Bib HTML PDF Slides

   

   High capacity and scalable memory systems play a vital role in enabling our desktops, smartphones, and pervasive technologies like Internet of Things (IoT). Unfortunately, memory systems are becoming increasingly prone to faults. This is because we rely on technology scaling to improve memory density, and at small feature sizes, memory cells tend to break easily. Today, memory reliability is seen as the key impediment towards using high-density devices, adopting new technologies, and even building the next Exascale supercomputer. To ensure even a bare-minimum level of reliability, present-day solutions tend to have high performance, power and area overheads. Ideally, we would like memory systems to remain robust, scalable, and implementable while keeping the overheads to a minimum. This dissertation describes how simple cross-layer architectural techniques can provide orders of magnitude higher reliability and enable seamless scalability for memory systems while incurring negligible overheads.

   @article{nair2017thesis,
     author = {Nair, Prashant J.},
     journal = {Ph.D. Thesis (arXiv preprint:1704.03991)},
     title = {Architectural Techniques to Enable Reliable and Scalable Memory Systems},
     year = {2017}
   }

2. ISCA

   DICE: Compressing DRAM Caches for Bandwidth and Capacity

   Vinson Young, Prashant J. Nair, and Moinuddin K. Qureshi

   In Proceedings of the 44th Annual International Symposium on Computer Architecture (ISCA), Toronto, ON, Canada, 2017

   Abs DOI Bib HTML PDF Slides

   

   This paper investigates compression for DRAM caches. As the capacity of DRAM cache is typically large, prior techniques on cache compression, which solely focus on improving cache capacity, provide only a marginal benefit. We show that more performance benefit can be obtained if the compression of the DRAM cache is tailored to provide higher bandwidth. If a DRAM cache can provide two compressed lines in a single access, and both lines are useful, the effective bandwidth of the DRAM cache would double. Unfortunately, it is not straight-forward to compress DRAM caches for bandwidth. The typically used Traditional Set Indexing (TSI) maps consecutive lines to consecutive sets, so the multiple compressed lines obtained from the set are from spatially distant locations and unlikely to be used within a short period of each other. We can change the indexing of the cache to place consecutive lines in the same set to improve bandwidth; however, when the data is incompressible, such spatial indexing reduces effective capacity and causes significant slowdown.Ideally, we would like to have spatial indexing when the data is compressible and TSI otherwise. To this end, we propose Dynamic-Indexing Cache comprEssion (DICE), a dynamic design that can adapt between spatial indexing and TSI, depending on the compressibility of the data. We also propose low-cost Cache Index Predictors (CIP) that can accurately predict the cache indexing scheme on access in order to avoid probing both indices for retrieving a given cache line. Our studies with a 1GB DRAM cache, on a wide range of workloads (including SPEC and Graph), show that DICE improves performance by 19.0% and reduces energy-delay-product by 36% on average. DICE is within 3% of a design that has double the capacity and double the bandwidth. DICE incurs a storage overhead of less than 1KB and does not rely on any OS support.

   @inproceedings{young2017dice,
     address = {New York, NY, USA},
     author = {Young, Vinson and Nair, Prashant J. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 44th Annual International Symposium on Computer Architecture (ISCA)},
     doi = {10.1145/3079856.3080243},
     isbn = {9781450348928},
     keywords = {bandwidth, compression, memory, Stacked DRAM},
     location = {Toronto, ON, Canada},
     numpages = {12},
     pages = {627–638},
     publisher = {Association for Computing Machinery},
     series = {ISCA '17},
     title = {DICE: Compressing DRAM Caches for Bandwidth and Capacity},
     url = {https://doi.org/10.1145/3079856.3080243},
     year = {2017}
   }

3. MICRO

   Taming the Instruction Bandwidth of Quantum Computers via Hardware-Managed Error Correction

   Swamit S. Tannu, Zachary A. Myers, Prashant J. Nair, Douglas M. Carmean, and Moinuddin K. Qureshi

   In Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Cambridge, Massachusetts, 2017

   Abs DOI Bib HTML PDF Slides

   

   A quantum computer consists of quantum bits (qubits) and a control processor that acts as an interface between the programmer and the qubits. As qubits are very sensitive to noise, they rely on continuous error correction to maintain the correct state. Current proposals rely on software-managed error correction and require large instruction bandwidth, which must scale in proportion to the number of qubits. While such a design may be reasonable for small-scale quantum computers, we show that instruction bandwidth tends to become a critical bottleneck for scaling quantum computers.In this paper, we show that 99.999% of the instructions in the instruction stream of a typical quantum workload stem from error correction. Using this observation, we propose QuEST (Quantum Error-Correction Substrate), an architecture that delegates the task of quantum error correction to the hardware. QuEST uses a dedicated programmable micro-coded engine to continuously replay the instruction stream associated with error correction. The instruction bandwidth requirement of QuEST scales in proportion to the number of active qubits (typically << 0.1%) rather than the total number of qubits. We analyze the effectiveness of QuEST with area and thermal constraints and propose a scalable microarchitecture using typical Quantum Error Correction Code (QECC) execution patterns. Our evaluations show that QuEST reduces instruction bandwidth demand of several key workloads by five orders of magnitude while ensuring deterministic instruction delivery. Apart from error correction, we also observe a large instruction bandwidth requirement for fault tolerant quantum instructions (magic state distillation). We extend QuEST to manage these instructions in hardware and provide additional reduction in bandwidth. With QuEST, we reduce the total instruction bandwidth by eight orders of magnitude.

   @inproceedings{tannu2017quantumhardwareecc,
     address = {New York, NY, USA},
     author = {Tannu, Swamit S. and Myers, Zachary A. and Nair, Prashant J. and Carmean, Douglas M. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1145/3123939.3123940},
     isbn = {9781450349529},
     keywords = {quantum control processor, quantum error correction},
     location = {Cambridge, Massachusetts},
     numpages = {13},
     pages = {679–691},
     publisher = {Association for Computing Machinery},
     series = {MICRO-50 '17},
     title = {Taming the Instruction Bandwidth of Quantum Computers via Hardware-Managed Error Correction},
     url = {https://doi.org/10.1145/3123939.3123940},
     year = {2017}
   }

2016

1. ISCA

   XED: Exposing On-Die Error Detection Information for Strong Memory Reliability

   Prashant J. Nair, Vilas Sridharan, and Moinuddin K. Qureshi

   In Proceedings of the 43rd International Symposium on Computer Architecture (ISCA), Seoul, Republic of Korea, 2016

   Commercial Adoption Abs DOI Bib HTML PDF Slides

   

   Incorporated into the HBM3 Memory Protocol by JEDEC

   Large-granularity memory failures continue to be a critical impediment to system reliability. To make matters worse, as DRAM scales to smaller nodes, the frequency of unreliable bits in DRAM chips continues to increase. To mitigate such scaling-related failures, memory vendors are planning to equip existing DRAM chips with On-Die ECC. For maintaining compatibility with memory standards, On-Die ECC is kept invisible from the memory controller.This paper explores how to design high reliability memory systems in presence of On-Die ECC. We show that if On-Die ECC is not exposed to the memory system, having a 9-chip ECC-DIMM (implementing SECDED) provides almost no reliability benefits compared to an 8-chip non-ECC DIMM. We also show that if the error detection of On-Die ECC can be exposed to the memory controller, then Chipkill-level reliability can be achieved even with a 9-chip ECC-DIMM. To this end, we propose eXposed On-Die Error Detection (XED), which exposes the On-Die error detection information without requiring changes to the memory standards or consuming bandwidth overheads. When the On-Die ECC detects an error, XED transmits a pre-defined "catch-word" instead of the corrected data value. On receiving the catch-word, the memory controller uses the parity stored in the 9-chip of the ECC-DIMM to correct the faulty chip (similar to RAID-3). Our studies show that XED provides Chipkill-level reliability (172x higher than SECDED), while incurring negligible overheads, with a 21% lower execution time than Chipkill. We also show that XED can enable Chipkill systems to provide Double-Chipkill level reliability while avoiding the associated storage, performance, and power overheads.

   @inproceedings{nair2016xed,
     author = {Nair, Prashant J. and Sridharan, Vilas and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 43rd International Symposium on Computer Architecture (ISCA)},
     doi = {10.1109/ISCA.2016.38},
     isbn = {9781467389471},
     keywords = {double-chipkill, chipkill, RAID-3, on-die ECC},
     location = {Seoul, Republic of Korea},
     numpages = {13},
     pages = {341–353},
     publisher = {IEEE Press},
     series = {ISCA '16},
     title = {XED: Exposing On-Die Error Detection Information for Strong Memory Reliability},
     url = {https://doi.org/10.1109/ISCA.2016.38},
     year = {2016}
   }

2. TACO

   Citadel: Efficiently Protecting Stacked Memory from TSV and Large Granularity Failures

   Prashant J. Nair, David A. Roberts, and Moinuddin K. Qureshi

   ACM Transactions on Architecture and Code Optimization (TACO), Jan 2016

   Abs DOI Bib HTML PDF Slides

   Stacked memory modules are likely to be tightly integrated with the processor. It is vital that these memory modules operate reliably, as memory failure can require the replacement of the entire socket. To make matters worse, stacked memory designs are susceptible to newer failure modes (e.g., due to faulty through-silicon vias, or TSVs) that can cause large portions of memory, such as a bank, to become faulty. To avoid data loss from large-granularity failures, the memory system may use symbol-based codes that stripe the data for a cache line across several banks (or channels). Unfortunately, such data-striping reduces memory-level parallelism, causing significant slowdown and higher power consumption.This article proposes Citadel, a robust memory architecture that allows the memory system to retain each cache line within one bank. By retaining cache lines within banks, Citadel enables a high-performance and low-power memory system and also efficiently protects the stacked memory system from large-granularity failures. Citadel consists of three components; TSV-Swap, which can tolerate both faulty data-TSVs and faulty address-TSVs; Tri-Dimensional Parity (3DP), which can tolerate column failures, row failures, and bank failures; and Dynamic Dual-Granularity Sparing (DDS), which can mitigate permanent faults by dynamically sparing faulty memory regions either at a row granularity or at a bank granularity. Our evaluations with real-world data for DRAM failures show that Citadel provides performance and power similar to maintaining the entire cache line in the same bank, and yet provides 700 \texttimes higher reliability than ChipKill-like ECC codes.

   @article{nair2016citadeltaco,
     address = {New York, NY, USA},
     articleno = {49},
     author = {Nair, Prashant J. and Roberts, David A. and Qureshi, Moinuddin K.},
     doi = {10.1145/2840807},
     issn = {1544-3566},
     issue_date = {January 2016},
     journal = {ACM Transactions on Architecture and Code Optimization (TACO)},
     keywords = {Reliability, system memory},
     month = jan,
     number = {4},
     numpages = {24},
     publisher = {Association for Computing Machinery},
     title = {Citadel: Efficiently Protecting Stacked Memory from TSV and Large Granularity Failures},
     url = {https://doi.org/10.1145/2840807},
     volume = {12},
     year = {2016}
   }

3. HPCA

   Low-Cost Inter-Linked Subarrays (LISA): Enabling Fast Inter-Subarray Data Movement in DRAM

   Kevin K. Chang, Prashant J. Nair, Donghyuk Lee, Saugata Ghose, Moinuddin K. Qureshi, and Onur Mutlu

   In Proceedings of the 22nd International Symposium on High Performance Computer Architecture (HPCA), Mar 2016

   Abs DOI Bib HTML PDF Slides

   

   This paper introduces a new DRAM design that enables fast and energy-efficient bulk data movement across subarrays in a DRAM chip. While bulk data movement is a key operation in many applications and operating systems, contemporary systems perform this movement inefficiently, by transferring data from DRAM to the processor, and then back to DRAM, across a narrow off-chip channel. The use of this narrow channel for bulk data movement results in high latency and energy consumption. Prior work proposed to avoid these high costs by exploiting the existing wide internal DRAM bandwidth for bulk data movement, but the limited connectivity of wires within DRAM allows fast data movement within only a single DRAM subarray. Each subarray is only a few megabytes in size, greatly restricting the range over which fast bulk data movement can happen within DRAM. We propose a new DRAM substrate, Low-Cost Inter-Linked Subarrays (LISA), whose goal is to enable fast and efficient data movement across a large range of memory at low cost. LISA adds low-cost connections between adjacent subarrays. By using these connections to interconnect the existing internal wires (bitlines) of adjacent subarrays, LISA enables wide-bandwidth data transfer across multiple subarrays with little (only 0.8%) DRAM area overhead. As a DRAM substrate, LISA is versatile, enabling an array of new applications. We describe and evaluate three such applications in detail: (1) fast inter-subarray bulk data copy, (2) in-DRAM caching using a DRAM architecture whose rows have heterogeneous access latencies, and (3) accelerated bitline precharging by linking multiple precharge units together. Our extensive evaluations show that each of LISA’s three applications significantly improves performance and memory energy efficiency, and their combined benefit is higher than the benefit of each alone, on a variety of workloads and system configurations.

   @inproceedings{chang2016lisa,
     author = {Chang, Kevin K. and Nair, Prashant J. and Lee, Donghyuk and Ghose, Saugata and Qureshi, Moinuddin K. and Mutlu, Onur},
     booktitle = {Proceedings of the 22nd International Symposium on High Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA.2016.7446095},
     issn = {2378-203X},
     keywords = {},
     month = mar,
     number = {},
     pages = {568-580},
     title = {Low-Cost Inter-Linked Subarrays (LISA): Enabling Fast Inter-Subarray Data Movement in DRAM},
     volume = {},
     year = {2016}
   }

2015

1. ASPLOS

   DEUCE: Write-Efficient Encryption for Non-Volatile Memories

   Vinson Young, Prashant J. Nair, and Moinuddin K. Qureshi

   In Proceedings of the 20th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), Istanbul, Turkey, 2015

   MICRO Top Picks (Honorable Mention) Abs DOI Bib HTML PDF Slides

   

   An honorable mention to be among the top papers in computer architecture in that year.

   Phase Change Memory (PCM) is an emerging Non Volatile Memory (NVM) technology that has the potential to provide scalable high-density memory systems. While the non-volatility of PCM is a desirable property in order to save leakage power, it also has the undesirable effect of making PCM main memories susceptible to newer modes of security vulnerabilities, for example, accessibility to sensitive data if a PCM DIMM gets stolen. PCM memories can be made secure by encrypting the data. Unfortunately, such encryption comes with a significant overhead in terms of bits written to PCM memory, causing half of the bits in the line to change on every write, even if the actual number of bits being written to memory is small. Our studies show that a typical writeback modifies, on average, only 12% of the bits in the cacheline. Thus, encryption causes almost a 4x increase in the number of bits written to PCM memories. Such extraneous bit writes cause significant increase in write power, reduction in write endurance, and reduction in write bandwidth. To provide the benefit of secure memory in a write efficient manner this paper proposes Dual Counter Encryption (DEUCE). DEUCE is based on the observation that a typical writeback only changes a few words, so DEUCE reencrypts only the words that have changed. We show that DEUCE reduces the number of modified bits per writeback for a secure memory from 50% to 24%, which improves performance by 27% and increases lifetime by 2x.

   @inproceedings{young2015deuce,
     address = {New York, NY, USA},
     author = {Young, Vinson and Nair, Prashant J. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 20th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)},
     doi = {10.1145/2694344.2694387},
     isbn = {9781450328357},
     keywords = {phase change memory, security, encryption, NVM, bitflip},
     location = {Istanbul, Turkey},
     numpages = {12},
     pages = {33–44},
     publisher = {Association for Computing Machinery},
     series = {ASPLOS '15},
     title = {DEUCE: Write-Efficient Encryption for Non-Volatile Memories},
     url = {https://doi.org/10.1145/2694344.2694387},
     year = {2015}
   }

2. DSN

   AVATAR: A Variable-Retention-Time (VRT) Aware Refresh for DRAM Systems

   Moinuddin K. Qureshi, Dae-Hyun Kim, Samira Khan, Prashant J. Nair, and Onur Mutlu

   In Proceedings of the 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2015

   Abs DOI Bib HTML PDF Slides

   

   Multirate refresh techniques exploit the non-uniformity in retention times of DRAM cells to reduce the DRAM refresh overheads. Such techniques rely on accurate profiling of retention times of cells, and perform faster refresh only for a few rows which have cells with low retention times. Unfortunately, retention times of some cells can change at runtime due to Variable Retention Time (VRT), which makes it impractical to reliably deploy multirate refresh. Based on experimental data from 24 DRAM chips, we develop architecture-level models for analyzing the impact of VRT. We show that simply relying on ECC DIMMs to correct VRT failures is unusable as it causes a data error once every few months. We propose AVATAR, a VRT-aware multirate refresh scheme that adaptively changes the refresh rate for different rows at runtime based on current VRT failures. AVATAR provides a time to failure in the regime of several tens of years while reducing refresh operations by 62%-72%.

   @inproceedings{qureshi2015avatar,
     address = {USA},
     author = {Qureshi, Moinuddin K. and Kim, Dae-Hyun and Khan, Samira and Nair, Prashant J. and Mutlu, Onur},
     booktitle = {Proceedings of the 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)},
     doi = {10.1109/DSN.2015.58},
     isbn = {9781479986293},
     keywords = {Memory Scrubbing, Variable Retention Time, Error Correcting Codes, Refresh Rate, Dynamic Random Access Memory, Performance},
     numpages = {11},
     pages = {427–437},
     publisher = {IEEE Computer Society},
     series = {DSN '15},
     title = {AVATAR: A Variable-Retention-Time (VRT) Aware Refresh for DRAM Systems},
     url = {https://ieeexplore.ieee.org/document/7266870},
     year = {2015}
   }

3. TACO

   FaultSim: A Fast, Configurable Memory-Reliability Simulator for Conventional and 3D-Stacked Systems

   Prashant J. Nair, David A. Roberts, and Moinuddin K. Qureshi

   ACM Transactions on Architecture and Code Optimization (TACO), Dec 2015

   Abs DOI Bib HTML PDF Code Slides

   

   As memory systems scale, maintaining their Reliability Availability and Serviceability (RAS) is becoming more complex. To make matters worse, recent studies of DRAM failures in data centers and supercomputer environments have highlighted that large-granularity failures are common in DRAM chips. Furthermore, the move toward 3D-stacked memories can make the system vulnerable to newer failure modes, such as those occurring from faults in Through-Silicon Vias (TSVs). To architect future systems and to use emerging technology, system designers will need to employ strong error correction and repair techniques. Unfortunately, evaluating the relative effectiveness of these reliability mechanisms is often difficult and is traditionally done with analytical models, which are both error prone and time-consuming to develop. To this end, this article proposes FaultSim, a fast configurable memory-reliability simulation tool for 2D and 3D-stacked memory systems. FaultSim employs Monte Carlo simulations, which are driven by real-world failure statistics. We discuss the novel algorithms and data structures used in FaultSim to accelerate the evaluation of different resilience schemes. We implement BCH-1 (SECDED) and ChipKill codes using FaultSim and validate against an analytical model. FaultSim implements BCH-1 and ChipKill codes with a deviation of only 0.032% and 8.41% from the analytical model. FaultSim can simulate 1 million Monte Carlo trials (each for a period of 7 years) of BCH-1 and ChipKill codes in only 34 seconds and 33 seconds, respectively.

   @article{nair2015faultsim,
     address = {New York, NY, USA},
     articleno = {44},
     author = {Nair, Prashant J. and Roberts, David A. and Qureshi, Moinuddin K.},
     doi = {10.1145/2831234},
     issn = {1544-3566},
     issue_date = {January 2016},
     journal = {ACM Transactions on Architecture and Code Optimization (TACO)},
     keywords = {Error correcting codes, reliability, monte carlo simulation, stacked memory, through silicon vias},
     month = dec,
     number = {4},
     numpages = {24},
     publisher = {Association for Computing Machinery},
     title = {FaultSim: A Fast, Configurable Memory-Reliability Simulator for Conventional and 3D-Stacked Systems},
     url = {https://doi.org/10.1145/2831234},
     volume = {12},
     year = {2015}
   }

4. HPCA

   Reducing Read Latency of Phase Change Memory via Early Read and Turbo Read

   Prashant J. Nair, Chiachen Chou, Bipin Rajendran, and Moinuddin K. Qureshi

   In Proceedings of the 21st International Symposium on High Performance Computer Architecture (HPCA), Feb 2015

   Abs DOI Bib HTML PDF Slides

   

   Phase Change Memory (PCM) is an emerging memory technology that can enable scalable high-density main memory systems. Unfortunately, PCM has higher read latency than DRAM, resulting in lower system performance. This paper investigates architectural techniques to improve the read latency of PCM. We observe that there is a wide distribution in cell resistance in both the SET state and the RESET state, and that the read latency of PCM is designed conservatively to handle the worst case cell. If PCM sensing can be tuned to exploit the variability in cell resistance, then we can get reduced read latency. We propose two schemes to enable better-than-worst-case read latency for PCM systems. Our first proposal, Early Read, reads the data earlier than the specified time period. Our key observation that Early Read causes only unidirectional errors (SET being read as RESET) allows us to efficiently detect data errors using Berger codes. In the uncommon case that Early Read causes data error(s), we simply retry the read operation with original latency. Our evaluations show that Early Read can reduce the read latency by 25% while incurring a storage overhead of only 10 bits per 64 byte line. Our second proposal, Turbo Read, reduces the sensing time for read operations by pumping higher current, at the expense of accidentally switching the PCM cell with small probability during the read operation. We analyze Error Correction Codes (ECC) and Probabilistic Row Scrubbing (PRS) for maintaining data integrity under Turbo Read. We show that a combination of Early Read and Turbo Read can reduce the PCM read latency by 30%, improve the system performance by 21%, and reduce the Energy Delay Product (EDP) by 28%, while requiring minimal changes to the memory system.

   @inproceedings{nair2015earlyandturboread,
     author = {Nair, Prashant J. and Chou, Chiachen and Rajendran, Bipin and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 21st International Symposium on High Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA.2015.7056042},
     issn = {2378-203X},
     keywords = {},
     month = feb,
     number = {},
     pages = {309-319},
     title = {Reducing Read Latency of Phase Change Memory via Early Read and Turbo Read},
     volume = {},
     year = {2015}
   }

5. DSN

   Reducing Refresh Power in Mobile Devices with Morphable ECC

   Chiachen Chou, Prashant J. Nair, and Moinuddin K. Qureshi

   In Proceedings of the 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Jun 2015

   Abs DOI Bib HTML PDF Slides

   

   Energy consumption is a primary consideration that determines the usability of emerging mobile computing devices such as smartphones. Refresh operations for main memory account for a significant fraction of the overall energy consumption, especially during idle periods, when processor can be switched off quickly, however, memory contents continue to get refreshed to avoid data loss. Given that mobile devices are idle most of the times, reducing refresh power in idle mode is critical to maximize the duration for which the device remains usable. The frequency of refresh operations in memory can be reduced significantly by using strong multi-bit error correction codes (ECC). Unfortunately, strong ECC codes incur high latency, which causes significant performance degradation (as high as 21%, and on average 10%). To obtain both low refresh power in idle periods and high performance in active periods, this paper proposes Morphable ECC (MECC). During idle periods, MECC keeps the memory protected with 6-bit ECC (ECC-6) and employs a refresh period of 1 second, instead of the typical refresh period of 64ms. During active operation, MECC reduces the refresh interval to 64ms, and converts memory from ECC-6 to weaker ECC (single-bit error correction) on a demand-basis, thus avoiding the high latency of ECC-6, except for the first access during the active mode. Our proposal reduces refresh operations during idle mode by 16x, memory power in idle mode by 2X, while retaining performance within 2% of a system that does not use any ECC.

   @inproceedings{chou2015morphableecc,
     author = {Chou, Chiachen and Nair, Prashant J. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)},
     doi = {10.1109/DSN.2015.33},
     issn = {2158-3927},
     keywords = {},
     month = jun,
     number = {},
     pages = {355-366},
     title = {Reducing Refresh Power in Mobile Devices with Morphable ECC},
     volume = {},
     year = {2015}
   }

2014

1. TACO

   Refresh Pausing in DRAM Memory Systems

   Prashant J. Nair, Chia-Chen Chou, and Moinuddin K. Qureshi

   ACM Transactions on Architecture and Code Optimization (TACO), Feb 2014

   Abs DOI Bib HTML PDF Slides

   

   Dynamic Random Access Memory (DRAM) cells rely on periodic refresh operations to maintain data integrity. As the capacity of DRAM memories has increased, so has the amount of time consumed in doing refresh. Refresh operations contend with read operations, which increases read latency and reduces system performance. We show that eliminating latency penalty due to refresh can improve average performance by 7.2%. However, simply doing intelligent scheduling of refresh operations is ineffective at obtaining significant performance improvement.This article provides an alternative and scalable option to reduce the latency penalty due to refresh. It exploits the property that each refresh operation in a typical DRAM device internally refreshes multiple DRAM rows in JEDEC-based distributed refresh mode. Therefore, a refresh operation has well-defined points at which it can potentially be Paused to service a pending read request. Leveraging this property, we propose Refresh Pausing, a solution that is highly effective at alleviating the contention from refresh operations. It provides an average performance improvement of 5.1% for 8Gb devices and becomes even more effective for future high-density technologies. We also show that Refresh Pausing significantly outperforms the recently proposed Elastic Refresh scheme.

   @article{nair2014refreshpausing,
     address = {New York, NY, USA},
     articleno = {10},
     author = {Nair, Prashant J. and Chou, Chia-Chen and Qureshi, Moinuddin K.},
     doi = {10.1145/2579669},
     issn = {1544-3566},
     issue_date = {February 2014},
     journal = {ACM Transactions on Architecture and Code Optimization (TACO)},
     keywords = {Memory scheduling, memory controller},
     month = feb,
     number = {1},
     numpages = {26},
     publisher = {Association for Computing Machinery},
     title = {Refresh Pausing in DRAM Memory Systems},
     url = {https://doi.org/10.1145/2579669},
     volume = {11},
     year = {2014}
   }

2. CAL

   Architectural Support for Mitigating Row Hammering in DRAM Memories

   Dae-Hyun Kim, Prashant J. Nair, and Moinuddin K. Qureshi

   IEEE Computer Architecture Letters (CAL), 2014

   Abs Bib HTML PDF

   

   DRAM scaling has been the prime driver of increasing capacity of main memory systems. Unfortunately, lower technology nodes worsen the cell reliability as it increases the coupling between adjacent DRAM cells, thereby exacerbating different failure modes. This paper investigates the reliability problem due to Row Hammering, whereby frequent activations of a given row can cause data loss for its neighboring rows. As DRAM scales to lower technology nodes, the threshold for the number of row activations that causes data loss for the neighboring rows reduces, making Row Hammering a challenging problem for future DRAM chips. To overcome Row Hammering, we propose two architectural solutions: First, Counter-Based Row Activation (CRA), which uses a counter with each row to count the number of row activations. If the count exceeds the row hammering threshold, a dummy activation is sent to neighboring rows proactively to refresh the data. Second, Probabilistic Row Activation (PRA), which obviates storage overhead of tracking and simply allows the memory controller to proactively issue dummy activations to neighboring rows with a small probability for all memory access. Our evaluations show that these solutions are effective at mitigating Row hammering while causing negligible performance loss (less than 1%).

   @article{dhkim2014rowhammer,
     author = {Kim, Dae-Hyun and Nair, Prashant J. and Qureshi, Moinuddin K.},
     journal = {IEEE Computer Architecture Letters (CAL)},
     number = {1},
     pages = {9--12},
     publisher = {IEEE},
     title = {Architectural Support for Mitigating Row Hammering in DRAM Memories},
     volume = {14},
     year = {2014}
   }

3. MICRO

   Citadel: Efficiently Protecting Stacked Memory from Large Granularity Failures

   Prashant J. Nair, David A. Roberts, and Moinuddin K. Qureshi

   In Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Dec 2014

   Abs DOI Bib HTML PDF Slides

   

   Stacked memory modules are likely to be tightly integrated with the processor. It is vital that these memory modules operate reliably, as memory failure can require the replacement of the entire socket. To make matters worse, stacked memory designs are susceptible to newer failure modes (for example, due to faulty through-silicon vias, or TSVs) that can cause large portions of memory, such as a bank, to become faulty. To avoid data loss from large-granularity failures, the memory system may use symbol-based codes that stripe the data for a cache line across several banks (or channels). Unfortunately, such data-striping reduces memory level parallelism causing significant slowdown and higher power consumption. This paper proposes Citadel, a robust memory architecture that allows the memory system to retain each cache line within one bank, thus allowing high performance, lower power and efficiently protects the stacked memory from large-granularity failures. Citadel consists of three components, TSV-Swap, which can tolerate both faulty data-TSVs and faulty address-TSVs, Tri Dimensional Parity (3DP), which can tolerate column failures, row failures, and bank failures, and Dynamic Dual Granularity Sparing (DDS), which can mitigate permanent faults by dynamically sparing faulty memory regions either at a row granularity or at a bank granularity. Our evaluations with real-world data for DRAM failures show that Citadel provides performance and power similar to maintaining the entire cache line in the same bank, and yet provides 700x higher reliability than Chip Kill-like ECC codes.

   @inproceedings{nair2014citadel,
     author = {Nair, Prashant J. and Roberts, David A. and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 47th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)},
     doi = {10.1109/MICRO.2014.57},
     issn = {2379-3155},
     keywords = {},
     month = dec,
     number = {},
     pages = {51-62},
     title = {Citadel: Efficiently Protecting Stacked Memory from Large Granularity Failures},
     volume = {},
     year = {2014}
   }

2013

1. ISCA

   ArchShield: Architectural Framework for Assisting DRAM Scaling by Tolerating High Error Rates

   Prashant J. Nair, Dae-Hyun Kim, and Moinuddin K. Qureshi

   In Proceedings of the 40th Annual International Symposium on Computer Architecture (ISCA), Tel-Aviv, Israel, 2013

   Abs DOI Bib HTML PDF Slides

   

   DRAM scaling has been the prime driver for increasing the capacity of main memory system over the past three decades. Unfortunately, scaling DRAM to smaller technology nodes has become challenging due to the inherent difficulty in designing smaller geometries, coupled with the problems of device variation and leakage. Future DRAM devices are likely to experience significantly high error-rates. Techniques that can tolerate errors efficiently can enable DRAM to scale to smaller technology nodes. However, existing techniques such as row/column sparing and ECC become prohibitive at high error-rates.To develop cost-effective solutions for tolerating high error-rates, this paper advocates a cross-layer approach. Rather than hiding the faulty cell information within the DRAM chips, we expose it to the architectural level. We propose ArchShield, an architectural framework that employs runtime testing to identify faulty DRAM cells. ArchShield tolerates these faults using two components, a Fault Map that keeps information about faulty words in a cache line, and Selective Word-Level Replication (SWLR) that replicates faulty words for error resilience. Both Fault Map and SWLR are integrated in reserved area in DRAM memory. Our evaluations with 8GB DRAM DIMM show that ArchShield can efficiently tolerate error-rates as higher as 10−4 (100x higher than ECC alone), causes less than 2% performance degradation, and still maintains 1-bit error tolerance against soft errors.

   @inproceedings{nair2013archshield,
     address = {New York, NY, USA},
     author = {Nair, Prashant J. and Kim, Dae-Hyun and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 40th Annual International Symposium on Computer Architecture (ISCA)},
     doi = {10.1145/2485922.2485929},
     isbn = {9781450320795},
     keywords = {hard faults, dynamic random access memory, error correction},
     location = {Tel-Aviv, Israel},
     numpages = {12},
     pages = {72–83},
     publisher = {Association for Computing Machinery},
     series = {ISCA '13},
     title = {ArchShield: Architectural Framework for Assisting DRAM Scaling by Tolerating High Error Rates},
     url = {https://doi.org/10.1145/2485922.2485929},
     year = {2013}
   }

2. HPCA

   A Case for Refresh Pausing in DRAM Memory Systems

   Prashant J. Nair, Chia-Chen Chou, and Moinuddin K. Qureshi

   In Proceedings of the 19th International Symposium on High Performance Computer Architecture (HPCA), 2013

   Abs DOI Bib HTML PDF Slides

   

   DRAM cells rely on periodic refresh operations to maintain data integrity. As the capacity of DRAM memories has increased, so has the amount of time consumed in doing refresh. Refresh operations contend with read operations, which increases read latency and reduces system performance. We show that eliminating latency penalty due to refresh can improve average performance by 7.2%. However, simply doing intelligent scheduling of refresh operations is ineffective at obtaining significant performance improvement. This paper provides an alternative and scalable option to reduce the latency penalty due to refresh. It exploits the property that each refresh operation in a typical DRAM device internally refreshes multiple DRAM rows in JEDEC-based distributed refresh mode. Therefore, a refresh operation has well defined points at which it can potentially be Paused to service a pending read request. Leveraging this property, we propose Refresh Pausing, a solution that is highly effective at alleviating the contention from refresh operations. It provides an average performance improvement of 5.1% for 8Gb devices, and becomes even more effective for future high-density technologies. We also show that Refresh Pausing significantly outperforms the recently proposed Elastic Refresh scheme.

   @inproceedings{nair2013caserefreshpausing,
     address = {USA},
     author = {Nair, Prashant J. and Chou, Chia-Chen and Qureshi, Moinuddin K.},
     booktitle = {Proceedings of the 19th International Symposium on High Performance Computer Architecture (HPCA)},
     doi = {10.1109/HPCA.2013.6522355},
     isbn = {9781467355858},
     numpages = {12},
     pages = {627–638},
     publisher = {IEEE Computer Society},
     series = {HPCA '13},
     title = {A Case for Refresh Pausing in DRAM Memory Systems},
     url = {https://doi.org/10.1109/HPCA.2013.6522355},
     year = {2013}
   }