This month’s tech transfer story is a powerful story of collaboration catalyzed by a member company’s liaison embedded at a university as a visiting researcher, where they are able to work alongside university researchers while maintaining strong ties to their own company.

PRISM researchers proposed and explored a heterogeneous Retrieval-Augmented Generation (RAG) cache architecture for document key-value (KV) caching used in Large Language Model (LLM)-based RAG systems. While current state-of-the-art approaches rely solely on DRAM for RAG document caching, PRISM researchers’ proposed architecture utilizes MRAM, leveraging its advantages in high-density, lower-power consumption, and lower latency, alongside DRAM in a heterogeneous memory hierarchy. Through this approach, they achieved up to a 15.48x energy reduction and up to a 1.58x latency improvement compared to conventional approaches.

This research was conducted in close collaboration with Kiseok Suh, a Visiting Researcher at UCSD from Samsung Electronics, a sponsor of JUMP 2.0. Collaborating with an MRAM expert from Samsung allowed PRISM researchers to quickly access the latest technological trends and real-world technical insights, enabling them to pursue a more practical and industry-aligned research direction. Additionally, Kiseok continuously updated Samsung on the development progress and obtained timely feedback, allowing the research to align rapidly with the sponsor’s goals and requirements. This close industry-academic collaboration significantly enhanced both the practicality and efficiency of the team’s research and development efforts. Read more about this collaborative research effort in their recently published paper, “HeteroRAGCache: Software-Hardware Co-Design for Efficient RAG Caching using MRAM,” submitted to the IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2025, and co-authored by Jangseon Park, Kiseok Suh, Flavio Ponzina, and PRISM Director Tajana Rosing.

Congratulations to both the PRISM researchers and Samsung for setting a great example on how visiting liaisons/researchers in residence can be a great tool for fostering collaboration that can lead to potential tech transfers down the road.

This project is related to PRISM project 3135.007 2.2 T2: Architecture, titled "Acceleration In and Near Memory and Storage (AIMS) Architecture", as well as project 3135.017 T4: Grand Challenges "Deep Insights."