SIGPLAN logoBenchmarking in the Data Center: Expanding to the Cloud

workshop held in conjunction with PPoPP 2021: Principles and Practice of Parallel Programming 2021

Workshop Scope

High performance computing (HPC) is no longer confined to universities and national research laboratories, it is increasingly used in industry and in the cloud. Education of users also needs to take this into account. Users need to be able to evaluate what benefits HPC can bring to their companies, what type of computational resources (e.g. multi-, many-core CPUs, GPUs, hybrid systems) would be best for their workloads and how they can evaluate what they should pay for these resources. Another issue that arises in shared computing environments is privacy: in commercial HPC environments, data produced and software used typically has commercial value, and so needs to be protected. Recent general adoption of machine learning has motivated migration of HPC workloads to cloud data centers, and there is a growing interest by the community on performance evaluation in this area, especially for end-to-end workflows. In addition to traditional performance benchmarking and high performance system evaluation (including absolute performance, energy efficiency), as well as configuration optimizations, this workshop will discuss issues that are of particular importance in commercial HPC. Benchmarking has typically involved running specific workloads that are reflective of typical HPC workloads, yet with growing diversity of workloads, theoretical performance modeling is also of interest to allow for performance prediction given a minimal set of measurements. The workshop will be composed of submitted papers, invited talks and a panel composed of representatives from industry.

Date and Location

Saturday, February 27, 17:00 - 21:00, UTC-5, Eastern Time (US & Canada)
Online Event
Co-located with PPoPP 2021


We invite novel, unpublished research paper submission within the scope of this workshop. Paper submission topics include, but are not limited to, the following areas:
Full paper: Up to 10 pages (not including references)
Short paper: Up to 5 pages (including references)

All submissions must be made electronically through the submission site (to be announced). Full paper submissions must be in PDF formatted printable on both A4 and US letter size paper. All papers must be prepared in ACM Conference Format using the 2-column acmart format: use the SIGPLAN proceedings template acmart-sigplanproc-template.tex for Latex, and interim-layout.docx for Word. You may also want to consult the official ACM information on the Master Article Template and related tools. Important note: The Word template (interim-layout.docx) on the ACM website uses 9pt font; you need to increase it to 10pt.

Paper text typeface should be no smaller than 10 point. References must include the name of all authors (not {et al.}), and there is no page limit for references in full paper submissions. Appendices are not allowed, but the authors may provide supplementary material (proofs or source code etc.) to the Program Chairs. Review of supplementary material is at the discretion of the reviewers; papers must be complete and self-contained.

Submission is double blind and authors will need to identify any potential conflicts of interest with PC and Extended Review Committee members, as defined here: ACM SIGPLAN policy.

Workshop submission site:

Workshop Program

Speaker abstracts and bios can be found here: View full program

Feb 27, 17:00 - 21:00, UTC-5, Eastern Time (US & Canada)
17:00: Welcome Remark, 2”, Juan (Jenny) Chen, Workshop Chair
Keynote Session (17:10 - 19:45)
17:10-17:35 Mr. Sven Breuner, VAST Data & Dr. Chin Fang, Zettar
Title: elbencho – A new storage benchmark for AI et al
Moderator: Benson Muite
17:35-17:45 Q&A
Moderator: Benson Muite
17:45-18:10 Dr. Wanling Gao, Institute of Computing Technology, Chinese Academy of Sciences
Title: AIBench Scenario: Scenario-distilling AI Benchmarking
Moderator: Juan (Jenny) Chen
18:10-18:20 Q&A
Moderator: Juan (Jenny) Chen
18:20-18:45 Dr. Jason (Zhixiang) Ren, Peng Cheng Laboratory, China
Title: AIPerf: AutoML as an AI-HPC benchmark
Moderator: Kevin Brown
18:45-18:55 Q&A
Moderator: Kevin Brown
18:55 - 19:10, Coffee Break
19:10-19:30 Dr. Dan Huang, Sun Yat-sen University, China
Title: A Comprehensive Study of In-Memory Computing on Large HPC Systems
Moderator: Samar Aseeri
19:35-19:45 Q&A
Moderator: Samar Aseeri
Panel Session
19:45-20:30 Industry Panel Discussion
Dr. Sreeram Potluri, NVIDIA, USA
Dr. Jithin Jose, Microsoft, USA
Ms. Verónica G. Melesse Vergara, Oakridge Leadership Computing Facility, USA
Moderators: Ammar Awan, Shahzeb Siddiqui
20:30-20:45 General Discussion
Close Remarks

Resgister for access to the workshop Zoom meeting.


Please register at the main conference website, here.


Paper submission deadline: 15 29 January 15 February 2021 AoE (EXTENDED)
Preliminary author notification on presentation: 822 February 2021
Workshop: 27 February 2021, 17:00 Easter Time (US & Canada)
Final manuscripts due: 29 March 2021 AoE

Organizing Committee

Juan (Jenny) Chen (National University of Defense Technology, China)

Kevin Brown (Argonne National Laboratory)

Contact: [email protected]

Industrial Panel Chairs

Ammar Awan (Microsoft Research)

Shahzeb Siddiqui (Lawrence Berkeley National Laboratory)

Contact: [email protected]


Xinxin (Alva) Qi (National University of Defense Technology, China)