As modern supercomputers continue to be increasingly heterogeneous with diverse computational accelerators (GPUs, FPGAs, ASICs, etc.), software becomes a critical design aspect. Exploiting this new computational power requires increased software design time and effort to make valuable scientific discovery in the face of the complicated programming environments introduced by these accelerators.
Ever increasing processing demands of modern applications can no longer be satisfied with traditional homogenous computing systems. These applications (e.g., big data analytics, artificial intelligence, etc.) are characterized by large data volume and non-traditional processing, often requiring computational accelerators to boost performance, maintain reliability, and meet energy efficiency goals. Recently, there has been an explosive emergence of these accelerators for AI, from large industry organizations to small startups, using acceleration techniques ranging from soft data processing units (DPUs) with high reconfigurability (e.g., FPGAs)to hard DPUs with less reconfigurability but high performance (e.g., ASICs). This diversity further compounds the complexity in the paradigm shift towards heterogeneous-accelerator-based platforms and the associated models of computation (parallel computation models) that guide programmability to simplify usage.
Given these emerging architecture trends, no single parallel computation model can effectively leverage heterogeneous-based platforms in a cluster environment due to the diverse user, application, and resource availability requirements. This diversity includes, but is not limited to, software, algorithmic, workflow, data transfer, curation, analysis and accelerator sharing requirements. Overcoming these complexity challenges will require transformative new designs for future supercomputers and systems. Concomitantly, innovative methods are required to flexibly map emerging adaptable AI algorithms to leverage heterogeneous hardware resources without necessitating drastic programming environment changes that would otherwise hinder productivity.
To address those challenges, PCS unifies multiple programming models into a single programming environment to facilitate large-scale, accelerator-aware, heterogeneous computing for next-generation scientific applications. A prototype PCS platform has been developed (with more work still to be done). The PCS prototype platform currently provides a programming abstraction that is accelerator-system agnostic, focusing on scalability and productivity to meet the demands of rapidly changing AI workloads and heterogeneous architectures. PCS draws motivation from the US Department of Energy (DOE) Summary Report from the June 2017 Summit on Extreme Heterogeneity, which describes the current state-of-affairs in heterogeneous systems for scientific computing.