Accelerating Discovery with Emerging AI Technologies
AI applications abound and are expanding rapidly, yet the methods, performance, and
understanding of AI are in their infancy. Researchers face vexing issues such as how
to improve performance, transferability, reliability, comprehensibility, and how
better to train AI models with only limited data.
GPU utilization while running tests for the Single Stage Detection MLPerf benchmark.
Open Compass is an exploratory research project to conduct academic pilot studies on an advanced
testbed for artificial intelligence, the Compass Lab, culminating in the development and publication
practices for the benefit of the broad scientific community. Open Compass includes the development
ontology to describe the complex range of existing and emerging AI hardware technologies and the
identification of benchmark problems that represent different challenges in training deep learning
These benchmarks are then used to execute experiments in alternative advanced hardware solution
architectures. Here we present the methodology of Open Compass and some preliminary results on
effects of different GPU types, memory, and topologies for popular deep learning models applicable
PSC Compass Program
PSC’s Compass Program focuses on the application of AI technologies to address
challenges in research.
It consists of four main components: the Compass Center, Compass Lab, Compass Consortium, and
The Compass Center brings together faculty, staff, and members from industry, government,
nonprofit organizations to advance AI research and development and accelerate the adoption of AI
various fields, industries, and sectors of society.
The Compass Lab provides unique access through PSC to new hardware and software technologies for
backed by human expertise and partnering with faculty thought leaders to develop solutions.
The Compass Consortium provides, depending on the level of membership, access to seminars,
presentations, publications, reports, case studies, technical briefings, and benchmark results;
to consortium projects; member-directed projects emphasizing the application of new AI
derive value from data; networking with faculty, students, vendors, and other members;
domain experts; training; input through an advisory board to define consortium projects; and
Open Compass brings the benefits of the Compass Lab to the academic research community. It also
expert assistance in applying Compass Lab resources and collaborative development of algorithms,
and software; develops and disseminates best practices; and provides training tailored to the
Figure: Example visualization of the Compass Ontology
This figure is focusing on an individual NVIDIA DGX-2 class server. Solid arcs indicate
relationships, and dashed arcs indicate object properties such as GPU Type and Internal
Open Compass addresses both training and inferencing, initially focusing on
networks for image processing for its familiarity to the community and to document our methodology.
Performance results for training five neural networks have been reported, InceptionV3, ResNet-50,
ResNet-152, and VGG16, and AlexNet, all using synthetic data. These networks differ significantly in
layers and topologies and therefore have different memory and computational requirements. In
has proven valuable for image classification.
Single Stage Detection
Single Shot Detection is an object detection model in the MLPerf benchmark; SSD is intended
reflect a simpler and lower latency model for interactive use cases such as in end-point and
non-server situations. Notably, SSD uses a pre-trained ResNet-34 backbone as part of the
architecture. SSD is trained and evaluated on the COCO dataset (COCO 2017 is large-scale
detection, segmentation and captioning dataset with 80 Object categories, 123,287 Train/val
and 886,284 instances).
Note the computational overhead of the SSD model is small compared with the
ResNet-50 model. The final target is to achieve mean Average Precision of 0.23.
The training is done using various NVIDIA-DGX2 multi-gpu configurations i.e. 1, 2, 4, 8 and
The results are shown below.
Figure: Training time vs number of GPUs used.
The green line is the target train time benchmarked by NVIDIA for 16 DGX2