Mxnet™ benchmark with LeaderGPU® servers

Attention: due to the newly amended License for Customer Use of Nvidia® GeForce® Sofware, the GPUs presented in the benchmark (GTX 1080, GTX 1080 TI) can not be used for training neural networks.(except blockchain processing). 

The purpose of the following article is to present results of testing mxnet™ on various GPU’s and compare costs of data processing on AWS vs LeaderGPU®.

The following table shows the performance test results, namely the number of images that can be processed per unit of time (measured in seconds).

Scoring results

We've used the official benchmark of mxnet™ benchmark_score.py and cuDNN 6.0

Results are following:

Ltbv 14 (GTX 1080 single):

Ltbv 19 (GTX 1080TI single):

Ltbv 20 (Tesla P100 single):

Below are testing results when using GPU’s on AWS K80 (EC2 p2.xlarge), M40 and P100 (DGX-1).

There used here an official benchmark of mxnet™ benchmark_score.py.

Test results are quoted from mxnet™ official web page.

Results are following:

K80 (single GPU):

M40(single GPU):

P100 (single GPU):

Next we would like to address to the pricing part of this test, we will calculate the cost price of processing 1,000,000 images and the time spent on the process, with batch_size 32:

Results are following:

Training results

In the following section, we will review test results of training networks in mxnet™.

These results are based on example/image-classification/train_imagenet.py and cuDNN 6.0. In addition testing script is available here. For the Alexnet network batch size increased by 8 times.

Ltbv 14 (GTX 1080 single):

Ltbv 19 (GTX 1080TI single):

Ltbv 20 (Tesla P100 single):

** available amount of GPU memory is not enough for batch processing.

K80 (single GPU)

M40(single GPU)

P100(single GPU)

Training results on Multiple Devices

This section will be devoted to the analysis of data collected from testing mxnet™ when using several GPU’s on the LeadersGPU's instances.

Ltbv 14 (4х GTX 1080):

Ltbv 19 (4х GTX 1080TI):

Ltbv 20 (2х Tesla® P100):

* Too many slices therefore some splits are empty

LEGAL WARNING:

PLEASE READ THE LICENSE FOR CUSTOMER USE OF NVIDIA® GEFORCE® SOFTWARE CAREFULLY BEFORE AGREEING TO IT, AND MAKE SURE YOU USE THE SOFTWARE IN ACCORDANCE WITH THE LICENSE, THE MOST IMPORTANT PROVISION IN THIS RESPECT BEING THE FOLLOWING LIMITATION OF USE OF THE SOFTWARE IN DATACENTERS:

«No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.»

BY AGREEING TO THE LICENSE AND DOWNLOADING THE SOFTWARE YOU GUARANTEE THAT YOU WILL MAKE CORRECT USE OF THE SOFTWARE AND YOU AGREE TO INDEMNIFY AND HOLD US HARMLESS FROM ANY CLAIMS, DAMAGES OR LOSSES RESULTING FROM ANY INCORRECT USE OF THE SOFTWARE BY YOU.