Running SAGECal on an NVIDIA Jetson Nano

Introduction

SAGECal is already being run on supercomputers, utilizing thousands of CPUs and with GPU acceleration. But what about running it on an embedded system? We have successfully installed and run SAGECal on the NVIDIA Jetson Nano system and this is a short description about it.

NVIDIA Jetson Nano — Figure 1: Jetson Nano system running headless, connection via the ethernet.

In Fig. 1, we show the system we are using. We do not need a monitor/mouse/keyboard for our experiments. You can follow this web page for setting up the system for first use. Our configuration is as follows:

sarod@sarod-jetson:~$ jetson-release -h
 - NVIDIA Jetson NANO/TX1
   * Jetpack 4.2 [L4T 32.1.0]
   * CUDA GPU architecture 5.3
 - Libraries:
   * CUDA 10.0.166
   * cuDNN 7.3.1.28-1+cuda10.0
   * TensorRT 5.0.6.3-1+cuda10.0
   * Visionworks 1.6.0.500n
   * OpenCV 3.3.1 compiled CUDA: NO
 - Jetson Performance: inactive

Installing SAGECal

The Jetson Nano installs Ubuntu Linux and you can follow the normal instructions to install the pre-requisites. In particular, the following command should do this for you:

sudo apt-get install -y git cmake g++ pkg-config libcfitsio-bin libcfitsio-dev libopenblas-base libopenblas-dev wcslib-dev wcslib-tools libglib2.0-dev libcasa-casa2 casacore-dev casacore-data casacore-tools

The default cmake is too old to work with CUDA 10.0, so you need to download the latest cmake source and compile it. We use cmake version 3.14.5 and it works. We install cmake in our home directory

 ./configure --prefix=/home/sarod/

and it will be installed at ~/cmake-3.14.5/. Once you have everything ready, check out the dev branch of SAGECal source. Since we have only one GPU available, we edit src/lib/Dirac/GPUtune.h and replace

#define MAX_GPU_ID 3

with

#define MAX_GPU_ID 0

We build SAGEcal without GPU support as follows:

~/cmake-3.14.5/bin/cmake .. -DHAVE_CUDA=OFF
make -j4

This will create the binary build/dist/bin/sagecal. We build SAGEcal with GPU support as follows:

~/cmake-3.14.5/bin/cmake .. -DHAVE_CUDA=ON -DCMAKE_CXX_FLAGS='-L/usr/local/cuda/lib64 -DMAX_GPU_ID=0' -DCUDA_NVCC_FLAGS='-gencode arch=compute_53,code=sm_53'
make -j4

This will create the binary build/dist/bin/sagecal_gpu.

Running SAGECal

You can run the example script provided test/Calibration/dosage.sh with both sagecal and sagecal_gpu to compare the performance. We use two tools, JTop and gpuGraphTX, for monitoring the GPU usage. See figures 2 and 3 below for an example. We see that SAGEcal uses the GPU almost fully and the CPU utilization is minimal.

GPU/CPU usage with JTop — Figure 2: SAGECal CPU and GPU usage using `JTop`.

SAGECal GPU usage with gpuGraphTX — Figure 3: SAGECal GPU usage using `gpuGraphTX`.

These results show that it is possible to run SAGECal even in embedded systems!! therefore paving the way for its use in various other applications such as in mobile devices, space interferometry, edge computing etc.