A lot of new customers ask us where they can run RealityServer since they don’t have their own server or workstation with NVIDIA GPU hardware available. Starting up RealityServer on Nimbix is covered in another article where everything is pre-configured for you, on AWS however you need to do a bit more setup yourself. We are assuming here that you are already familiar with Amazon Web Services and starting instances on Amazon EC2, along with basic concepts like security groups. We won’t cover the basics of how to start an instance here however there is lots of good information about that online, including this guide from Amazon. So, let’s get started.
Before you dive in to launch some instances on AWS to run RealityServer, you should first think about where you want to run your server (which region) as well as which instance type you want to use. While RealityServer can work on pure CPU instances to access all features and get the best performance, you should really use AWS GPU instances. There are currently two types of GPU instances available on AWS that support RealityServer:
The g2 instance types are previous generation GPU instance types and are available in most regions. They use GRID GPU hardware and in general do not perform very well, however they are fairly inexpensive. The p2 instance types in contrast use Tesla GPU hardware and offer more memory and greater performance. The g3 instances are the most recently released and use the Maxwell architecture offering further improved performance. Of course Amazon offerings might change in the future. You can see our benchmark data on how these instance types perform. When selecting which region to start your node in consider how far it is likely to be from you; closer servers mean lower latency.
When you start your instances, ensure you open TCP ports 8080, 8081 and 1935 in your security group, in addition to the standard SSH port. We generally use the standard Amazon Linux AMI (ami-0b33d91d for US East Coast, Virginia); for GPU instances you must use HVM AMIs. These instructions assume you are using the standard Amazon Linux AMI; other distributions can also be used with RealityServer, but you will need to sort out the installation process yourself. In general since Amazon Linux is Red Hat Enterprise based, CentOS and RHEL should behave in a similar way.
Once you have launched your instance and are ready to connect you can ssh into your instance and start setting up the GPU drivers and RealityServer. First let’s install some dependencies we will need:
sudo yum update -y sudo yum groupinstall -y "Development tools" sudo yum install -y wget kernel-devel-$(uname -r) kernel-headers-$(uname -r) libX11 libGLU libSM
In order to obtain your licenses for RealityServer you need to set the SPM_HOST environment variable. You can do this by hand each time you login, but to keep it persistent you can do the following:
sudo sh -c 'echo setenv SPM_HOST licensing.migenius.com > /etc/profile.d/spm.csh' sudo sh -c 'echo export SPM_HOST=licensing.migenius.com > /etc/profile.d/spm.sh'
One additional step is required to obtain your licenses; you need to tell SPM which port to connect to our servers on (we definitely realise this licensing process is a bit complex and are working on ways to simplify this in the future). Add the licensing port to your /etc/services file as follows (replacing [port] with the licensing port provided by migenius):
sudo sh -c 'echo -e "mi-spm\t[port]/tcp\t# migenius SPM License Server" >> /etc/services'
Make sure you use the double redirections, you don’t want to overwrite your /etc/services file! Next let’s get the NVIDIA drivers installed. You can check for the latest Linux version here. Here we used the versions that worked well at the time this was written (on the p2 instance types), substitute as needed:
wget http://us.download.nvidia.com/XFree86/Linux-x86_64/384.69/NVIDIA-Linux-x86_64-384.69.run sudo sh ./NVIDIA-Linux-x86_64-384.69.run -a
If you are running the g2 instance type you will need to install an older driver. 367.57 is the most recent version to support the hardware on the g2 instances while still having the CUDA 8.0 support needed for RealityServer. Future releases of RealityServer may require more recent CUDA versions and therefore might not be supported on the g2 instance types. If you are running on a p2 instance type then you should additionally run the following commands to force the clock speed and enable persistence mode so the driver stays loaded.
sudo nvidia-smi -pm 1 sudo nvidia-smi -acp 0 sudo nvidia-smi --auto-boost-permission=0 sudo nvidia-smi -ac 2505,875
Follow all of the prompts, its pretty self-explanatory. When running RealityServer you may get warnings about ECC being enabled, unfortunately AWS do not offer the ability to disable this GPU feature, so you may get a slight performance hit and you will lose 12.5% of your GPU memory capacity. This is just due to the way AWS works and sadly not something we can fix with RealityServer. With the GPU drivers installed (if you want to confirm they detect the GPU just run nvidia-smi and check out the output) you can now download and install RealityServer:
sudo mkdir -p /usr/local/migenius cd /usr/local/migenius sudo wget http://download.migenius.com/releases/RealityServer/rsws-50-1806.184.tgz sudo tar zxvf rsws-50-1806.184.tgz sudo chown -R ec2-user:ec2-user rsws-50-1806.184
Obviously replace the RealityServer link with that to the release you want to use. If all goes well you can now do a test run and get RealityServer going:
cd rsws-50-1806.184 . /etc/profile.d/spm.sh ./realityserver_ws
You should see a lot of log output. You can then connect to RealityServer and explore the examples and documentation by going to the following address (where ec2-ip is the public IP of your AWS instance):
If you want to start RealityServer so that it keeps running after you log out you can use nohup for that:
nohup ./realityserver_ws > rs.log 2>&1 &
Here we are sending the log output to rs.log so you won’t see anything on screen. To see the log output you can just tail the file:
tail -f rs.log
When you’re done with your instance and don’t want to leave it running, you can just power it off:
On AWS this puts the instance into a stopped state and you can restart it at anytime. You don’t get charged for the instance while it’s stopped (only for the storage associated with it), so this can be a handy way to keep a RealityServer development node ready to run. There is a lot we haven’t covered here and, in a production system, you will of course want to set up RealityServer for systemd, init or init.d depending on how you configure your machines. We don’t cover this here however don’t hesitate to contact us if you want more details on how to do this. We also haven’t covered creating an AMI from your work here so you can start more nodes. This is essentially no different to any other Amazon instance type, so there is a lot of good information out there on that.
One note, if you want to try clustering multiple nodes on AWS, you will need to use TCP based clustering (without the UDP discovery option) because AWS does not support UDP multicast on its network.