Scientific Computing & Data Analysis Section

Connect to the Clusters

Here is how to get access to the Deigo and Saion clusters, how to connect to them, and how to transfer files to and from the research storage.

Note that you do not need an account just to access the storage; you can access the login nodes and mount bucket remotely without it. See the page on transferring data.

Here are the steps to use the cluster:

Get an Account. First, get an account by applying from the forms here.
Use SSH to access the cluster. With instructions for Linux, for OSX and for Windows.
Access the clusters from outside. To access Deigo and Saion from outside OIST you use a cryptographic key instead of a password to log in. Here is how to set it up.
Create Aliases. Create convenient aliases to the clusters and storage.

Get an account

The first thing we'll need is an account on Deigo or Saion. It's very easy to apply for a general account: Click on the "Open Resources" form on this page, tell us that you'd like access and press submit. All OIST members can get access to the clusters. We will normally set it up for you within a working day.

You can ask for access to the open partitions on Deigo and Saion with no restrictions. We also have other partitions available with large amounts of memory or with GPU accellerators. You need to apply for those separately (use the "HPC Restricted Resources" form), and you need to give us a reason for using them. The partitions are summarized on this page.

Set up SSH

Now that we have access, we need to log in. We do that with a program called 'ssh'. It stands for "Secure Shell" and is the most common way to connect to remote servers and cluster computers. With SSH you can connect interactively, run graphical applications and transfer files securely over a network.

Your login name and password is your OIST user name and password. Try to log in as we show below. You should be able to log in even without access to the clusters.

Linux

On Linux you already have all software installed.

Open a terminal and type:

ssh -X your-name@deigo.oist.jp

You can log out again either by typing exit followed by a return, or press CTRL+d.

The '-X' parameter tells ssh to allow graphical applications to display their windows on your local computer. If you want to access Saion, use "saion.oist.jp"

You can now connect to the cluster from within OIST. But you can't log in from outside OIST, and you need to enter your OIST password every time. You can set up SSH so that you don't need to type your password any longer. Please proceed to the next section on using public key authentication.

Mac OSX

SSH is already installed on the Mac. If you want to display graphics, you first need to install a piece of software called XQuartz. The installation is straightforward:

download Xquartz from this page
click it to open.
agree to the install.
You may need to restart the computer to make it work.

To use SSH to access the clusters, Open a terminal (in the Utilities folder) and run:

ssh -X your-name@deigo.oist.jp

You can log out again either by typing exit followed by a return, or press CTRL+d.

The '-X' parameter tells ssh to allow graphical applications to display their windows on your local computer. If you want to access Saion, use "saion.oist.jp"

Windows

Windows lacks the software needed to connect to the cluster and to run graphical applications remotely. We need to install an SSH client.

There are several popular clients, and if you already have a favourite then please use that. We currently recommend MobaXTerm, as it allows you to display graphics without any extra installation.

MobaXTerm: go to the download page here. Some users have experienced problems using the installer, so select the "portable" version:

Click to unpack somewhere. You start it by clicking the program itself, or you can right-click, create a shortcut, move the shortcut to the desktop, then click that instead.

When the window appears, start a local terminal session:

Click "Start a local terminal", or press the return key. If it doesn't appear, you can select "Session", then "local terminal" in the list.

Once the terminal appears, you log in to Deigo using:

ssh -X your-name@deigo.oist.jp

You can log out again either by typing exit followed by a return, or press CTRL+d.

he '-X' parameter tells ssh to allow graphical applications to display their windows on your local computer. If you want to access Saion, use "saion.oist.jp".

Public Key Authentication

SSH can use public key authentication. SSH will create a pair of keys (long random numbers), one public and one private. You will copy the public key over to your account on the cluster. When you log in, SSH will use those keys to determine that you are who you are and log you in without a password. If your public key is installed on a machine, you can access it from any computer that has the corresponding private key.

On your own local computer — not on Deigo — you create a new key-pair with the following command (for Windows, enter the commands in the MobaXTerm terminal window). "ecdsa" is a more secure key type than the default:

ssh-keygen

Follow the prompts and accept all the defaults. Note that if it says that a key already exists, then stop. You already have a key and don't need to generate another one.

This creates two keys, one private and one public. You copy the public key to machines that you want to access. A computer that has the private key can then log in to those machines without a password. The public key is like a door with a lock that you install, and the private key is the key that opens that particular lock.
Copy the public key over to Deigo. SSH has a convenient command to do this:

ssh-copy-id your-name@deigo.oist.jp

ssh-copy-id should ask you for a password and copy your public key to the cluster. Enter your OIST password.
try to log in again. You should now be able to so without giving a password:

ssh -X your-name@deigo.oist.jp

If you can log in without a password locally, you should now also be able to log in from outside OIST as we explain in the next section.

Optional: Create keys on Deigo

Optionally, we want to do something that may seem a little odd: On Deigo we create another key pair, then copy the public key right back to Deigo itself. This lets you login directly to compute nodes if you ever need to; some software makes use of this.

ssh-keygen
ssh-copy-id your-name@deigo.oist.jp

Get Access from outside OIST

You can access the OIST clusters in two ways. The first is to apply for the OIST VPN. When the VPN is active you can access Deigo and Saion in the same way as when you are at the OIST campus.

The other way is to use SSH to access the HPC systems directly from outside, without using the VPN. The benefit is that the connection will be faster and more reliable. The drawback is that you can only use the connection through SSH. If you want to access the storage through remote folders on your desktop you need the VPN access.

To access the clusters with SSH you need to set up key-based login, as we explained in the previous section.

The clusters themselves are not accessible from the outside, but we have a proxy server that you can use to first log in to OIST, then log in to the clusters from that server.

From outside OIST (you can test this using the "OIST Public" wifi network), log in to "login.oist.jp":

ssh -X username@login.oist.jp

Once you've logged in to login.oist.jp, you can proceed to the clusters:

ssh -X username@deigo.oist.jp

You now have full access to the clusters and storage through SSH.

To copy data to and from storage, please refer to our page on transferring data.
You can streamline your access and avoid typing by using aliases.
If you want to run a computation on the cluster, please read our Slurm guide.

Create Aliases

It gets fairly tiresome to have to type out your user name and the full address of the systems all the time. Fortunately, we can take care of this.

On MobaXterm (and mobile clients such as JuiceSSH) you set up connection profiles so you can log in with a single click. Please see the documentation for the application that you use.

On the command line client on Linux and OSX we can set up ssh aliases. In the directory ~/.ssh/ (this is a subdirectory to your home) on your laptop or personal computer you can create a file "config" where you can set up your ssh configuration:

cd                          # returns you to your home
nano .ssh/config            # nano is a simple editor, we use it to create
                            # and edit the config file

In this file, add this text - replace the user name with your own:

Host deigo
    Hostname deigo.oist.jp
    User <username>
    ForwardX11 yes

Host saion
    Hostname saion.oist.jp
    User <username>
    ForwardX11 yes

Host oist
    Hostname login.oist.jp
    User <username>
    ForwardX11 yes

The "ForwardX11" setting is the same as the "-X" option you used with ssh earlier. You can set other ssh connection parameters here as well. For instance, if you have multiple keys for different sites, you can set which crypto key you want to use for a given site here.

Now you can log in simply by doing:

ssh deigo

Set up direct access to the clusters from outside

As we saw earlier, to log in from the outside you need to first ssh to login.oist.jp, then from there ssh to either Deigo or Saion. This is slightly inconvenient. Also, /bucket is not available from login.oist.jp. If you want to copy files, you would first need to copy them to your /home on login.oist.jp, then login to Deigo and move the data to its final destination. Not convenient at all.

Fortunately, a nifty ssh config file setup will let us connect and access Deigo and Saion, and copy data directly from the outside to Bucket. In your local ~/.ssh/config file on your personal computer, add this:

host oist-ext
    Controlmaster auto
    User <username>
    hostname login.oist.jp

host deigo-ext
    User <username>
    ForwardX11 yes
    ProxyCommand ssh -q -W deigo.oist.jp:22 oist-ext

host saion-ext
    User <username>
    ForwardX11 yes
    ProxyCommand ssh -q -W saion.oist.jp:22 oist-ext

Now you can use deigo-ext and saion-ext as the destination for all ssh-related commands remotely just as if you were right on campus. You no longer need to use login.oist.jp directly any longer. To login to Deigo from home you can now do:

ssh deigo-ext

And you can copy a file to Bucket like this:

scp my_local_file deigo-ext:/bucket/MyunitU/