Jupyter 
=======

.. note::
   A special thank you to Mateusz Doliński for his help in
   preparing the solution described on this page.

Running an instance of Jupyter Notebook on a compute node
is possible using SSH port forwarding. Direct loginng in into
each compute node is disabled (it should not be possible
to circumvent the queue system), but the outgoing traffic
is not restricted. Accessing a compute node is possible
using *interactive* ``srun`` sessions.

Installing Jupyter software
---------------------------

Each user of the cluster can install Jupyter using
Python's ``virtualenv``.

The full documentation on how to install Jupyter can be found
`here <https://jupyter.org/>`_.

Local SSH setup
---------------

Accessing a JupyterLab session involves **SSH port forwarding**.
In general, two tunnels have to be set up: from a user's computer to the ``entropy``,
and from a compute node to the ``entropy`` (later, during an interactive
``srun`` session).

.. note::
   To avoid port conflicts each user should use the port number which is
   his ``${UID}`` plus ``10 000``.

The most convenient way to setup the first tunnel (from a user's machine
to ``entropy``) is to create ``~/.ssh/config`` file and add an entry
which specifies all required parameters. For example, to forward a local
port ``12222`` (which is the port number for the user whose ``${UID}``
is ``2222``) to the ``entropy`` server:

   .. code-block:: shell
      :linenos:
      
      Host entropy
          HostName entropy.mimuw.edu.pl
          User entropy_login
          Preferredauthentications publickey
          IdentityFile ~/.ssh/a_private_key
          LocalForward 12222 localhost:12222

This configuration will forward the local ``12222`` port to the ``entropy``
server each time a user will use SSH to connect to the machine.

Cluster account setup
---------------------

First, start by creating a **passwordless** SSH key pair on the ``entropy`` machine:

   .. code-block:: shell
      :linenos:
      
      ssh-keygen -t ed25519 -C "your_comment"

Then, add the **public** key from this pair to the ``authorized_keys`` file on the
``entropy`` machine.

   .. code-block:: shell
      :linenos:
      
      cat ~/.ssh/your_public_key.pub >> ~/.ssh/authorized_keys

Finally create the ``~/.ssh/config`` file similar to the one previously
created on your computer (using the key pair that has just been created):

   .. code-block:: shell
      :linenos:
      
      Host entropy
          HostName entropy.mimuw.edu.pl
          User entropy_login
          Preferredauthentications publickey
          IdentityFile ~/.ssh/a_private_key
          IdentitiesOnly yes
          RemoteForward 12222 localhost:12222
          RequestTTY no

This configuration will forward the ``12222`` port **from** a compute node
to the ``entropy`` server each time a user will start an SSH connection to
the machine. This connection will not reserve a terminal and serves only
the purpose of forwarding the specified port.

Finally, we can start a job and a ``jupyter notebook`` instance. First, start
an interactive ``srun`` session (note the ``--pty`` option):

   .. code-block:: shell
      :linenos:

      srun --qos=student --partition=common --gres=gpu:1 --time=1-0 --pty /bin/bash -i

This will open an interactive session on a machine with the home directory available through
the NFS protocol. Then, activate the previously created ``virtualenv`` and start Jupyter.

   .. code-block:: shell
      :linenos:

      source path/to/user/virtualenv
      jupyter-lab --no-browser --port="1${UID}" &
      ssh entropy

A ``jupyter-lab`` session should start; it should be possible to access the notebook's URL
in the user's local browser.