- Welcome FAQ
- Secure Shell SSH
- Available Software
- Installing Software
- Guidelines and Policies
- Grant Support
- Sharing Data
- Containers & Singularity
- UserGroup Presentations
- Jupyter Notebook Usage
Lilac Cluster Guide
Juno Cluster Guide
Backup Policy on server/node local drives
Information about submitting LSF jobs is at http://mskcchpc.org/display/CLUS/LSF+Primer. Your interactive LSF job should request the resources your Jupyter notebook will need.
Once you are comfortable to submitting lsf jobs:
These are the steps for starting and connecting to your jupiter-notebook on the cluster:
- Download miniconda and install Jupyter notebook with miniconda.
- Start an interactive session with bsub and start
- Pay attention to the message showing on the screen once you start
- jupyter-notebook. It will tell you the actual node you are on
- , the port number
- and the URL.
- Set up SSH tunnels to the node running jupiter-notebook.
- Connect using the browser on your local machine.
Here is an example of a simple LSF interactive job submission:
username@lilac:~>bsub -n 2 -W 4:00 -R 'span[hosts=1] rusage[mem=8]' -Is /bin/bash
This example requests a GPU for NLP(natural language processing) on Lilac:
username@lilac:~> bsub -n 2 -W 2:00 -q gpuqueue -gpu "num=1" -R "span[hosts=1]" -Is /bin/bash
You can connect to jupyter-notebook by setting up a SSH tunnel. When you create a SSH tunnel, it associates a specific port on the remote system to one on the local system. SSH authentication forwarding must be configured for this to work. The basic command to create a SSH tunnel that passes through the login node to the compute node on your laptop or local server is:
"ssh -L<Port-A>:localhost:<Port-B> user@Host-B -t ssh -L<Port-B>:localhost:<Port-C> user@Host-C"