Jupyter Notebook is kind of a Python (and some other languages) sandbox, where you can mix text (Markdown) with code blocks and “run” all that in a web-browser. So it’s like running Python interpreter in interactive mode, but more convenient and user-friendly.

Jupyter Notebook

So, here’s the plan:

  • install and run a local instance of Jupyter Notebook on Linux server
    • local here means that Jupyter instance itself will be bound to localhost (not available from the internet)
    • also create a systemd service
  • interface it with the internet via NGINX as a reverse proxy
    • using HTTPS, of course, so the existing certificate needs to be extended
  • host it on a subdomain of existing website, so there will be a new DNS record

The example of a Jupyter Notebook from the screenshot above is of course a very silly one. If you want to see some more sophisticated ones, here’s a whole gallery of them (like this one or that one).

Setup

My environment:

$ lsb_release -a
Description:	Ubuntu 19.10

$ python --version
Python 3.7.5

$ pip --version
pip 20.1.1 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)

$ nginx -v
nginx version: nginx/1.16.1 (Ubuntu)

$ mysql --version
mysql  Ver 8.0.20-0ubuntu0.19.10.1 for Linux on x86_64 ((Ubuntu))

Installing Jupyter

Jupyter itself is easy:

$ pip install jupyter

But then comes a number of infrastructural things. First of all, it’s better if you won’t run it as root, so:

  1. Create a new user for it
  2. Pick a folder where to store notebooks (I went with /var/www/jupyter)
  3. Generate a config, as you’ll need to overwrite some default settings
$ sudo useradd -m jupyter
$ sudo mkdir /var/www/jupyter
$ sudo chown -R jupyter:jupyter /var/www/jupyter
$ sudo -u jupyter jupyter notebook --generate-config

Edit the generated config:

$ nano /home/jupyter/.jupyter/jupyter_notebook_config.py
# I think it's better to keep remote access disabled
#c.NotebookApp.allow_remote_access = False

# allow requests from your domain (works in combo with $http_host from NGINX)
c.NotebookApp.local_hostnames = ['localhost','jupyter.YOUR-DOMAIN.com']

systemd service

Create a service:

$ sudo nano /etc/systemd/system/jupyter.service
[Unit]
Description=Jupyter Notebook

[Service]
WorkingDirectory=/var/www/jupyter/
ExecStart=/usr/local/bin/jupyter notebook --no-browser --config=/home/jupyter/.jupyter/jupyter_notebook_config.py
Restart=always
RestartSec=10
SyslogIdentifier=jupyter-notebook
User=jupyter

[Install]
WantedBy=multi-user.target

Enable and run the service:

$ sudo systemctl enable jupyter.service
$ sudo systemctl start jupyter.service
$ sudo systemctl status jupyter.service

NGINX config

$ sudo nano /etc/nginx/sites-available/jupyter
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 80;
    listen [::]:80;

    # so it will be available at this subdomain of the main website
    server_name jupyter.YOUR-DOMAIN.com;

    location / {
        # or whichever port you've set for your Jupyter
        proxy_pass http://localhost:8888;
        # $http_host is important for accessing Jupyter locally
        proxy_set_header Host $http_host;
        # http://nginx.org/en/docs/http/websocket.html
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }
}
$ sudo ln -s /etc/nginx/sites-available/jupyter /etc/nginx/sites-enabled/
$ sudo systemctl restart nginx.service

Let’s Encrypt certificate for the subdomain

If you already have a certificate for your main domain, here’s how to extend it to the new subdomain:

$ certbot --expand -d jupyter.YOUR-DOMAIN.com

It will also make required changes in the NGINX config.

DNS-record

Go to your hoster and add a new subdomain under your main domain. Then add a new DNS record for it (just one A record):

jupyter.YOUR-DOMAIN.com 	A 	YOUR-SERVER-IP-ADDRESS

Now as soon as DNS records update on servers, your Jupyter instance will be available on https://jupyter.YOUR-DOMAIN.com.

Authentication

Conveniently enough, Jupyter already has authentication out of the box, so anonymous users won’t be able to open your notebooks. Here’s how the login page looks like:

Jupyter login with token

With token

At least for the first time you’ll need to log in using token. To get your current token value run the following:

$ sudo -u jupyter jupyter notebook list
Currently running servers:
http://localhost:8888/?token=h8g4a81416de5ofcc3nmb9ao32de80206ae640dq8agee9nn :: /var/www/jupyter

After successful authentication you will get a listing of your working folder (/var/www/jupyter) where you can manage your notebooks:

Jupyter files

With password

You can keep logging in with token, but its value will be different with every service restart, so you might want to switch to using password. Use the Setup a Password form in the bottom of the login page - your password will be hashed and saved to /home/jupyter/.jupyter/jupyter_notebook_config.json:

{
  "NotebookApp": {
    "password": "sha1:bd10epf67nc8:1fg568845sgdeb3377bf90eccc0d34337ddhcc94"
  }
}

Sadly, it doesn’t seem to support multiple users accounts, but for us that is fine for now.

Restart the service. From now on your login page will have just the password field:

Jupyter login with password

And if you list the running Jupyter server now, it will no longer have the token value:

$ sudo -u jupyter jupyter notebook list
Currently running servers:
http://localhost:8888/ :: /var/www/jupyter

Working with SQL database

In our case we needed to do some analytics/statistics on certain values in our database. At first we started adding views for that in our .NET Core MVC admin interface, but that was veeeeery time consuming, especially given the dynamic nature of our momentary needs.

Jupyter Notebook, on the other hand, turned out to be a super convenient and fast tool for such purpose. So instead of piling up admin pages (or exposing our MySQL server over the internet) we can just run SQL queries from a notebook and process results with simple Python operations.

Packages needed for that:

$ pip install mysql-connector
$ pip install tabulate

If you will be getting this error:

mysql.connector.errors.NotSupportedError: Authentication plugin 'caching_sha2_password' is not supported

then install this package:

$ pip install mysql-connector-python

When it comes to executing SQL, I would not recommend connecting to the database from Jupyter Notebook with a user that has any writing privileges, so it’s better to create a new user with read-only privileges:

CREATE USER 'jupyter'@'localhost' IDENTIFIED BY 'YOUR-PASSWORD';
GRANT SELECT ON YOUR_DATABASE.SOME_TABLE to 'jupyter'@'localhost';
SHOW GRANTS FOR 'jupyter'@'localhost';

Now you can start your notebook with something like:

Jupyter Notebook SQL database connection

and then add as many blocks with queries as you want, formatting their results as nice tables:

Jupyter Notebook SQL query

Notebook in Visual Studio Code

Amazingly enough, you can work with Jupyter Notebook from Visual Studio Code. Here’s their documentation about that.

While it does support connecting to a remote server, sadly, you’ll need to download a local copy of your notebook file. Put it into some folder on your computer and open that folder in VS Code.

Open Command Palette and run Python: Specify local or remote Jupyter server for connections. Provide your https://jupyter.YOUR-DOMAIN.com and password/token when asked for it. If it’s all good, you’ll have .vscode/settings.json file in the workspace folder with the following contents:

{
    "python.dataScience.jupyterServerURI": "https://jupyter.YOUR-DOMAIN.com/"
}

Now you can open your sandbox.ipynb, and VS Code will connect to your Jupyter server:

Jupyter Notebook in VS Code

Just to make it clear: you launch the code from VS Code on your local computer, but it is executed on your remote Jupyter server. How awesome is that.

But I will still have to use the browser, at least until something like this feature is implemented, because, like I said, we need everyone in the team to work with the same version of the notebook.