WebMO Help - JupyterLab (Enterprise)

JupyterLab Provides Web-Based Python Programming

Results from WebMO calculations can be acessed programmatically via a REST-JSON interface. Although this can be done from any programming language with http communication capabilities, Python is widely available, very popular, and particularly convenient for this purpose. To simplify this process, a Python Interface for WebMO has been written that allows user, job, result information to be retrieved. Jobs may also be submitted to WebMO from Python.

Python typically needs to be installed on a user's local computer as a programming language, from which it can be accessed in a terminal session or through a Integrated Programming Environment (IDE). An increasingly popular Python programming environment is JupyterLab, which includes both Python and a notebook interface, so that programs and explanatory comments can exist within one interactive document.

Similar to WebMO, JupyterLab is web-based so all interaction is done through a web-browser. JupyterLab allows one to customize and extend the viewing of WebMO results. In addition to running Python programs, JupyterLab can load and execute notebooks written by others, allowing WebMO users to share their results and/or analysis methods.

JupyterLab is a web-based Python programming environment. The JupyterLab server is often installed on a user's local computer, which has the advantage of being able to locally save Jupyter notebooks (programs and results). Alternatively, a JupyterLab server can also be accessed on the web, which has the advantage of not having to install any software on a user's computer, but any resulting notebooks are saved on the server and would need to be downloaded for local saving and further distribution.

Prerequisites: The WebMO webserver must be capable of processing REST requests. JupyterLab (or notebook) must be available to the WebMO user and running. JupyterLab typically runs on the user's local computer so that notebooks can be saved, although installations on remote servers are also common.

Several options exist for allowing WebMO users to access to WebMO output files via JupyterLab:

Option 1: JupyterLab is usually installed on a local machine

The easiest and recommended way to install JupyterLab is by installing Anaconda, which includes Python, JupyterLab, and Navigator to optionally install additional conda packages for numerical analysis:

  1. Download Anaconda for your local operating system. Use the latest Python 3.X version.
  2. Install the downloaded version of Anaconda.
  3. From a command prompt (Windows) or Terminal (Mac/Linux), navigate to a jupyter folder, and run JupyterLab on the local machine:
        $ jupyter-lab
  4. Open and use an existing Python3 notebook (*.ipynb)
    or
    Click the Python 3 Notebook icon to open a new notebook
  5. If not already done, install the webmo python library by typing "%pip install webmo" in a cell and clicking the Run icon (or pressing control-Enter)
  6. When done, choose File:Shutdown to exit JupyterLab

Option 2: MyBinder.Org can host an instance of JupyterLab and reference a GitHub repository of notebooks

MyBinder.org provides an executable Jupyter notebook environment for a GitHub repository, thereby eliminating the need for each user to install JupyterLab on their local machine:

  1. In a web browser, open
    https://mybinder.org/v2/gh/<github_user>/<github_repository>/master
    which will open JupyterLab. This is most easily done by clicking the Jupyter link in the navigation bar, followed by Open Jupyter button. Servers pre-configured for WebMO are available at
    https://mybinder.org/v2/gh/webmodemo/demoserver/master
  2. Open and use an existing Python3 notebook (*.ipynb)
    or
    Click the Python 3 Notebook icon to open a new notebook
  3. When done, choose File:Shutdown to exit JupyterLab. Note that results will not be saved locally, as they were stored on the virtual JupyterLab server which is being deleted!

Additional notebooks can be saved as *.ipynb files in the GitHub repository, which must directly reference your WebMO server. Required python modules can be specified in a requirements.txt file in the repository.

Option 3: JupyterLab can be hosted on the WebMO server (under development)

A development version of The Littlest Jupyterhub (TLJB) is available, though it is untested by WebMO. TLJB allows up to 100 users to access Jupyter notebooks running on the WebMO server itself.

Use System Manager to specify whether the JupyterLab server is expected on a user's local machine or on a web-based host. The Open Jupyter button will open the location defined by Jupyter notebook URL.

Running WebMO Python Program Templates in a JupyterLab Notebook

WebMO Enterprise includes a built-in interface to JupyterLab. When viewing the results of a WebMO job, several demonstration program templates are available within WebMO to illustrate downloading job results into a JupyterLab notebook performing subsequent calculations:

These demonstration templates are available on the View Results page from the Jupyter link in the navigation panel. The WebMO administrator can add additonal Python programs by placing them in the <cgiBase>/REST/templates directory.

While viewing a job within the the WebMO interface, click the Jupyter link to open the Jupyter Notebook Templates dialog box


Jupyter Notebook Templates

To run a WebMO python program in your Jupyter notebook:

  1. Select a script from the dropdown list
  2. Click Copy to Clipboard to copy the %load template command
  3. Click Open Jupyter to open Jupyter in your web browser; it will either be a local or remote Jupyter server, depending on the Jupyter notebook URL specification in System Manager
  4. Click the Notebook: Python3 icon to open a notebook
  5. Paste in the %load template command into a code cell, and click the Run icon (or press control-Enter) to load the Python program
  6. Click Run again to execute the program

MyBinder Jupyter Server
JupyterLab

 


%load Template
Python Program

 


Python Output
Python Output, continued

Python scripts added to the <cgiBase>/REST/templates directory will be dynamically displayed to in the dropdown list for users to select.

Writing and Running JupyterLab Notebooks that Access WebMO

Pre-Requisites

For JupyterLab notebook development, it is most convenient to have JupyterLab installed on your local machine. See option 1 above.

To interact with a WebMO instance from within JupyterLab, it is assumed that the webmo Python library has been installed as part of the JupyterLab environment.

$ pip install webmo

Simple.ipynb

Because Jupyter notebooks contains both code and data, you can open them locally in JupyterLab and view what they look like before running them on your jobs. Here is Simple.ipynb, which demonstrates connecting to a WebMO instance, obtaining the results of a job, and displaying an image.


Simple.ipynb demonstrates basic access to job results and images

Header information

JupyterLab notebooks that interact with WebMO need to:

The webmo library must already be installed in the Jupyter notebook environment, after which it can be imported.
import webmo # required
import json # optional for formatting values from JSON output
import matplotlib.pyplot as plt # optional for creating and displaying plots

A REST session is invoked by specifying the URL to a WebMO instance.
URL = "https://server.university.edu/~webmo/cgi-bin/webmo/rest.cgi"
user_name = input("Enter WebMO username: ")
rest = webmo.WebMOREST(URL, username=user_name)
The user will be prompted for their WebMO password.

Some additional information can be obtained from the user, such as job number
job_number = 72
or
job_number = int(input("Enter job number: "))

Numeric Results

Parsed information contained on the View Results page is accessible inh a JupyterLab notebook. These results are in JSON format, allowing them to be extracted and printed in a customized manner.

results = rest.get_job_results(job_number)
energy = results['properties']['rhf_energy']['value']
units = results['properties']['rhf_energy']['units']
print("Energy: {} {}".format(energy, units))

Images

In addition to numeric quantities, a wide variety of images can be created, including geometry, dipole moment, partial charges, vibrational modes, molecular orbitals, and spectra (IR, Raman, VCD, UV-Vis, and NMR). Images require IPython, which is implemented in JupyterLab.

Images are generated by an async process that requires using the await command. Images can be saved as a python variable for subsequent processing and/or written to disk.

await rest.display_job_property(job_number, "geometry")


Use await for obtaining images
Images can be saved as variables

One can tweak the images by specifying size, rotation, background, etc. See Image_Simple.ipynb and Documentation.ipynb for details.

Running Jobs

New jobs can be submitted to WebMO from within JupyterLab notebooks.

Input files can be directly submitted to an engine:

input_contents = """#N HF/6-31G(d) OPT

water

0 1
O
O 0.00 0.00 0.00
H 1.05 0.00 0.00
H -0.35 0.00 1.00

"""
job_number = rest.submit_job("H2O", input_contents, "gaussian")

Or variables for a specific Calculation Type can be defined and submitted for WebMO to create an input file, which then can be submitted to run:

from webmo.util import xyz_from_name

templates = rest.get_templates('gaussian')
template = templates['Geometry Optimization']
input_contents = rest.generate_input(template, variables={
'jobName' : 'H2O',
'geometry' : xyz_from_name('water'),
'theory' : 'HF',
'basisSet' : '6-31G(d)'})
job_number = rest.submit_job("H2O", input_contents, "gaussian")

For interactive notebooks, it can be useful to wait for a job to complete before moving to subsequent cells:

rest.wait_for_job(job_number)

Examples

The following JupyterLab notebooks (*.ipynb) illustrate additional applications:

Tips on Using Jupyter Notebooks

What is Jupyter?

Jupyter notebooks are similar to Mathematica notebooks, Maple worksheets, or MathCAD documents, which allow for interactive analysis and documentation of scientific questions. Jupyter extends these prior examples from stand-alone applications to the web.

A Jupyter Notebook Document (*.ipynb), or simply "Jupyter notebook", contains both code and rich text elements (figures, links, equations, ...). Because of the mix of code and text elements, a Jupyter notebook can bring together data, description of analysis, real-time analysis and results. JupyterLab allows you to edit and run notebooks via a web browser.

The two main components of JupyterLab are the Notebooks and a kernel:

Starting JupyterLab

After installation (see above), JupyterLab can be started from a shortcut or a terminal session:
$ jupyter-lab

A webpage containing JupyterLab will open up with a menu, file navigator, and launcher. Note that the URL for a locally-installed instance will be something like:
    http://localhost:8888/lab
"localhost" is not a website, but your own computer. "lab" is your home or root JupyterLab directory.


JupyterLabe

Running an existing notebook

Navigate to folder with your *.ipynb ("iPython notebook") files and run the desired file. A kernel will be started to run notebook, and the existing notebook will appear in a new tab.

The notebook can be edited, etc. Changes are automatically saved.

To exit the document, File: Close Tab.

Creating a new notebook

In the Launcher, select a Python 3 Notebook. A kernel will be started to run notebook, and the blank notebook will appear in a new webbrowser tab.


Launch Python3 Notebook
Empty Python 3 Notebook

Interacting with a JupyterLab notebook

A notebook consists of cells, which may be of the following types:

There are two modes of interacting with a JupyterLab notebook page:

Within Edit mode there are several important keyboard shortcuts:

Within Command mode there are several important keyboard shortcuts:

A Code cell contains python code to be executed by the kernel. Output will appear at the bottom of the cell. It has a label "[ ]" on the left. The contents of "[ ]" is blank if not yet run, * if running, or a number representing the order of processing by the kernel. The bar on the left of the cell is blue if unedited, but orange of edited since it was last run.

A Markdown cell contains text formatted in markdown language and is editable in edit mode. It displays its formatted output when run from command mode. It does not have a label on the left.

IMPORTANT: You must run the contents of a cell to see the code output or formatted display!

Example

Open a new Python3 notebook. In code cell, type
    print ('Hello World!')
and click the run icon (or select Run: Run Selected Cells, or press Shift+Enter). The program output will appear below the code cell.


JupyterLab Python Program

Magic commands

There are also "magic" commands that execute outside of Python:

Exiting JupyterLab

Save your notebook. Choose File:Shutdown from the menu system.