FreeSWMM Build Notes
Contents
FreeSWMM Build Notes#
FreeSWMM has several parts, all interrelated, but separate components including:
The website itself (http://freeswmm.ddns.net), which is built and hosted on an Ubuntu Linux server, which resides on x86-64 architecture.
A JupyterHub environment - mostly for file management via a web interface, also an easy way to move files across networks, and allows pySWMM builds and runs in a near native enviornment.
An XFCE Desktop Environment (any Desktop would suffice, GNOME, KDE, etc.; XFCE is a pretty lightweight choice)
A WINE application layer to allow simple access to the EPA supplied GUI.
The SWMM and EPANET programs (download the installers, and install using WINE)
A NoVNC server to access SWMM and EPANET via a web browser. This component allows one to remotely access the XFCE desktop and run the programs as if it were a windows environment (with some differences, but darned close)
The Host Server#
The FreeSWMM website is hosted on Amazon Web Services(AWS) as a Lightsail instance, with a public facing “static IP address”.
As far as replication the instance is a standard Ubuntu 19+ LTS install just uses the default Ubuntu packages supplied on AWS. The web server is an ordinary Apache server installed using the Ubuntu apt package menager. The server is configured so that the webroot houses freeswmm-webroot which contains the build and deploy files for this book.
The server is configured with several users, one is the shared user that allows ccess to the FreeSWMM desktop to run SWMM online. Another user is reserved for maintaining content of the webroot.
Note
Ports 22,80,8080, and 5901 are all public facing and set to accept all traffic. This level of exposure allows the website itself to function and allows remote access to the shared desktop. Other than thesed ports all others are left closed.
A few other special packages are installed: Jupyter and supporting packages; XFCE and supporting packages; WINE and its supporting packages; NoVNC and its supporting packages; Firefox and its supporting packages; LibreOffice and its supporting packages; R and TexLive. These are all done as complete, but default installs. Upon success this creates a useable desktop and modeling environment to use SWMM over the wire.
The transaction record used to build the server is listed at the end of this section.
JupyterHub#
JupyterHub is an open source tool that lets you host a distributed Jupyter Notebook environment. With JupyterHub, users can log in to the server, and write Python code in a web browser, without having to install software on their local machine.
Quoting from the development Wiki
JupyterHub is the best way to serve Jupyter notebook for multiple users. Because JupyterHub manages a separate Jupyter environment for each user, it can be used in a class of students, a corporate data science group, or a scientific research group. It is a multi-user Hub that spawns, manages, and proxies multiple instances of the single-user Jupyter notebook server.
In the FreeSWMM instances it was installed by following the instructions at https://tljh.jupyter.org/en/latest/install/custom-server.html
The primary use in FreeSWMM is to support file transfers from the remote FreeSWMM website to the user’s machine (either open a Jupyter connection on the remote to the FreeSWMM site or the other way to move files to/from the remote). The secondary use is to access and run pySWMM models (pending). The JupyterHub is also used to manage this JupyterBook (using a different userid and webroot).
XFCE Desktop Environment#
The SWMM and EPANET programs using the EPA supplied interface, need a desktop environment to effectively use the GUI. They were originally designed for Microsoft environments running on Intel architecture, but can be coerced onto other enviornments (as done here).
For the FreeSWMM demonstrator, XFCE was chosen because:
It installed natively with early attempts to create an on-line implementation of the two programs.
It is promoted as a lightweight desktop environment (see below)
Quoting from https://www.vpsserver.com/gnome-vs-xfce-vs-kde/
KDE, GNOME, and Xfce are the three most widely used desktop environments for Linux. Many users use the primary desktop environment with their chosen Linux distribution, which is part of the Unix operating system. However, professionals usually favor one over the other due to accessibility, performance, layout, file manager, etc. An other option is a RDP Windows Desktop (which is paid)
… Xfce, or XForms everyday environment, offers many configuration options and prioritizes system resource protection. Linux distributions and Unix-based operating systems frequently use it as the default environment.
… When comparing GNOME vs. Xfce, the latter has fewer features. The design is essentially the same, and it satisfies the criteria for being quick, simple to adjust, and incredibly gorgeous once customized. Since everything is straightforward, you can customize Xfce via your Configuration center.
… The Xfce environment is designed for users who don’t need to utilize the system for complex activities and want a straightforward environment with fast access to all their apps. Authors, reporters, and users of computers for everyday tasks can have it installed as it’s a mix between power and speed.
On FreeSWMM the default XFCE was installed during the OS build, with some mandatory changes in the desktop window manager in Linux.
WINE application layer#
Quoting from the development Wiki
Wine (originally an acronym for “Wine Is Not an Emulator”) is a compatibility layer capable of running Windows applications on several POSIX-compliant operating systems, such as Linux, macOS, & BSD. Instead of simulating internal Windows logic like a virtual machine or emulator, Wine translates Windows API calls into POSIX calls on-the-fly, eliminating the performance and memory penalties of other methods and allowing you to cleanly integrate Windows applications into your desktop.
The WINE layer is employed in the FreeSWMM container as a convienent way to use the EPA supplied graphical interfaces already built for SWMM and EPANET. In the FreeSWMM container, the underlying OS is an Ubuntu Linux 19+ LTS distribution. It is an ordinary Linux OS, installed onto an x86-64 hardware host
Note
The use of non-AMD/Intel hardware does not seem capable to handle the graphics properly. At one time an EXAGEAR emulator installed on a Raspberry Pi (arm7-64 architecture) was able to properly deliver EPANET with its GUI. But EXAGEAR is out of business, and no longer able to support arm-7 installs
After a server is built and has a working desktop (i.e. XFCE) then install WINE using usual root-level package manager commands.
The SWMM and EPANET programs#
These programs were downloaded directly from https://www.epa.gov/water-research/storm-water-management-model-swmm and https://www.epa.gov/water-research/epanet as self-extracting installers. They are then invoked from WINE as the root user. At the time of this writing FreeSWMM only supported the 32-bit implementations. WINE-64 was still under development.
NoVNC server to access SWMM and EPANET#
Quoting from https://novnc.com/info.html
noVNC - the open source VNC client - noVNC is both a VNC client JavaScript library as well as an application built on top of that library. noVNC runs well in any modern browser including mobile browsers (iOS and Android).
NoVNC server running on FreeSWMM listens for requests from the web server to connect to a desktop. When credentials are sent the desktop is shared through the web browser and a remote user (a short course student) has access to a working desktop and SWMM or EPANET through WINE on that desktop - literally allowing a fully provisioned computation enviornment to be shared and run on the internet.
Note
In a short course, the instructor would activate multiple instances on Amazon Web Services (AWS) and share public IP with each student (1 student, 1 IP address). At the end of the course the instances could be transferred to the students (they would create their own AWS accounts), or destroyed. The original instance that was cloned to generate the student instances remains untouched.
Server Build Transactions:#
# freeswmm.ddns.net
# initial updates
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
# change hostname
sudo hostnamectl set-hostname freeswmm
# create users
sudo useradd -m -d /home/sensei -s /bin/bash sensei # this is the maintenance user
sudo useradd -m -d /home/antares -s /bin/bash antares # this is the shared user
# set passwords
sensei == *******
antares == freeswmm
root == ***********
ubuntu == AWS default # don't change or lose the instance!
# install and configure apache
sudo apt install apache2
sudo systemctl status apache2
sudo systemctl stop apache2
# install R
sudo apt-get install r-base-core
sudo apt-get install r-base
# install and configure JupyterHub
sudo apt install -y python3-pip
sudo apt install -y build-essential libssl-dev libffi-dev python3-dev
sudo apt-get install python3-venv
sudo python3 -m venv /opt/jupyterhub/
sudo apt install nodejs npm
sudo npm install -g configurable-http-proxy
sudo /opt/jupyterhub/bin/python3 -m pip install wheel
sudo /opt/jupyterhub/bin/python3 -m pip install jupyterhub jupyterlab
sudo /opt/jupyterhub/bin/python3 -m pip install ipywidgets
sudo mkdir -p /opt/jupyterhub/etc/jupyterhub/
cd /opt/jupyterhub/etc/jupyterhub/
sudo /opt/jupyterhub/bin/jupyterhub --generate-config
EDIT THE CONFIG "c.Spawner.default_url = '/lab'"
sudo mkdir -p /opt/jupyterhub/etc/systemd
sudo nano /opt/jupyterhub/etc/systemd/jupyterhub.service
INSERT <--
[Unit]
Description=JupyterHub
After=syslog.target network.target
[Service]
User=root
Environment="PATH=/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/jupyterhub/bin"
ExecStart=/opt/jupyterhub/bin/jupyterhub -f /opt/jupyterhub/etc/jupyterhub/jupyterhub_config.py
[Install]
WantedBy=multi-user.target
-->
sudo ln -s /opt/jupyterhub/etc/systemd/jupyterhub.service /etc/systemd/system/jupyterhub.service
sudo systemctl daemon-reload
sudo systemctl enable jupyterhub.service
sudo systemctl status jupyterhub.service
sudo systemctl start jupyterhub.service
# enable https
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/apache-selfsigned.key -out /etc/ssl/certs/apache-selfsigned.crt
SSLCertificateFile /etc/ssl/certs/apache-selfsigned.crt
SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key
--> dependencies to get nbconvert to work ---------
texlive-lang-french texlive-latex-base texlive-latex-recommended
python-pil-doc python3-pil-dbg python-pygments-doc ttf-bitstream-vera
python-pyparsing-doc dvipng imagemagick-6.q16 latexmk libjs-mathjax
python3-sphinx-rtd-theme python3-stemmer sphinx-doc texlive-fonts-recommended
texlive-latex-extra texlive-plain-generic sgml-base-doc debhelper
gdb-doc python3-doc python3-pil.imagetk python3-gdbm-dbg python3-tk-dbg
ghostscript-x imagemagick-doc autotrace cups-bsd | lpr | lprng enscript ffmpeg gimp gnuplot grads graphviz hp2xx html2ps
libwmf-bin mplayer povray radiance sane-utils transfig ufraw-batch colord libfftw3-bin libfftw3-dev libgd-tools gvfs fonts-mathjax-extras fonts-stix libjs-mathjax-doc inkscape libjxr-tools librsvg2-bin libwmf0.2-7-gtk www-browser zathura-ps zathura-djvu zathura-cb
<---- end dependencies ----
sudo systemctl status apache2.service
# install Xfce and TightVNC for a desktop
sudo apt update
sudo apt install xfce4 xfce4-goodies
sudo apt install tightvncserver
In ~/.vnc/xstartup
....
#!/bin/bash
xrdb $HOME/.Xresources
startxfce4 &
....
sudo ufw allow from 192.168.1.1/24 to any port 5901 # modify IP address to public facing IP
sudo ufw allow 'Apache Full'
sudo ufw allow 'OpenSSH'
## also need to open same ports in AWS hypervisor
## need to work on port forwarding, does not quite work right yet
# install Wine (future for x86-64 emulation)
# install snap
# install NoVNC
#VNC Services (Run as root or sudo)
sudo snap set novnc services.n6081.listen=6081 services.n6081.vnc=localhost:5901<br>
sudo snap set novnc services.n6082.listen=6082 services.n6082.vnc=localhost:5902<br>