Enabling software using environment modules with `Lmod`

Environment modules allow us to specify which environment variables have to change for a software package to function and/or the search paths that need to be added to your current environment.

A modern implementation of environment modules that we will use is Lmod (https://github.com/tacc/lmod), which was developed by TACC (Texas Advanced Computing Center), one of the large computing centers in the United States.

Installation of LMOD

Download the latest version of LMOD

[install@master ~]$ wget https://github.com/TACC/Lmod/archive/refs/tags/8.7.37.tar.gz
[install@master ~]$ tar xvzf 8.7.37.tar.gz

Install prerequisites

You will need the following RPM packages installed on both your master and your compute nodes.
- lua
- lua-devel
- lua-posix
- lua-filesystem

Configure and install LMOD

We will install LMOD into our /data/opt folder.

[install@master ~]$ cd Lmod-8.7.37
[install@master Lmod-8.7.37]$ ./configure --prefix=/data/opt/apps
[install@master Lmod-8.7.37]$ make pre-install

Create necessary symbolic links as root user

The reason we are using make pre-install and not make install is that the latter would try to create symbolic links at locations which require root rights.

Instead we will be doing these links manually. Exit the install user session with exit and create the following symbolic links as root:
```
# go to /data/opt/apps/lmod/ and create a symbolic link for the current version
cd /data/opt/apps/lmod/
ln -s 8.7.37 lmod

# module support for bash
ln -s /data/opt/apps/lmod/lmod/init/profile /etc/profile.d/z00_lmod.sh

# module support for csh
ln -s /data/opt/apps/lmod/lmod/init/cshrc /etc/profile.d/z00_lmod.csh
```
Note

The symbolic links for z00_lmod.sh and z00_lmod.csh also have to be created on each compute node.

Why do we create the first lmod symbolic link? This allows updating to a new Lmod version on the entire cluster without having to update all symbolic links on all compute nodes. You only need to update the /data/opt/apps/lmod/lmod symbolic links to point to the new version.

Log out and back into your master node. If you set up Lmod correctly, you should be able to type module avail.

[install@master ~]$ module avail
------------------------------ /data/opt/apps/lmod/8.7.37/modulefiles/Core ------------------------------
lmod    settarg

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys"

Create a folder hierarchy to organize your software

Lmod loads module files from arbitrary locations. First we create some sub-directories inside of /data/opt for common software categories. E.g., we already installed Python into the /data/opt/tools sub-directory. The corresponding module files of each category are placed in a modulefiles sub-directory. In case of tools this will be /data/opt/tools/modulefiles.

# make sure to be 'install' user
su - install

# create directories
mkdir -p /data/opt/{apps,base,libs,tools}/modulefiles

To add these modulefiles directories to the Lmod search path we need to set the MODULEPATH environment variable. This need to be a permanent change. This can be achieved by creating a file /etc/profile.d/modules.sh and adding the following export:

export MODULEPATH=/data/opt/base/modulefiles:/data/opt/libs/modulefiles:/data/opt/apps/modulefiles:/data/opt/tools/modulefiles

Log out and back in to apply this change to your session.

Note

Copy the /etc/profile.d/modules.sh to all compute nodes too to ensure they see the same module files.

Add a module files for Python 2 & 3

Module file basics

Lmod searches all MODULEPATH directories for LUA script files (.lua) and interprets them as module files. Module files can be organized in folders to represent software packages. Each module file in a folder is seen as a version of that software, which is why these files usually have a filename representing the version number.

For Python 3.9.19 we create a python folder in /data/opt/tools/modulefiles and add an empty text file with the name 3.9.19.lua.

[install@master ~]$ mkdir -p /data/opt/tools/modulefiles/python
[install@master ~]$ touch /data/opt/tools/modulefiles/python/3.9.19.lua

show directory tree of tools/modulefiles

[install@master ~]$ tree /data/opt/tools/modulefiles/
/data/opt/tools/modulefiles/
└── python
    └── 3.9.19.lua

1 directory, 1 file

Once the module file exists, running module avail will show the new module in its list.

[install@master ~]$ module avail

------------------------- /data/opt/tools/modulefiles --------------------------
   python/3.9.19

----------------- /data/opt/apps/lmod/8.7.37/modulefiles/Core ------------------
   lmod    settarg

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching
any of the "keys".

The module list command will show you all modules that have been loaded so far, which should be empty now.

[install@master ~]$ module list
No modules loaded

Try loading the python module and run module list again.

[install@master ~]$ module load python
[install@master ~]$ module list

Currently Loaded Modules:
  1) python/3.9.19

You will see that module load python is the same as module load python/3.9.19 since it is the only version and therefore the default.

Create an empty Python 2.7.16 module file and inspect the output of module avail:

[install@master ~]$ touch /data/opt/tools/modulefiles/python/2.7.16.lua
[install@master ~]$ tree /data/opt/tools/modulefiles/
/data/opt/tools/modulefiles/
└── python
    ├── 2.7.16.lua
    └── 3.9.19.lua

1 directory, 2 files

[install@master ~]$ module avail

------------------------- /data/opt/tools/modulefiles --------------------------
   python/2.7.16    python/3.9.19 (D)

----------------- /data/opt/apps/lmod/8.7.37/modulefiles/Core ------------------
   lmod    settarg

  Where:
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching
any of the "keys".

If a folder has multiple versions installed and follows a common naming scheme that follows a version number such as X, X.Y or X.Y.Z, Lmod is able to detect the latest version and make it the default. It will mark this version with a (D). Any module load without a specific version number will load the default module.

An alternative to the automatic detection is to create a symbolic link with the name default inside of the python folder and point it to a specific version’s module file. Create a symbolic link to 2.7.16.lua and call it default to see the difference in module avail.

[install@master ~]$ ln -s /data/opt/tools/modulefiles/python/2.7.16.lua /data/opt/tools/modulefiles/python/default
[install@master ~]$ tree /data/opt/tools/modulefiles/
/data/opt/tools/modulefiles/
└── python
    ├── 2.7.16.lua
    ├── 3.9.19.lua
    └── default -> 2.7.16.lua

0 directories, 3 files

[install@master ~]$ module avail

------------------------- /data/opt/tools/modulefiles --------------------------
   python/2.7.16 (D)    python/3.9.19

----------------- /data/opt/apps/lmod/8.7.37/modulefiles/Core ------------------
   lmod    settarg

  Where:
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching
any of the "keys".

You can also create other symbolic links to create other defaults, e.g. if you have multiple version of Python 2 and 3 installed, you can define a default one by creating a 3.lua and 2.lua symbolic link to a specific default version.

[install@master ~]$ module avail

------------------------ /data/opt/tools/modulefiles ---------------------------
   python/2    python/2.7.16 (D)    python/3    python/3.9.19

------------------------ /data/opt/apps/lmod/8.7.37/modulefiles/Core -----------
   lmod    settarg

  Where:
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching
any of the "keys".

Note that loading a different version of the same software package will perform a switch, which means the other version will be unloaded.

[install@master ~]$ module list
No modules loaded
[install@master ~]$ module load python/2
[install@master ~]$ module list

Currently Loaded Modules:
  1) python/2

[install@master ~]$ module load python/3

The following have been reloaded with a version change:
  1) python/2 => python/3

[install@master ~]$ module list

Currently Loaded Modules:
  1) python/3

Writing a module file

Module scripts in Lmod are just Lua scripts. That means you can use any valid Lua code and execute in such a module. E.g., you can use variables to parameterize your Lua scripts.

Lmod defines several functions such as whatis(), help() and prepend_path() which allow you to manipulate the environment or control help output. Here is a short summary of some useful ones:

help(text)
Output presented to users if they request module help MODULENAME

whatis(text)
Output presented to users if they request module whatis MODULENAME and used for searches with module keyword TEXT

setenv(name, value)
Set an environment variable value. Value is unset if module is unloaded.

pushenv(name, value)
Set the value of environment variable and push old value to a internal stack. If this module is unloaded again the previous value is popped from the stack and restored.

prepend_path(name, path)
Prepend a path to an environment variable. Unloading the module reverses this.

append_path(name, path)
Append a path to an environment variable. Unloading the module reverses this.

depends_on(module_name)
Specifies the name of a module that must be loaded for this module to function.

Below you can see the contents of such a module file. The Lua language allows you to define variables such as version and prefix and perform string operations such as concatenation with .. to build more complex strings.
```
-- -*- lua -*-

version = "3.9.19"
prefix = "/data/opt/tools/python-" .. version

whatis("Python Programming Language")

help([[
This module provides the Python 3 programming language.

Python is an interpreted, interactive, object-oriented programming
language often compared to Tcl, Perl, Scheme or Java. Python includes
modules, classes, exceptions, very high level dynamic data types and
dynamic typing.

The environment variables $PATH and $MANPATH are updated as needed.
]])

prepend_path("PATH",            prefix .. "/bin")
prepend_path("LD_LIBRARY_PATH", prefix .. "/lib")
prepend_path("MANPATH",         prefix .. "/share/man")
```
Save the file as /data/opt/tools/modulefiles/python/3.9.19.lua and type module load python/3.9.19. Python 3 is now available in your PATH and can run python3.
```
[install@master ~]$ module load python/3.9.19
[install@master ~]$ python3
Python 3.9.19
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
```
Create an equivalent module for Python 2. You’ll quickly see why variables and concatenation are useful tools. Ensure that python2 runs the Python 2.7.16 binary, not the system default.

Enabling software using environment modules with Lmod

Installation of LMOD

Create a folder hierarchy to organize your software

Add a module files for Python 2 & 3

Enabling software using environment modules with `Lmod`