Installing Lua and Lmod

Environment modules simplify customizing the users’ shell environment and it can be done dynamically. Users load modules as they see fit. It is completely under their control. Environment Modules or simply modules provide a simple contract or interface between the system administrators and users. System administrators provide modules and users get to choose which to load.

There have been environment module systems for quite a while. See http://modules.sourceforge.net/ for a TCL based module system and see http://www.lysator.liu.se/cmod for another module system. Here we describe Lmod, which is a completely new module system written in Lua. For those who have used modules before, Lmod automatically reads TCL modulefiles. Lmod has some important features over other module system, namely a built-in solution to hierarchical modulefiles and provides additional safety features to users as described in the User Guide.

The hierarchical modulefiles are used to solve the issue of system pre-built libraries. User applications using these libraries must be built with the same compiler as the libraries. If a site provides more than one compiler, then for each compiler version there will be separate versions of the libraries. Lmod provides built-in control making sure that compilers and pre-built libraries stay matched. The rest of the pages here describe how to install Lmod, how to provide the module command to users during the login process and some discussion on how to install optional software and the associated modules.

The goal of installing Lmod is when completed, any user will have the module command defined and a preset list of modules will be loaded. The module command should work without modifying the users startup files (~/.bashrc, ~/.profile, ~/.cshrc, or ~/.zshenv). The module command should be available for login shells, interactive shells, and non-interactive shells. The command ssh YOUR_HOST module list should work. This will require some understanding of the system startup procedure for various shells which is covered here.

Installing Lua

In this document, it is assumed that all optional software is going to be installed in /opt/apps. The installation of Lmod requires installing lua as well. On some system, it is possible to install Lmod directly with your package manager. It is available with recent fedora, debian and ubuntu distributions.

Install lua-X.Y.Z.tar.gz

One choice is to install the lua-X.Y.Z.tar.gz file. This tar ball contains lua and the required libraries. This can be downloaded from https://sourceforge.net/projects/lmod/files/:

$ wget https://sourceforge.net/projects/lmod/files/lua-5.1.4.9.tar.bz2

The current version is 5.1.4.9 but it may change in the future. If the above wget doesn’t work then please go to sourceforce.net and down from the web interface. The lua package can be installed using the following commands:

$ tar xf lua-X.Y.Z.tar.bz2
$ cd lua-X.Y.Z
$ ./configure --prefix=/opt/apps/lua/X.Y.Z
$ make; make install
$ cd /opt/apps/lua; ln -s X.Y.Z lua
$ mkdir /usr/local/bin; ln -s /opt/apps/lua/lua/bin/lua /usr/local/bin

The last command is optional, but you will have to somehow put the lua command in your path. Also obviously, please replace X.Y.Z with the actual version (say 5.1.4.9)

If you use this option you do not need to use your package manager or install luarocks. Instead please read the section on how to install Lmod.

Using Your Package Manager for Redhat/Centos

If you didn’t install the lua tar ball described above then you can use your package manager for your OS to install Lua. You will also need the luaposix package

Centos may require looking the EPEL repo. At TACC we install the following rpms for our Centos based systems:

lua-5.1.4-15.el7.x86_64
lua-bitop-1.0.2-3.el7.x86_64
lua-devel-5.1.4-15.el7.x86_64
lua-json-1.3.2-2.el7.noarch
lua-lpeg-0.12-1.el7.x86_64
lua-posix-32-2.el7.x86_64
lua-term-0.03-3.el7.x86_64

You will also need the tcl and tcl-devel packages as well.:

tcl-8.5.13-8.el7.x86_64
tcl-devel-8.5.13-8.el7.x86_64

Please note that the devel packages such as tcl-devel and lua-devel are only required to build Lmod. They are not required to run the lmod package. Note as well that the tcl-devel for Centos or tcl-dev for ubuntu is only required if you configure Lmod using –with-fastTCLInterp=yes.

Using Your Package Manager for Ubuntu or Debian systems

For modern debian and debian derivatives, you can get the necessary dependencies by executing the following commands. Note that for this to work you may need to repair your /etc/apt/sources.list. Check to see if your sources.list file as uncommmented “deb-src” lines. Once this is correct, please execute the following lines:

$ sudo apt update
$ sudo apt -y build-dep lmod
$ lua_ver=$(which lua | xargs realpath -e | xargs basename)
$ sudo apt -y install lib${lua_ver}-dev tcl-dev

Otherwise please install the following packages:

lua5.3
lua-bit32:amd64
lua-posix:amd64
lua-posix-dev
liblua5.3-0:amd64
liblua5.3-dev:amd64
tcl
tcl-dev
tcl8.6
tcl8.6-dev:amd64
libtcl8.6:amd64

For Ubuntu 18.04, you will need to make lua 5.3 the default using update-alternatives and fix a lua posix symlink:

sudo update-alternatives --install /usr/bin/lua \
    lua-interpreter /usr/bin/lua5.3 130 \
    --slave /usr/share/man/man1/lua.1.gz lua-manual \
    /usr/share/man/man1/lua5.3.1.gz
sudo update-alternatives --install /usr/bin/luac \
    lua-compiler /usr/bin/luac5.3 130 \
    --slave /usr/share/man/man1/luac.1.gz lua-compiler-manual \
    /usr/share/man/man1/luac5.3.1.gz
sudo ln -s /usr/lib/x86_64-linux-gnu/liblua5.3-posix.so \
    /usr/lib/x86_64-linux-gnu/lua/5.3/posix.so

Using Luarocks

If you have installed lua but still need luaposix, you can install the luarocks program from your package manager or directly from https://luarocks.org/. The luarocks programs can install many lua packages including the ones required for Lmod.

$ luarocks install luaposix

Now you have to make the lua packages installed by luarocks to be known by lua. On our Centos system, Lua knowns about the following for *.lua files:

$ lua -e 'print(package.path)'
./?.lua;/usr/share/lua/5.1/?.lua;/usr/share/lua/5.1/?/init.lua;/usr/lib64/lua/5.1/?.lua;/usr/lib64/lua/5.1/?/init.lua;

and the following for shared libraries:

$ lua -e 'print(package.cpath)'
./?.so;/usr/lib64/lua/5.1/?.so;/usr/lib64/lua/5.1/loadall.so;

Assuming that luarocks has installed things in its default location (/usr/local/…) then you’ll need to do:

LUAROCKS_PREFIX=/usr/local
export LUA_PATH="$LUAROCKS_PREFIX/share/lua/5.1/?.lua;$LUAROCKS_PREFIX/share/lua/5.1/?/init.lua;;"
export LUA_CPATH="$LUAROCKS_PREFIX/lib/lua/5.1/?.so;;"

Please change LUAROCKS_PREFIX to match your site. The exporting of LUA_PATH and LUA_CPATH must be done before any module commands. It is very important that the double trailing semicolon are there. They are replaced by the built-in system path for Lua.

Using Ansible

There is a ready-to-use Ansible role <https://galaxy.ansible.com/idiv-biodiversity/lmod/> that allows you to install Lmod conveniently from Ansible. The role was written with installation on HPC clusters in mind, i.e. it is possible to install Lmod into a global, networked file system share on only a single host, while all other hosts install just the Lmod dependencies and the shell configuration files. Nevertheless, it is of course possible to install Lmod with this role on a single server. Also, the role supports the transition to Lmod as described in How to Transition to Lmod (or how to test Lmod without installing it for all).

You can find the complete role documentation here <https://github.com/idiv-biodiversity/ansible-role-lmod#ansible-role-lmod>.

Why does Lmod install differently?

Lmod automatically creates a version directory for itself. So, for example, if the installation prefix is set to /apps, and the current version is X.Y.Z, installation will create /apps/lmod and /apps/lmod/X.Y.Z. This way of configuring is different from most packages. There are two reasons for this:

1. Lmod is designed to have just one version of it running at one time. Users will not be switching version during the course of their interaction in a shell.

2. By making the symbolic link the startup scripts in /etc/profile.d do not have to change. They just refer to /apps/lmod/lmod/... and not /apps/lmod/X.Y.Z/...

Installing Lmod

Lmod has a large number of configuration options. They are discussed in the Configuring Lmod Guide. This section here will describe how to get Lmod installed quickly by using the defaults:

Note

If you have a large number of modulefiles or a slow parallel filesystem please read the Configure Lmod Guide on how to set-up the spider caching system. This will greatly speed up module avail and module spider

To install Lmod, you’ll want to carefully read the following. If you want Lmod version X.Y installed in /opt/apps/lmod/X.Y, just do:

$ ./configure --prefix=/opt/apps
$ make install

The installation will also create a link to /apps/lmod/lmod. The symbolic link is created to ease upgrades to Lmod itself, as numbered versions can be installed side-by-side, testing can be done on the new version, and when all is ready, only the symbolic link needs changing.

To create such a testing installation, you can use:

$ make pre-install

which does everything but create the symbolic link.

In the init directory of the source code, there are profile.in and cshrc.in templates. During the installation phase, the path to lua is added and profile and cshrc are written to the /apps/lmod/lmod/init directory. These files are created assuming that your modulefiles are going to be located in /apps/modulefiles/$LMOD_sys and /apps/modulefiles/Core, where $LMOD_sys is what the command “uname” reports, (e.g., Linux, Darwin). The layout of modulefiles is discussed later.

Note

Obviously you will want to modify the profile.in and cshrc.in files to suit your system.

Sites that want to use the .modulespath file have 3 choices on how to specify where the .modulespath file is located in order of priority:

1. Set the LMOD_MODULEPATH_INIT environmant variable to point to a file.

2. Use /etc/lmod/.modulespath

3. /apps/lmod/lmod/init/.modulespath` or configure with –with-ModulePathInit=…` to point to any file.

The format of this file is:

# comments are allowed as well as wildcards
/apps/modulefiles/\*
/apps/other_modulefiles

If this file exists then MODULEPATH_ROOT method is not used.

Another way for a site to add their own directories $MODULEPATH is to define it before z00_lmod.* is sourced. Care is required so that $MODULEPATH is changed on the login shell but not subsequent sub-shells.

Also sites can set the environment variable $LMOD_SITE_MODULEPATH with a colon separate list of directories which will be prepended to $MODULEPATH. This variable is used in /etc/profile.d/z00_lmod.* So it must be defined before the z00_lmod.* file is source. ($LMOD_SITE_MODULEPATH is new in Lmod 8.5.18)

Note

It is important to define $MODULEPATH before z00_lmod.* is run by either using .modulepath or setting $LMOD_SITE_MODULEPATH or $MODULEPATH. Do not use module use … statements in later /etc/profile.d/* files. This is because module reset returns $MODULEPATH to the value defined when lmod is first executed, which will be when z00_lmod.* is run.

The profile file is the Lmod initialization script for the bash and zsh shells, the cshrc file is for tcsh and csh shells, and the profile.fish file is for the fish shell, etc. Please copy or link the profile and cshrc files to /etc/profile.d, and optionally the fish file to /etc/fish/conf.d:

$ ln -s /opt/apps/lmod/lmod/init/profile        /etc/profile.d/z00_lmod.sh
$ ln -s /opt/apps/lmod/lmod/init/cshrc          /etc/profile.d/z00_lmod.csh
$ ln -s /opt/apps/lmod/lmod/init/profile.fish   /etc/fish/conf.d/z00_lmod.fish

To test the setup, you just need to login as a user. Or if you are already logged in, please logout and log back in so that the startup files in /etc/profile.d/*.sh will be sourced. The module command should be set and MODULEPATH should be defined. Bash or Zsh users should see something like:

$ type module
module ()
{
  eval $($LMOD_CMD bash $*)
}

$ echo $LMOD_CMD
/opt/apps/lmod/lmod/libexec/lmod

$ echo $MODULEPATH
/opt/apps/modulefiles/Linux:/opt/apps/modulefiles/Core

Similar for csh users:

% which module
module: alias to eval `/opt/apps/lmod/lmod/libexec/lmod tcsh !*`

% echo $MODULEPATH
/opt/apps/modulefiles/Linux:/opt/apps/modulefiles/Core

If you do not see the module alias then please read the next section.

Integrating module Into Users’ Shells

Bash:

On login, the bash shell first reads /etc/profile, and if profiles.d is activated, that in turn should source all the *.sh files in /etc/profile.d with something like:

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
fi

Similarly, the system BASHRC file should source all the *.sh files in /etc/profile.d as well.

Bash under Ubuntu:

Sites that run Ubuntu and have bash users should consider adding the following to their /etc/bash.bashrc:

if ! shopt -q login_shell; then
  if [ -d /etc/profile.d ]; then
    for i in /etc/profile.d/*.sh; do
      if [ -r $i ]; then
        . $i
      fi
    done
  fi
fi

This is useful because non-login interactive shells only source /etc/bash.bashrc and this file doesn’t normally source the files in /etc/profile.d/*.sh.

Bash Shell Scripts:

Bash shell scripts do not source any system or user files before starting execution. Instead it looks for the environment variable BASH_ENV. It treats the contents as a filename and sources it before starting a bash script.

Bash Script Note:

It is important to remember that all bash scripts should start with:

#!/bin/bash

Starting with:

#!/bin/sh

won’t define the module command, even if sh is linked to bash. Bash will run those scripts in shell emulation mode and won’t source the file that BASH_ENV points to.

Csh:

Csh users have an easier time with the module command setup. The system cshrc file is always sourced on every invocation of the shell. The system cshrc file is typically called: /etc/csh.cshrc. This file should source all the *.csh files in /etc/profile.d:

if ( -d /etc/profile.d ) then
  set nonomatch
  foreach i (/etc/profile.d/*.csh)
    source $i
  end
  unset nonomatch
endif

Zsh:

Zsh users have an easy time with the module command setup as well. The system zshenv file is sourced on all shell invocations. This system file can be in a number of places but is typically in /etc/zshenv or /etc/zsh/zshenv and should have:

if [ -d /etc/profile.d ]; then
  setopt no_nomatch
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
  setopt nomatch
fi

Ksh:

Ksh users as of Lmod version 8.3.12+ now have full support as long as the site installs Lmod with –supportKsh=yes. Lmod now defines FPATH to be the directory for the shell function commands such as module and ml that provide the module commands.

Note: Zsh users who wish to run ksh scripts that have module commands in them will have to export the FPATH variable as FPATH is normally a local variable and not exported in zsh.

Rc:

Rc shells ignore the /etc/profile.d directory and source /lib/rcmain on startup. Global initialization should be placed in that file. Login shells (started with rc -l) additionally read the user’s $HOME/.rcrc file.

Running the following line on startup will set up the module commands for rc users:

` mod=/opt/apps/lmod/lmod/init/profile.rc if(test -r $mod) . $mod `

Fish:

Fish users have several standard places searched for scripts. The system location is usually /etc/fish/conf.d and the user location is usually ~/.config/fish/conf.d/. Fish users are provided a special profile file, init/profile.fish, that should be linked into one of these places with a suitable name. For example, a local user for fish might want:

$ ln -s /opt/apps/lmod/lmod/init/profile.fish ~/.config/fish/conf.d/z00_lmod.fish

A site might set:

$ ln -s /opt/apps/lmod/lmod/init/profile.fish /etc/fish/conf.d/z00_lmod.fish

Issues with Bash

Interactive Non-login shells

The Bash startup procedure for interactive non-login shells is complicated and varies between Operating Systems. In particular, Redhat & Centos distributions of Linux as well as Mac OS X have no system bashrc read during startup whereas Debian based distributions do source a system bashrc. One easy way to tell how bash is set up is to execute the following:

$ strings `type -p bash` | grep bashrc

If the entire result of the command is:

~/.bashrc

then you know that your bash shell doesn’t source a system bashrc file.

If you want to have the same behavior between both interactive shells (login or non-login) and your system doesn’t source a system bashrc, then you have two choices:

1. Patch bash so that it does source a system bashrc. See contrib/bash_patch for details on how to do that.

2. Expect all of your bash users to have the following in their ~/.bashrc

   if [ -f /etc/bashrc ]; then
      . /etc/bashrc
   fi
   

As a side note, we at TACC patched bash for a different reason which may apply to your site. When an MPI job starts, it logs into each node with an interactive non-login shell. When we had no system bashrc file, many of our fortran 90 programs failed because they required ulimit -s unlimited which makes the stack size unlimited. By patching bash, we could guarantee that it was set by the system on each node. Sites will have to chose which of the two above methods they wish to deal with this deficiency in bash.

You may have to also change the /etc/bashrc (or /etc/bash.bashrc) file so that it sources /etc/profile.d/*.sh for non-login shells.

Bash Shell Scripts

Bash shell scripts, unlike Csh or Zsh scripts, do not source any system or user files. Instead, if the environment variable, BASH_ENV is set and points to a file then this file is sourced before the start of bash script. So by default Lmod sets BASH_ENV to point to the bash script which defines the module command.

It may seem counter-intuitive but Csh and Zsh users running bash shell scripts will want BASH_ENV set so that the module command will work in their bash scripts.

A bash script is one that starts as the very first line:

#!/bin/bash

A script that has nothing special or starts with:

#!/bin/sh

is a shell script. And even if /bin/sh points to /bin/bash bash runs in a compatibility mode and doesn’t honor BASH_ENV.

To combat this Lmod exports the definition of the module command. This means that even /bin/sh scripts will have the module command defined when run by a Bash User. However, a Csh or Zsh user running a bash script will still need to set BASH_ENV and run bash scripts. They won’t have the module command defined if they run a sh script.