## Setup

Before you set up the evoland-plus HPC pipeline, you should make sure to satisfy
a few requirements.
This section will go over [hardware](#hardware) and [software](#software)
requirements, and then guide you through
the [evoland-plus HPC repository](#future-ei-repository) setup.
The following pages guide through the details of each step in the pipeline,
before concluding with the [execution](running.html) of the pipeline.


### Requirements

We are using a Linux cluster
with [SLURM](https://slurm.schedmd.com/overview.html) as scheduler.
If your cluster uses a different scheduler, you can see if it is compatible
with the SLURM syntax, or you can adapt the scripts to your scheduler.

::: {#cau-linux-cluster .callout-note}
#### What is a Linux cluster?
As this pipeline is specifically designed to simulate a large number of
scenarios,
it has been optimized for high-performance computing (HPC) environments.
If you only need to run a few scenarios, it might be easier to run the
steps manually.
Otherwise, you do need to have access to a Linux cluster with SLURM.
Feel free to reach out to a technically savvy colleague or your local HPC
support for help.
:::

#### Hardware
The minimum memory and CPU requirements cannot generally be stated, as they
depend on the area of interest, input data, and the number of scenarios.
A viable starting point for a country with the size of Switzerland, using a
resolution of 100 m, is 16 GB of memory and 4 CPUs.
This is the case for a few scenarios and no parallelization within the steps.
Scaling up to around 1000 scenarios, we suggest at least 128 GB of memory
and 16 CPUs, to achieve a viable runtime.
As this is an estimate, it is essential to monitor runtime before scaling up.


#### Software


Additionally, you need to install the following software:

##### Micromamba/Conda

For some pipeline steps, we use conda environments.
[Conda](https://docs.conda.io/projects/conda/en/stable/) is a package manager
that helps you manage dependencies in isolated environments.
We recommend using [`micromamba`](https://mamba.readthedocs.io/en/latest/),
which does the same job as Conda, but resolves dependencies much faster,
with the flexibility of
[`miniconda`](https://docs.conda.io/en/latest/miniconda.html) (CLI of Conda).
Find the installation instructions for Micromamba
[here](https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html).
We have added compatibility for `micromamba`, `mamba` and `conda`,
in this order of preference, but only tested with `micromamba`[^conda].

[^conda]: The installed CLI is identified via bash variables in
    [`src/bash_common.sh`](https://github.com/ethzplus/evoland-plus-HPC/tree/main/src/bash_common.sh).
    If none is found, an error highlights the issue.

We have chosen [`conda-forge`](https://conda-forge.org/) as the default
channel for the conda environments,
as it is a single source for our `R`, Python, and lower-level dependencies
(e.g., `gdal`, `proj`).
This is independent of the modules and applications provided by the
HPC environment.

##### Apptainer

Running containerized applications on HPCs can be challenging.
To simplify the process, we use the [Apptainer](https://apptainer.org/)
(formerly Singularity) container runtime.
Make sure your HPC environment supports Apptainer, and that you have the
necessary permissions to run containers.
If this is not the case, contact your HPC support team for help.

##### Docker

Building the LULCC container requires Docker[^docker-version]
before converting it to the Apptainer format.
The `lulcc` container uses
the [`dinamica-ego-docker`](https://github.com/cbueth/dinamica-ego-docker/)
container (version `7.5`).

This step can be done on a local machine, and will be explained in the
[LULCC](10_LULCC.html) step.

[^docker-version]: The Docker version used is `24.0.7`, but the container
    should be compatible with most versions.

##### Dinamica EGO

Dinamica EGO is an environmental modeling platform used in the LULCC step.
It is available on [the project website](https://dinamicaego.com/).
But as aforementioned, it will be used from the LULCC docker image,
as it is only integrated from the command line interface (CLI),
not with the usual graphical user interface (GUI).

##### Yaml Parser `yq`

For the `bash` scripts, we use [`yq`](https://mikefarah.gitbook.io/yq/) to parse
the `yaml` configuration file.
`yq` needs to be available in the `PATH` variable of the shell.
To install the latest version[^yq-version], run the following command:
```bash
bin_dir=/usr/bin &&\
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O $bin_dir/yq
chmod +x $bin_dir/yq
```
Other [installation options](https://github.com/mikefarah/yq/#install)
and binaries can be found on the repository's README.
To make `yq` available in the `PATH` variable, make sure the `$bin_dir` is
in the `PATH` variable.
To check the parser is installed correctly, run `yq --version` in the shell.

[^yq-version]: We have used `yq` `v4.40.3`, but any version `>=4.18.1`
    should work.

##### LULCC Repository {#lulcc-repository}

The version used for evoland-plus HPC is a reduced version of the original model,
adapted for containerized execution on HPCs,
and can be found on
the [`hpc` branch](https://github.com/blenback/LULCC-CH/tree/hpc)
of the repository.
Clone the repository to the HPC using git or download the repository as a zip.
If you have never used git before, search online for a guide on how to clone a
repository.


### evoland-plus HPC Repository

After you have set up the requirements, you can clone
the [evoland-plus HPC repository](https://github.com/ethzplus/evoland-plus-HPC).
This repository contains the pipeline and all necessary scripts to run it.

Before you start the pipeline, you need to configure the pipeline.
These settings are centralized in the `config.yml` file.
There are only a few mandatory changes we will highlight,
but you can find more settings with descriptive names in the file.

```{.yaml filename="src/config.yml" code-copy="false"}
# Bash variables
bash_variables:
  FUTURE_EI_CONFIG_FILE: ~/evoland-plus HPC/src/config.yml
  FUTURE_EI_OUTPUT_DIR: ~/evoland-plus HPC-Output
  ...
  # LULCC HPC version
  LULCC_CH_HPC_DIR: ~/LULCC-CH
  ...
  # Overwrites $TMPDIR if not set by the system. $TMPDIR is used by Dinamica EGO.
  # and conda/libmamba
  ALTERNATIVE_TMPDIR: /scratch
...
```

For each script, [`src/bash_common.sh`](https://github.com/ethzplus/evoland-plus-HPC/tree/main/src/bash_common.sh)
is sourced to set the environment variables.
First, `FUTURE_EI_CONFIG_FILE` needs to be set to the absolute
path of this configuration file.
`FUTURE_EI_OUTPUT_DIR` is the directory where the outputs of the pipeline will
be stored.
As the pipeline needs a multiple more temporary space than the output itself,
having a fast and large temporary directory is crucial.
If the HPC does not set the `$TMPDIR` variable, you can set it to a
different directory using `ALTERNATIVE_TMPDIR`.
This will be used in the LULCC and NCP steps for temporary files.
Finally, `LULCC_CH_HPC_DIR` is the directory where the LULCC repository is
stored,
which was cloned in the [previous step](#lulcc-repository).



```{.yaml filename="src/config.yml" code-copy="false"}
# Focal LULC
FocalLULCC:
  ...

# LULC check
CheckLULCC:
  ...
```

To mention the `FocalLULCC` and `CheckLULCC` sections,
these are settings dedicated to separate steps in the pipeline and are
specifically loaded in the respective scripts.
We will touch on these settings in the respective steps.
To see the current settings (and test `yq`),
print the contents of `config.yml` as idiomatic YAML to stdout:
```bash
yq -P -oy src/config.yml
```

As a last general note, make sure to set the permissions of the scripts to
executable.
To make all bash scripts in the source executable, give them the permission as
follows:
```bash
# possibly activate globstar: shopt -s globstar
chmod +x src/**/*.sh
```

The next sections will guide you through the setup of each step in the pipeline.
