# Step-by-step tutorial on a Rocky Linux VM

In this tutorial, we will see how to setup {{SSH_Monitor}} in order to monitor a virtual machine (VM).\
Here, the VM will both be the {{target}} machine to monitor, and the machine {{host}}ing {{SSH_Monitor}}.

We will do so using the [SSH Monitor training VM on Rocky Linux](sshmonitor-gitlab-url:packages/57879141),
in order to make this tutorial nicer.

```{important}
In this turorial, the [Security configuration](../how-to-guides/basics/security_how_to.md) step
will be ignored\
only to "ease" the overall process.

*This step otherwise almost always recommended\
and should not be skipped for production/real-world use!*
```

---

## 0. Prerequisites

* Ideally, install the latest version of [VirtualBox](https://www.virtualbox.org/wiki/Downloads).
  At least `v7+`.

* Download the [SSH Monitor training VM on Rocky](sshmonitor-gitlab-url:packages/57879141)
  (the `SSH-Monitor-training-VM-on-Rocky.ova` file)\
  and import it with VirtualBox ({guilabel}`File` → {guilabel}`Import appliance`, or `Ctrl+I`).

* You don't need to modify the VM settings, the `.ova` file comes with the VM pre-configured.

* Start the VM.

```{admonition} Troubleshooting
If VirtualBox doesn't launch the VM and throws the `VERR_NAT_REDIR_SETUP` error,\
then this is probably because a program is using TCP and/or UDP port `5064`,\
or because a program is using TCP and/or UDP port `5065`.

You can check what program is using those ports,
e.g. on Linux with: `$ sudo netstat -tunlp`.

Sometimes, [CSS](https://controlssoftware.sns.ornl.gov/css_rcp/products.html)
or [Phoebus](https://controlssoftware.sns.ornl.gov/css_phoebus/)
can still be using those ports even after being closed.\
In this case, do not hesitate to kill their associated Java process(es).
```

### 0.1. SSH access to the VM, from Linux

On the Linux computer used to run VirtualBox, please follow the below instructions:

* Check that you can access <https://gitlab.com> (hosting {{SSH_Monitor}})
  using {{SSH}}:

  ```{code} bash
  $ ssh -T git@gitlab.com
  Welcome to GitLab, @you-are-such-a-nice-person!
  ```

  * If you don't have access,
    then please follow the
    [Gitlab documentation about how to use SSH keys to communicate with GitLab](https://docs.gitlab.com/ee/user/ssh.html).

* Check that your GitLab {{SSH}} key is included in your {{SSH}} agent:

  ```{code} bash
  $ ssh-add -l
  ```

* Check that the `ForwardAgent` option is **not** set to `no`, in `$HOME/.ssh/config`.
  E.g.:

  ```{code} bash
  Host localhost
    ForwardAgent yes
    ...

  ```

* Check that the `ForwardAgent` option is **not** set to `no`, in `/etc/ssh/sshd_config`.

* Connect to the VM via {{SSH}} like so: 

  ```{code} bash
  $ ssh -A -p 2222 notroot@localhost
  ```

  > Password is `notroot`

* On the VM launched by VirtualBox,\
  check that you can still access <https://gitlab.com> using {{SSH}}:

  ```{code} bash
  VM $ ssh -T git@gitlab.com
  Welcome to GitLab, @you-are-such-a-nice-person!
  ```

  * If you don't have access,
    then you somehow probably have a `ForwardAgent` issue.\
    Please refeer to the
    [GitLab documentation about how to troubleshoot SSH](https://docs.gitlab.com/ee/user/ssh_troubleshooting.html).

* On the VM launched by VirtualBox,\
  try to {{SSH}} to yourself in order to add yourself to the `~/.ssh/known_hosts` file\
  (this is important because otherwise, errors will appears when running your {{SSH_Monitor}} {{IOC_program}}):

  ```{code} bash
  VM $ ssh notroot@localhost

    The authenticity of host localhost (::1) can t be established.
    ED25519 key fingerprint is SHA256:<...>
    This key is not known by any other names.
    Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
    Warning: Permanently added localhost (ED25519) to the list of known hosts.
  ```

* 🎉 You are ready to start the tutorial!


### 0.2. SSH access to the VM, from Windows
  
On the Windows computer used to run VirtualBox, please follow the below instructions:

* Open a terminal (`CMD` or `PowerShell`)
  and check that [OpenSSH](https://www.openssh.com/) is installed,
  e.g. like so:

  ```{code} bash
  $ ssh -V
  ```

  [OpenSSH](https://www.openssh.com/) should be installed by default on Windows\
  (since Windows 10 build 1809 and Windows server 2019).\
  See [the OpenSSH for Windows overview](web-archive:20240930123624/https://learn.microsoft.com/en-us/windows-server/administration/OpenSSH/openssh-overview)
  for more details.

  *If [OpenSSH](https://www.openssh.com/) is not installed,
  then you will have to [install it yourself](https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH).*

* Follow
  [the official microsoft prerequisites (and prerequisites check)](web-archive:20241004183908/https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse?tabs=gui#prerequisites).

* Make sure the `ssh-agent` service is enabled. E.g. run in a PowerShell terminal:

  ```{code} bash
  # By default the ssh-agent service is disabled. Configure it to start automatically.
  # Make sure you're running as an Administrator.
  Get-Service ssh-agent | Set-Service -StartupType Automatic
  
  # Start the service
  Start-Service ssh-agent
  
  # This should return a status of Running
  Get-Service ssh-agent
  
  # Now load your key file (the one associated to your GitLab account) into ssh-agent
  ssh-add $env:USERPROFILE\.ssh\<your-ssh-key-file>
  ```

  ```{admonition} Troubleshooting
  After running the `ssh-add` command,\
  if you are confronted to the `No such file or directory` error message,\
  then you will have to specify the full absolute path to your SSH key file by hand.\
  E.g. like so:

        ssh-add C:\Users\username\.ssh\id_rsa
  ```

  ```{admonition} Troubleshooting
  After running the `ssh-add` command,\
  if you are confronted to an error message similar to the one below:

        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
        @         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
        Permissions for 'C:\\Users\\username\\.ssh\\id_rsa' are too open.
        It is required that your private key files are NOT accessible by others.
        This private key will be ignored.

  Then you should change the permissions of the SSH key file.\
  Follow [this link](https://superuser.com/questions/1296024/windows-ssh-permissions-for-private-key-are-too-open)
  for help about how to do it.
  ```

  For more details,\
  see [the official microsoft documentation about how to start the ssh-agent service](web-archive:20241006133220/https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement#user-key-generation)\
  (at the end of the "User key generation" section).

* Check that you can access <https://gitlab.com> (hosting {{SSH_Monitor}})
  using {{SSH}}:

  ```{code} bash
  $ ssh -T git@gitlab.com
  Welcome to GitLab, @you-are-such-a-nice-person!
  ```

  * If you don't have access,\
    then please follow the
    [Gitlab documentation about how to use SSH keys to communicate with GitLab](https://docs.gitlab.com/ee/user/ssh.html).

* Check that your GitLab {{SSH}} key is included in your {{SSH}} agent:

  ```{code} bash
  $ ssh-add -l
  ```

* Check that the `ForwardAgent` option is **not** set to `no`,\
  in the `config` file located in the `.ssh` directory of your home.
  E.g.:

  ```{code} bash
  Host localhost
    ForwardAgent yes
    ...

  ```

* Connect to the VM via {{SSH}} like so: 

  ```{code} bash
  $ ssh -A -p 2222 root@localhost
  ```

  > Password is `notroot`

* On the VM launched by VirtualBox,\
  check that you can still access <https://gitlab.com> using {{SSH}}:

  ```{code} bash
  VM $ ssh -T git@gitlab.com
  Welcome to GitLab, @you-are-such-a-nice-person!
  ```

  * If you don't have access,
    then you somehow probably have a `ForwardAgent` issue.\
    Please refeer to the
    [GitLab documentation about how to troubleshoot SSH](https://docs.gitlab.com/ee/user/ssh_troubleshooting.html).

* On the VM launched by VirtualBox,\
  try to {{SSH}} to yourself in order to add yourself to the `~/.ssh/known_hosts` file\
  (this is important because otherwise, errors will appears when running your {{SSH_Monitor}} {{IOC_program}}):

  ```{code} bash
  VM $ ssh root@localhost

    The authenticity of host localhost (::1) can t be established.
    ED25519 key fingerprint is SHA256:<...>
    This key is not known by any other names.
    Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
    Warning: Permanently added localhost (ED25519) to the list of known hosts.
  ```

* 🎉 You are ready to start the tutorial!

---

```{important}
From now on, the rest of the turorial happens on the VM!\
All the following commands should be run inside the VM, not on the computer hosting it.
```

---

## 1. Install, patch and build EPICS

```{seealso}
This section is inspired from the [install, update, remove how-to guide](../how-to-guides/basics/install_update_remove_how_to.md).\
Feel free to check this documentation for more details.
```

- Install the {{EPICS}} dependencies
  (using `dnf` because the [DNF package manager](https://en.wikipedia.org/wiki/DNF_(software)) 
   is used on this Rocky Linux base training VM):

  - First, install the [Core DNF Plugins](https://github.com/rpm-software-management/dnf-plugins-core/)
    (used to enhance [DNF](wikipedia:DNF_(software)) with additional useful fonctionnalities):

    ```{code} bash
    $ sudo dnf install dnf-plugins-core
    ```

  - Install [EPEL](https://www.redhat.com/en/blog/whats-epel-and-how-do-i-use-it)
    ("Extra Packages for Enterprise Linux"),
    a package repository containing a lot of interesting packages not included in the default repositories:

    ```{code} bash
    $ sudo dnf install epel-release
    $ sudo dnf repolist | grep epel && echo OK
    ```

  - Activate
    [CRB](web-archive:20241010075212/https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/package_manifest/codereadylinuxbuilder-repository#CodeReadyLinuxBuilder-repository)
    (CRB is another useful package repository, mostly oriented towards developpment tools):

    ```{code} bash
    $ sudo crb enable
    $ dnf repolist | grep crb && echo OK
    ```

  - Finally, install the dependencies:

    ```{code} bash
    $ sudo dnf install git wget               # needed to retrieve the EPICS base
    $ sudo dnf install make perl gcc gcc-c++  # needed to build the EPICS base
    ```

```{admonition} Side note
On other Linux distributions, using other package managers,
the {{EPICS}} dependencies might have different names.

- E.g. on Debian based distribution (Debian, Ubuntu, XUbuntu, Lubuntu, Linux Mint, etc):

        sudo apt install git wget           # needed to retrieve the EPICS base
        sudo apt install make perl gcc g++  # needed to build the EPICS base

- E.g. on Arch based distribution (Arch Linux, Manjaro, Parabola, Artix, SystemRescue, etc)

        sudo pacman -S git wget # needed to retrieve the EPICS base
        sudo pacman -S make perl gcc # needed to build the EPICS base

- E.g. on Gentoo based distribution (Gentoo, Calculate, Funtoo, ChromiumOS, CoreOS, etc)

        sudo emerge -a dev-vcs/git net-misc/wget2 # needed to retrieve the EPICS base
        sudo emerge -a dev-build/make dev-lang/perl sys-devel/gcc # needed to build the EPICS base

- E.g. on other RedHat based distribution (RHEL, CentOS, Fedora, Alma Linux, etc):

  - First, install the [Core DNF Plugins](https://github.com/rpm-software-management/dnf-plugins-core/)
    (used to enhance [DNF](wikipedia:DNF_(software)) with additional useful fonctionnalities):

        sudo dnf install dnf-plugins-core

  - Install [EPEL](https://www.redhat.com/en/blog/whats-epel-and-how-do-i-use-it)
    ("Extra Packages for Enterprise Linux"),
    a package repository containing a lot of interesting packages not included in the default repositories:

        sudo dnf install epel-release
        sudo dnf repolist | grep epel && echo OK

  - If using CentOS stream 9+, or RHEL 7+, or Rocky 9+, or Alma 9+, etc:
    Activate
    [CRB](web-archive:20241010075212/https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/package_manifest/codereadylinuxbuilder-repository#CodeReadyLinuxBuilder-repository)
    (CRB is another useful package repository, mostly oriented towards developpment tools):

        sudo crb enable
        dnf repolist | grep crb && echo OK

  - If using CentOS 7, or CentOS stream 8, or Rocky 7 or 8, or Alma 7 or 8, etc:
    Activate
    [PowerTools](https://archive.download.redhat.com/pub/redhat/linux/7.1/kr/doc/RH-DOCS/rhl-rg-en-7.1/ch-powertools.html)
    (PowerTools is another useful package repository, mostly oriented towards developpment tools):

        sudo dnf config-manager --set-enabled powertools
        sudo dnf repolist | grep powertools && echo OK

  - Finally, install the dependencies:

        sudo dnf install git wget               # needed to retrieve the EPICS base
        sudo dnf install make perl gcc gcc-c++  # needed to build the EPICS base

- For other Linux distributions, check <https://repology.org/> and/or <https://pkgs.org/>
  to find equivalent packages names for your distro.
```

* Create a dedicated `epics` group and include yourself inside that group:

  ```{code} bash
  $ sudo groupadd epics
  $ sudo sermod -aG epics $USER
  $ newgrp epics
  ```

* Create a dedicated `epics` installation location (e.g. in `/opt/epics`, but it could be anywhere
  else you want):

  ```{code} bash
  $ sudo mkdir -p /opt/epics
  $ sudo chown root:epics /opt/epics && sudo chmod 775 /opt/epics
  $ sudo chmod g+s /opt/epics # set setgid bit for files/dirs under /opt/epics to inherit group rights
  ```

* Clone the {{EPICS}} base:

  ```{code} bash
  $ cd /opt/epics
  $ sudo git clone --recursive -b 7.0 https://git.launchpad.net/epics-base base-7.0
  $ sudo ln -s base-7.0 base
  ```

* Patch the {{EPICS}} base in order to increase the maximum size of an {{EPICS}}
  {{macro}}. By default, the maximum size of a {{macro}} is
  256 characters long, which is too short for the common {{SSH_Monitor}} use case. Let's increase it to
  65535:

  ```{code} bash
  $ cd /opt/epics/base

  $ sudo sed -i 's/MAC_SIZE 256/MAC_SIZE 65535/' modules/libcom/src/macLib/macLib.h
  $ sudo sed -i 's/MY_BUFFER_SIZE 1024/MY_BUFFER_SIZE 65535/' modules/database/src/ioc/dbStatic/dbLexRoutines.c
  $ sudo sed -i 's/MAX_VAR_FACTOR 50/MAX_VAR_FACTOR 1000/' modules/database/src/ioc/dbtemplate/dbLoadTemplate.y
  $ sudo sed -i 's/MAXLINE BUFSIZ/MAXLINE 65535/' modules/libcom/src/flex/flexdef.h
  $ sudo sed -i 's/YY_TRAILING_MASK 0x2000/YY_TRAILING_MASK 0x20000/' modules/libcom/src/flex/flexdef.h
  $ sudo sed -i 's/YY_TRAILING_HEAD_MASK 0x4000/YY_TRAILING_HEAD_MASK 0x40000/' modules/libcom/src/flex/flexdef.h
  $ sudo sed -i 's/YY_READ_BUF_SIZE 8192/YY_READ_BUF_SIZE 65535/' modules/libcom/src/flex/flex.skel
  $ sudo sed -i 's/YY_READ_BUF_SIZE 8192/YY_READ_BUF_SIZE 65535/' modules/libcom/src/flex/scan.c
  $ sudo sed -i 's/YY_READ_BUF_SIZE 8192/YY_READ_BUF_SIZE 65535/' modules/libcom/src/flex/flex.skel.static
  ```

* Compile the {{EPICS}} base:

  ```{code} bash
  $ cd /opt/epics/base
  $ sudo make clean && echo OK || echo KO
  $ sudo make && echo OK || echo KO
    ...
    OK
  ```

  ```{tip}
  Compiling the {{EPICS}} base can take some time (up to 10+ minutes depending on your hardware).\
  Do not hesitate to take a break or do something else in the meantime.
  ```

* Export some environment variables in your shell configuration file.\
  Depending on your Linux shell (Bash, ZSH, Fish, Dash, etc),
  your shell configuration file can be found at different locations:\
  `$HOME/bashrc`, `$HOME/.zshrc`, `$HOME/.zshenv`, `$HOME/.config/fish/config.fish`,`$HOME/.dashcore/dash.conf`, etc.\
  Since this Rocky Linux training VM is based on Bash, edit the `$HOME/bashrc` file like so:

  ```{code} bash
  $ vi $HOME/.bashrc
      > # .bashrc
      > 
      > # Source global definitions
      > if [ -f /etc/bashrc ]; then
      >         . /etc/bashrc
      > fi
      > 
      > # User specific environment
      > if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]]
      > then
      >     PATH="$HOME/.local/bin:$HOME/bin:$PATH"
      > fi
      > export PATH
      > 
      > # Uncomment the following line if you don't like systemctl's auto-paging feature:
      > # export SYSTEMD_PAGER=
      > 
      > # User specific aliases and functions
      > if [ -d ~/.bashrc.d ]; then
      >         for rc in ~/.bashrc.d/*; do
      >                 if [ -f "$rc" ]; then
      >                         . "$rc"
      >                 fi
      >         done
      > fi
      > 
      > unset rc
    + >
    + > # EPICS
    + > export EPICS_BASE=/opt/epics/base
    + > export EPICS_HOST_ARCH=$(${EPICS_BASE}/startup/EpicsHostArch)
    + > export PATH=${EPICS_BASE}/bin/${EPICS_HOST_ARCH}:${PATH}
    + > export LD_LIBRARY_PATH=${EPICS_BASE}/lib/${EPICS_HOST_ARCH}:${LD_LIBRARY_PATH}

  $ source $HOME/.bashrc
  ```

---

## 2. Install and build SSH Monitor

```{seealso}
This section is inspired from the [install, update, remove how-to guide](../how-to-guides/basics/install_update_remove_how_to.md).\
Feel free to check this documentation for more details.
```

* Clone {{SSH_Monitor}}:

    ```{code} bash
    $ cd /opt/epics
    $ sudo mkdir support
    $ cd support
    $ sudo -E git clone git@gitlab.com:sshmonitor/sshmonitor.git
    ```

* Checkout to the latest tag/release:

    ```{code} bash
    $ cd /opt/epics/support/sshmonitor
    $ latestTag=$(git describe --tags `git rev-list --tags --max-count=1`)
    $ sudo git checkout $latestTag # checkout to the latest release
    ```

* Configure your {{EPICS}} base location in {{SSH_Monitor}}:

    ```{code} bash
    $ sudo vi /opt/epics/support/sshmonitor/configure/RELEASE # configure the path to your EPICS base location
        > ...
        >
        > # EPICS_BASE should appear last so earlier modules can override stuff:
      ~ > EPICS_BASE = /opt/epics/base
        >
        > # Set RULES here if you want to use build rules from somewhere
        > # other than EPICS_BASE:
        > #RULES = $(MODULES)/build-rules
        >
        > # These lines allow developers to override these RELEASE settings
        > # without having to modify this file directly.
        > -include $(TOP)/../RELEASE.local
        > -include $(TOP)/../RELEASE.$(EPICS_HOST_ARCH).local
        > -include $(TOP)/configure/RELEASE.local
    ```

* Build {{SSH_Monitor}}:

    ```{code} bash
    $ cd /opt/epics/support/sshmonitor
    $ sudo make && echo OK || echo KO
        ...
        OK
    ```

---

## 3. Create your Top

```{seealso}
This section is inspired from the [install, update, remove how-to guide](../how-to-guides/basics/install_update_remove_how_to.md).\
Feel free to check this documentation for more details.
```

* Create the {{Top}} that will use {{SSH_Monitor}} in order to monitor your own
  machine:

    ```{code} bash
    $ cd /opt/epics
    $ mkdir tops
    $ cd tops
    $ mkdir selfMonitorTop
    $ cd selfMonitorTop
    $ makeBaseApp.pl -a linux-x86_64 -t ioc selfMonitor
    $ makeBaseApp.pl -a linux-x86_64 -i -p selfMonitor SelfMonitor
    ```

* Import {{SSH_Monitor}} into your {{Top}}, and select the correct {{EPICS}} base:

    ```{code} bash
    $ cd /opt/epics/tops/selfMonitorTop
    ```

    ```{code} bash
    $ vi configure/RELEASE
        > ...
        >
        > # Variables and paths to dependent modules:
        > #MODULES = /path/to/modules
        > #MYMODULE = $(MODULES)/my-module
        >
        > # If using the sequencer, point SNCSEQ at its top directory:
        > #SNCSEQ = $(MODULES)/seq-ver
      + >
      + > SSHMONITOR = /opt/epics/support/sshmonitor
        >
        > # EPICS_BASE should appear last so earlier modules can override stuff:
      ~ > EPICS_BASE = /opt/epics/base
        >
        > # Set RULES here if you want to use build rules from somewhere
        > # other than EPICS_BASE:
        > #RULES = $(MODULES)/build-rules
        >
        > # These lines allow developers to override these RELEASE settings
        > # without having to modify this file directly.
        > -include $(TOP)/../RELEASE.local
        > -include $(TOP)/../RELEASE.$(EPICS_HOST_ARCH).local
        > -include $(TOP)/configure/RELEASE.local
    ```

    ```{code} bash
    $ vi selfMonitorApp/Db/Makefile
        > TOP=../..
        > include $(TOP)/configure/CONFIG
        > #----------------------------------------
        > #  ADD MACRO DEFINITIONS AFTER THIS LINE
        >
        > #----------------------------------------------------
        > # Create and install (or just install) into <top>/db
        > # databases, templates, substitutions like this
        > #DB += xxx.db
        >
      + > DB_INSTALLS += $(SSHMONITOR)/db/ssh_monitor_core.template
      + > DB_INSTALLS += $(wildcard $(SSHMONITOR)/db/*.substitutions)
        >
        > #----------------------------------------------------
        > # If <anyname>.db template is not named <anyname>*.template add
        > # <anyname>_template = <templatename>
        >
        > include $(TOP)/configure/RULES
        > #----------------------------------------
        > #  ADD RULES AFTER THIS LINE
    ```

    ```{code} bash
    $ vi selfMonitorApp/src/Makefile
        > TOP=../..
        >
        > include $(TOP)/configure/CONFIG
        > #----------------------------------------
        > #  ADD MACRO DEFINITIONS AFTER THIS LINE
        > #=============================
        >
        > #=============================
        > # Build the IOC application
        >
        > PROD_IOC = selfMonitor
        > # selfMonitor.dbd will be created and installed
        > DBD += selfMonitor.dbd
        >
        > # selfMonitor.dbd will be made up from these files:
        > selfMonitor_DBD += base.dbd
      + > selfMonitor_DBD += system.dbd
        >
        > # Include dbd files from all support applications:
        > #selfMonitor_DBD += xxx.dbd
      + > selfMonitor_DBD += sshmonitor.dbd
        >
        > # Add all the support libraries needed by this IOC
        > #selfMonitor_LIBS += xxx
      + > selfMonitor_LIBS += sshmonitorSupport
        >
        > # selfMonitor_registerRecordDeviceDriver.cpp derives from selfMonitor.dbd
        > selfMonitor_SRCS += selfMonitor_registerRecordDeviceDriver.cpp
        >
        > # Build the main IOC entry point on workstation OSs.
        > selfMonitor_SRCS_DEFAULT += selfMonitorMain.cpp
        > selfMonitor_SRCS_vxWorks += -nil-
        >
        > # Add support from base/src/vxWorks if needed
        > #selfMonitor_OBJS_vxWorks += $(EPICS_BASE_BIN)/vxComLibrary
        >
        > # Finally link to the EPICS Base libraries
        > selfMonitor_LIBS += $(EPICS_BASE_IOC_LIBS)
        >
        > #===========================
        >
        > include $(TOP)/configure/RULES
        > #----------------------------------------
        > #  ADD RULES AFTER THIS LINE
    ```

* Build your {{Top}}:

    ```{code} bash
    $ cd /opt/epics/tops/selfMonitorTop
    $ make && echo OK || echo KO
        ...
        OK
    ```

---

## 4. Configure the Firewall

```{seealso}
This section is inspired from the [firewall how-to guide](../how-to-guides/basics/firewall_how_to.md).\
Feel free to check this documentation for more details.
```

* Make sure that port 22 (default {{SSH}} port) is open:

```{code} bash
$ sudo firewall-cmd --add-port=22/tcp --permanent
$ sudo firewall-cmd --reload
```

* Make sure port `5064` and port `5065` (default {{CA}} ports) are open for both TCP and UDP:

```{code} bash
$ sudo firewall-cmd --add-port=5064/tcp --permanent
$ sudo firewall-cmd --add-port=5064/udp --permanent
$ sudo firewall-cmd --add-port=5065/tcp --permanent
$ sudo firewall-cmd --add-port=5065/udp --permanent
$ sudo firewall-cmd --reload
```

```{important}
If -- after completing this tutorial -- you intend to run more than one {{IOC_program}} on this VM,\
then *you have to* read
the [TCP ports limitation explanations](../explanations/tcp_ports_limitation_explanations.md)!\
This is important because in this case,
you will have to open more ports\
(usually an additional TCP port and UDP port per {{IOC_program}})\
and this will have an impact on the {{IOC}}s clients configuration.\
To open additional ports: just repeat the previous step with the desired ports numbers.
```

---

## 5. Configure SSH

```{seealso}
This section is inspired from the [SSH how-to guide](../how-to-guides/basics/ssh_how_to.md).\
Feel free to check this documentation for more details.
```

* Check the status of the `sshd` service, and make sure everything is fine:
    ```{code} bash
    $ systemctl status sshd
    ```

* Create a dedicated {{SSH_Monitor}} user
  (the user you will connect to, in order to monitor your own VM):

    ```{code} bash
    $ sudo useradd --create-home sshmonitor-target-user
    $ sudo usermod -aG epics sshmonitor-target-user
    $ sudo passwd sshmonitor-target-user
    ```

* Generate an {{SSH}} key pair, deploy it and test it:

    ```{code} bash
    $ ssh-keygen -o -t ed25519 -f "$HOME/.ssh/ssh_monitor_ed25519_key" -N ""
    $ eval `ssh-agent -s`
    $ ssh-add $HOME/.ssh/ssh_monitor_ed25519_key
    $ ssh-add -l # make sure your key appears here
    $ ssh-copy-id -i $HOME/.ssh/ssh_monitor_ed25519_key.pub sshmonitor-target-user@localhost
    $ echo $?
      0
    $ ssh sshmonitor-target-user@localhost "echo 'test OK'"
    ```

    ```{seealso}
    Check the [SSH how-to guide](../how-to-guides/basics/ssh_how_to.md)
    troubleshooting notes for more details if you run into any difficulties.
    ```

---

## 6. Configure your `.cmd` and `.substitutions`

```{seealso}
This section is inspired from the [`.cmd` and `.substitutions` how-to guide](../how-to-guides/basics/cmd_and_sub_how_to.md).\
Feel free to check this documentation for more details.
```

* Edit your `/opt/epics/tops/selfMonitorTop/iocBoot/iocSelfMonitor/st.cmd`,\
  in order for this file to have to following content:

    ```{code} bash
    #!../../bin/linux-x86_64/selfMonitor
    
    < envPaths
    
    epicsEnvSet("IOCSH_PS1", "Self Monitor> ")
    epicsEnvSet("EPICS_CAS_SERVER_PORT", "5064")
    
    epicsEnvSet("PREFIX", "tests:")
    epicsEnvSet("SSH_OPTS_AND_ARGS", "ssh sshmonitor-target-user@localhost -i /home/notroot/.ssh/ssh_monitor_ed25519_key -o BatchMode=yes -o PasswordAuthentication=no -o ConnectionAttempts=3 -x -o ControlMaster=auto -o ControlPath=~/.ssh/%C -o ControlPersist=120s")
    
    dbLoadDatabase("${EPNIX_SSHMONITOR}/dbd/menuScan.dbd")
    dbLoadDatabase("${TOP}/dbd/selfMonitor.dbd")
    
    selfMonitor_registerRecordDeviceDriver(pdbbase)
    
    var(dbRecordsOnceOnly, 1)

    #dbLoadTemplate("${TOP}/db/example_monitoring.substitutions", "PREFIX_MACRO=${PREFIX}, SSH_OPTS_AND_ARGS_MACRO=${SSH_OPTS_AND_ARGS}")
    dbLoadTemplate("${TOP}/db/processors_monitoring.substitutions", "PREFIX_MACRO=${PREFIX}, SSH_OPTS_AND_ARGS_MACRO=${SSH_OPTS_AND_ARGS}")
    dbLoadTemplate("${TOP}/db/memory_monitoring.substitutions", "PREFIX_MACRO=${PREFIX}, SSH_OPTS_AND_ARGS_MACRO=${SSH_OPTS_AND_ARGS}")
    dbLoadTemplate("${TOP}/db/partitions_monitoring.substitutions", "PREFIX_MACRO=${PREFIX}, SSH_OPTS_AND_ARGS_MACRO=${SSH_OPTS_AND_ARGS}")
    dbLoadTemplate("${TOP}/db/connection_monitoring.substitutions", "PREFIX_MACRO=${PREFIX}, SSH_OPTS_AND_ARGS_MACRO=${SSH_OPTS_AND_ARGS}")
    
    iocInit()
    
    ```

    For more details about this {{cmd}} configuration, 
    see the [`.cmd` and `.substitutions` how-to guide](../how-to-guides/basics/cmd_and_sub_how_to.md).

* Don't forget to make your {{cmd}} executable:

    ```{code} bash
    $ chmod +x /opt/epics/tops/selfMonitorTop/iocBoot/iocSelfMonitor/st.cmd
    ```

```{important}
In a few words, how does it work internally?

Let's look at the [processors_monitoring.substitutions file](sshmonitor-file-gitlab-url:sshmonitorApp/db/processors_monitoring.substitutions).\
When we take a closer look e.g. to the `PV_NAME_SCALAR_1` {{macro}} and the `SCALAR_1_INSTRUCTION` {{macro}},\
we can see that the {{SSH_shell_instruction}} used to retrieve the number of CPUs is `lscpu | grep '^CPU(s):' | awk '{print $2}'`.

This instruction is passed to the {{SSH_Monitor}} core {{template}} file:
[ssh_monitor_core.template](sshmonitor-file-gitlab-url:sshmonitorApp/db/ssh_monitor_core.template),\
where it is processed by an [aSubRecord](docs-epics-controls:aSubRecord.html).

Note that the {{SSH_opts_and_args}} sourroundig the {{SSH_shell_instruction}} are specified with the `SSH_OPTS_AND_ARGS` {{macro}}.\
The combination of the {{SSH_opts_and_args}} and the {{SSH_shell_instruction}} gives us the full {{SSH_command}}.

Every {{SSH_command}} is defined like so.
```

---

## 7. Run your IOC program

* Run your monitoring {{Top}}:

    ```{code} bash
    $ cd /opt/epics/tops/selfMonitorTop/iocBoot/iocSelfMonitor
    $ ./st.cmd
    ```

```{admonition} Troubleshooting
When running your {{IOC_program}},
if you encounter errors messages similar to the ones below:

    Host key verification failed.
    SSH Monitor ERROR: the full SSH command including the shell instruction (ssh sshmonitor-target-user@localhost -i /home/notroot/.ssh/ssh_monitor_ed25519_key -o BatchMode=yes -o PasswordAuthentication=no -o ConnectionAttempts=3 -x -o ControlMaster=auto -o ControlPath=~/.ssh/%C -o ControlPersist=120s "free -m | grep 'Swap:' | awk '{print \$2}'" >| /root/.cache/sshmonitor/root_tests:mem:ASubCore) returns an error when executed!
    
    Host key verification failed.
    SSH Monitor ERROR: the full SSH command including the shell instruction (ssh sshmonitor-target-user@localhost -i /home/notroot/.ssh/ssh_monitor_ed25519_key -o BatchMode=yes -o PasswordAuthentication=no -o ConnectionAttempts=3 -x -o ControlMaster=auto -o ControlPath=~/.ssh/%C -o ControlPersist=120s "free -m | grep 'Swap:' | awk '{print \$3}'" >| /root/.cache/sshmonitor/root_tests:mem:ASubCore) returns an error when executed!
    
    Host key verification failed.
    SSH Monitor ERROR: the full SSH command including the shell instruction (ssh sshmonitor-target-user@localhost -i /home/notroot/.ssh/ssh_monitor_ed25519_key -o BatchMode=yes -o PasswordAuthentication=no -o ConnectionAttempts=3 -x -o ControlMaster=auto -o ControlPath=~/.ssh/%C -o ControlPersist=120s "free -m | grep 'Swap:' | awk '{print \$4}'" >| /root/.cache/sshmonitor/root_tests:mem:ASubCore) returns an error when executed!

then you should try to {{SSH}} to `sshmonitor-target-user@localhost` in order to add it to the known hosts:

    $ ssh sshmonitor-target-user@localhost

    The authenticity of host 'localhost (::1)' can't be established.
    ED25519 key fingerprint is SHA256:<...>
    This key is not known by any other names.
    Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
    Warning: Permanently added 'localhost' (ED25519) to the list of known hosts.

Another possible explanation is that the `SSH_OPTS_AND_ARGS` {{macro}} -- specified in the {{cmd}} file -- contains e.g. a syntax error.
```

```{admonition} Troubleshooting
When running your {{IOC_program}},
if you encounter the following message: 

    muxclient: master hello exchange failed

then you can ignore it.

This might be due to the fact that your are forwarding your ssh-agent to the VM with more than one SSH key.

See the first note of section 5.1 of the following link:\
<web-archive:20241007123652/https://docs.releng.linuxfoundation.org/_/downloads/common-packer/en/latest/pdf/>
```

```{seealso}
For more troubleshooting, see [the troubleshooting guide](../troubleshooting_guide.md).
```

## 8. Check the GUI

* On the computer used to launch VirtualBox (not on the VM),\
  please retrieve {{SSH_Monitor}} default GUIs: <sshmonitor-directory-gitlab-url:gui>.

  * E.g. on Linux:
  
    ```{code} bash
    $ cd /tmp
    $ git archive --remote git@gitlab.com:sshmonitor/sshmonitor.git master | tar -xvf - gui
    ```

* Run [Phoebus](https://controlssoftware.sns.ornl.gov/css_phoebus/)
  or [CSS](https://controlssoftware.sns.ornl.gov/css_rcp/products.html)
  in order to display the GUI.

  * If using Phoebus, check if its associated `settings.ini` file\
    includes either `127.0.0.1` or `localhost` in the `org.phoebus.pv.ca/addr_list` variable.

  * If using CSS, check if the "Address List" property, located in\
    {guilabel}`Edit` → {guilabel}`Preference` → {guilabel}`CSS Core` → {guilabel}`Data Sources` → {guilabel}`Channel Access` → {guilabel}`Address List`,\
    includes either `127.0.0.1` or `localhost`.

* Execute `main_display.bob` (if using Phoebus) or run `main_display.opi` (if using Phoebus).

* Note that some views might be full of disconnected {{PV}}s.\
  E.g. the `Archiver Appliance monitoring` view.\
  Indeed, its associated `archiver_appliance_monitoring.substitutions`\
  has not been loaded in the {{cmd}},\
  because there is no archiving system running on the training VM.

---

## 9. Success

🎉 Congratulations! Your are now monitoring your own VM through {{SSH_Monitor}}!

---

## 10. Exercise: Monitor things more securely

Like said in the introduction:\
the [Security configuration](../how-to-guides/basics/security_how_to.md) step has been ignored\
only to "ease" the overall process.

It's time to do things the right way,
by following the [Security how-to guide](../how-to-guides/basics/security_how_to.md).

---

## 11. Exercise: Monitor your own things

In order to specify your own shell instruction's
and retrieve the exact data you want, the way you want,\
follow the
[Custom shell instructions how-to guide](../how-to-guides/basics/custom_shell_instructions_how_to.md).

---

## ∞. Monitor even more things

A lot more can be done with {{SSH_Monitor}},\
do not hesite to explore other [how-to guides](../how-to-guides/index.md)
and [explanations](../explanations/index.md) to find out!