..
 #
 # Critical Application Monitoring (CAM)
 #
 # SPDX-FileCopyrightText: <text>Copyright 2023-2024 Arm Limited
 # and/or its affiliates <open-source-office@arm.com></text>
 #
 # SPDX-License-Identifier: BSD-3-Clause
 #

.. _getting_started:

###############
Getting Started
###############

This section of documentation describes how to download, build and execute CAM
project on Linux.

.. note::

  All command examples on this page from the HTML document format can be copied
  by clicking the copy button.

****************************
Build Host Environment Setup
****************************

System Requirements
===================

* AArch64 or x86_64 host to build the software
* Ubuntu Desktop or Server 20.04 Linux distribution

Install Dependencies
====================

The following dependencies are required to build CAM:

* CMake (3.16 or later)
* GCC (9.4 or later)
* Python (3.8.10 or later)
* Python-venv (3.8.10 or later)
* Pip (20.0 or later)
* pkg-config (0.29.1 or later)
* CUnit (2.1.3 or later)
* Doxygen (1.8.17 or later): Generate API documentation from source code.
* Sphinx (6.1.3 or later): Build the project documentation.
* Git version control system.
* Python virtual environment (3.8.2 or later).

Install essential packages required for the build host:

.. code-block:: console

  sudo apt-get install -y --no-install-recommends \
       git \
       build-essential \
       cmake \
       doxygen \
       pkg-config \
       python3 \
       python3-venv \
       python3-pip \
       libcunit1-dev

.. note::

  CAM project has Dockerfiles to build docker images which will contain all
  dependencies needed by CAM. Refer to :cam-repo:`Dockerfiles` for more
  information on the Dockerfiles.

********
Download
********

Download the CAM repository using Git, via:

.. code-block:: console
  :substitutions:

  mkdir ~/cam
  git clone |cam remote| --branch |cam refspec| ~/cam/critical-app-monitoring

.. _getting_started_build_and_install:

*****
Build
*****

Create a Python virtual environment then install the Python dependencies:

.. code-block:: console

  cd ~/cam/critical-app-monitoring
  python3 -m venv ~/cam/venv
  source ~/cam/venv/bin/activate
  pip install -r requirements.txt

Create a build directory and run cmake:

.. code-block:: console

  mkdir ~/cam/build
  cd ~/cam/build
  cmake ~/cam/critical-app-monitoring

Build and install cam-app-example, cam-service and documentation:

.. code-block:: console

  make
  cmake --install . --prefix ~/cam/cam-packages

Install cam-tool:

.. code-block:: console

  cd ~/cam/critical-app-monitoring
  pip install cam-tool/ --prefix ~/cam/cam-packages

.. note::

  When running the command above, a warning message might appear
  ``"WARNING: The scripts futurize and pasteurize are installed in
  '/home/<user>/cam/cam-packages/bin' which is not on PATH."`` where ``<user>``
  might vary depending on the user system.

The following components get installed as a result of the above steps:

~/cam/cam-packages/bin
  * **cam-app-example**: An application example that introduces how to integrate
    the project.
  * **cam-service**: The service executable used to monitor critical
    applications.
  * **cam-tool**: Python based **cam-tool** binary.
~/cam/cam-packages/share/doc/CAM/
  * Includes all CAM documentations of the project, it can be rendered and
    explored pointing the browser to
    ``file:///home/<user>/cam/cam-packages/share/doc/CAM/index.html`` where
    ``<user>`` might vary depending on the user system.

.. _getting_started_setup_terminal_environment:

**************************
Setup terminal environment
**************************

The terminal environment can be setup by running the following commands:

.. code-block:: console

  source ~/cam/venv/bin/activate
  python_path=$(find ~/cam/cam-packages -maxdepth 3 -name "site-packages")
  export PYTHONPATH=${PYTHONPATH}:${python_path}
  export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:~/cam/cam-packages/lib/

***
Run
***

This demo requires two Linux terminals: ``Terminal 1`` for running **cam-tool**,
and **cam-app-example** and ``Terminal 2`` for running **cam-service**. Make
sure to follow the steps listed in the
:ref:`Build <getting_started_build_and_install>` section, afterwards make sure
to follow the step listed in
:ref:`Setup terminal environment <getting_started_setup_terminal_environment>`
section for both ``Terminal 1`` and ``Terminal 2``.

One of the prerequisites of CAM demo is the creation and deployment of CSD files
to the system where **cam-service** runs.

1. From ``Terminal 1``, start **cam-app-example** in calibration mode.

   .. code-block:: console

     mkdir ~/cam/cam-run
     cd ~/cam/cam-run
     ~/cam/cam-packages/bin/cam-app-example \
       --processing-period 3000 \
       --processing-count 2 \
       --stream-count 4 \
       --enable-calibration-mode \
       --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
       --service-address 127.0.0.1 \
       --service-port 21604

   This mode generates one CSEL file for each stream. The CSEL file contains the
   stream metadata and event timestamps for further analysis. The output
   should look like as below:

   .. code-block:: console

     Cam application configuration:
         Service IP address: 127.0.0.1
         Service port: 21604
         UUID base: 94085ddc-bc10-11ed-9a44-7ef9696e
         Stream count: 4
         Processing period (ms): 3000
         Processing count: 2
         Multiple connection support: false
         Calibration mode support: true
         Calibration directory: ./[uuid].csel
         Fault injection support: false
         Event(s) interval time (ms): 0
     Using libcam v0.1
     Starting activity...
     Starting activity...
     Starting activity...
     Starting activity...
         Stream 0 sends event 0
         Stream 1 sends event 0
         Stream 2 sends event 0
         Stream 3 sends event 0
         ...

   List the files generated:

   .. code-block:: console

     ls -1 *.csel

   The CSEL files can be shown as below:

   .. code-block:: console

     94085ddc-bc10-11ed-9a44-7ef9696e0000.csel
     94085ddc-bc10-11ed-9a44-7ef9696e0001.csel
     94085ddc-bc10-11ed-9a44-7ef9696e0002.csel
     94085ddc-bc10-11ed-9a44-7ef9696e0003.csel

2. From ``Terminal 1``, run **cam-tool** to analyze the CSEL files and convert
   them to CSC files.

   .. code-block:: console

     ~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0000.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml
     ~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0001.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0001.csc.yml
     ~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0002.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0002.csc.yml
     ~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0003.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0003.csc.yml

   The analysis result is reported by **cam-tool** as below:

   .. code-block:: shell

     CAM event log analyze report:
     Input event log file:                   94085ddc-bc10-11ed-9a44-7ef9696e0000.csel
     Output configuration file:              94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml
     Stream UUID:                            94085ddc-bc10-11ed-9a44-7ef9696e0000
     Stream name:                            CAM STREAM  0
     Timeout between init and start:         300000
     Timeout between start and event:        450000
     Application running times:              1
     Processing count in each run:           [2]

     Event ID        timeout
     0               3500103
     ...

   .. note::

     The CSC files contain human-readable settings corresponding to the stream
     metadata and event timestamps of the streams. Users can modify this
     configuration, for example to fine tune timeout values depending on the
     system capabilities.

   List the files generated:

   .. code-block:: console

     ls -1 *.csc.yml

   The CSC files can be shown as below:

   .. code-block:: console

     94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml
     94085ddc-bc10-11ed-9a44-7ef9696e0001.csc.yml
     94085ddc-bc10-11ed-9a44-7ef9696e0002.csc.yml
     94085ddc-bc10-11ed-9a44-7ef9696e0003.csc.yml

   Run **cam-tool** along with ``pack`` option for each of the CSC files to
   generate the corresponding CSD files:

   .. code-block:: console

     ~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0000.csd
     ~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0001.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0001.csd
     ~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0002.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0002.csd
     ~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0003.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0003.csd

   List the files generated:

   .. code-block:: console

     ls -1 *.csd

   The CSD files can be shown as below:

   .. code-block:: console

     94085ddc-bc10-11ed-9a44-7ef9696e0000.csd
     94085ddc-bc10-11ed-9a44-7ef9696e0001.csd
     94085ddc-bc10-11ed-9a44-7ef9696e0002.csd
     94085ddc-bc10-11ed-9a44-7ef9696e0003.csd

3. From ``Terminal 2``, run **cam-service** to monitor the multiple streams
that will be generated by **cam-app-example**.

   .. code-block:: console

     cd ~/cam/cam-run/
     mkdir -p ~/cam/cam-packages/usr/share/cam-data/
     export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:~/cam/cam-packages/lib/
     ~/cam/cam-packages/bin/cam-service --config-dir ~/cam/cam-packages/usr/share/cam-data/ --log-level info --address 127.0.0.1 --port 21604

   Users have the flexibility to customize the socket on which **cam-service**
   listens by modifying the IP address and port using command-line parameters.
   Once critical applications connect to **cam-service** and initiate event
   streams, **cam-service** can receive and process stream events through this
   socket. The following messages are expected from the terminal:

   .. code-block:: shell

     Cam service configuration:
         IP address: 127.0.0.1
         port: 21604
         log level: info
         configuration directory: ~/cam/cam-packages/usr/share/cam-data/
         fault action: none

4. From ``Terminal 1``, run **cam-tool** along with ``deploy`` option to deploy
   the CSD files into the system where **cam-service** is running.

   .. code-block:: console

     ~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0000.csd --address 127.0.0.1 --port 21604
     ~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0001.csd --address 127.0.0.1 --port 21604
     ~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0002.csd --address 127.0.0.1 --port 21604
     ~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0003.csd --address 127.0.0.1 --port 21604

   The output on the ``Terminal 2`` should look like as below, the connection
   number might change:

   .. code-block:: console

     Connection 1 is created.
     Deploy Message

     Connection 1 is closed.
     ...

   List the CSD files deployed:

   .. code-block:: console

     ls -1 ~/cam/cam-packages/usr/share/cam-data/*.csd

   The CSD files can be shown as below, the output might change because the
   ``~`` is expanded in different ways on different systems, usually to
   ``/home/<user>``:

   .. code-block:: console

     ~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0000.csd
     ~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0001.csd
     ~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0002.csd
     ~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0003.csd

5. From ``Terminal 1``, start **cam-app-example** to run the Critical
   Application with four streams. Each stream sends an event message 10 times
   with a period of 3000ms.

   .. code-block:: console

     ~/cam/cam-packages/bin/cam-app-example \
       --processing-period 3000 \
       --processing-count 10 \
       --stream-count 4 \
       --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
       --service-address 127.0.0.1 \
       --service-port 21604

   The following messages are expected on ``Terminal 2``:

   .. code-block:: console

     Cam application configuration:
         Service IP address: 127.0.0.1
         Service port: 21604
         UUID base: 94085ddc-bc10-11ed-9a44-7ef9696e
         Stream count: 4
         Processing period (ms): 3000
         Processing count: 10
         Multiple connection support: false
         Calibration mode support: false
         Fault injection support: false
         Event(s) interval time (ms): 0
     Using libcam v0.1
     Starting activity...
     Starting activity...
     Starting activity...
     Starting activity...

   And the log of event messages send are shown repeatedly:

   .. code-block:: console

     Stream 0 sends event 0
     Stream 1 sends event 0
     Stream 2 sends event 0
     Stream 3 sends event 0
     Stream 0 sends event 0
     Stream 1 sends event 0
     Stream 2 sends event 0
     Stream 3 sends event 0
     ...

   As observed from the ``Terminal 2``, **cam-service** loads four CSD files for
   monitoring. In the following log, the stream messages are received and
   processed by it:

   .. code-block:: console

     Connection 4 is created.
     Init Message
     Stream 94085ddc-bc10-11ed-9a44-7ef9696e0001 configuration is loaded.
     Init Message
     Stream 94085ddc-bc10-11ed-9a44-7ef9696e0000 configuration is loaded.
     Init Message
     Stream 94085ddc-bc10-11ed-9a44-7ef9696e0002 configuration is loaded.
     Init Message
     Stream 94085ddc-bc10-11ed-9a44-7ef9696e0003 configuration is loaded.
     Start Message
     Start Message
     Start Message
     Start Message
     Event Message
     Event Message
     Event Message
     Event Message
     Event Message
     # Repeated event messages
     ...

6. CAM provides a mechanism to generate a fault from within **cam-app-example**
   that can be detected by **cam-service**. From ``Terminal 1``, run
   **cam-app-example** with fault injection set to event stream 0:

   .. code-block:: console

     ~/cam/cam-packages/bin/cam-app-example \
       --processing-period 3000 \
       --processing-count 10 \
       --stream-count 4 \
       --enable-fault-injection \
       --fault-injection-stream 0 \
       --fault-injection-time 1000 \
       --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
       --service-address 127.0.0.1 \
       --service-port 21604

   The fault happens 1000ms after stream initialization. At that time
   **cam-service** should detect a stream temporal error with the following
   output on ``Terminal 2``.

   .. code-block:: console

     #Repeated event messages
     ...
     ERROR: Stream temporal error:
     ERROR: stream_name: CAM STREAM 0
     ERROR: stream_uuid: 94085ddc-bc10-11ed-9a44-7ef9696e0000
     ERROR: event_id: 0
     ERROR: time_received: 0
     ERROR: time_expected: 1701066141314201
     ...

   The following diagram describes the fault injection option:

   .. image:: images/fault_injection.svg
     :align: center

   After fault is injected, **cam-app-example** stops performing new processing
   activity. The interval time from stream initialization to the first activity
   is 500ms.

.. note::

  In addition to sending only one event during each activity,
  **cam-app-example** supports sending a specific number of events within one
  activity.

  .. code-block:: console

    ~/cam/cam-packages/bin/cam-app-example \
      --processing-period 3000 \
      --processing-count 10 \
      --stream-count 4 \
      --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
      --service-address 127.0.0.1 \
      --service-port 21604 \
      --event-interval "<t0,t1,...tn>"

  With the command line above, cam-app-example sends n events in each processing
  activity. The interval time between events is defined by t\ :sub:`0` -
  t\ :sub:`n`. The following diagram shows the definition of the event interval
  option.

  .. image:: images/event_interval.svg
    :align: center

  t\ :sub:`n`: The list of event intervals(in ms) in each processing activity.

  t\ :sub:`0` is the interval time from activity start to event 0.

8. To stop **cam-service** running on ``Terminal 2``, type ``Ctrl-C``.

.. _getting_started_cam_integration_with_kronos:

***************************
CAM Integration with Kronos
***************************

A complete integration of CAM can be found in Kronos Reference Software Stack.
This integration demonstrates the feasibility of monitoring Primary Compute
applications by **cam-service** running on Safety Island.
Kronos makes use of CAM CMake option **CAM_TARGET=<Linux> | <Zephyr>** to build
**cam-service** for Zephyr.

Refer to `Kronos Reference Software Stack CAM Documentation`_ for more
information on the integration.