Getting Started

This section of documentation describes how to download, build and execute CAM project on Linux.

Note

All command examples on this page from the HTML document format can be copied by clicking the copy button.

Build Host Environment Setup

System Requirements

  • AArch64 or x86_64 host to build the software

  • Ubuntu Desktop or Server 20.04 Linux distribution

Install Dependencies

The following dependencies are required to build CAM:

  • CMake (3.16 or later)

  • GCC (9.4 or later)

  • Python (3.8.10 or later)

  • Python-venv (3.8.10 or later)

  • Pip (20.0 or later)

  • pkg-config (0.29.1 or later)

  • CUnit (2.1.3 or later)

  • Doxygen (1.8.17 or later): Generate API documentation from source code.

  • Sphinx (6.1.3 or later): Build the project documentation.

  • Git version control system.

  • Python virtual environment (3.8.2 or later).

  • [optional] tcpdump (4.9.3 or later)

Install essential packages required for the build host:

sudo apt-get install -y --no-install-recommends \
     git \
     build-essential \
     cmake \
     doxygen \
     pkg-config \
     python3 \
     python3-venv \
     python3-pip \
     libcunit1-dev

Note

CAM project has Dockerfiles to build docker images which will contain all dependencies needed by CAM. Refer to Dockerfiles for more information on the Dockerfiles.

Download

Download the CAM repository using Git, via:

mkdir ~/cam
git clone https://git.gitlab.arm.com/automotive-and-industrial/safety-island/critical-app-monitoring.git --branch v1.1 ~/cam/critical-app-monitoring

Build

Create a Python virtual environment then install the Python dependencies:

cd ~/cam/critical-app-monitoring
python3 -m venv ~/cam/venv
source ~/cam/venv/bin/activate
pip install -r requirements.txt

Create a build directory and run cmake:

mkdir ~/cam/build
cd ~/cam/build
cmake ~/cam/critical-app-monitoring

Build and install cam-app-example, cam-service and documentation:

make
cmake --install . --prefix ~/cam/cam-packages

Install cam-tool:

cd ~/cam/critical-app-monitoring
pip install cam-tool/ --prefix ~/cam/cam-packages

Note

When running the command above, a warning message might appear "WARNING: The scripts futurize and pasteurize are installed in '/home/<user>/cam/cam-packages/bin' which is not on PATH." where <user> might vary depending on the user system.

The following components get installed as a result of the above steps:

~/cam/cam-packages/bin

  • cam-app-example: An application example that introduces how to integrate the project.

  • cam-service: The service executable used to monitor critical applications.

  • cam-tool: Python based cam-tool binary.

~/cam/cam-packages/share/doc/CAM/

  • Includes all CAM documentations of the project, it can be rendered and explored pointing the browser to file:///home/<user>/cam/cam-packages/share/doc/CAM/index.html where <user> might vary depending on the user system.

Setup terminal environment

The terminal environment can be setup by running the following commands:

source ~/cam/venv/bin/activate
python_path=$(find ~/cam/cam-packages -maxdepth 3 -name "site-packages")
export PYTHONPATH=${PYTHONPATH}:${python_path}
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:~/cam/cam-packages/lib/

Run

This demo requires two Linux terminals: Terminal 1 for running cam-tool, and cam-app-example and Terminal 2 for running cam-service. Make sure to follow the steps listed in the Build section, afterwards make sure to follow the step listed in Setup terminal environment section for both Terminal 1 and Terminal 2.

One of the prerequisites of CAM demo is the creation and deployment of CSD files to the system where cam-service runs.

  1. From Terminal 1, start cam-app-example in calibration mode.

    mkdir ~/cam/cam-run
cd ~/cam/cam-run
~/cam/cam-packages/bin/cam-app-example \
  --processing-period 3000 \
  --processing-count 2 \
  --stream-count 4 \
  --enable-calibration-mode \
  --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
  --service-address 127.0.0.1 \
  --service-port 21604

    This mode generates one CSEL file for each stream. The CSEL file contains the stream metadata and event timestamps for further analysis. The output should look like as below:

    Cam application configuration:
    Service IP address: 127.0.0.1
    Service port: 21604
    UUID base: 94085ddc-bc10-11ed-9a44-7ef9696e
    Stream count: 4
    Processing period (ms): 3000
    Processing count: 2
    Multiple connection support: false
    Calibration mode support: true
    Calibration directory: ./[uuid].csel
    Fault injection support: false
    Event(s) interval time (ms): 0
Using libcam v0.1
Starting activity...
Starting activity...
Starting activity...
Starting activity...
    Stream 0 sends event 0
    Stream 1 sends event 0
    Stream 2 sends event 0
    Stream 3 sends event 0
    ...

    List the files generated:

    ls -1 *.csel

    The CSEL files can be shown as below:

    94085ddc-bc10-11ed-9a44-7ef9696e0000.csel
94085ddc-bc10-11ed-9a44-7ef9696e0001.csel
94085ddc-bc10-11ed-9a44-7ef9696e0002.csel
94085ddc-bc10-11ed-9a44-7ef9696e0003.csel

  2. From Terminal 1, run cam-tool to analyze the CSEL files and convert them to CSC files.

    ~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0000.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml
~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0001.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0001.csc.yml
~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0002.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0002.csc.yml
~/cam/cam-packages/bin/cam-tool analyze --margin 500000 --input 94085ddc-bc10-11ed-9a44-7ef9696e0003.csel --output 94085ddc-bc10-11ed-9a44-7ef9696e0003.csc.yml

    The analysis result is reported by cam-tool as below:

    CAM event log analyze report:
Input event log file:                   94085ddc-bc10-11ed-9a44-7ef9696e0000.csel
Output configuration file:              94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml
Stream UUID:                            94085ddc-bc10-11ed-9a44-7ef9696e0000
Stream name:                            CAM STREAM  0
Timeout between init and start:         300000
Timeout between start and event:        450000
Application running times:              1
Processing count in each run:           [2]

Event ID        timeout
0               3500103
...

    Note

    The CSC files contain human-readable settings corresponding to the stream metadata and event timestamps of the streams. Users can modify this configuration, for example to fine tune timeout values depending on the system capabilities.

    List the files generated:

    ls -1 *.csc.yml

    The CSC files can be shown as below:

    94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml
94085ddc-bc10-11ed-9a44-7ef9696e0001.csc.yml
94085ddc-bc10-11ed-9a44-7ef9696e0002.csc.yml
94085ddc-bc10-11ed-9a44-7ef9696e0003.csc.yml

    Run cam-tool along with pack option for each of the CSC files to generate the corresponding CSD files:

    ~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0000.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0000.csd
~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0001.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0001.csd
~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0002.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0002.csd
~/cam/cam-packages/bin/cam-tool pack --input 94085ddc-bc10-11ed-9a44-7ef9696e0003.csc.yml --output 94085ddc-bc10-11ed-9a44-7ef9696e0003.csd

    List the files generated:

    ls -1 *.csd

    The CSD files can be shown as below:

    94085ddc-bc10-11ed-9a44-7ef9696e0000.csd
94085ddc-bc10-11ed-9a44-7ef9696e0001.csd
94085ddc-bc10-11ed-9a44-7ef9696e0002.csd
94085ddc-bc10-11ed-9a44-7ef9696e0003.csd

3. From Terminal 2, run cam-service to monitor the multiple streams that will be generated by cam-app-example.

cd ~/cam/cam-run/
mkdir -p ~/cam/cam-packages/usr/share/cam-data/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:~/cam/cam-packages/lib/
~/cam/cam-packages/bin/cam-service --config-dir ~/cam/cam-packages/usr/share/cam-data/ --log-level info --address 127.0.0.1 --port 21604

Users have the flexibility to customize the socket on which cam-service listens by modifying the IP address and port using command-line parameters. Once critical applications connect to cam-service and initiate event streams, cam-service can receive and process stream events through this socket. The following messages are expected from the terminal:

Cam service configuration:
    IP address: 127.0.0.1
    port: 21604
    log level: info
    configuration directory: ~/cam/cam-packages/usr/share/cam-data/
    fault action: none

  1. From Terminal 1, run cam-tool along with deploy option to deploy the CSD files into the system where cam-service is running.

    ~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0000.csd --address 127.0.0.1 --port 21604
~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0001.csd --address 127.0.0.1 --port 21604
~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0002.csd --address 127.0.0.1 --port 21604
~/cam/cam-packages/bin/cam-tool deploy --input 94085ddc-bc10-11ed-9a44-7ef9696e0003.csd --address 127.0.0.1 --port 21604

    The output on the Terminal 2 should look like as below, the connection number might change:

    Connection 1 is created.
Deploy Message

Connection 1 is closed.
...

    From Terminal 1, list the CSD files deployed:

    ls -1 ~/cam/cam-packages/usr/share/cam-data/*.csd

    The CSD files can be shown as below, the output might change because the ~ is expanded in different ways on different systems, usually to /home/<user>:

    ~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0000.csd
~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0001.csd
~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0002.csd
~/cam/cam-packages/usr/share/cam-data/94085ddc-bc10-11ed-9a44-7ef9696e0003.csd

  2. From Terminal 1, start cam-app-example to run the Critical Application with four streams. Each stream sends an event message 10 times with a period of 3000ms.

    ~/cam/cam-packages/bin/cam-app-example \
  --processing-period 3000 \
  --processing-count 10 \
  --stream-count 4 \
  --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
  --service-address 127.0.0.1 \
  --service-port 21604

    The following messages are expected on Terminal 1:

    Cam application configuration:
    Service IP address: 127.0.0.1
    Service port: 21604
    UUID base: 94085ddc-bc10-11ed-9a44-7ef9696e
    Stream count: 4
    Processing period (ms): 3000
    Processing count: 10
    Multiple connection support: false
    Calibration mode support: false
    Fault injection support: false
    Event(s) interval time (ms): 0
Using libcam v0.1
Starting activity...
Starting activity...
Starting activity...
Starting activity...

    And the log of event messages send are shown repeatedly:

    Stream 0 sends event 0
Stream 1 sends event 0
Stream 2 sends event 0
Stream 3 sends event 0
Stream 0 sends event 0
Stream 1 sends event 0
Stream 2 sends event 0
Stream 3 sends event 0
...

    As observed from the Terminal 2, cam-service loads four CSD files for monitoring. In the following log, the stream messages are received and processed by it:

    Connection 4 is created.
Init Message
Stream 94085ddc-bc10-11ed-9a44-7ef9696e0001 configuration is loaded.
Init Message
Stream 94085ddc-bc10-11ed-9a44-7ef9696e0000 configuration is loaded.
Init Message
Stream 94085ddc-bc10-11ed-9a44-7ef9696e0002 configuration is loaded.
Init Message
Stream 94085ddc-bc10-11ed-9a44-7ef9696e0003 configuration is loaded.
Start Message
Start Message
Start Message
Start Message
Event Message
Event Message
Event Message
Event Message
Event Message
# Repeated event messages
...

  3. CAM provides a mechanism to generate a fault from within cam-app-example that can be detected by cam-service. From Terminal 1, run cam-app-example with fault injection set to event stream 0:

    ~/cam/cam-packages/bin/cam-app-example \
  --processing-period 3000 \
  --processing-count 10 \
  --stream-count 4 \
  --enable-fault-injection \
  --fault-injection-stream 0 \
  --fault-injection-time 1000 \
  --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
  --service-address 127.0.0.1 \
  --service-port 21604

    The fault happens 1000ms after stream initialization. At that time cam-service should detect a stream temporal error with the following output on Terminal 2.

    #Repeated event messages
...
ERROR: Stream temporal error:
ERROR: stream_name: CAM STREAM 0
ERROR: stream_uuid: 94085ddc-bc10-11ed-9a44-7ef9696e0000
ERROR: event_id: 0
ERROR: time_received: 0
ERROR: time_expected: 1701066141314201
...

    The following diagram describes the fault injection option:

    _images/fault_injection.svg

    After fault is injected, cam-app-example stops performing new processing activity. The interval time from stream initialization to the first activity is 500ms.

Note

In addition to sending only one event during each activity, cam-app-example supports sending a specific number of events within one activity.

~/cam/cam-packages/bin/cam-app-example \
  --processing-period 3000 \
  --processing-count 10 \
  --stream-count 4 \
  --uuid-base 94085ddc-bc10-11ed-9a44-7ef9696e \
  --service-address 127.0.0.1 \
  --service-port 21604 \
  --event-interval "<t0,t1,...tn>"

With the command line above, cam-app-example sends n events in each processing activity. The interval time between events is defined by t0 - tn. The following diagram shows the definition of the event interval option.

_images/event_interval.svg

tn: The list of event intervals(in ms) in each processing activity.

t0 is the interval time from activity start to event 0.

  1. To stop cam-service running on Terminal 2, type Ctrl-C.

CAM Integration with Kronos

A complete integration of CAM can be found in Kronos Reference Software Stack. This integration demonstrates the feasibility of monitoring Primary Compute applications by cam-service running on Safety Island. Kronos makes use of CAM CMake option CAM_TARGET=<Linux> | <Zephyr> to build cam-service for Zephyr.

Refer to Kronos Reference Software Stack CAM Documentation for more information on the integration.