User Guide¶
Download¶
The complete BOSP sources are downloaded and compiled into a user-defined working directory. That folder could be created and properly initialized by just cloning the GIT repository as it follows:
$ git clone https://bitbucket.org/bosp/bosp.git BOSP
For the purposes of this tutorial, thereafter let’s assume your working directory is ~/BOSP.
The actual download of the source trees of the submodules included in BOSP, however, must be performed by running the following command:
$ ./bosp setup
Getting updates¶
To synchronize the source trees with the latest stable release, use the following command:
$ ./bosp update-stable
To synchronize the source code with the development release:
$ ./bosp update-devel
Compilation¶
The BOSP source tree includes the sources of the BarbequeRTRM daemon, along with sample applications, libraries and additional tools. Further dependencies must be solved by installing them through the distribution package manager. The list of such dependencies includes:
Essential builing tools (e.g., the “build-essential” packages on Debian based systems)
GIT
CMake (>= v3.0)
GCC/G++ and libstdc++-v3 (both >= v5.4)
Autotools (i.e. autoconf, automake and libtoolize)
Doxygen (>= v1.8)
Texinfo
On Debian-based Linux distributions you can type the following apt line to install the required libraries and tools, not included in the BOSP source tree:
$ sudo apt install build-essential autoconf automake libtool gperf \
cmake git-core doxygen graphviz texinfo ncurses-dev \
byacc flex bison moreutils gawk curl premake aterm hwloc \
bc dialog criu
Build configuration¶
Before proceeding with the actual compilation of the framework, you are asked to properly configure it, for the purposes of your target system. To this aim, the configuration step is based on the well-known Kconfig framework, and it is triggered by executing:
$ make menuconfig
This will open the following menu:
Check Build Configuration with Kconfig for more details.
Build¶
Once the configured and selected the output directory (Deployment options). You can proceed with the actual compilation and installation via make:
$ make
Hint
After each update we recommend you to re-run the make menuconfig and the cleanup targets (as following), before launching a new build.
$ make distclean
Usage¶
The BOSP Shell¶
The BOSP Shell is a convenient environment providing shortcuts to start, stop and interact with the BarbequeRTRM daemon. It is initialized by sourcing the bosp_init.env file:
$ . ~/BOSP/out/etc/bbque/bosp_init.env
The BOSP Shell will appear with the appearance shown in the figure.
The command bosp-help will dump a short list of all the available commands.
Tip
All the BOSP/BarbequeRTRM related commands start with either bosp- or
bbque-. This allows you to exploit the shell completion to get a list of
available commands.
Commands starting with bosp- target the management of the projects, such as
compile a different flavor, while bbque- starting commands target the
management of the BarbequeRTRM daemon, such as starting and stopping it.
So, to launch the execution of the BarbequeRTRM daemon, just type:
$ bbque-startd
or
$ bbque-startd --nocg
…in case you configured the target platform with the Host Emulated option enabled (see Build configuration).
Pay attention on the first launch of the deamon. In the first case, a further script is invoked, to parse the hardware description of the target platform, as explained later in Platform description.
Warning
Starting from Ubuntu 15.04, the for the coexistance of the BarbequeRTRM daemon
and systemd you are asked to apply a patch to the systemd configuration,
by running the bbque-systemd-setup, answer yes and reboot the system.
The expected console output should look like this:
[BOSPShell bosp] \> bbque-startd
[ .019] Sudo user: giuseppe
[ .026] Checking for run-time R/W directories [/home/giuseppe/Development/bosp/barbeque/../out/var/bbque/...]
[ .033] Application task-graph location: /tmp/
[ .040] Checking platform description file...
[ .047] Platform description directory: /home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/pil
[ .055] Platform description: systems.xml
[ .067] Host name: jupiter
[ .149] PIL generator:
[ .156] Platform description already available
[ .163] Cgroup initialization...
[ .173] GCroup root: /sys/fs/cgroup/cpu,cpuset,cpuacct,memory,net_cls
[ .197] Umount pre-existing unified hierarchy
[ .262] CPU/Memory exclusive: YES
[ .273] CGroups configuration: /home/giuseppe/Development/bosp/barbeque/../out/var/bbque/jupiter.cgroup
[ .286] Moving all (user-space) tasks into HOST node
[ .871] Setup Release Agent: /home/giuseppe/Development/bosp/barbeque/../out/sbin/bbque-release-agent
[ .894] LOG file: /home/giuseppe/Development/bosp/barbeque/../out/var/bbque.log
[ .899] Starting the BarbequeRTRM Daemon...
[DBG] Using Log4CppLogger configuration file [/home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/bbque.conf]
But to actually check a successfull start of the daemon, open the log file. It should report a series of messages like the following ones:
tail -n 500 /home/giuseppe/Development/bosp/barbeque/../out/var/bbque.log
2020-12-11 11:48:49,584 - NOTICE bq : Starting BarbequeRTRM (v1.2.0-stable) as daemon [bbqued], running with uid [root]...
2020-12-11 11:48:49,584 - NOTICE bq : Using configuration file [/home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/bbque.conf]
2020-12-11 11:48:49,591 - INFO bq.sig : System signals installed, signal catcher thread [27640]
2020-12-11 11:48:49,611 - INFO bq.bm : BatteryManager initialization...
2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Technology : Li-ion
2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Full Charge : 3204 mAh
2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Level : [==========] 100%
2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Charge : 3204 mAh
2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Rate : 1 mA
2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Voltage : 8638 mV
2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Est. time : 11534400 s
2020-12-11 11:48:49,629 - INFO bq.bm : Batteries : 1
2020-12-11 11:48:49,629 - INFO bq.eym : EnergyMonitor initialization...
2020-12-11 11:48:49,632 - INFO bq.sm : Loading optimization policy [bq.sp.]...
2020-12-11 11:48:49,635 - INFO bq.cm : Registering command [bq.rm.sys_status]: Dump the status of each registered EXC
2020-12-11 11:48:49,635 - INFO bq.cm : Registering command [bq.rm.opt_force]: Force a new scheduling event
2020-12-11 11:48:49,636 - INFO bq.rm : RM: Registered plugins:
2020-12-11 11:48:49,636 - INFO bq.rm : * bq.pl.rxml
2020-12-11 11:48:49,636 - INFO bq.rm : * bq.rl.rxml
2020-12-11 11:48:49,636 - INFO bq.rm : * bq.rpc.fif
2020-12-11 11:48:49,636 - INFO bq.rm : * bq.sp.test
2020-12-11 11:48:49,636 - INFO bq.rm : * bq.ym.sp.sasb
2020-12-11 11:48:49,637 - INFO bq.pl.rxml : Loading 'jupiter.xml' platform file...
2020-12-11 11:48:49,637 - INFO bq.pl.rxml : System 0: IP=127.0.0.1
2020-12-11 11:48:49,639 - INFO bq.cm : Registering command [bq.wm.datalog]: Start/stop power monitor data logging
2020-12-11 11:48:49,639 - NOTICE bq.wm : ================================================================================
2020-12-11 11:48:49,639 - NOTICE bq.wm : | THRESHOLDS | HIGH | LOW | MARGIN | TRIGGER TYPE |
2020-12-11 11:48:49,639 - NOTICE bq.wm : +------------------------+------------+-----------+---------+------------------+
2020-12-11 11:48:49,639 - NOTICE bq.wm : | Temperature | 80°C | 75°C | 0% | over_threshold |
2020-12-11 11:48:49,639 - NOTICE bq.wm : ================================================================================
2020-12-11 11:48:49,646 - INFO bq.plm : LoadPlatformData: platform description loaded
2020-12-11 11:48:49,646 - NOTICE bq.ra : Report on state view: 0
2020-12-11 11:48:49,646 - NOTICE bq.ra : ===========================================================================
2020-12-11 11:48:49,646 - NOTICE bq.ra : | RESOURCES ON | MOD | USED | AVAILABLE | TOTAL |
2020-12-11 11:48:49,646 - NOTICE bq.ra : |-------------------------------+-----+-----------+-----------+-----------|
2020-12-11 11:48:49,646 - NOTICE bq.ra : | sys0.cpu0.pe7 Y | | 0 % | 100 % | 100 % |
2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.mem0 Y | | 0 | 16.52 G | 16.52 G |
2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.io0 Y | | 0 | 104.85 M | 104.85 M |
2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.cpu0.pe2 Y | | 0 % | 100 % | 100 % |
2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.cpu0.pe3 Y | | 0 % | 100 % | 100 % |
2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.cpu0.pe6 Y | | 0 % | 100 % | 100 % |
2020-12-11 11:48:49,647 - NOTICE bq.ra : ===========================================================================
Finally, as you may expect, you can stop the deamon with:
$ bbque-stopd
Platform description¶
The description of the hardware resources available on the target platform is
provided by the systems.xml file located under the <OUT>/etc/bbque/pil
directory. Since the BarbequeRTRM can be developed to work on distributed systems, this
file can includes a multiple inclusion lines, each one pointing to the
node-specific description file.
<!-- systems.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<systems version="1.0">
<include local="true">node0.xml</include>
</systems>
Important
In case of Emulated Host (no cgroup) option enabled, the root description file is symsys.xml.
The file node.xml will include general information about the system node (e.g., hostname, network address, etc…) along with the actual hierarchy of hardware resources. In the given example, the node comes with two dual-core CPUs, one memory and two network interfaces.
<?xml version="1.0" encoding="UTF-8"?>
<system hostname="plutone" address="10.168.67.165">
<cpu arch="x86_64" id="0" socket_id="0" mem_id="0">
<pe id="0" core_id="0" share="100" partition="host"/>
<pe id="1" core_id="0" share="100" partition="host"/>
</cpu>
<cpu arch="x86_64" id="1" socket_id="0" mem_id="0">
<pe id="2" core_id="1" share="100" partition="mdev"/>
<pe id="3" core_id="1" share="100" partition="mdev"/>
</cpu>
<mem id="0" quantity="8066028" unit="KB"/>
<netif id="3" name="wlp3s0" />
<netif id="4" name="enp6s0" />
</system>
If the target platform is the host system itself, these files can be generated
by following the step-by-step procedure triggered by the first execution of the
BarbequeRTRM daemon. The launch script (bbque-startd) in fact, checks if
such files have been already provided. In do not, the generation script is invoked.
Configuration file¶
The file is located at <BOSP_PREFIX>/etc/bbque/bbque.conf.
Important
The file is re-generated after each compilation/installation run, from the
source file located in BOSP/barbeque/config/bbque.conf.in.
It is made by several sections (identified by square-brackets enclosed words
e.g., [ModuleName]), each one devoted to group the options related to a
specific module of the daemon.
For each commented line (starting with #), the default value is taken. If you want to specify your value, uncomment the line.
General options¶
The first three options are probably not worth to touch unless you are playing very confusingly with the BOSP installation. They point to three directories: the one containing the plugin binaries, the one from which to retrieve the application recipes and the one into which to create the FIFOs for the inter-process communication between the BarbequeRTRM daemon and the RTLib integration applications.
[bbque]
#plugins = /home/giuseppe/Development/bosp/barbeque/../out/lib/bbque/plugins
[rloader]
#xml.recipe_dir = /home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/recipes
[rpc]
#fif.dir = /home/giuseppe/Development/bosp/barbeque/../out/var/bbque
Daemon execution parameters¶
The following ones allow us to customize the execution of the resource
allocation policy. First, we can decide to discard the default event-based
invocation of the policy, in favour of a periodical invocation. In this case,
we have to uncomment and set the value for opt_interval (in milliseconds),
to set the invocation rate of the policy.
Then, the choice of the policy itself. By leaving the policy line
commented, we lead the resource manager to load the plugin specified by default
at compile-time, through the Default policy option. To the contrary, we can
uncomment the line and set the name of the policy we want to use, starting from
the next launch of the daemon.
The third option (domains) these first three regards the binding domains,
i.e., the type of resources for which the policy may have to perform the
assignments. In most of the cases, the default setting covers all the cases,
whatever is the selected target hardware.
Similarly, the fourth one (SynchronizationManager.policy) is another option
for which we can typically rely on the default value unless we are developing a
new synchronization policy.
[ResourceManager]
# opt_interval is in milliseconds
#opt_interval = 4000
[SchedulerManager]
#policy = test
[BindingManager]
domains = cpu,gpu,acc,grp.gpu,grp.acc,sys
[SynchronizationManager]
#policy = sasb
The following set of options touches the power management and power monitoring functionalities of the resource manager.
First, we can exploit PowerManager.temp.sockets to provide the path to the
filesystem location of the CPU thermal sensors files. By default, this value is
set at compile time. You are asked to overwrite it only something does not work
as expected.
The power monitoring functionality instead is the most tunable of the
framework. Here we enable the logging of values to read at run-time
(PowerMonitor.log.enabled) and store them into files that will be saved
into the .log.dir directory. Obviously, it is also possible to redefine the
monitoring period (.period_ms) and the number of threads to spawn for this
(.nr_threads). This one makes sense in case of target platform featuring a wide
number of processors, or resources in general.
The last set options for the PowerMonitor allows us to trigger the invocation of the resource allocation policy, in case the monitored values of temperature, power consumption or battery charge falls out a range of threshold. In this case, it is worth to select a policy that actually takes into account the status of the hardware from this point of view.
[PowerManager]
temp.sockets = /sys/devices/platform/coretemp.0/hwmon/hwmon1
[PowerMonitor]
# log enabled at starting time
log.enabled = 0
# output directory for the power monitor logs
log.dir = /tmp/bbque/power
# monitoring period
period_ms = 4000
# number of monitoring threads to spawn
nr_threads = 1
# trigger types: over_threshold, ...
temp.threshold_high = 80
temp.threshold_low = 75
temp.trigger = over_threshold
#temp.margin = 0.01
#power.threshold_high = 16000
#power.threshold_low = 15000
power.trigger = over_threshold
#power.margin = 0.05
#batt.threshold_rate = 16000
#batt.margin_rate = 0.05
#batt.threshold_level = 15
batt.trigger = under_threshold
The last set options from this block, regard the interfaces of the daemon with
external entities. First, the AgentProxy, through which we put remote
instances of the BarbequeRTRM in communication. Currently, the only option to
set is the number of the server port (.port), for which the default value
comes from the compile-time configuration.
Then, the DataManager (if compiled) enables the BarbequeRTRM to act as a
data provider for external tools or managers. In fact, this is the interface
through which it can send data about the status of the managed applications and
hardware resources. Its settings include the server port (.server_port) and
the number of attempts, after which a client is considered disconnected
(.client_attempts).
[AgentProxy]
#port = 8850
[DataManager]
server_port = 30200
client_attempts = 3
Logger configuration¶
This section is extremely useful to tune the verbosity of daemon output. The logger is a never-without instrument for diagnostic and development purposes.
First, the .conf_file option points by default to the BarbequeRTRM
configuration file itself, to drive the logger through its specific
configuration options, that follows.
Here, we can set the log level for the overall the resource manager
(rootCategory =) and the where the output must be directed (file, console,
syslog or rsyslog). For each of this option, further details can be provided
through the “.appender.*” set of options. In the following example, we want the
daemon to write out to a file (appender.raFile.fileName =) log messages
starting from the most verbose level: DEBUG.
However, a log file with debug messages coming from all the BarbequeRTRM
components could be extremely verbose, and in some cases, overwhelming and not
very easy to read. Therefore, what we may want is to select the exceptions or
to define, component by component, the desired log level. To do this, we can
uncomment the specific category.bq.X line to overwrite the root value.
The level value can be chosen between DEBUG, INFO, WARN, ERROR, NOTICE, CRIT and FATAL.
[logger]
#log4cpp.conf_file = /home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/bbque.conf
[log4cpp]
###
# Set root category priority and its appender
# By default all logs are only from level greater than or equal to INFO
###
rootCategory = DEBUG, raFile
#rootCategory = DEBUG, raConsole
#rootCategory = ERROR, raSyslog
#rootCategory = ERROR, raRSyslog
###
# Uncomment each of the following lines for subsystems you want
# to redefine the category priority to a value diffent from that of
# the rootCategory
###
#category.bq = INFO
category.bq.ap = NOTICE
#category.bq.app = NOTICE
category.bq.am = INFO
...
###
# Root Appender
###
...
appender.raFile = org.apache.log4j.FileAppender
appender.raFile.fileName = /home/giuseppe/Development/bosp/barbeque/../out/var/bbque.log
appender.raFile.layout = org.apache.log4j.PatternLayout
appender.raFile.layout.ConversionPattern = %d{%Y-%m-%d %H:%M:%S,%l} - %-6p %-15c : %m%n
Application execution management¶
Once started the BarbequeRTRM daemon, and checked that it is properly running by looking at the log file, we actually use it by launching manageable applications. In this regard, we can distinguish between managing Not-integrated applications and Adaptive Execution Model integrated applications.
In the former case, we refer to legacy application for which we do not want to perform any porting activity. In the latter instead, we talk about applications implemented according to the execution model described in the Adaptive Execution Model.
If we want to quickly test the behaviour of the resource manager, we can select one of the sample applications from the configuration menu AEM-integrated applications and run it.
[BOSPShell bosp] \> hellocpp
[DBG] Using Log4CppLogger configuration file [/home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/HelloCPP.conf]
17:40:13,734 - INFO hellocpp : .:: HelloCPP (ver. HEAD-HASH-NOTFOUND) ::.
17:40:13,734 - INFO hellocpp : Built: Dec 11 2020 11:48:19
17:40:13,734 - INFO hellocpp : STEP 0. Initializing RTLib, application [hellocpp]...
17:40:13,737 - NOTICE rtl.ocl : OCL: Platform 0 = Intel(R) CPU Runtime for OpenCL(TM) Applications
17:40:13,738 - INFO hellocpp : STEP 1. Registering EXC with recipe <HelloCPP>...
17:40:13,740 - WARN exc : New HelloCPP::HelloCPP()
17:40:13,740 - INFO hellocpp : STEP 2. Starting EXC control thread...
17:40:13,740 - INFO hellocpp : STEP 3. Waiting for EXC completion...
17:40:13,740 - WARN exc : HelloCPP::onSetup()
17:40:13,833 - WARN exc : HelloCPP::onConfigure(): EXC [HelloCPP] => AWM [02]
17:40:13,833 - NOTICE exc : MayApp::onConfigure(): EXC [HelloCPP], AWM[02] => R<PROC_quota>=400, R<PROC_nr>= 4, R<MEM>= 0
17:40:13,833 - NOTICE exc : HelloCPP::onRun() : Hello World!
17:40:13,833 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [1], CPS = 0.00
17:40:13,834 - NOTICE rpc : ForwardRuntimeProfile: [HelloCPP] {CPS_GGAP=0.00, CPU=0.00 (round=0), Cycle-time=0.00ms}
17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World!
17:40:13,834 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [2], CPS = 1987.04
17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World!
17:40:13,834 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [3], CPS = 2379.82
17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World!
17:40:13,834 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [4], CPS = 2893.79
17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World!
17:40:13,835 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [5], CPS = 3284.68
17:40:13,835 - WARN exc : HelloCPP::onRelease() : exit
17:40:13,837 - INFO hellocpp : STEP 4. Disabling EXC...
Cumulative execution stats for 'HelloCPP':
TotCycles : 5
StartLatency : 90 [ms]
AwmWait : 90 [ms]
Configure : 2 [ms]
Process : 0 [ms]
# EXC AWM Uses Cycles Total | Min Max | Avg Var
#==================================+===================+==================
HelloCPP 002 1 5 0 | 0.171 0.503 | 0.278 0.016
#-------------------------+ +-------------------+------------------
HelloCPP 002 onRun 0 | 0.114 0.406 | 0.199 0.016
HelloCPP 002 onMonitor 0 | 0.057 0.097 | 0.079 0.000
#-------------------------+--------+-------------------+------------------
HelloCPP 002 onConfigure 2 | 2.027 2.027 | 2.027 0.000
[DBG] RTLIB_Exit: thread tid=15082 terminating...
[DBG] RTLIB_Exit: unregistering all the contexts...
[DBG] BbqueRPC_FIFO_Client dtor
[DBG] Releasing FIFO RPC channel
[DBG] Tx [APP_EXIT] Request FIFO_HDR [sze: 20, off: 4, typ: 1], RPC_HDR [typ: 1, pid: 15082, eid: 0], Bytes: 20...
[DBG] Waiting for FIFO header...
[DBG] Rx FIFO_HDR [sze: 20, off: 4, typ: 1]
[DBG] ChannelTrd [PID: 15083] END
17:40:13,838 - INFO hellocpp : ===== HelloCPP DONE! =====
If everything is properly working, the application will print a sequence of log
messages about the Abstract Execution Model (AEM) calls executed and the number
of onRun() cycles, plus a final report of the execution times.
In case of errors, the application may stuck, in such a case check the BarbequeRTRM log file for more information.