.. _user-guide: User Guide ========== * :ref:`download` * :ref:`compilation` * :ref:`usage` .. _download: Download -------- The complete BOSP sources are downloaded and compiled into a user-defined working directory. That folder could be created and properly initialized by just cloning the GIT repository as it follows: .. code-block:: shell $ git clone https://bitbucket.org/bosp/bosp.git BOSP For the purposes of this tutorial, thereafter let's assume your working directory is ~/BOSP. The actual download of the source trees of the submodules included in BOSP, however, must be performed by running the following command: .. code-block:: shell $ ./bosp setup Getting updates ^^^^^^^^^^^^^^^ To synchronize the source trees with the latest stable release, use the following command: .. code-block:: shell $ ./bosp update-stable To synchronize the source code with the development release: .. code-block:: shell $ ./bosp update-devel .. _compilation: Compilation ----------- The BOSP source tree includes the sources of the BarbequeRTRM daemon, along with sample applications, libraries and additional tools. Further dependencies must be solved by installing them through the distribution package manager. The list of such dependencies includes: - Essential builing tools (e.g., the “build-essential” packages on Debian based systems) - GIT - CMake (>= v3.0) - GCC/G++ and libstdc++-v3 (both >= v5.4) - Autotools (i.e. autoconf, automake and libtoolize) - Doxygen (>= v1.8) - Texinfo On Debian-based Linux distributions you can type the following **apt** line to install the required libraries and tools, not included in the BOSP source tree: .. code-block:: bash $ sudo apt install build-essential autoconf automake libtool gperf \ cmake git-core doxygen graphviz texinfo ncurses-dev \ byacc flex bison moreutils gawk curl premake aterm hwloc \ bc dialog criu .. _build_configuration: Build configuration ^^^^^^^^^^^^^^^^^^^ .. toctree:: :maxdepth: 2 :hidden: guide-user-kconfig Before proceeding with the actual compilation of the framework, you are asked to properly configure it, for the purposes of your target system. To this aim, the configuration step is based on the well-known Kconfig framework, and it is triggered by executing: .. code-block:: bash $ make menuconfig This will open the following menu: .. image:: img/bbque_kconfig_root.png Check :ref:`build-kconfig` for more details. .. _build: Build ^^^^^ Once the configured and selected the output directory (*Deployment* options). You can proceed with the actual compilation and installation via **make**: .. code-block:: bash $ make .. hint:: After each update we recommend you to re-run the **make menuconfig** and the cleanup targets (as following), before launching a new build. .. code-block:: bash $ make distclean .. _usage: Usage --------- .. _bosp-shell: The BOSP Shell ^^^^^^^^^^^^^^ The **BOSP Shell** is a convenient environment providing shortcuts to start, stop and interact with the BarbequeRTRM daemon. It is initialized by sourcing the *bosp_init.env* file: .. code-block:: bash $ . ~/BOSP/out/etc/bbque/bosp_init.env The BOSP Shell will appear with the appearance shown in the figure. .. image:: img/bosp_shell.png The command ``bosp-help`` will dump a short list of all the available commands. .. tip:: All the BOSP/BarbequeRTRM related commands start with either ``bosp-`` or ``bbque-``. This allows you to exploit the shell completion to get a list of available commands. Commands starting with ``bosp-`` target the management of the projects, such as compile a different flavor, while ``bbque-`` starting commands target the management of the BarbequeRTRM daemon, such as starting and stopping it. So, to launch the execution of the BarbequeRTRM daemon, just type: .. code-block:: bash $ bbque-startd or .. code-block:: bash $ bbque-startd --nocg ...in case you configured the target platform with the *Host Emulated* option enabled (see :ref:`build_configuration`). Pay attention on the first launch of the deamon. In the first case, a further script is invoked, to parse the hardware description of the target platform, as explained later in :ref:`hw-description`. .. warning:: Starting from Ubuntu 15.04, the for the coexistance of the BarbequeRTRM daemon and **systemd** you are asked to apply a patch to the systemd configuration, by running the ``bbque-systemd-setup``, answer yes and reboot the system. The expected console output should look like this: .. code-block:: bash [BOSPShell bosp] \> bbque-startd [ .019] Sudo user: giuseppe [ .026] Checking for run-time R/W directories [/home/giuseppe/Development/bosp/barbeque/../out/var/bbque/...] [ .033] Application task-graph location: /tmp/ [ .040] Checking platform description file... [ .047] Platform description directory: /home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/pil [ .055] Platform description: systems.xml [ .067] Host name: jupiter [ .149] PIL generator: [ .156] Platform description already available [ .163] Cgroup initialization... [ .173] GCroup root: /sys/fs/cgroup/cpu,cpuset,cpuacct,memory,net_cls [ .197] Umount pre-existing unified hierarchy [ .262] CPU/Memory exclusive: YES [ .273] CGroups configuration: /home/giuseppe/Development/bosp/barbeque/../out/var/bbque/jupiter.cgroup [ .286] Moving all (user-space) tasks into HOST node [ .871] Setup Release Agent: /home/giuseppe/Development/bosp/barbeque/../out/sbin/bbque-release-agent [ .894] LOG file: /home/giuseppe/Development/bosp/barbeque/../out/var/bbque.log [ .899] Starting the BarbequeRTRM Daemon... [DBG] Using Log4CppLogger configuration file [/home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/bbque.conf] But to actually check a successfull start of the daemon, open the log file. It should report a series of messages like the following ones: .. code-block:: bash tail -n 500 /home/giuseppe/Development/bosp/barbeque/../out/var/bbque.log 2020-12-11 11:48:49,584 - NOTICE bq : Starting BarbequeRTRM (v1.2.0-stable) as daemon [bbqued], running with uid [root]... 2020-12-11 11:48:49,584 - NOTICE bq : Using configuration file [/home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/bbque.conf] 2020-12-11 11:48:49,591 - INFO bq.sig : System signals installed, signal catcher thread [27640] 2020-12-11 11:48:49,611 - INFO bq.bm : BatteryManager initialization... 2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Technology : Li-ion 2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Full Charge : 3204 mAh 2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Level : [==========] 100% 2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Charge : 3204 mAh 2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Rate : 1 mA 2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Voltage : 8638 mV 2020-12-11 11:48:49,628 - INFO bq.pm.bat.BAT0 : Est. time : 11534400 s 2020-12-11 11:48:49,629 - INFO bq.bm : Batteries : 1 2020-12-11 11:48:49,629 - INFO bq.eym : EnergyMonitor initialization... 2020-12-11 11:48:49,632 - INFO bq.sm : Loading optimization policy [bq.sp.]... 2020-12-11 11:48:49,635 - INFO bq.cm : Registering command [bq.rm.sys_status]: Dump the status of each registered EXC 2020-12-11 11:48:49,635 - INFO bq.cm : Registering command [bq.rm.opt_force]: Force a new scheduling event 2020-12-11 11:48:49,636 - INFO bq.rm : RM: Registered plugins: 2020-12-11 11:48:49,636 - INFO bq.rm : * bq.pl.rxml 2020-12-11 11:48:49,636 - INFO bq.rm : * bq.rl.rxml 2020-12-11 11:48:49,636 - INFO bq.rm : * bq.rpc.fif 2020-12-11 11:48:49,636 - INFO bq.rm : * bq.sp.test 2020-12-11 11:48:49,636 - INFO bq.rm : * bq.ym.sp.sasb 2020-12-11 11:48:49,637 - INFO bq.pl.rxml : Loading 'jupiter.xml' platform file... 2020-12-11 11:48:49,637 - INFO bq.pl.rxml : System 0: IP=127.0.0.1 2020-12-11 11:48:49,639 - INFO bq.cm : Registering command [bq.wm.datalog]: Start/stop power monitor data logging 2020-12-11 11:48:49,639 - NOTICE bq.wm : ================================================================================ 2020-12-11 11:48:49,639 - NOTICE bq.wm : | THRESHOLDS | HIGH | LOW | MARGIN | TRIGGER TYPE | 2020-12-11 11:48:49,639 - NOTICE bq.wm : +------------------------+------------+-----------+---------+------------------+ 2020-12-11 11:48:49,639 - NOTICE bq.wm : | Temperature | 80°C | 75°C | 0% | over_threshold | 2020-12-11 11:48:49,639 - NOTICE bq.wm : ================================================================================ 2020-12-11 11:48:49,646 - INFO bq.plm : LoadPlatformData: platform description loaded 2020-12-11 11:48:49,646 - NOTICE bq.ra : Report on state view: 0 2020-12-11 11:48:49,646 - NOTICE bq.ra : =========================================================================== 2020-12-11 11:48:49,646 - NOTICE bq.ra : | RESOURCES ON | MOD | USED | AVAILABLE | TOTAL | 2020-12-11 11:48:49,646 - NOTICE bq.ra : |-------------------------------+-----+-----------+-----------+-----------| 2020-12-11 11:48:49,646 - NOTICE bq.ra : | sys0.cpu0.pe7 Y | | 0 % | 100 % | 100 % | 2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.mem0 Y | | 0 | 16.52 G | 16.52 G | 2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.io0 Y | | 0 | 104.85 M | 104.85 M | 2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.cpu0.pe2 Y | | 0 % | 100 % | 100 % | 2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.cpu0.pe3 Y | | 0 % | 100 % | 100 % | 2020-12-11 11:48:49,647 - NOTICE bq.ra : | sys0.cpu0.pe6 Y | | 0 % | 100 % | 100 % | 2020-12-11 11:48:49,647 - NOTICE bq.ra : =========================================================================== Finally, as you may expect, you can stop the deamon with: .. code-block:: bash $ bbque-stopd .. _hw-description: Platform description ^^^^^^^^^^^^^^^^^^^^ The description of the hardware resources available on the target platform is provided by the **systems.xml** file located under the ``/etc/bbque/pil`` directory. Since the BarbequeRTRM can be developed to work on distributed systems, this file can includes a multiple inclusion lines, each one pointing to the node-specific description file. .. code-block:: xml node0.xml .. important:: In case of Emulated Host (no cgroup) option enabled, the root description file is **symsys.xml**. The file **node.xml** will include general information about the system node (e.g., *hostname*, *network address*, etc...) along with the actual hierarchy of hardware resources. In the given example, the node comes with two dual-core CPUs, one memory and two network interfaces. .. code-block:: xml If the target platform is the host system itself, these files can be generated by following the step-by-step procedure triggered by the first execution of the BarbequeRTRM daemon. The launch script (``bbque-startd``) in fact, checks if such files have been already provided. In do not, the generation script is invoked. .. _run-configuration: Configuration file ^^^^^^^^^^^^^^^^^^ The file is located at **/etc/bbque/bbque.conf**. .. important:: The file is re-generated after each compilation/installation run, from the source file located in ``BOSP/barbeque/config/bbque.conf.in``. It is made by several sections (identified by square-brackets enclosed words e.g., ``[ModuleName]``), each one devoted to group the options related to a specific module of the daemon. For each commented line (starting with **#**), the default value is taken. If you want to specify your value, uncomment the line. General options """"""""""""""" The first three options are probably not worth to touch unless you are playing very confusingly with the BOSP installation. They point to three directories: the one containing the plugin binaries, the one from which to retrieve the application recipes and the one into which to create the FIFOs for the inter-process communication between the BarbequeRTRM daemon and the RTLib integration applications. .. code-block:: [bbque] #plugins = /home/giuseppe/Development/bosp/barbeque/../out/lib/bbque/plugins [rloader] #xml.recipe_dir = /home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/recipes [rpc] #fif.dir = /home/giuseppe/Development/bosp/barbeque/../out/var/bbque Daemon execution parameters """"""""""""""""""""""""""" The following ones allow us to customize the execution of the resource allocation policy. First, we can decide to discard the default *event-based* invocation of the policy, in favour of a periodical invocation. In this case, we have to uncomment and set the value for ``opt_interval`` (in milliseconds), to set the invocation rate of the policy. Then, the choice of the policy itself. By leaving the ``policy`` line commented, we lead the resource manager to load the plugin specified by default at compile-time, through the *Default policy* option. To the contrary, we can uncomment the line and set the name of the policy we want to use, starting from the next launch of the daemon. The third option (``domains``) these first three regards the *binding domains*, i.e., the type of resources for which the policy may have to perform the assignments. In most of the cases, the default setting covers all the cases, whatever is the selected target hardware. Similarly, the fourth one (``SynchronizationManager.policy``) is another option for which we can typically rely on the default value unless we are developing a new synchronization policy. .. code-block:: [ResourceManager] # opt_interval is in milliseconds #opt_interval = 4000 [SchedulerManager] #policy = test [BindingManager] domains = cpu,gpu,acc,grp.gpu,grp.acc,sys [SynchronizationManager] #policy = sasb The following set of options touches the power management and power monitoring functionalities of the resource manager. First, we can exploit ``PowerManager.temp.sockets`` to provide the path to the filesystem location of the CPU thermal sensors files. By default, this value is set at compile time. You are asked to overwrite it only something does not work as expected. The power monitoring functionality instead is the most tunable of the framework. Here we enable the logging of values to read at run-time (``PowerMonitor.log.enabled``) and store them into files that will be saved into the ``.log.dir`` directory. Obviously, it is also possible to redefine the monitoring period (``.period_ms``) and the number of threads to spawn for this (``.nr_threads``). This one makes sense in case of target platform featuring a wide number of processors, or resources in general. The last set options for the **PowerMonitor** allows us to trigger the invocation of the resource allocation policy, in case the monitored values of *temperature*, *power consumption* or *battery charge* falls out a range of threshold. In this case, it is worth to select a policy that actually takes into account the status of the hardware from this point of view. .. code-block:: [PowerManager] temp.sockets = /sys/devices/platform/coretemp.0/hwmon/hwmon1 [PowerMonitor] # log enabled at starting time log.enabled = 0 # output directory for the power monitor logs log.dir = /tmp/bbque/power # monitoring period period_ms = 4000 # number of monitoring threads to spawn nr_threads = 1 # trigger types: over_threshold, ... temp.threshold_high = 80 temp.threshold_low = 75 temp.trigger = over_threshold #temp.margin = 0.01 #power.threshold_high = 16000 #power.threshold_low = 15000 power.trigger = over_threshold #power.margin = 0.05 #batt.threshold_rate = 16000 #batt.margin_rate = 0.05 #batt.threshold_level = 15 batt.trigger = under_threshold The last set options from this block, regard the interfaces of the daemon with external entities. First, the **AgentProxy**, through which we put remote instances of the BarbequeRTRM in communication. Currently, the only option to set is the number of the server port (``.port``), for which the default value comes from the compile-time configuration. Then, the **DataManager** (if compiled) enables the BarbequeRTRM to act as a data provider for external tools or managers. In fact, this is the interface through which it can send data about the status of the managed applications and hardware resources. Its settings include the server port (``.server_port``) and the number of attempts, after which a client is considered disconnected (``.client_attempts``). .. code-block:: [AgentProxy] #port = 8850 [DataManager] server_port = 30200 client_attempts = 3 Logger configuration """""""""""""""""""" This section is extremely useful to tune the verbosity of daemon output. The logger is a never-without instrument for diagnostic and development purposes. First, the ``.conf_file`` option points by default to the BarbequeRTRM configuration file itself, to drive the logger through its specific configuration options, that follows. Here, we can set the log level for the overall the resource manager (``rootCategory =``) and the where the output must be directed (file, console, syslog or rsyslog). For each of this option, further details can be provided through the ".appender.*" set of options. In the following example, we want the daemon to write out to a file (``appender.raFile.fileName =``) log messages starting from the most verbose level: *DEBUG*. However, a log file with debug messages coming from all the BarbequeRTRM components could be extremely verbose, and in some cases, overwhelming and not very easy to read. Therefore, what we may want is to select the exceptions or to define, component by component, the desired log level. To do this, we can uncomment the specific ``category.bq.X`` line to overwrite the root value. The level value can be chosen between DEBUG, INFO, WARN, ERROR, NOTICE, CRIT and FATAL. .. code-block:: [logger] #log4cpp.conf_file = /home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/bbque.conf [log4cpp] ### # Set root category priority and its appender # By default all logs are only from level greater than or equal to INFO ### rootCategory = DEBUG, raFile #rootCategory = DEBUG, raConsole #rootCategory = ERROR, raSyslog #rootCategory = ERROR, raRSyslog ### # Uncomment each of the following lines for subsystems you want # to redefine the category priority to a value diffent from that of # the rootCategory ### #category.bq = INFO category.bq.ap = NOTICE #category.bq.app = NOTICE category.bq.am = INFO ... ### # Root Appender ### ... appender.raFile = org.apache.log4j.FileAppender appender.raFile.fileName = /home/giuseppe/Development/bosp/barbeque/../out/var/bbque.log appender.raFile.layout = org.apache.log4j.PatternLayout appender.raFile.layout.ConversionPattern = %d{%Y-%m-%d %H:%M:%S,%l} - %-6p %-15c : %m%n .. _test: Application execution management ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Once started the BarbequeRTRM daemon, and checked that it is properly running by looking at the log file, we actually use it by launching manageable applications. In this regard, we can distinguish between managing :ref:`generic-processes` and *Adaptive Execution Model integrated* applications. In the former case, we refer to legacy application for which we do not want to perform any porting activity. In the latter instead, we talk about applications implemented according to the execution model described in the :ref:`aem-applications`. If we want to quickly test the behaviour of the resource manager, we can select one of the sample applications from the configuration menu :ref:`kconfig-applications` and run it. .. code-block:: [BOSPShell bosp] \> hellocpp [DBG] Using Log4CppLogger configuration file [/home/giuseppe/Development/bosp/barbeque/../out/etc/bbque/HelloCPP.conf] 17:40:13,734 - INFO hellocpp : .:: HelloCPP (ver. HEAD-HASH-NOTFOUND) ::. 17:40:13,734 - INFO hellocpp : Built: Dec 11 2020 11:48:19 17:40:13,734 - INFO hellocpp : STEP 0. Initializing RTLib, application [hellocpp]... 17:40:13,737 - NOTICE rtl.ocl : OCL: Platform 0 = Intel(R) CPU Runtime for OpenCL(TM) Applications 17:40:13,738 - INFO hellocpp : STEP 1. Registering EXC with recipe ... 17:40:13,740 - WARN exc : New HelloCPP::HelloCPP() 17:40:13,740 - INFO hellocpp : STEP 2. Starting EXC control thread... 17:40:13,740 - INFO hellocpp : STEP 3. Waiting for EXC completion... 17:40:13,740 - WARN exc : HelloCPP::onSetup() 17:40:13,833 - WARN exc : HelloCPP::onConfigure(): EXC [HelloCPP] => AWM [02] 17:40:13,833 - NOTICE exc : MayApp::onConfigure(): EXC [HelloCPP], AWM[02] => R=400, R= 4, R= 0 17:40:13,833 - NOTICE exc : HelloCPP::onRun() : Hello World! 17:40:13,833 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [1], CPS = 0.00 17:40:13,834 - NOTICE rpc : ForwardRuntimeProfile: [HelloCPP] {CPS_GGAP=0.00, CPU=0.00 (round=0), Cycle-time=0.00ms} 17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World! 17:40:13,834 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [2], CPS = 1987.04 17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World! 17:40:13,834 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [3], CPS = 2379.82 17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World! 17:40:13,834 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [4], CPS = 2893.79 17:40:13,834 - NOTICE exc : HelloCPP::onRun() : Hello World! 17:40:13,835 - WARN exc : HelloCPP::onMonitor() : EXC [HelloCPP] @ AWM [02] => cycles [5], CPS = 3284.68 17:40:13,835 - WARN exc : HelloCPP::onRelease() : exit 17:40:13,837 - INFO hellocpp : STEP 4. Disabling EXC... Cumulative execution stats for 'HelloCPP': TotCycles : 5 StartLatency : 90 [ms] AwmWait : 90 [ms] Configure : 2 [ms] Process : 0 [ms] # EXC AWM Uses Cycles Total | Min Max | Avg Var #==================================+===================+================== HelloCPP 002 1 5 0 | 0.171 0.503 | 0.278 0.016 #-------------------------+ +-------------------+------------------ HelloCPP 002 onRun 0 | 0.114 0.406 | 0.199 0.016 HelloCPP 002 onMonitor 0 | 0.057 0.097 | 0.079 0.000 #-------------------------+--------+-------------------+------------------ HelloCPP 002 onConfigure 2 | 2.027 2.027 | 2.027 0.000 [DBG] RTLIB_Exit: thread tid=15082 terminating... [DBG] RTLIB_Exit: unregistering all the contexts... [DBG] BbqueRPC_FIFO_Client dtor [DBG] Releasing FIFO RPC channel [DBG] Tx [APP_EXIT] Request FIFO_HDR [sze: 20, off: 4, typ: 1], RPC_HDR [typ: 1, pid: 15082, eid: 0], Bytes: 20... [DBG] Waiting for FIFO header... [DBG] Rx FIFO_HDR [sze: 20, off: 4, typ: 1] [DBG] ChannelTrd [PID: 15083] END 17:40:13,838 - INFO hellocpp : ===== HelloCPP DONE! ===== If everything is properly working, the application will print a sequence of log messages about the Abstract Execution Model (AEM) calls executed and the number of ``onRun()`` cycles, plus a final report of the execution times. In case of errors, the application may stuck, in such a case check the BarbequeRTRM log file for more information.