1. Introduction
Xenomai 3 is the new architecture of the Xenomai real-time framework, which can run seamlessly side-by-side Linux as a co-kernel system, or natively over mainline Linux kernels. In the latter case, the mainline kernel can be supplemented by the PREEMPT-RT patch to meet stricter response time requirements than standard kernel preemption would bring.
One of the two available real-time cores is selected at build time. The dual kernel core is codenamed Cobalt, the native Linux implementation is called Mercury.
If you are looking for detailed information about installing a legacy Xenomai 2.x release, please refer to this document instead. Please note that Xenomai 2.x is discontinued and not maintained anymore. |
2. Installation steps
Xenomai follows a split source model, decoupling the kernel space support from the user-space libraries.
To this end, kernel and user-space Xenomai components are respectively
available under the kernel/
and lib/
sub-trees. Other top-level
directories, such as scripts/
, testsuite/
and utils/
, provide
additional scripts and programs to be used on either the build host,
or the runtime target.
The kernel/
sub-tree which implements the in-kernel support code is
seen as a built-in extension of the Linux kernel. Therefore, the
standard Linux kernel configuration process should be used to define
the various settings for the Xenomai kernel components. All the
kernel code Xenomai currently introduces, implements the Cobalt core
(i.e. dual kernel configuration). As of today, the Mercury core
needs no Xenomai-specific code in kernel space.
The lib/
sub-tree contains the various user-space libraries exported
by the Xenomai framework to the applications. This tree is built
separately from the kernel support. Libraries are built in order to
support the selected core, either Cobalt or Mercury.
3. Installing the Cobalt core
3.1. Preparing the Cobalt kernel
Xenomai/cobalt provides a real-time extension kernel seamlessly
integrated to Linux, therefore the first step is to build it as part
of the target kernel. To this end, scripts/prepare-kernel.sh
is a
shell script which sets up the target kernel properly. The syntax is
as follows:
$ scripts/prepare-kernel.sh [--linux=<linux-srctree>]
[--ipipe=<ipipe-patch>] [--arch=<target-arch>]
-
--linux
-
specifies the path of the target kernel source tree. Such kernel tree may be already configured or not, indifferently. This path defaults to $PWD.
-
--ipipe
-
specifies the path of the interrupt pipeline (aka I-pipe) patch to apply against the kernel tree. Suitable patches are available from the project’s download area. This parameter can be omitted if the I-pipe has already been patched in, or the script shall suggest an appropriate one. The script will detect whether the interrupt pipeline code is already present into the kernel tree, and skip this operation if so.
-
--arch
-
tells the script about the target architecture. If unspecified, the build host architecture suggested as a reasonable default.
For instance, the following command would prepare the Linux tree
located at /home/me/linux-3.10-ipipe
in order to patch the Xenomai
support in:
$ cd xenomai-3
$ scripts/prepare-kernel.sh --linux=/home/me/linux-3.10
Note: The script will infer the location of the Xenomai kernel code
from its own location within the Xenomai source tree. For instance, if
/home/me/xenomai-3/scripts/prepare-kernel.sh
is executing, then
the Xenomai kernel code available from
/home/me/xenomai-3/kernel/cobalt
will be patched in the target
Linux kernel.
3.2. Configuring and compiling the Cobalt kernel
Once prepared, the target kernel can be configured as usual. All Xenomai configuration options are available from the "Xenomai" toplevel Kconfig menu.
There are several important kernel configuration options, documented in the TROUBLESHOOTING guide.
Once configured, the kernel can be compiled as usual.
If you want several different configs/builds at hand, you may reuse
the same source by adding O=../build-<target>
to each make
invocation.
In order to cross-compile the Linux kernel, pass an ARCH and CROSS_COMPILE variable on make command line. See sections "Building a Cobalt/arm kernel", "Building a Cobalt/powerpc kernel", "Building a Cobalt/x86 kernel", for examples.
3.3. Cobalt kernel parameters
The Cobalt kernel accepts the following set of parameters, which should be passed on the kernel command line by the boot loader.
NAME | DESCRIPTION | DEFAULT |
---|---|---|
xenomai.allowed_group=<gid> |
Enable non-root access to Xenomai services from user-space. <gid> is the ID of the Linux user group whose members should be allowed such access by the Cobalt core. |
None |
xenomai.sysheap_size=<kbytes> |
Set the size of the memory heap used internally by the Cobalt core to allocate runtime objects. This value is expressed in kilo-bytes. |
256 |
xenomai.state=<state> |
Set the initial state of the Cobalt core at boot up, which may be enabled, stopped or disabled. See the documentation about the corectl(1) utility for a description of these states. |
enabled |
xenomai.clockfreq=<hz-freq> |
Override the real-time clock frequency used in measuring time intervals with the given value. The most accurate value is normally determined by the Cobalt core automatically when initializing. It is strongly recommended not to use this option unless you really know what you are doing. This value is expressed in HZ. |
0 (=calibrated) |
xenomai.timerfreq=<hz-freq> |
Override the real-time timer frequency used in programming timer shots with the given value. The most accurate value is normally determined by the Cobalt core automatically when initializing. It is strongly recommended not to use this option unless you really know what you are doing. This value is expressed in HZ. |
0 (=calibrated) |
xenomai.smi=<state> |
x86-specific: Set the state of the SMI workaround. The possible values are disabled, detect and enabled. See the discussion about SMIs for a description of these states. |
detect |
xenomai.smi_mask=<source-mask> |
x86-specific: Set of bits to mask in the SMI control register. |
1 (=global disable) |
3.4. Examples of building the Cobalt kernel
The examples in following sections use the following conventions:
-
$linux_tree
-
path to the target kernel sources
-
$xenomai_root
-
path to the Xenomai sources
3.4.1. Building a Cobalt/x86 kernel (32/64bit)
Building Xenomai/cobalt for x86 is almost the same for 32bit and 64bit platforms. You should note, however, that it is not possible to run Xenomai libraries compiled for x86_32 on a kernel compiled for x86_64, and conversely.
Assuming that you want to build natively for a x86_64 system (x86_32 cross-build options from x86_64 appear between brackets), you would typically run:
$ cd $linux_tree
$ $xenomai_root/scripts/prepare-kernel.sh --arch=x86 \
--ipipe=ipipe-core-X.Y.Z-x86-NN.patch
$ make [ARCH=i386] xconfig/gconfig/menuconfig
…configure the kernel (see also the recommended settings here).
Enable Xenomai options, then build with:
$ make [ARCH=i386] bzImage modules
Now, let’s say that you really want to build Xenomai for a Pentium-based x86 32bit platform, using the native host toolchain; the typical steps would be as follows:
$ cd $linux_tree
$ $xenomai_root/scripts/prepare-kernel.sh --arch=i386 \
--ipipe=ipipe-core-X.Y.Z-x86-NN.patch
$ make xconfig/gconfig/menuconfig
…configure the kernel (see also the recommended settings here).
Enable Xenomai options, then build with:
$ make bzImage modules
Similarly, for a 64bit platform, you would use:
$ cd $linux_tree
$ $xenomai_root/scripts/prepare-kernel.sh --arch=x86_64 \
--ipipe=ipipe-core-X.Y.Z-x86-NN.patch
$ make xconfig/gconfig/menuconfig
…configure the kernel (see also the recommended settings here).
Enable Xenomai options, then build with:
$ make bzImage modules
The remaining examples illustrate how to cross-compile a Cobalt-enabled kernel for various architectures. Of course, you would have to install the proper cross-compilation toolchain for the target system first.
3.4.2. Building a Cobalt/powerpc kernel
A typical cross-compilation setup, in order to build Xenomai for a ppc-6xx architecture running a 3.10.32 kernel. We use the DENX ELDK cross-compiler:
$ cd $linux_tree
$ $xenomai_root/scripts/prepare-kernel.sh --arch=powerpc \
--ipipe=ipipe-core-3.10.32-powerpc-1.patch
$ make ARCH=powerpc CROSS_COMPILE=ppc_6xx- xconfig/gconfig/menuconfig
…select the kernel and Xenomai options, save the configuration
$ make ARCH=powerpc CROSS_COMPILE=powerpc-linux- uImage modules
…manually install the kernel image and modules to the proper location
3.4.3. Building Cobalt/arm kernel
Using codesourcery toolchain named arm-none-linux-gnueabi-gcc
and
compiling for a CSB637 board (AT91RM9200 based), a typical compilation
will look like:
$ cd $linux_tree
$ $xenomai_root/scripts/prepare-kernel.sh --arch=arm \
--ipipe=ipipe-core-X.Y.Z-x86-NN.patch
$ mkdir -p $build_root/linux
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- O=$build_root/linux \
csb637_defconfig
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- O=$build_root/linux \
bzImage modules
…manually install the kernel image, system map and modules to the proper location
4. Installing the Mercury core
For Mercury, you need no Xenomai-specific kernel support so far,
beyond what your host Linux kernel already provides. Your kernel
should at least provide high resolution timer support
(CONFIG_HIGH_RES_TIMERS
), and likely complete preemption
(PREEMPT_RT) if your application requires short and bounded
latencies.
Kernels with no real-time support can be used too, likely for basic debugging tasks, and/or running applications which do not have strict response time requirements.
Therefore, unlike with Cobalt, there is no additional steps for preparing and/or configuring the kernel for Mercury.
5. Installing the Xenomai libraries and tools
5.1. Prerequisites
5.1.1. Generic requirements (both cores)
-
GCC must have support for legacy atomic builtins (__sync form).
-
GCC should have a (sane/working) support for TLS preferably, although this is not mandatory if building with
--disable-tls
. -
If you plan to enable the user-space registry support (i.e.
--enable-registry
), then CONFIG_FUSE_FS must be enabled in the target kernel running the real-time applications. In addition, the FUSE development libraries must be available from the toolchain. -
If you plan to build from the sources available from the Xenomai GIT tree (git.xenomai.org), the autoconf (>= 2.62), automake and libtool packages must be available on your build system. This is not required when building from a source tree extracted from a release tarball.
5.1.2. Cobalt-specific requirements
-
The kernel version must be 3.10 or better.
-
An interrupt pipeline (I-pipe) patch must be available for your target kernel. You can find the official patches issued by the Xenomai project there. Only patches from the ipipe-core series are appropriate, legacy patches from the adeos-ipipe series are not.
-
A timestamp counter (TSC) is required from running on a x86_32 hardware. Unlike with Xenomai 2.x, TSC-emulation using a PIT register is not available.
5.1.3. Mercury-specific requirement
-
There is no particular requirement for Mercury setups, although using a NPTL-based glibc or uClibc is recommended.
5.2. Configuring
If building the source obtained from the Xenomai GIT tree
(git.xenomai.org), the configure
script and Makefiles must be
generated in the Xenomai source tree. The recommended way is to run
the automatic reconfiguration script shipped, from the top of the
source tree:
$ ./scripts/bootstrap
If building from a release tarball, a set of autoconf-generated file will be readily available from the extracted source tree, and therefore reconfiguring will not be required.
When run, the generated configure
script prepares for building the
libraries and programs, for both the Cobalt and Mercury cores. The
core-specific code which may be needed internally is automatically and
transparently selected at compilation-time by the build process.
The options listed below can be passed to this script.
5.2.1. Generic configuration options (both cores)
--with=core=<type>
|
Indicates which real-time core you want to build the support libraries for, namely cobalt or mercury. This option defaults to cobalt. |
--prefix=<dir>
|
Specifies the root installation path for libraries, include
files, scripts and executables. Running |
--enable-debug[=partial]
|
This switch controls the debug level. Three levels are available, with varying overhead:
|
--disable-debug
|
Fully turns off all consistency checks and assertions, turns on the optimizer and disables debug symbol generation. |
--enable-assert
|
A number of debug assertion statements are present into the
Xenomai libraries, checking the internal consistency of the
runtime system dynamically (see man assert(3)). Passing
|
--enable-pshared
|
Enable shared multi-processing. When enabled, this option allows multiple processes to share real-time objects (e.g. tasks, semaphores). |
--enable-registry[=/registry-root-path]
|
Xenomai APIs can export their internal state through a pseudo-filesystem, which files may be read to obtain information about the existing real-time objects, such as tasks, semaphores, message queues and so on. This feature is supported by FUSE, which must be available on the target system. Building the Xenomai libraries with the registry support requires the FUSE development libraries to available from the toolchain. In addition, CONFIG_FUSE_FS must be enabled in the target kernel. |
When this option is enabled, the system creates a file hierarchy at
<user>/<session>/<pid>
under the registry root path, where you
can access the internal state of the active real-time objects. The
session label is obtained from the --session runtime switch. If no
session name is specified, anon@<pid>
will be used. E.g. looking at
the properties of a VxWorks task could be done as follows:
If not specified in the configuration switch, the registry root path
will be /var/run/xenomai
.
$ cat /var/run/xenomai/root/anon@12656/12656/vxworks/tasks/windTask
name = windTask
errno = 0
status = ready
priority = 70
lock_depth = 0
You may override the default root of the registry hierarchy
either statically at build time by passing the desired root
path to the --enable-registry configuration switch, or
dynamically by using the --registry-root
runtime option
passed to the application.
When running over Xenomai/cobalt, the /proc/xenomai interface is
also available for inspecting the core system state. |
- --enable-lores-clock
-
Enables support for low resolution clocks. By default, libraries are built with no support for tick-based timing. If you need such support (e.g. for pSOS ™ or VxWorks ™ APIs), then you can turn it on using this option.
The POSIX API does not support tick-based timing. Alchemy may use it optionally. |
- --enable-clock-monotonic-raw
-
The Xenomai libraries requires a monotonic clock to be available from the underlying POSIX interface. When
CLOCK_MONOTONIC_RAW
is available on your system, you may want to pass this switch, otherwiseCLOCK_MONOTONIC
will be used by default.
The Cobalt core implements CLOCK_MONOTONIC_RAW , so this switch is
turned on by default when building with --with-core=cobalt . On the
contrary, this option is turned off by default when building for the
Mercury core, since we don’t know in advance whether this feature
does exist on the target kernel. |
- --enable-tls
-
Xenomai can use GCC’s thread local storage extension (TLS) to speed up the retrieval of the per-thread information it uses internally. This switch enables TLS, use the converse
--disable-tls
to prevent this.Due to GCC bugs regarding this feature with some release,architecture combinations, whether TLS is turned on by default is a per-architecture decision. Currently, this feature is enabled for x86 and powerpc by default, other architectures will require
--enable-tls
to be passed to the configure script explicitly.Unless
--enable-dlopen-libs
is present, the initial-exec TLS model is selected.When TLS is disabled, POSIX’s thread-specific data management services are used internally (i.e. pthread_set/getspecific()).
- --enable-dlopen-libs
-
This switch allows programs to load Xenomai-based libraries dynamically, using the
dlopen(3)
routine. Enabling dynamic loading introduces some overhead in TLS accesses when enabled (see--enable-tls
), which might be noticeable depending on the architecture.To support dynamic loading when
--enable-tls
is turned on, the global-dynamic TLS model is automatically selected.Dynamic loading of Xenomai-based libraries is disabled by default.
- --enable-async-cancel
-
Enables fully asynchronous cancellation of Xenomai threads created by the real-time APIs, making provision to protect the Xenomai implementation code accordingly.
When disabled, Xenomai assumes that threads may exit due to cancellation requests only when they reach cancellation points (like system calls). Asynchronous cancellation is disabled by default.
Fully asynchronous cancellation can easily lead to resource leakage, silent corruption, safety issues and all sorts of rampant bugs. The only reason to turn this feature on would be aimed at cancelling threads which run significantly long, syscall-less busy loops with no explicit exit condition, which should probably be revisited anyway. |
- --enable-smp
-
Turns on SMP support for Xenomai libraries.
SMP support must be enabled in Xenomai libraries when the client applications are running over a SMP-capable kernel. |
- --disable-sanity
-
Turns off the sanity checks performed at application startup by the Xenomai libraries. This option sets a default, which can later be overridden using the --[no-]sanity options passed to a Copperplate-based Xenomai application. Sanity checks are enabled by default when configuring.
- --enable-fortify
-
Enables
_FORTIFY_SOURCE
when building the Xenomai code unless --enable-debug=full is also given on the command line, in which case --enable-fortify is silently ignored. - --disable-valgrind-client
-
Turns off the Valgrind client support, forcing
CONFIG_XENO_VALGRIND_API
off in the Xenomai configuration header. - --enable-doc-build
-
Causes the inline Xenomai documentation based on the Doxygen markup language to be produced as PDF and HTML documents. Additional documentation like manpages based on the Asciidoc markup language is produced too.
5.2.2. Cobalt-specific configuration options
NAME | DESCRIPTION | DEFAULT |
---|---|---|
|
Use the x86/vsyscall interface for issuing syscalls. If disabled, the legacy 0x80 vector will be used. Turning on this option requires NPTL. |
enabled |
|
Enable ARM TSC emulation.
|
kuser |
|
Enable quirks for specific ARM SOCs Currently sa1100 and xscale3 are supported. |
disabled |
5.2.3. Mercury-specific configuration options
NAME | DESCRIPTION | DEFAULT |
---|---|---|
|
Enable workaround for broken priority inheritance with condition variables in glibc. This option adds some overhead to RTOS API emulators. |
disabled |
[Each option enabled by default can be forcibly disabled by passing --disable-<option>
to the configure script]
5.3. Cross-compilation
In order to cross-compile the Xenomai libraries and programs, you will
need to pass a --host
and --build
option to the configure
script. The --host
option allow to select the architecture for which
the libraries and programs are built. The --build
option allows to
choose the architecture on which the compilation tools are run,
i.e. the system running the configure script.
Since cross-compiling requires specific tools, such tools are
generally prefixed with the host architecture name; for example, a
compiler for the PowerPC architecture may be named
powerpc-linux-gcc
.
When passing --host=powerpc-linux
to configure, it will
automatically use powerpc-linux-
as a prefix to all
compilation tools names and infer the host architecture name from this
prefix. If configure is unable to infer the architecture name from the
cross-compilation tools prefix, you will have to manually pass the
name of all compilation tools using at least the CC and LD, variables
on configure command line.
The easiest way to build a GNU cross-compiler might involve using crosstool-ng, available here.
If you want to avoid to build your own cross compiler, you might if find easier to use the ELDK. It includes the GNU cross development tools, such as the compilers, binutils, gdb, etc., and a number of pre-built target tools and libraries required on the target system. See here for further details.
Some other pre-built toolchains:
6. Examples of building the Xenomai libraries and tools
The examples in following sections use the following conventions:
-
$xenomai_root
-
path to the Xenomai sources
-
$build_root
-
path to a clean build directory
-
$staging_dir
-
path to a directory that will hold the installed file temporarily before they are moved to their final location; when used in a cross-compilation setup, it is usually a NFS mount point from the target’s root directory to the local build host, as a consequence of which running
make{nbsp}DESTDIR=$staging_dir{nbsp}install
on the host immediately updates the target system with the installed programs and libraries.
In the examples below, make sure to add --enable-smp to the
configure script options if building for a SMP-enabled kernel. |
6.1. Building the x86 libraries (32/64bit)
Assuming that you want to build the Mercury libraries natively for a x86_64/SMP system, enabling shared multi-processing support. You would typically run:
$ mkdir $build_root && cd $build_root
$ $xenomai_root/configure --with-core=mercury --enable-smp --enable-pshared
$ make install
Conversely, cross-building the Cobalt libraries from x86_64 with the same feature set, for running on x86_32 could be:
$ mkdir $build_root && cd $build_root
$ $xenomai_root/configure --with-core=cobalt --enable-smp --enable-pshared \
--host=i686-linux CFLAGS="-m32 -O2" LDFLAGS="-m32"
$ make install
After installing the build tree (i.e. using "make install"), the
installation root should be populated with the libraries, programs
and header files you can use to build Xenomai-based real-time
applications. This directory path defaults to /usr/xenomai
.
The remaining examples illustrate how to cross-compile Xenomai for various architectures. Of course, you would have to install the proper cross-compilation toolchain for the target system first.
6.2. Building the PPC32 libraries
A typical cross-compilation setup, in order to build the Cobalt libraries for a ppc-6xx architecture. In that example, we want the debug symbols to be generated for the executable, with no runtime overhead though. We use the DENX ELDK cross-compiler:
$ cd $build_root
$ $xenomai_root/configure --host=powerpc-linux --with-core=cobalt \
--enable-debug=symbols
$ make DESTDIR=$staging_dir install
6.3. Building the ARM libraries
Using codesourcery toolchain named arm-none-linux-gnueabi-gcc
and
compiling for a CSB637 board (AT91RM9200 based), a typical cross-compilation
from a x86_32 desktop would look like:
$ mkdir $build_root/xenomai && cd $build_root/xenomai
$ $xenomai_root/configure CFLAGS="-march=armv4t" LDFLAGS="-march=armv4t" \
--build=i686-pc-linux-gnu --host=arm-none-linux-gnueabi- --with-core=cobalt
$ make DESTDIR=$staging_dir install
Unlike previous releases, Xenomai no longer passes any arm
architecture specific flags, or FPU flags to gcc, so, users are
expected to pass them using the CFLAGS and LDFLAGS variables as
demonstrated above, where the AT91RM9200 is based on the ARM920T core,
implementing the armv4 architecture. The following table summarizes
the CFLAGS and options which were automatically passed in previous
revisions and which now need to be explicitly passed to configure,
for the supported SOCs: |
SOC | CFLAGS | configure options |
---|---|---|
at91rm9200 |
|
|
at91sam9x |
|
|
imx1 |
|
|
imx21 |
|
|
imx31 |
|
|
imx51/imx53 |
|
|
imx6q |
|
|
ixp4xx |
|
|
omap3 |
|
|
omap4 |
|
|
orion |
|
|
pxa |
|
|
pxa3xx |
|
|
s3c24xx |
|
|
sa1100 |
|
|
It is possible to build for an older architecture version (v6 instead of v7, or v4 instead of v5), if your toolchain does not support the target architecture, the only restriction being that if SMP is enabled, the architecture should not be less than v6.
7. Testing the installation
7.1. Booting the Cobalt kernel
In order to test the Xenomai installation over Cobalt, you should first try to boot the patched kernel. Check the kernel boot log for messages like these:
$ dmesg | grep -i xenomai
I-pipe: head domain Xenomai registered.
[Xenomai] Cobalt vX.Y.Z enabled
If the kernel fails booting, or the log messages indicates an error status instead, see the TROUBLESHOOTING guide.
7.2. Testing the real-time system (both cores)
First, run the latency test:
$ /usr/xenomai/bin/latency
The latency test should display a message every second with minimum, maximum and average latency values. If this test displays an error message, hangs, or displays unexpected values, see the TROUBLESHOOTING guide.
If the latency test succeeds, you should try next to run the
xeno-test
test in order to assess the worst-case latency of your
system. Try:
$ xeno-test --help
7.3. Calibrating the Cobalt core timer
The accuracy of the Cobalt timing services depends on proper calibration of its core timer. Sound factory-default calibration values are defined for each platform Xenomai supports, but it is recommended to calibrate the core timer specifically for the target system.
See the documentation about the autotune(1) utility.
8. Building and running Xenomai 3 applications
Once the latency test behaves as expected on your target system, it is deemed ready to run real-time applications.
You may want to have a look at this document for details about the application build process.
In addition, you may refer to this document to learn about the command line options available with Xenomai 3 applications.
9. Migrating applications to Xenomai 3
If you plan to port an existing application based on Xenomai 2.x to Xenomai 3.x, you should have a look at this migration guide.