//Life with Adeos
PhilippeGerum
RevisionB
Copyright? 2005
Copyright ? 2005Philippe Gerum
Permission is granted to copy, distribute and/or modify thisdocument under the terms of the GNU Free Documentation License, Version 1.2 orany later version published by the Free Software Foundation; with no InvariantSections, no FrontCover Texts,
and no Back-Cover Texts. A copy of the licenseis published on gnu.org: "GNU FreeDocumentation License" [http://www.gnu.org/licenses/fdl.html].
10/09/2005
Abstract
This article gives an overview of the Adeos technology and what itprovides to Xenomai. After an introduction addressing the event pipeline, theimplementation of interrupts dispatching as well as the system eventpropagation, we will then study
the interactions between Xenomai and Adeos. Thenotions of primary and secondary domains will be explained and we will also beinterested in studying how interrupts get propagated. We will finally concludeon a few tips and tricks to disable/enable interrupts
sources and to shareinterrupts between domains.
Table of Contents
1. What is Adeos..............................................................................................2
1.1. The event pipeline..............................................................................2
1.2. Optimistic interrupt protection.............................................................. 4
1.3. System-event propagation.................................................................... 6
2. What does Adeos provide toXenomai? ............................................................. 6
2.1. Xenomai‘s primary and secondarydomains .............................................. 7
2.2. System call interception..................................................................... 10
2.3. Interrupt propagation......................................................................... 11
3. Tips and tricks............................................................................................11
3.1. Enabling/Disabling interruptsources ..................................................... 11
3.2. Sharing interrupts betweendomains ..................................................... 12
3.3. Interrupt sharing and latency............................................................... 13
4. Conclusion.................................................................................................14
5. Useful links................................................................................................14
1. What is Adeos
Adeos is a resource virtualization layer available as a Linux kernelpatch, which general design has been proposed(提议) by
Karim Yaghmour in a technical paper,back in 2001.
The current incarnation of this proposal makes it a simple, yetefficient real-time system enabler, providing a mean to run a regular GNU/Linuxenvironment and a RTOS, side-by side on the same hardware.
这个提议提供一个一种方法,使平常的GNU/Linux环境或者一个RTOS下运行在同样的硬件环境下,变得简单,而是一个有效的实时系统的使能者。
To this end, Adeos enables multiple entities — called domains — toexist simultaneously on the same machine. These domains do not necessarily seeeach other, but all of them see Adeos. A domain is most probably a complete OS,but there is no assumption
being made regarding the sophistication of what isin a domain. However, all domains are likely to compete for processing externalevents (e.g. interrupts) or internal ones (e.g. traps, exceptions), accordingto the system-wide priority they have been given.
为了达到这个目的,Adeos以被称作domains的多实例,同时运行在同一台机器上。这些domains不需要看到彼此,但是可以都看到Adeos。一个domain可以是一个完整的OS,但是并没有假定一种复杂的存在。但是,按照给定的系统范围内的优先级,所有的domains都可以竞争处理器外部的事件(中断),或者内部的traps或者异常。
Additionally to its straightforward virtualization capabilities,another key advantage of Adeos relies in its ability to export a generic API toclient domains, which does not depend on the CPU architecture. Therefore, muchif not most of the porting
effort for client domains occurs at the Adeos level.
除了本身简单的虚拟化的兼容性,Adeos的其他优势在于能够依赖自己的能力输出一组通用的API给客户端的domains。这并不依赖于CPU架构。因此,不会有很多的移植修改发生在Adeos level上。
1.1. The event pipeline
The fundamental Adeos structure one must keep in mind is the chainof client domains asking for event control. A domain is a kernel-based softwarecomponent which can ask the Adeos layer to be notified of:
Adeos结构的基础是维持一个客户端domains的链表来请求事件控制。一个domain就是一个内核基本的软件成员,可以请求adeos层通知以下信号:
? Every incoming externalinterrupt, or auto-generated virtual interrupt;
? 每一次的外部中断,或者自产生的虚拟中断
? Every system call issued byLinux applications,
? 每一次由linux user端发起的系统调用
? Other system events triggeredby the kernel code (e.g. Linux task switching, signal notification, Linux taskexits etc.)
? 内核代码产生的触发事件(比如任务切换,信号通知,任务退出等)
Adeosensures that events are dispatched in an orderly manner to the various clientdomains according to their respective priority in the system, so it is possibleto provide for timely and predictable delivery of such events. This is achievedby
assigning each domain a static priority. This priority value strictlydefines the delivery order of events to the domains. All active domains arequeued according to their respective priority, forming the pipeline abstractionused by Adeos to make the events
flow, from the highest to the lowest prioritydomain. Incoming events (including interrupts) are pushed to the head of thepipeline (i.e. to the highest priority domain) and progress down to its tail(i.e. to the lowest priority domain). The figure below gives
a general view ofsome Adeos-based system, where multiple domains are sharing events through thepipeline abstraction:
Adeos可以保证,将事件会以一种循序的方式分发给不同的客户端domains,遵循各自的系统优先级。所以周期性的和预测性的传递事件是可能的。通过分配给每个domain一个静态优先级,可以达到这个目的。优先级的值严格于事件传递的顺序相关。所有激活的domains按照优先级排队,形成了一种pipeline抽象。Adeos利用这种抽象产生事件流,从最高优先级到最低优先级传递。到来的事件被传递到pipeline链表头(最高优先级),依次传递到链表尾(最低优先级)。下面的图标给出了一个adeos基础的系统的一般视图,通过pipeline抽象,多domains可以共享事件。
Event pipeline representation
In the figure above, the position of the Linux kernel with respectto the pipelining order might be anywhere; this said, the Linux kernel stillhas a special role since it stands for the root domain, because all otherdomains need Linux to install
them, usually by mean of loading the kernelmodules which embody them.
上图图标中,linuxkernel的位置不固定;也就是说,尽管替代了root domain,由于其他的domain需要linux来安装自己,kernel仍然扮演一种特殊的角色,通常这意味着通过以linux module的方式来体现自己。
1.2. Optimistic interrupt protection
In order to dispatch interrupts in an prioritized manner, whilestill allowing domains to create interrupt-free sections, Adeos implements theso-called Optimistic interrupt protection scheme as described by Stodolsky,Chen, and Bershad, in the
following paper. [http://citeseer.nj.nec.com/stodolsky93fast.html]
为了实现中断以优先级方式的分发,需要允许domain去常见自由中断,adeos实现了所谓的优化中断保护调度。
The stage of the pipeline occupied by any given domain can be stalled,which means that the next incoming interrupt will not be delivered to thedomain‘s handler, and will be prevented from flowing down to the lowestpriority domain(s) in the same
move. While a stage is stalled, pendinginterrupts accumulate in the domain‘s interrupt log, and eventually get playedwhen the stage is unstalled, by an internal operation called synchronization.Domains use this feature to protect their own critical sections
from unwantedpreemption by their own interrupt handlers. However, thanks to thevirtualization of the interrupt control Adeos brings, a higher priority domaincan still receive interrupts, and eventually preempt any lower priority domain.Practically, this means
that, albeit an Adeos-enabled Linux kernel regularlystalls its own stage to perform critical operations, a real-time system runningahead of it in the pipeline would still be able to receive interrupts any time,with no incurred delay.
Pipeline的平台可以被任何给定的domain占有,也就是说下一个到来的中断将不会传递给domain的handler,同时阻止它传递到低优先级的domain。当平台被安装时,通过内部同步动作将中断保存在domain中断日志,卸载时被执行。Domain利用这种方式来保护自己的临界部分,防止被自己的中断handler抢占。无论怎样,感谢中断控制adeos的虚拟机的诞生,实现了一个更高优先级的domain仍然可以接收中断,同时抢占任何其他低优先级的domain。一个使能adeos的linux
kernel 安装自己的平台实现临界操作,在pipeline的实时系统跑在前面,可以接收任意时刻的中断,而不遭受延迟。
When a domain has finished processing all the pending interrupts ithas received, it then calls a special Adeos service which yields the CPU to thenext domain down the pipeline, so the latter can process in turn the pendingevents it has been
notified of, and this cycle continues down to the lowestpriority domain of the pipeline.
当domain已经完成了所有的悬挂中断处理,然后调用一个特殊的adeos服务,释放cpu占有给下一个domain,反过来之后可以处理一个被通知的悬挂事件,同时这个循环可以继续到最低优先级的domain。
Thefigure below illustrates how several domains running on mutiple CPUs can sharethe incoming interrupts through the Adeos pipeline abstraction. Of course, acorrect bookkeeping of pending interrupts must be done while a stage isstalled: this
is achieved by a perdomain, per-CPU interrupt log, as illustratedbelow:
下表展现了,当一个共享中断到来时,通过adeos pipeline如何在一个多cpu运行多domain的实体上的实现通信。当然,当一个平台被安装时,一个正确的悬挂终端的日志已经完成了。
The optimisticinterrupt protection scheme
1.3. System-event propagation
Interrupts are not the only kind of events which can flow throughthe pipeline abstraction; internal events triggered by the Linux kernel itselfor its applications can generate what is called system events. Basically,system events are synchronous
notifications of trap, exception, or some actionperformed by the Linux kernel, and which gets notified to any interestedparties through the pipeline.
事件不仅仅是包括中断,同样也指其他的pipeline抽象。由linux kernel发出的内部触发事件或者 应用产生的系统调用。基本来说,系统事件就是指同步通知的陷阱和异常,或者是一些通过pipeline发出内核的动作通知给伙伴domain。
Since those events are synchronous by essence, there is no way todefer their notification through the stall operation, just because you cannotdelay their handling. The rationale behind such design decision comes from thefact that a code triggering
a system event might just not be able to continuewithout immediate intervention from the corresponding handler: e.g. the pagefault handler must run immediately upon memory addressing exception, delayingit makes no sense. In other words, stall/unstall operations
on any given domainonly concern interrupts, either real or virtual.
本质上说,由于事件都是同步的,也就没有办法通过安装的操作,来延迟通知,由于你也不能延迟动作的处理。如此原理设计的目的背后,带来这样一种事实,系统的代码触发事件,在没有即时相应handler处理时,是不能继续操作的。内存地址访问异常导致的页错误,必须立即执行,延迟执行毫无意义。换句话说,在任何给定的domain进行安装和卸载操作时,仅仅关心中断,或者真实的或者虚拟的。
2. What does Adeos provide toXenomai?
Conversely, this question could also be: what basic guarantees doesXenomai need to provide real-time services? The answer is simple andstraightforward: it must be allowed to handle all incoming interrupts first,before the Linux kernel has had
the opportunity to notice them, and it must beable to handle them immediately, regardless of any current attempt from theLinux kernel to lock them out using the CPU interrupt mask. It must also makesure to always enforce the proper priority management for
its threads,regardless of their current execution domain.
相反的,这个问题可以是:xenomai需要提供给实时服务的基本保证是什么?答案是简单且直观的:在内核有机会通知他们之前,它必须允许去处理所有的到来的中断,同时它必须能够立即执行,而无论当前是谁是否利用cpu的中断屏蔽字,试图从内核去获取内核锁。也必须保证对于自己的线程总是可以调整合适的优先级管理,而不管当前执行的domain。
Theseguarantees give Xenomai predictable interrupt latencies in the lowestmicro-second level range whatever activity Linux is undergoing, which oncecoupled with Xenomai‘s fast co-scheduling technique of Linux tasks (i.e. shadowthreading), provide
for deterministic scheduling latencies of real-timethreads. The figure below illustrates the position of the Adeos layer in theXenomai architecture:
这些保证给了xenomai可以预测中断以微秒级的中断延迟,而无论linux是否同意,基于伴随linux任务的xenomai的快速协任务技术提供了确定性的实时线程的调度延迟。下表解释了adeos的在xenomai架构中的位置
Position of Adeos in the Xenomai architecture
Youwill notice that the Adeos interface is directly exposed to the HardwareAbstraction Layer which underlies the Xenomai core. Therefore, most of therequests for Adeos services are issued from the HAL, which implementation canbe found in the
relevant arch/<archname>/haldirectories, the generic bits being available under arch/generic/ hal. Having a look at the latter is the best approach to understand howXenomai makes use of Adeos for its own purposes.
注意到,adeos的接口直接链接到HAL。因此大部分的请求服务都会通过HAL,实现的相关代码在arch/archname/hal文件夹下。一般的文件在arch/generic/hal。后边的最好的方法理解xenomain如何利用adeos基于自己的目的。
2.1. Xenomai‘s primary and secondarydomains
Xenomai allows to run real-time threads either strictly in kernelspace, or within the address space of a Linux process. In the rest of thisarticle, we will refer to the latter as the Xenomai threads, not to be confusedwith regular Linux tasks
(even when they belong to the SCHED_FIFO class). All threads managed by Xenomai are known from the real-timenucleus.
Xenomain允许允许实时线程在内核控件或者在进程的地址空间。剩下的部分,我们将参照后者作为xenomai线程,(此处不要与通常的linux任务混淆)。所有的线程由xenomai中的实时内核nucleus管理。
Support for real-time threads exclusively running in kernel space isa reminiscence of the mere co-kernel era, before the advent of true real-timesupport in user-space, when real-time applications would only run embodied intokernel modules; this
feature has been kept in Xenomai mainly for the purpose ofsupporting legacy applications, and won‘t be discussed here.
仅仅支持linux内核控件实时线程是在协内核的年代,在这之前,用户空间的实时支持还未出现。当时,实时应用只能以内核模块的方式呈现。这种特点保持在xenomai里支持老的app,此处不做讨论。
What is more interesting is that Xenomai has a symbiotic approachwith respect to Linux; this is, for instance, what makes it different from theRTAI/LXRT implementation. To this end, Xenomai threads are not only able to runover the context of
the highest priority domain in the pipeline (i.e. theprimary domain) like kernel-based Xenomai threads, but also in the regularLinux space (i.e. the secondary domain), while still being considered asreal-time by Xenomai, albeit suffering higher scheduling
latencies. InXenomai‘s jargon, the former are said to run in primary execution mode, whilstthe latter undergo the secondary execution mode.
更感兴趣的是xenomai在linux中存在一个公用的方法。比如,他不同于RTAI/LXRT的实现方式。基于这个目的,xenomai线程不仅能运行在高优先级的上下文切换比如内核xenomai线程。同样,作为xenomai的实时考虑,虽然承受了更高的调度延迟,在传统的linux空间也一样支持。按照xenomai的专业术语来说,前者是运行主执行模式,后边运行在次执行模式。
In order to provide a complete real-time support to threads runningin the secondary domain, Xenomai needs the following to be achieved:
为了提供一种完整的实时支持在次模式domain中,xenomai需要实现以下部分:
Common priority scheme.Scheduling-wise, we need a way to have both the real- time nucleus and theLinux kernel share the same priority scheme with respect to the set of threadsthey share the control of; in other words, a Xenomai thread should
have itspriority properly enforced at any time, regardless of its current domain, amongall existing Xenomai threads. Xenomai applies what it calls the root thread‘smutable priority technique, by which the Linux kernel automatically inheritsthe priority of
the Xenomai thread controlled by the real-time nucleus whichhappens to enter the secondary domain. Practically, this means that Xenomaithreads currently running in the primary domain won‘t necessarily preempt thoserunning in the secondary one, unless their
effective priority is actuallyhigher. The above behaviour is to be opposed to what happens with RTAI/LXRT forinstance, where threads migrating to the Linux space actually lose theirreal-time priority in the same move, by inheriting the lowest priority definedby
the RTAI scheduler. This said, regular Linux tasks unknown to Xenomai, andwhich only happen to belong to the SCHED_FIFO class, will always be preemptedwhen competing for the CPU with Xenomai threads from the primary Xenomaidomain, albeit they will still compete
priority-wise with Xenomai threadsrunning in the secondary domain.
相同的优先级调度。调度方面,我们需要一种方式,在实时微内核和linux内核共享相同的优先级调度,共享相关的线程集合的控制;换句话说,xenomai线程应该在任意时刻,在所有的现成的xenomai线程中间,具有合适的优先级去执行,而无论当前的domain是什么。Xenomai应用在它调用root线程的不变的优先级,通过这样linux内核自动的继承xenomai线程的优先级,通过实时内核的控制进入到次domain。实际上,这意味着xenomai线程当前运行在主domain,没有必要抢占在运行在次domain的线程,除非优先级确实比这个更高。例如,以上的行为在RTAI/LXRT是相反的,此处线程同样的移动到内核空间会丢失他们的实时优先级,这仅仅发生在隶属于SCHED_FIFO类中,当与cpu与xenomai线程从主xenomai
domain中竞争时总是被抢占的,在secondary domain竞争也是存在。
Predictability of program executiontimes. When a Xenomai thread runs over the Linux ( i.e. secondary) domain,either executing kernel or application code, its timing should not beperturbated by non real-time Linux interrupt activities, and generallyspeaking,
by any, low priority, asynchronous activity occurring at kernellevel. A simple way to prevent most opportunities for the latter to happen isto starve the Linux kernel from interrupts when a Xenomai thread is running inthe Linux domain, so that no deferred
post-processing could be triggered fromtop-halves interrupt handlers during this period of time. A simple way tostarve the Linux kernel from interrupts is to block them when needed inside anintermediate Adeos domain, sitting between those occupied by the real-timenucleus
and the Linux kernel, which is called the interrupt shield in Xenomai‘sjargon. This shield is engaged whenever a Xenomai thread is scheduled in by theLinux kernel, and disengaged in all other cases. It should be noted that theshielding support can be enabled/disabled
on a per-thread basis, or on asystem-wide basis at Xenomai build time; by default, it is disabled for Xenomaithreads and not built in.
程序运行时间的可预测性。当xenomai线程运行在linux domain下,执行内核代码或者应用代码,他的时序不应该被非实时的linux中断活动、一般的低优先级、异步的发生在内核级的活动所打断。为了降低发生的概率,一个简单的方式就是,当xenomai线程运行在linux domain时,让linux kernel等待,在这段期间,从顶层到来的中断处理会没有延迟的被触发。让linux 内核等待的一种简单方式,即当内部媒介adeos domain被需要时,在被实时微内核和linux
内核占用之间保持将其锁住,即中断屏蔽。xenomai线程被linux 内核调度时中断屏蔽,其他情况下中断打开。注意到,掩码可以是使能或者失能在一个单线程的基本,或者在xenomai 建立时的系统范围内的基本;默认的,在线程中和没有建立时是失能的。
Fine-grained Linux kernel. In orderto get the best from the secondary execution mode, we need the Linux kernel toexhibit the shortest possible non-preemptible section, so that reschedulingopportunities are taken as soon as possible after a Xenomai
thread running inthe secondary domain becomes ready-to-run. Additionally, this ensures thatXenomai threads can migrate from the primary to the secondary domain within ashort and time bounded period of time, since this operation involves reaching akernel rescheduling
point. For this reason, Xenomai benefits from thecontinuous trend of improvements regarding the overall preemptibility of theLinux kernel, including Ingo Molnar‘s PREEMPT_RT extension. Of course, Xenomaithreads which only run in the primary domain are not
impacted by the level ofgranularity of the Linux kernel, and always benefit from ultralow and boundedlatencies, since they do not need to synchonize in any way with the undergoingLinux operations, which they actually always preempt unconditionally.
更细粒度的linux 内核。
Priority inversion management. Boththe real-time nucleus and the Linux kernel should handle the case where a highpriority thread is kept from running because a low priority one holds acontended resource for a possibly unbounded amount of time.
Xenomai providesthis support, but only the PREEMPT_RT variant does so for the Linux kernel. Forthis reason, Xenomai keeps an eye and provides support for the currentdevelopments of PREEMPT_RT, albeit the mainline kernel still remains the systemof reference
for now.
优先级反转管理。
As a consequence of the requirements above, when the Xenomai core isloaded, the underlying Adeos pipeline contains three stages, through which allinterrupts are flowing, by order of priority:
基于以上的描述,当xenomai核心被加载,潜在的adeospipeline 包含3种平台,通过它所有的中断流向的顺序如下:
? Xenomai‘s primary domain, whichis the home of the real-time nucleus;
? Xenomai主domain,是实时内核的home
? The interrupt shield domain;
? 中断掩码的domain
? The Linux domain.
? Linux 的domain
The three stages of the Adeos pipeline
2.2. System call interception(拦截)
Since real-time APIs (i.e. skins) which arestackable over the Xenomai nucleus, can export their own set of services toXenomai threads in user-space, there must be a way to properly dispatch thecorresponding system calls, and the regular Linux kernel system
callsaltogether, to the proper handlers. To this end, Xenomai intercepts everysystem call trap/exception issued on behalf of any of the Xenomai or Linuxdomains by the Xenomai threads. This is made possible by subscribing an eventhandler using the proper Adeos
service (for more on this, see the adeos_catch_event() service, whenspecifying the
ADEOS_SYSCALL_PROLOGUE/IPIPE_EVENT_SYSCALL event).
由于实时API(皮肤)是附着在xenomai内核之上,在用户空间,它可以输出一系列的服务集合以xenomai线程的方式。必须存在一种合适的方式分发相应的系统调用,包括传统的内核系统调用一起被合适的处理。基于这个目的,xenomai线程发出的每一个系统调用的陷阱和异常,代表了xenomai或者linux domain,都将会被拦截。通过预定一个事件处理函数,利用一个合适的adeos服务,拦截将成为可能。更多细节参考adeos_catch_event()服务,具体表征的是ADEOS_SYSCALL_PROLOGUE/IPIPE+EVENT_SYSCALL事件。
Xenomai uses this capability to:
? Dispatch the real-time servicesrequests from applications to the proper system call handlers, which areimplemented by the various APIs running over the real-time nucleus;
? 分发应用的请求的实时服务给合适系统调用处理者。这可以通过运行在实时微内核的不同的API来实现。
? Ensure that every system callis performed under the control of the proper domain, either Xenomai or Linux,by migrating the caller seamlessly to the target domain as required. Forinstance, a Linux system call issued from a Xenomai thread running
in theXenomai domain will cause the automatic migration of the caller to the Linuxdomain, before the request is relayed to the regular Linux system call handler.Conversely, a Xenomai thread which invokes a possibly blocking Xenomai systemcall will be moved
to the Xenomai domain before the service is eventuallyperformed, so that the caller may sleep under the control of the real-timenucleus.
? 保证每一个系统调用都能够在合适的domain控制下,或者xenomai 或者linux中,按照需求移植调用到目标domain。例如,运行在xenomai domain的xenomai线程发布一个linux系统调用将会自动的发生迁徙到linux的domain,动作在转发到传统的linux系统调用处理者之前已经完成。相反的,一个xenomai线程,涉及到可能锁住xenomai系统调用,在最后实现之前,将会被移动到xenomai domain,所以调用者可能会在实时内核的控制下进行休眠。
The combinationof both makes Xenomai threads particularly well integrated into the Linuxrealm. For instance, a common system call path for Xenomai and the regularLinux applications makes the former appear as a natural extension of thelatter. As an illustration
of this, Xenomai threads both support the full Linuxsignals semantics and ptracing feature, which in turn enables the GDB supportnatively for them.
两种的方式的集合,使xenomai线程良好的集成到linux中。例如,针对xenomai和传统linux应用的一般的系统调用,使前者作为后者的一种自然扩展。基于这种解释,xenomai线程同时支持全部的linux信号语义和追踪特点,反过来说即使能GDB即能原始的支持他们。
2.3. Interrupt propagation
Because it isahead in the pipeline, the real-time nucleus which lives in the Xenomai domainis first notified of any incoming interrupt of interest, processes it, thenmarks such interrupt to be passed down the pipeline, eventually to the Linuxkernel domain,
if needed.
由于pipeline中的领头羊位置,实时微内核作为xenomai domain的第一个被通知的实体,第一时间接收中断并处理,然后标记中断是否传递给下一个pipeline,如果需要的话也会传递给linux kernel domain。
When notifiedfrom an incoming interrupt, the real-time nucleus reschedules after the outerinterrupt handler has returned (in case interrupts are piling up), and switchesin the highest priority runnable thread it controls.
The Xenomaidomain yields the CPU to the interrupt shield domain when no real-time activityis pending, which in turn let them through whenever it is disengaged to theLinux kernel, or block them if engaged. Adeos has two basic propagation modesfor interrupts
through the pipeline:
当接收到中断通知后,在外部中断处理返回后,实时微内核被重新调度,切换到一个最高的优先级线程中。当没有实时活动被悬挂时,Xenomai domain掩码cpu到interrupt domain,反过来即让他们注册时锁住,卸载时恢复。通过pipeline针对中断,Adeos有两种基本的分发模式。
? In the implicit mode, anyincoming interrupt is automatically marked as pending by Adeos into each andevery receiving domain‘s log accepting the interrupt source.
? 隐含模式,任何一个到来的中断都自动被标记为悬挂,收进每一个domain的接收中断源的日志log中。
? In the explicit mode, aninterrupt must be propagated "manually" if needed by the interrupthandler to the neighbour domain down the pipeline.
? 显性模式,如果需要中断处理函数传递给相邻的domain即更低的pipeline,一个中断必须被标记为手动。
This setting isdefined on a per-domain, per-interrupt basis. Xenomai always uses the explicitmode for all interrupts it intercepts. This means that each handler must callthe explicit propagation service to pass an incoming interrupt down thepipeline. When
no handler is defined for a given interrupt by Xenomai, theinterrupt is unconditionally propagated down to the Linux kernel: this keepsthe system working when the real-time nucleus does not intercept suchinterrupt.
这种设置为定义单domain,单中断基础。Xenomai总是利用显性模式针对所有的拦截的中断。这就意味着,每一个处理函数必须调用显式的分发服务,传递一个到来的中断到低pipeline。对于一个给定的中断,没有中断处理函数时,中断会无条件的传递给linux 内核。当实时微内核不能拦截如此的中断时,仍然能保持系统稳定的工作。
3. Tips and tricks
3.1. Enabling/Disabling interruptsources
In addition tobeing able to stall a domain entirely so that no interrupt could flow throughit anymore until it is explicitly unstalled, Adeos allows to selectivelydisable, and conversely re-enable, the actual source of interrupts, at hardwarelevel.
除了能够安装完整的domain,以至于没有中断能够传递给任何人,直到它显式的被卸载硬件层面的实际的中断源,adeos允许去选择性的使能和重新使能。
After havingtaken over the box, Adeos handles the interrupt disabling requests for alldomains, including for the Linux kernel and the real-time nucleus. This meansdisabling the interrupt source at the hardware PIC level, and locking out any interruptdelivery
from this source to the current domain at the pipeline level.Conversely, enabling interrupts means reactivating the interrupt source at thePIC level, and allowing further delivery from this source to the currentdomain. Therefore, a domain enabling an interrupt
source must be the same asthe one which disabled it, because such operation is domain-dependent.
在接管了box之后,adeos处理了所有的domain的中断失能请求,包括linux内核和实时内核。这就意味着,失能硬件层面的中断源和锁住pipeline层面的中断传递。相反的,使能中断重新激活中断源,允许中断源的传递。因此,一个domain使能的中断源和另外一个失能中断源的调用者是一致的,因为这样的动作都是domain依赖的。
Practically, thismeans that, when used in pair, the rthal_irq_disable() and rthal_irq_enable() services which encapsulate the relevant Adeos calls inside thereal-time HAL underlying Xenomai, must be issued from the same Adeos domain.For instance, if a real-time
interrupt handler connected to some interruptsource using the rthal_irq_request() service, disables the source using rthal_irq_disable(), then suchsource will be blocked for the Xenomai domain until rthal_irq_enable() is called for the same interrupt, and
from the same domain. Failingto deal with this requirement usually leads to the permanent loss of theaffected interrupt channel.
实际上,这就意味着,成成对的调用rthal_irq_disable和rthal_irq_enable服务必须是从同一个adeos domain发出的。例如,如果一个事实处理函数链接到了中断源,通过接口rthal_irq_request,通过rthal_irq_disable失能,然后关乎这个中断源的所有中断将会被阻塞,除非该domain调用rthal_irq_enable。请求失败会导致永久丢失相关的中断通道。
3.2. Sharing interrupts betweendomains
A typical example of mis-using the Adeospipeline when sharing hardware interrupts between domains is as follows:
voidrealtime_eth_handler (unsigned irq, void *cookie) {
/*
* Thisinterrupt handler has been installed using
* rthal_irq_request(),so it will always be invoked on behalf o * the Xenomai (primary) domain. */
rthal_irq_disable(irq); /* The Xenomai domain won‘t receive thisirq anymore */ rthal_irq_host_pend(irq); /*This irq has been marked as pending for Linux */ }
voidlinux_eth_handler (int irq, void *dev_id, struct pt_regs *regs) {
/*
* Thisinterrupt handler has been installed using
* rthal_irq_host_request(),so it will always be invoked on
* behalf of the Linux (secondary) domain,as a shared interrupt
* handler(Linux-wise). */ rthal_irq_enable(irq);
/*
* BUG:This won‘t work as expected: we are only unlocking the
* interruptsource for the Linux domain which is current here, * not for the Xenomai domain!
*/
In thenon-working example above, since Xenomai always uses the explicit propagationmode for all interrupts it intercepts, the next ethernet interrupt will bemarked as pending in the Xenomai log only, waiting for the Xenomai handler topossibly propagate it
manually down to Linux. But since the interrupt is stilllocked at pipeline level for Xenomai (remember that nobody actually issued theexpected rthal_irq_enable() from the Xenomaidomain, but only mistakenly from the Linux one), this won‘t happen, because theXenomai
handler won‘t run until the lock is removed. Therefore, well,chickenandegg problem: we are toast.
Fortunately,there is a solution for sharing interrupts properly, between domains which needto keep the interrupt source disabled until the final processing is done (e.g.dealing with level-triggered interrupts is one of those issues): actually, youdon‘t need
to do anything, because Adeos already masks any incoming interruptsat PIC level before feeding the pipeline with it. Therefore, you only need toprocess the interrupt as you see fit in the relevant domain handler, and makesure to re-enable the interrupt source
from the last one using rthal_irq_enable(). Whenever the Linux kernel is one of those recipients, the regularkernel handler will do this re-enabling automatically, so basically, you justneed to bother calling rthal_irq_enable() in handlers which don‘t propagate
the incoming interruptsdownstream to the Linux kernel.
Specifically onthe x86 architecture, it happens that the timer interrupt is not being maskedupon receipt by Adeos, for performances reasons. This said, the timer source isnot one you may want to disable in any way, so this is a nonissue.
3.3. Interrupt sharing and latency
However, keepingan interrupt source masked while the propagation takes place through the entirepipeline may increase the latency.
但是,通过整体的pipeline当分发发生时,保持中断源的屏蔽可能会增加延迟。
Since Adeosguarantees that no stack overflow can occur due to interrupts piling up overany given domain, and because it also stalls the current stage before firing aninterrupt handler, there is no need to disable the interrupt source in theXenomai handler.
Instead you may even want to re-enable it, so that furtheroccurrences can be immediately logged, and will get played immediately afterthe current handler invocation returns.
尽管adeos保证不会发生栈溢出,由于中断发生在任何给定的domain,同时由于它也会在获取中断处理之前安装到当前的平台,没有必要去失能中断源。而不是你可能再使能一遍,以至于更远的事件能被立即记录,在当前处理返回之后立即被执行。
So, the solutionis to re-write the previous example this way, this time properly:
所以,解决方法是,重新写入之前的例子,这次合适的:
voidrealtime_eth_handler (unsigned irq, void *cookie) {
rthal_irq_enable(irq);
rthal_irq_host_pend(irq);
/*This irq has been marked as pending for Linux */
}
voidlinux_eth_handler (int irq, void *dev_id, struct pt_regs *regs) {
/* process the IRQ normally. */
}
4. Conclusion
Adeos is a rathersimple piece of code, with very interesting properties when used properly. Thebackbone of the Adeos scheme is the event pipeline, and as such, it brings allthe critical features we need in Xenomai:
Adeos不仅仅是简单的几行代码,伴随着非常有趣的属性。Adeos的核心就是事件驱动的pipeline,它创造了所有xenomai需要的重要的特点。
? Predictable interruptlatencies;
? 可预测性的中断延迟
? Precise interruptvirtualization control (per-domain and per-interrupt handler registration,per-domain and per-cpu interrupt masking);
? 精确的中断虚拟控制(单域和单中断处理注册,单域和单cpu的中断屏蔽)
? Uniform, prioritized anddomain-oriented propagation scheme for events;
? 统一的、优先级的和域导向的事件分发调度机制
? A generic and straightforwardAPI to ease portability of client code.
? 通用简单的API方便客户端代码的移植。
Xenomai usesthese features to seek the best possible integration of the real-time servicesit brings with the Linux kernel. Xenomai‘s primary mode delivers true real-timeperformances in the lowest micro-second latency range. Additionally, Xenomaibets on future
evolutions of Linux, like PREEMPT_RT, to improve the kernel‘soverall granularity, so that the secondary mode will still be real-time in thedeterministic sense, with bounded albeit higher worst-case latencies. This isthe reason why Xenomai is working hard since
day one to reach a tightintegration level with the Linux kernel. Think symbiotic, seek mutualism.
5. Useful links
? Karim Yaghmour‘s Adeosproposal: http://www.opersys.com/ftp/pub/Adeos/adeos.pdf
? Optimistic InterruptProtection: http://citeseer.nj.nec.com/stodolsky93fast.html
? The Adeos project workspace onGNA: https://gna.org/projects/adeos/
? Where to download the latestAdeos patches: http://download.gna.org/adeos/patches/
? The Adeos API reference manual:http://home.gna.org/adeos/doc/api/globals.html