Subject: tasklet latency and system calls on mips

From: "Sirotkin, Alexander" <[email-masked]>
Date: Wed, 13 Aug 2003 16:46:04 +0300

Hello dearest all.

I have a question regarding tasklets on MIPS. I suspect that there is a
bug in generic MIPS kernel, but I'm not sure yet.

Linux kernel has a couple of so called "checkpoints" when the system
should check if there are tasklets to
run and run them in the following way :

if (softirq_pending(cpu))
                    do_softirq();

One of these places is at the end of interrupt handler (do_IRQ()),
however this is not the only place. I was under
impression that this code should be called after system call too. The
caveat here is that on MIPS (contrary to
other architectures, such as x86) system call is not an interrupt (it's
a different exception) and has completely
different handler. So in x86 it is sufficient to call

if (softirq_pending(cpu))
                    do_softirq();

at the end of do_IRQ because do_IRQ handles system call too, but on MIPS
it is not. Therefore I believe
these lines should be added to the end of sys_syscall function on MIPS.

What do you think ?

P.S. The whole issue started when we noticed that user process making
many system calls has very
significant impact on device drivers running in tasklet mode and no
impact whatsoever on device
drivers running in interrupt mode.

---

From: Jun Sun <[email-masked]>
Date: Wed, 13 Aug 2003 09:54:46 -0700

> these lines should be added to the end of sys_syscall function on MIPS.

softirq/tasklet/bottom_half/etc should only be raised from interrupt
context.  Checking at the end of do_IRQ should be good enough.

One possible mistake in MIPS porting is that if the board uses its private
time interrupt routine poeple may forget to put the above two lines
at the end.  Check against that.

> significant impact on device drivers running in tasklet mode 

What kind of impact?  On i386? Or on MIPS?

Jun

---

From: "Sirotkin, Alexander" <[email-masked]>
Date: Thu, 14 Aug 2003 12:17:52 +0300

Hi.

See my comments below.

Jun Sun wrote:

Hello dearest all.

                    do_softirq();

                    do_softirq();

What do you think ?

On mips interrupt is an exception and system call is a different
exception. Different exceptions has different exception handlers,
at least that's what I was able to figure from entry.S file. So the
system call does not go through do_IRQ and do_softirq
is not called.

One possible mistake in MIPS porting is that if the board uses its
private

In our kernel port we do have these lines in the timer interrupt, that
is not a problem. 

  

Jun

The impact is that if our driver works in tasklet mode then some user
mode application making system calls
causes some (although quite small) packet loss. It does not happen if we
don't use tasklet and do everything in the
ISR.

I suspect that what happens is as follows :

system call arrives and while it's being processed and interrupt to one
of the drivers arrives. This interrupt 
schedules a tasklet which however is not executed after the system call
finishes, only after the next timer
interrupt which causes up to 10 ms latency (not all the time, only when
somebody makes a system call).

It only happens on MIPS. There is no easy way to check this on x86.
-- 

Alexander Sirotkin

SW Engineer

Texas Instruments

Broadband Communications Israel (BCIL)

Tel:  +972-9-9706587

________________________________________________________________________

"Those who do not understand Unix are condemned to reinvent it, poorly."

      -- Henry Spencer

---

From: Jun Sun <[email-masked]>
Date: Thu, 14 Aug 2003 09:45:15 -0700

On Thu, Aug 14, 2003 at 12:17:52PM +0300, Sirotkin, Alexander wrote:
> finishes, 

The tasklet should be executed at the return of interrupt handling.
If not, there is a bug.

> only after the next timer

BTW, make sure tasklet_schedule() is indeed called in an interrupt handler.
I am not sure why will happen otherwise.

If you suspect it is a bug, you can easily trace them.  You may my
little tracing tool useful,

	http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.patch

Jun

---

From: "Sirotkin, Alexander" <[email-masked]>
Date: Mon, 18 Aug 2003 18:29:52 +0300

Jun Sun wrote:

finishes, 

If not, there is a bug.

I have a feeling that we are going in circles. Tasklets are executed at
the return of interrupt handler.
However, I suspect that this is not enough. On mips (contrary to x86),
system call is NOT an interrupt.
It's a different exception with different handler. Therefore I suspect
that tasklets are NOT called at 
the end of system call exception handler (which is a different handler,
not do_IRQ).

    

BTW, make sure tasklet_schedule() is indeed called in an interrupt
handler.

I can try to trace it, I just wanted to ensure that what I was saying
makes sense. See above.

http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.pa
tch
<http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.p
atch> 

Jun

-- 

Alexander Sirotkin

SW Engineer

Texas Instruments

Tel:  +972-9-9706587

      -- Henry Spencer

---

From: Jun Sun <[email-masked]>
Date: Mon, 18 Aug 2003 10:08:31 -0700

On Mon, Aug 18, 2003 at 06:29:52PM +0300, Sirotkin, Alexander wrote:
> However, I suspect that this is not enough. 

If you follow this, plus "tasklet_schedule() is indeed called in an interrupt
handler", you will should see "executing tasklet at the return of interrupt
handler"  is _obviously_ enough.

> On mips (contrary to x86),

... which is fine, if you can follow the above logic.

Jun