Subject: Re: Linux: the next step

From: Steve Alexander <[email-masked]>
Date: Mon, 12 Aug 1996 13:32:01 -0700

lm@gate1-neteng (Larry McVoy) writes:
>The fine grained locking is a lose.  It costs us way too much and what I
>am seeing is a trend backwards towards coarser grained locking.  If this
>wasn't true, why did we build lego?  Why are we building Nexus?

I'd like to see some hard numbers for these assertions.  All of the bottlenecks
in ficus (that I am aware of) have to do with context-switching due to use of
mutexes rather than spinlocks.  The locking overhead itself isn't even on the
map as far as I can tell.

However, I agree that more locking is not the answer.  Eliminating as much
shared, global data as possible is the answer.  Nexus doesn't really do
anything towards that end that I can see.

>Linux already has a RT kernel.  I just reviewed a fantastic Usenix paper
>that added RT to Linux - fully preemptable, hard real time to < 100 usecs,
>and less than 3K lines of code change.  Their test case was a 100Khz clock
>pulse on a parallel port; they got it every time, with less than 15usecs
>skew, while the system was tar-ing one file system to another, running 
>netscape, and generally doing same old Unix stuff just like normal.  It's
>really impressive and very uninvasive.

I'd like to see that when it comes out.

>Finally, think about this:  how many times have you had a "great idea"
>for better IRIX performance, gone off an prototyped it, only to find
>that it makes no difference?  Linux lets you test out those ideas and
>see the real performance difference, unadulterated by any surrounding
>bloat.  That's cool.  We want that.

If it makes no difference, what difference does it make what platform you try
it on?  If it does make a difference, you would be able to measure it despite
the underlying bloat, and again the underlying platform makes no difference.

I'm in favor of some amount of Linux work on SGI gear, but this is a pretty
weak argument.

-- Steve

---

From: Bob English <[email-masked]>
Date: Mon, 12 Aug 96 13:52:47 -0700

>am seeing is a trend backwards towards coarser grained locking...

>map as far as I can tell.

If you look at larry's context switch and pipe benchmarks, you'll see
that they run much faster on a single CPU than on two or more.  A lot of
the difference is due to fine-grained locking, which causes more dirty
cache lines to thrash than is actually necessary for the operations
involved.  Pipe communication takes locks at the vnode and inode
level.  Context switches take locks on threads, accounting structures,
and queues.

It all adds up.

>the underlying bloat, and again the underlying platform makes no difference.

Back to the previous discussion.  Getting rid of any one of those locks
wouldn't make much difference.  Getting rid of all of them would, but is
a much taller order.

--bob--