Subject: lmbench with new checksum code...

From: "David S. Miller" <[email-masked]>
Date: Thu, 16 May 1996 02:53:50 -0400

It's not as hot as I wanted it to be, ho hum... Just for flavor I've
included numbers for my 115MHZ hypersparc running SunOS so no none
forgets that this is all running on a shitty SparcClassic. ;-)

I'm probably stalling the chip stupidly in my code or not touching the
cache lines in the correct order, ugh this is pissing me off, I'll
check it out some more tomorrow when my head doesn't hurt so much...
I doubt I can get 2mb/s more out of my code to beat solaris, sigh,
where the heck could all this overhead possibly be???

                    L M B E N C H  1 . 0   S U M M A R Y
                    ------------------------------------

            Processor, Processes - times in microseconds
            --------------------------------------------
Host                 OS  Mhz    Null    Null  Simple /bin/sh Mmap 2-proc 8-proc
                             Syscall Process Process Process  lat  ctxsw  ctxsw
--------- ------------- ---- ------- ------- ------- ------- ---- ------ ------
trombetas  Linux 1.99.3   50      15    8.8K   40.9K     75K  350     83    100
geneva.ru     SunOS 5.5   50      31   33.7K  148.2K    274K  596    174    205
negro.rut SunOS 4.1.3_U   49     124   18.3K   63.9K    110K  470    152    262
huahaga.r   SunOS 4.1.4  115      32   10.4K   34.3K     59K  169     96    104

            *Local* Communication latencies in microseconds
            -----------------------------------------------
Host                 OS  Pipe       UDP    RPC/     TCP    RPC/
                                            UDP             TCP
--------- ------------- ------- ------- ------- ------- -------
trombetas  Linux 1.99.3     295    1016    1756    1408    2564
geneva.ru     SunOS 5.5     530    1563    2080    1354    2398
negro.rut SunOS 4.1.3_U     890    1375    2287    1573    2804
huahaga.r   SunOS 4.1.4     306     616     956     667    1161

            *Local* Communication bandwidths in megabytes/second
            ----------------------------------------------------
Host                 OS Pipe  TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
                                  reread reread (libc) (hand) read write
--------- ------------- ---- ---- ------ ------ ------ ------ ---- -----
trombetas  Linux 1.99.3    8  4.8   25.0   17.4     18     24   41    36
geneva.ru     SunOS 5.5    8  7.0   12.6   19.5     18     18   40    36
negro.rut SunOS 4.1.3_U    4  2.0   19.5    8.2     18     24   41    36
huahaga.r   SunOS 4.1.4   14  5.3   30.2   19.9     20     22   48    37

            Memory latencies in nanoseconds
            (WARNING - may not be correct, check graphs)
            --------------------------------------------
Host                 OS   Mhz  L1 $   L2 $    Main mem    TLB    Guesses
--------- -------------   ---  ----   ----    --------    ---    -------
trombetas  Linux 1.99.3    50    20    170         180    600    No L2 cache?
geneva.ru     SunOS 5.5    49     -      -           -      -    Bad mhz?
negro.rut SunOS 4.1.3_U    49    20    175         183    659    No L2 cache?
huahaga.r   SunOS 4.1.4   115    17     17         510    842    No L1 cache?

---

From: Alan Cox <[email-masked]>
Date: Thu, 16 May 1996 10:35:04 +0100 (BST)

> where the heck could all this overhead possibly be???

What makes you think the Solaris loopback is even doing checksums or a memcpy
via kernel space ? They only way you'll beat solaris at the loopback network
game is to cheat as they do. Make tcp_connect spot a localhost connection
change the socket method to something akin to af_unix but streamlined a bit
(only special case is urgent data).

Alan

---

From: Linus Torvalds <[email-masked]>
Date: Thu, 16 May 1996 13:08:04 +0300 (EET DST)

> (only special case is urgent data).

One day we might actually want to do something like that. No, I'm not
suggesting special-casing loopback (we don't need it any more, we're getting
good enough performance as-is), but I would suggest that at some point we
integrate the networking code a bit tighter in the VFS model of open/close. 

One problem with the networking code right now is that we can't short-circuit
some of the decisions, so we're doing unnecessary run-time checks. This is
not really much of a performance problem, I consider it more of a beaty wart
right now. 

For example, when we open a normal pipe (be it unnamed or named), we 
never go through the filesystem code for that pipe any more - we just 
make the file operations point directly to the pipe code, and when we 
read/write to that pipe we don't have any filesystem overhead.

In contrast, when we open a network connection, we always go through the 
network layer, even at run-time. Admittedly there are some reasons for 
this, but most of them aren't really valid any more.

For example, sockets used to really be totally separate entities from 
inodes, so we couldn't consider sockets to be part of the VFS layer. But 
that isn't true any more (and hasn't been for a long whole): sockets 
really _are_ implemented as a part of the inode these days. So a "socket" 
is really just the network-specific part of an inode (the same way the 
"normal" filesystems have their own private parts - look at the inode "u" 
union to see what I mean.

However, due to historical reasons, the internal socket routines do not 
use the VFS "inode" abstraction, but instead they use only the socket 
sub-part. Sometime in the future I would really like to get rid of that, 
and make the low-level socket code use the "inode" the same way everybody 
else does.

This is _definitely_ not a huge issue - as I said it's more cleanlyness 
and encapsulation than performance. It will require making the 
socket-specific IO operations (recvmsg etc) be first-class members in the 
VFS layer etc, and in general it requires a lot of minor modifications. 
Nothing terribly hard (and some things get cleaner thanks to it), but 
it's a lot of code that has to be worked out.

Merging the sockets more tightly into the VFS layer gets rid of the current
"struct socket" that we don't really need (as opposed to the "struct sock",
which is a different beast altogether) and at least partly the "struct
proto_ops" (which would just be part of the "struct file_operations").  We'd
get rid of one (slightly confusing) layer of abstraction, and some cases
could be streamlined a bit. 

Finally, let me say again that I don't think we should short-circuit loopback
TCP like Solaris does. I used to think it was a nifty feature, but I got
better. When I talk about streamlining above, I'm thinking of similar
short-circuiting, but on a much smaller scale (getting rid of run-time checks
that can be done when the state of the thing changes instead). 

For an example of what I'm talking about, look at how the tty layer uses the
operations pointers to handle hangup etc. It just changes the file operations
pointer, which automagically means that the files start behaving differently
once they've been hung up (without having the actual routines do any extra
"am I hung up" checking).  That's the kind of thing that the network layer
could do too if it was better integrated with the VFS layer. 

[ Alan has heard some of this before - I've been talking about these changes
  for a long time. I've never felt they have been important enough to really
  do something radical about it, but I still think it's the right thing to do
  eventually ]

		Linus

---

From: "Greg Chesson" <[email-masked]>
Date: Thu, 16 May 1996 09:03:58 -0700

folks have special-cased the loopback driver in the past to improve
X performance.

g