From owner-linux@cthulhu.engr.sgi.com Wed Jul 31 17:50:24 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id RAA09318; Wed, 31 Jul 1996 17:50:24 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id AAA26701 for linux-list; Thu, 1 Aug 1996 00:50:18 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id RAA26688 for ; Wed, 31 Jul 1996 17:50:17 -0700 Received: (from ariel@localhost) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id RAA10293 for linux; Wed, 31 Jul 1996 17:48:43 -0700 From: ariel@yon.engr.sgi.com (Ariel Faigon) Message-Id: <199608010048.RAA10293@yon.engr.sgi.com> Subject: Linux: the next step To: linux@yon.engr.sgi.com Date: Wed, 31 Jul 1996 17:48:43 +1700 (PDT) Reply-To: ariel@cthulhu.engr.sgi.com Organization: Silicon Graphics Inc. X-Mailer: ELM [version 2.4 PL24 ME5a] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 544 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Hi Linuxies, So soon the port will be "over". What's next? Can we turn this from a neat hacker's adventure into something really beneficial to SGI? Thanks to Larry and David, I started collecting some thoughts in: http://info.engr/linux/next-step.html P.S. I also talked to some people who oppose all this so I made their voices heard in: http://info.engr/linux/skeptic.html I hope you'll find these two mildly interesting. I wish TJ could join us in this new endeavor. Any constructive input is as always welcome. -- Peace, Ariel From owner-linux@cthulhu.engr.sgi.com Thu Aug 1 13:48:04 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA07790; Thu, 1 Aug 1996 13:48:04 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id UAA21059 for linux-list; Thu, 1 Aug 1996 20:47:57 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA21054 for ; Thu, 1 Aug 1996 13:47:55 -0700 Received: (from ariel@localhost) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id NAA11583 for linux; Thu, 1 Aug 1996 13:46:21 -0700 From: ariel@yon.engr.sgi.com (Ariel Faigon) Message-Id: <199608012046.NAA11583@yon.engr.sgi.com> Subject: Those FreeBSD guys... To: linux@yon.engr.sgi.com Date: Thu, 1 Aug 1996 13:46:21 -0700 (PDT) Reply-To: ariel@cthulhu.engr.sgi.com Organization: Silicon Graphics Inc. X-Mailer: ELM [version 2.4 PL24 ME5a] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 2001 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Yesterday, I went to the MindSource monthly meeting (Michael's at Shoreline) where three FreeBSD people gave a talk. Jordan Hubbard (he is on a visit from Ireland), Sameer Parekh (sp?), and Matt Dillon. Of course there were persistent questions from the audience to the tune of "I have this other system at home that starts with L..., why should I switch to FreeBSD?" Here are some of the answers given (from memory): "Our impression of the Linux community is that they are a bunch of Cowboys, they don't even use a source-control system to coordinate and split the work. We have these great development control tools: cvsup and sup ... we work really well together. all the core team has write permission to the source tree" There are 47 people doing major contributions. 14 of them are core. "Linux might give you the "feeling" it is faster, but it is because we gave a lot of thought in the design to scalability and SMP, when you load Linux the great performance suddenly drops down sharply while BSD scales nicely." "If you want a single user Unix to play at home, Linux is fine but if you are a commercial entity or ISP and you count on reliability and solid network performance, complex routing etc. use FreeBSD" "Linux is just a kernel, when you go to a full distribution things are much more complex, there are too many Linux distributions there's only one FreeBSD source base and it is complete with all utilities" "They try to run on too many architectures and the code is not clean We (FreeBSD as opposed to NetBSD) focus on Intel only, do it cleanly and make sure it works well" "They have the momentum and we don't underestimate this... we learned that we need to be nicer towards newbie questions to establish a larger user base" The future of FreeBSD is really promising: Clustering & Log filesystems are coming. SMP is already here (although it is not exactly symmetrical)" Sounds like these guys haven't seen linux since 0.99... Interesting nevertheless. -- Peace, Ariel From owner-linux@cthulhu.engr.sgi.com Thu Aug 1 18:14:49 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id SAA18865; Thu, 1 Aug 1996 18:14:49 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id BAA29537 for linux-list; Fri, 2 Aug 1996 01:14:37 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id SAA29527 for ; Thu, 1 Aug 1996 18:14:35 -0700 Received: from neteng.engr.sgi.com (gate5-neteng.engr.sgi.com [150.166.61.33]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id SAA11931 for ; Thu, 1 Aug 1996 18:13:01 -0700 Received: (from dm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id SAA18859; Thu, 1 Aug 1996 18:14:34 -0700 Date: Thu, 1 Aug 1996 18:14:34 -0700 Message-Id: <199608020114.SAA18859@neteng.engr.sgi.com> From: "David S. Miller" To: ariel@cthulhu.engr.sgi.com CC: linux@yon.engr.sgi.com In-reply-to: <199608012046.NAA11583@yon.engr.sgi.com> (ariel@yon) Subject: Re: Those FreeBSD guys... Reply-to: dm@sgi.com Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk From: ariel@yon (Ariel Faigon) Date: Thu, 1 Aug 1996 13:46:21 -0700 (PDT) I can't help, I have to comment on some of this ;-) My comments probably look like flame fare, but statements like the following make my stomach turn. Enter this mail at your own risk. ;) Yesterday, I went to the MindSource monthly meeting (Michael's at Shoreline) where three FreeBSD people gave a talk. Jordan Hubbard (he is on a visit from Ireland), Sameer Parekh (sp?), and Matt Dillon. My pinhead radar was on high lately, this must have been what set it off ;) "Our impression of the Linux community is that they are a bunch of Cowboys, " Yee haw! "they don't even use a source-control system to coordinate and split the work. We have these great development control tools: cvsup and sup ... we work really well together. all the core team has write permission to the source tree" There are 47 people doing major contributions. 14 of them are core. Mostly untrue. Fact: Linus is the only person who can get at the master tree, of what use are source control tools when this is the case is beyond me. Fact: The entire world has write permission to the Linux sources, because of the GPL. The FreeBSD tree can be encorporated into things where the sources aren't publicly available. Fact: Those subprojects (mostly the ports) do use source control systems (the same ones the FreeBSD people use, sans sup which is truly braindamaged and useless, cvs does everything sup claims to be doing) when multiple people are working on tree, because it makes _sense_ here. Fact: FreeBSD people are jealous that we have allowed one person to drive the firetruck "by himslef" and provide free QA for all kernel development. In Linus we trust. "Linux might give you the "feeling" it is faster, but it is because we gave a lot of thought in the design to scalability and SMP, when you load Linux the great performance suddenly drops down sharply while BSD scales nicely." Fact: FreeBSD has been talking/toying with an SMP implementation for many years now, currently the best it can do is run user processes on one cpu and service interrupts on another on one particular dual-pentium system. Linux has SMP working on all Intel platforms which at least closely adhere to the Intel SMP specification. We also have SMP fully working on Sparc machines, MIPS should be next. Fact: It not only feels faster, it is faster, and on all supported platforms. We have shown this with benchmarks, and real life examples. On my 4 processor SparcStation I got Sparc/ Linux smp working on, I had a load of 600 for 3 hours going on that machine. The jobs were a mixture of lat_tcp's, bw_tcp's parallel kernel makes on both local disk and over NFS, 300 invocations of crashme, and my X session locally on the machine. The machine had filled up %92 of it's swap space (the machine had 168mb of ram if I recall), typing commands in my xterms were still spiffy. Next. "If you want a single user Unix to play at home, Linux is fine but if you are a commercial entity or ISP and you count on reliability and solid network performance, complex routing etc. use FreeBSD" Fact: Linux survives crashme and other stress tests longer than any commercial or free operating system out there. Fact: FreeBSD is faster over loopback when compared to Linux over the wire. ;-) "Linux is just a kernel, when you go to a full distribution things are much more complex, there are too many Linux distributions there's only one FreeBSD source base and it is complete with all utilities" Fact: Entities like RedHat and others provide prepackaged, plug in and go, click your mouse button on this and it works, Linux installations and full support. Fact: More distributions give Linux more flavor. If I don't like the way the distribution scheme is layed out in the FreeBSD "one true" distribution, I'm out of luck because "I have no choice." "They try to run on too many architectures and the code is not clean We (FreeBSD as opposed to NetBSD) focus on Intel only, do it cleanly and make sure it works well" I don't think I even need to say how bolixed this statement is. "They have the momentum and we don't underestimate this... we learned that we need to be nicer towards newbie questions to establish a larger user base" I am happy that they do not understand the real reason we have so much momentum. The future of FreeBSD is really promising: Clustering & Log filesystems are coming. SMP is already here (although it is not exactly symmetrical)" Linux has real symmetrical SMP today, we have MPP support and work on multicomputers, we are X/Open and POSIX branded. Fine grained SMP, IPV6, and more + faster support on more architectures in in the works. People are working on clustering and logging filesystems for Linux as well. Sounds like these guys haven't seen linux since 0.99... Interesting nevertheless. I agree. dm@engr.sgi.com 'Ooohh.. "FreeBSD is faster over loopback, when compared to Linux over the wire". Film at 11.' -Linus From owner-linux@cthulhu.engr.sgi.com Thu Aug 1 20:38:22 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id UAA20608; Thu, 1 Aug 1996 20:38:22 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id DAA23655 for linux-list; Fri, 2 Aug 1996 03:38:15 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id UAA23648 for ; Thu, 1 Aug 1996 20:38:14 -0700 Received: from soyuz.wellington.sgi.com (soyuz.wellington.sgi.com [155.11.228.1]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id UAA12032 for ; Thu, 1 Aug 1996 20:36:30 -0700 Received: from windy.wellington.sgi.com by soyuz.wellington.sgi.com via ESMTP (951211.SGI.8.6.12.PATCH1042/940406.SGI) id PAA04878; Fri, 2 Aug 1996 15:37:39 +1200 Received: (alambie@localhost) by windy.wellington.sgi.com (950413.SGI.8.6.12/8.6.9) id PAA16834; Fri, 2 Aug 1996 15:37:38 +1200 From: "Alistair Lambie" Message-Id: <9608021537.ZM16832@windy.wellington.sgi.com> Date: Fri, 2 Aug 1996 15:37:37 +0000 In-Reply-To: ariel@yon.engr.sgi.com (Ariel Faigon) "Linux: the next step" (Aug 1, 12:51pm) References: <199608010048.RAA10293@yon.engr.sgi.com> X-Mailer: Z-Mail (3.2.3 08feb96 MediaMail) To: ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk On Aug 1, 12:51pm, Ariel Faigon wrote: > Subject: Linux: the next step > Hi Linuxies, > > So soon the port will be "over". > > What's next? > > Can we turn this from a neat hacker's adventure into > something really beneficial to SGI? > > Thanks to Larry and David, I started collecting some > thoughts in: > > http://info.engr/linux/next-step.html > My 2c...maybe it isn't even worth that much! I know that there is a lot of politics here, so some of this may be just dreaming.... I'll break it into the same sections as Ariel has: We need more contributors to keep momentum: 1. We will HAVE to seed machines to people as I'm guessing most people have ~$0 budget for hardware. 2. We need to make sure the basic tools (gcc/glibc) make it as seamless as possilble to port things (ie, just configure/make and it happens). I think David is doing a great job here, but we are probably going to need an ongoing effort from a real good libc hacker who can get any compatability/conflict type problems sorted intelligently. What is David's commitment when he leaves...or are we going to let him leave?? 3. We need to get some of the neat Indy features supported. We need an SGI graphics wizard to make X fly, then things like ISDN and audio. These are the sorts of things that will make people want to use Indy's as their primary dev environments. Fast and great functionality. 4. We need to have some people who are able to commit to help contributors who need to find out things. It needs to be easy for people to find out information they need. This is a tricky area as sometimes people really will need to know confidential info...how can we handle this. I know Ralph has found it real difficult at times to find things out...we need to make this easier so as not to slow things down. We should start thinking marketing 1. I believe RedHat would be the best candidate for distribution. I would like to see it 'launched' at some high profile conference (Linux related), have it on a CD that has 'awesome' artwork (the old WebForce CD's are a good example) and becomes a great collectors item afterwards, and is packaged with a real cool T-Shirt for the first X copies. This stuff should be firmly Linux relevant from the design point (penguins??) but at the same time be SGI through and through. SGI should probably contribute this part. 2. Don't let marketing name it or we'll get Cosmo Linux....just kidding :-) I think we need to emphasis that SGI has put real dollars into this and is committed to make it work, rather than the 'faster' bandwagon. To emphasis this point I feel we need to have it run on any new platforms at first shipment, and support the neat features. Nothing will kill it quicker than if people get this feeling that this is second tier and nobody cares. We need a number of us to commit to give quality help to people on mailing lists/newgroups. Shucks...maybe if we even had 5% of the budget IRIX has... 3. Sponsor Linux events etc....give away Indy's etc...and other things like apparel and mugs. These should all have professional art work that is to die for....this stuff works when people really want it. It should really put SGI in front of people. My feeling is that we can really get in front of the technical people in a lot of places and get them on our side. While they might not be the decision makers, it sure helps sales people if the techo's in the organisation are on side rather than fighting against you! 4. A Web/FTP server is a great idea. Let's get some real great design on the Web server though so that the quality is similar to Silicon Surf. Again, it gets the message across that we are serious, committed etc. We also want people accessing as many things with .sgi.com in them!! 5. If we can't get people to come to sgi, how about we sponsor good net connections for them that are in a domain linux.sgi.com. Subtle, but it does get sgi associated with them on the net....and we do have a backbone to most parts of the world. > > P.S. I also talked to some people who oppose all this > so I made their voices heard in: > > http://info.engr/linux/skeptic.html > One comment I would make here is that people really need to work out whether we are a hardware or software company. I realise that this is probably contentious (sp?), but my feeling is that we are a company who make hardware, and the software is really a means to an end...selling more hardware. If this is true, then people need to look at where we are going to get the greatest number of hardware sales from....Irix or Linux. We need both I think to hit different markets, and maybe we even need NT (don't want to get into that argument!). Maybe we should be getting agreement from management for funding based on sales, and I'll bet we can get a lot more mileage from the funding than Irix will :-) > I hope you'll find these two mildly interesting. It's great having someone pull it all together, thanks. Cheers, Alistair -- Alistair Lambie alambie@wellington.sgi.com Silicon Graphics New Zealand SGI Voicemail: 56791 Level 5, Walsh Wrightson Tower, Ph: +64-4-802 1455 94-96 Dixon St, Wellington, NZ Fax: +64-4-802 1459 From owner-linux@cthulhu.engr.sgi.com Fri Aug 2 16:09:19 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA16192; Fri, 2 Aug 1996 16:09:18 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id XAA08028 for linux-list; Fri, 2 Aug 1996 23:07:57 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA08021 for ; Fri, 2 Aug 1996 16:07:55 -0700 Received: from aa5b.engr.sgi.com (aa5b.engr.sgi.com [192.102.117.24]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA13474 for ; Fri, 2 Aug 1996 16:06:18 -0700 Received: from aa5b (localhost [127.0.0.1]) by aa5b.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via SMTP id QAA13029; Fri, 2 Aug 1996 16:07:41 -0700 Message-ID: <32028A3C.2781@engr.sgi.com> Date: Fri, 02 Aug 1996 16:07:40 -0700 From: Nigel Gamble X-Mailer: Mozilla 2.02S (X11; I; IRIX 6.2 IP22) MIME-Version: 1.0 To: Alistair Lambie CC: ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step References: <199608010048.RAA10293@yon.engr.sgi.com> <9608021537.ZM16832@windy.wellington.sgi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Alistair Lambie wrote: > To > emphasis this point I feel we need to have it run on any new platforms > at first shipment, and support the neat features. Dream on! Here in ISD (Indigo2 land), we're working flat out just to get IRIX to run on any new platforms at first shipment, with all the neat features supported. I think you'll find the same is true of all the other product divisions. > One comment I would make here is that people really need to work out whether > we are a hardware or software company. I realise that this is probably > contentious (sp?), but my feeling is that we are a company who make hardware, > and the software is really a means to an end...selling more hardware. If this > is true, then people need to look at where we are going to get the greatest > number of hardware sales from....Irix or Linux. We need both I think to hit > different markets, and maybe we even need NT (don't want to get into that > argument!). Maybe we should be getting agreement from management for funding > based on sales, and I'll bet we can get a lot more mileage from the funding > than Irix will :-) We are neither a hardware company nor a software company. We are a *systems* company. We build the best computer systems on the planet through a combination of hardware and software working together. If we replaced IRIX with NT, we would have to cancel most of our high performance hardware projects, because NT will not support the new systems. As for Linux, when will it support true symmetric multiprocessing with fine-grained semaphore and mutex locking, and a fully preemptible kernel? At the San Diego Usenix earlier this year, Linus Torvalds thought that the probable answer was "not anytime soon". (By the way, in order to do this correctly, every device driver would have to be rewritten.) Linux would need this *at a minimum* before there is any hope that it would supplant IRIX on the current generation of hardware, let alone any future new platforms. -- Nigel Gamble "Are we going to push the edge of the envelope, Brain?" Silicon Graphics "No, Pinky, but we may get to the sticky part." nigel@sgi.com (415) 933-3109 From owner-linux@cthulhu.engr.sgi.com Fri Aug 2 16:55:31 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA19672; Fri, 2 Aug 1996 16:55:30 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id XAA15732 for linux-list; Fri, 2 Aug 1996 23:54:10 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA15725 for ; Fri, 2 Aug 1996 16:54:09 -0700 Received: from neteng.engr.sgi.com (gate5-neteng.engr.sgi.com [150.166.61.33]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA13519 for ; Fri, 2 Aug 1996 16:52:32 -0700 Received: (from dm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id QAA19624; Fri, 2 Aug 1996 16:54:06 -0700 Date: Fri, 2 Aug 1996 16:54:06 -0700 Message-Id: <199608022354.QAA19624@neteng.engr.sgi.com> From: "David S. Miller" To: nigel@cthulhu.engr.sgi.com CC: alambie@wellington.sgi.com, ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com In-reply-to: <32028A3C.2781@engr.sgi.com> (message from Nigel Gamble on Fri, 02 Aug 1996 16:07:40 -0700) Subject: Re: Linux: the next step Reply-to: dm@sgi.com Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Date: Fri, 02 Aug 1996 16:07:40 -0700 From: Nigel Gamble Dream on! Here in ISD (Indigo2 land), we're working flat out just to get IRIX to run on any new platforms at first shipment, with all the neat features supported. I think you'll find the same is true of all the other product divisions. I am one person, I am getting close to bootstrapping a complete Linux system on the INDY in the course of 3 months. The only way I could be dreaming is due to the fact that I have been hacking for 30 hours straight with no sleep at all and this isn't even flat out for me actually. We are neither a hardware company nor a software company. We are a *systems* company. Although this may be true, what ever sells your hardware could foot the purpose correct? The margin comes from the hardware, and as you say the software is there to make that hardware capable of doing something. If Linux could do this, and sell SGI hardware, SGI is doing the same thing it is today in my estimation. As for Linux, when will it support true symmetric multiprocessing with fine-grained semaphore and mutex locking, and a fully preemptible kernel? At the San Diego Usenix earlier this year, Linus Torvalds thought that the probable answer was "not anytime soon". Linus, myself, and some other key developers have come to the conclusion that the current way of doing SMP is no more than an intellectual curiosity and nothing more. What he had in the back of his mind when he made that statement at Usenix was really, "Linux is not going to scale it's SMP in that way." But being the person that he is, he decided not to put it that way until he had a cast in stone proof of concept that another way was indeed possible. With Linux, thankfully, we have the luxury of really thinking about how to correctly go about something like a scalable SMP kernel architecture without a deadline hovering over our heads. However, I will be frank, and say that no we do not have this SMP of all SMP figured out yet. But I do believe that we will come up with something not short of impressive and cutting edge. If I had to guess at how we'd go about it, generally? I would venture one of the following or a combination thereof perhaps: 1) Scale to 4 cpus and nothing more, cluster past that, run seperate kernels on each processor group of 4, which live in their own little world and do not have shared resources with the other cluster kernels. With the exception that clusters can pass processes between each other when load on one get much too high than the others. (I'm a bit sketchy on any of the real specifics on how we would pull this one off entirely) 2) Away with the locks completely, if you are going to do SMP then don't add a brick wall to the equation as a means to an end. Through numerous conversations with Linus I have deteremined that he and I agree that locks all over the place is not the way to go and neither is pre-emption. We believe that the purpose of locks can be replaced entirely with carefully designed data structures that are atomic via the way they are accessed. Again these are just off the top of my head, unfortunately I haven't had the opportunity to get into the zone and put all of my senses into getting these ideas out of perpetual beta. I was hoping that if I got the initial INDY port out of the way and functional very quickly I'd finally have a chance to think about these things and come up with something concrete I could actually implement and start playing with. This is along the lines of what I wanted my big contribution back to SGI to be for allowing me to have this internship, however it was obviously necessary to get a proof of concept Linux implementation out the door on a machine before I could start doing things like that. My real point is, yes we cannot scale very well on the type of machines you mention, we do not have fine grained SMP and kernel preemtion. The current view I propose is that this method of doing SMP is one of many oysters that could have been opened to satisfy the needs of the task at hand. However I feel that the oyster with the real pearl of an SMP implementation in it has yet to have been found. Putting Linux down because it cannot do what Solaris2.5, SVR4.2MP, and IRIX can do "today" on very high end hardware I would consider to be an oxy-moron. All of it's code has been written by contributors in their spare time and out of their own good hearts on the low end hardware they could only get their hands on. And even with those constraints, we have things which the other freely available operating systems simply do not have an probably will not for a very long time. But now all of this is slowly but surely going to change, as more and more people like myself get the opportunity to express their hacking talents on high end hardware (the systems where the capabilities you mention even matter) via contributions such as my internship and hardware loans to the various developers, this gap will get smaller and smaller especially if the current pace of Linux keeps up. This is one thing I can assure you of. Maybe the real issue is, now that I think about, we haven't put our efforts into crossing that bridge, because we have not gotten to it yet. When we do get there, it'll show time baby. dm@engr.sgi.com 'Ooohh.. "FreeBSD is faster over loopback, when compared to Linux over the wire". Film at 11.' -Linus From owner-linux@cthulhu.engr.sgi.com Fri Aug 2 17:14:15 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id RAA19986; Fri, 2 Aug 1996 17:14:14 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id AAA18041 for linux-list; Sat, 3 Aug 1996 00:12:54 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id RAA18029 for ; Fri, 2 Aug 1996 17:12:52 -0700 Received: from soyuz.wellington.sgi.com (soyuz.wellington.sgi.com [155.11.228.1]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id RAA13570 for ; Fri, 2 Aug 1996 17:11:13 -0700 Received: by soyuz.wellington.sgi.com (951211.SGI.8.6.12.PATCH1042/940406.SGI) id MAA02013; Sat, 3 Aug 1996 12:12:32 +1200 From: alambie@wellington.sgi.com (Alistair Lambie) Message-Id: <199608030012.MAA02013@soyuz.wellington.sgi.com> Subject: Re: Linux: the next step To: nigel@cthulhu.engr.sgi.com (Nigel Gamble) Date: Sat, 3 Aug 1996 12:12:32 +1200 (NZT) Cc: alambie@wellington.sgi.com, ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com In-Reply-To: <32028A3C.2781@engr.sgi.com> from "Nigel Gamble" at Aug 2, 96 04:07:40 pm X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit Content-Length: 3870 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk > > Alistair Lambie wrote: > > To > > emphasis this point I feel we need to have it run on any new platforms > > at first shipment, and support the neat features. > > Dream on! Here in ISD (Indigo2 land), we're working flat out just to > get IRIX to run on any new platforms at first shipment, with all > the neat features supported. I think you'll find the same > is true of all the other product divisions. > Yeah, I know!! This may be a dream, but I think it is to some degree achievable. The thing with IRIX is that EVERYTHING has to be integrated and commercial quality on first ship. What we really need to do in Linux is expose the interfaces to things. If we give enough info to people it will generally happen. I don't know what is involved to do this...maybe a skeleton driver or something. It doesn't really matter whether it is optimised, or even bug free, just as long as it gives those with the ability to write code like this enough info to work with. As for applications and things to use this stuff, thats up to the Linux community to do! > > One comment I would make here is that people really need to work out whether > > we are a hardware or software company. I realise that this is probably > > contentious (sp?), but my feeling is that we are a company who make hardware, > > and the software is really a means to an end...selling more hardware. If this > > is true, then people need to look at where we are going to get the greatest > > number of hardware sales from....Irix or Linux. We need both I think to hit > > different markets, and maybe we even need NT (don't want to get into that > > argument!). Maybe we should be getting agreement from management for funding > > based on sales, and I'll bet we can get a lot more mileage from the funding > > than Irix will :-) > > We are neither a hardware company nor a software company. > We are a *systems* company. We build the best computer systems > on the planet through a combination of hardware and software > working together. If we replaced IRIX with NT, we would have > to cancel most of our high performance hardware projects, because > NT will not support the new systems. As for Linux, > when will it support true symmetric multiprocessing with fine-grained > semaphore and mutex locking, and a fully preemptible kernel? > At the San Diego Usenix earlier this year, Linus Torvalds > thought that the probable answer was "not anytime soon". > (By the way, in order to do this correctly, every device driver would > have to be rewritten.) Linux would need this *at a minimum* before > there is any hope that it would supplant IRIX on the current > generation of hardware, let alone any future new platforms. > Yup, you're right, we are a systems company. I don't expect that Linux will replace IRIX even in the distant future. IRIX isn't going to stand still, but nor is Linux. One of the things I have noticed about Linux is that when someone shows the way, things happen fairly quickly. Probably Linus's comment about the symmetric multiprocessing was based on available resource, rather than a lack of desire to do it. Maybe this is an opportunity for SGI to provide some input into the Linux community...if we were to do some starter work on this (on one of our boxes of course!!) I'm sure that others would pretty soon catch on. The big thing with Linux is that we don't have to do all the work....a bit of work on an initial framework and a push in the right direction is often all that will be necessary. Let's take the ISDN port on the Indy....if we somehow show people how to use this, I'm sure it won't be long before someone outside of SGI gets it up and going! Same with audio. Let's not think in terms of supplanting IRIX, let's look at growing the number of boxes we ship. That's got ot be good for us!! Cheers, Alistair From owner-linux@cthulhu.engr.sgi.com Fri Aug 2 18:13:13 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id SAA20537; Fri, 2 Aug 1996 18:13:13 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id BAA29306 for linux-list; Sat, 3 Aug 1996 01:11:45 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id SAA29301 for ; Fri, 2 Aug 1996 18:11:44 -0700 Received: from neteng.engr.sgi.com (gate5-neteng.engr.sgi.com [150.166.61.33]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id SAA13659 for ; Fri, 2 Aug 1996 18:10:07 -0700 Received: from localhost (lm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via SMTP id SAA20491; Fri, 2 Aug 1996 18:11:31 -0700 Message-Id: <199608030111.SAA20491@neteng.engr.sgi.com> To: Nigel Gamble From: lm@gate1-neteng.engr.sgi.com (Larry McVoy) cc: Alistair Lambie , ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step Date: Fri, 02 Aug 1996 18:11:31 -0700 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk : Dream on! Here in ISD (Indigo2 land), we're working flat out just to : get IRIX to run on any new platforms at first shipment, with all : the neat features supported. I think you'll find the same : is true of all the other product divisions. It's hard work. I know people are working hard to keep IRIX going. And there is a lot in IRIX that isn't in Linux. However, there's a lot in IRIX that shouldn't be in IRIX, let alone Linux. One of the cool things that David has done this summer was to keep the generic code free of #ifdef *_WAR stuff that is littered throughout the IRIX kernel. : We are neither a hardware company nor a software company. : We are a *systems* company. I agree. But answer these questions: . would anyone buy our hardware if we didn't put the software on it? . would anyone buy our software on some other hardware? I think that gives you an answer to what kind of comapny we really are. : As for Linux, : when will it support true symmetric multiprocessing with fine-grained : semaphore and mutex locking, and a fully preemptible kernel? The fine grained locking is a lose. It costs us way too much and what I am seeing is a trend backwards towards coarser grained locking. If this wasn't true, why did we build lego? Why are we building Nexus? Linux already has a RT kernel. I just reviewed a fantastic Usenix paper that added RT to Linux - fully preemptable, hard real time to < 100 usecs, and less than 3K lines of code change. Their test case was a 100Khz clock pulse on a parallel port; they got it every time, with less than 15usecs skew, while the system was tar-ing one file system to another, running netscape, and generally doing same old Unix stuff just like normal. It's really impressive and very uninvasive. : At the San Diego Usenix earlier this year, Linus Torvalds : thought that the probable answer was "not anytime soon". : (By the way, in order to do this correctly, every device driver would : have to be rewritten.) Almost none of the drivers need a rewrite for the RT stuff. That work was a good example of what I call "orthogonal thinking" or using a completely different approach to solve the problem. : Linux would need this *at a minimum* before : there is any hope that it would supplant IRIX on the current : generation of hardware, let alone any future new platforms. Linux isn't going to replace IRIX any time soon. However, Linux is going to show off MTI's chips and SGI's hardware in a somewhat more positive light. If all that happens is that people get psyched about the "hidden" performance in SGI hardware and get IRIX to show off that performance, then the paltry amount we've spent on Linux will be worth it. Finally, think about this: how many times have you had a "great idea" for better IRIX performance, gone off an prototyped it, only to find that it makes no difference? Linux lets you test out those ideas and see the real performance difference, unadulterated by any surrounding bloat. That's cool. We want that. From owner-linux@cthulhu.engr.sgi.com Sun Aug 4 00:54:21 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id AAA26709; Sun, 4 Aug 1996 00:54:21 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id HAA02098 for linux-list; Sun, 4 Aug 1996 07:53:51 GMT Received: from neteng.engr.sgi.com (neteng.engr.sgi.com [192.26.80.10]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id AAA02093 for ; Sun, 4 Aug 1996 00:53:50 -0700 Received: (from dm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id AAA26656; Sun, 4 Aug 1996 00:53:49 -0700 Date: Sun, 4 Aug 1996 00:53:49 -0700 Message-Id: <199608040753.AAA26656@neteng.engr.sgi.com> From: "David S. Miller" To: linux@cthulhu.engr.sgi.com Subject: more on real-time functionality for Linux Reply-to: dm@sgi.com Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Because Larry mentioned some things regarding real time under Linux just to other day, I thought the following text would be interesting to people. A Vision for Linux 2.2 -- POSIX.1b Compatibility and Real-Time Support ---------------------------------------------------------------------- Markus Kuhn -- 1996-07-22 [The latest version of this text can be found on .] Today, the Linux kernel and libc are quite well compatible with the POSIX.1 and POSIX.2 standards, which specify system calls, library functions and shell command compatibility for UNIX-style operating systems. Some Linux distributions have even passed POSIX.1 conformance tests. However, the POSIX.1 system calls and library functions define only a minimum core functionality required by anything that looks like UNIX. Many slightly more advanced functions like mmap(), fsync(), timers, modifiable scheduling algorithms, IPC, etc., which are essential for many real world applications, have not been standardized by POSIX.1 in 1990. The new POSIX.1b standard (now officially called IEEE Std 1003.1b-1993, ISBN 1-55937-375-X, during development of the standard, it was called POSIX.4) corrects this and I believe POSIX.1b contains a large number of useful ideas for further development on Linux. In the very short introduction below, I hope to rise your interest in POSIX.1b and in real-time problems in general and to stimulate some development work in this direction. Happy reading! The new POSIX extensions focus on the requirements of real-time applications and on applications which have to perform high performance I/O. Many applications like interactive video games, high performance database servers, multimedia players and control software for all kinds of hardware require more deterministic scheduling, paging, signaling, timing and inter process communication mechanisms than what is available on traditional UNIX systems like BSD4.3. The functionality of systems like BSD4.3 has been optimized with mainframe multi-user time-sharing scenarios in mind, while operating systems for personal computers should also support real-time applications in addition. On a personal computer, it is often acceptable and desired that for example interactive games or CPU and memory intensive multimedia applications are excluded from the normal paging and scheduling strategies that try to be as fair as possible to all users of a large mainframe. The lack of real-time capability in Linux 1.2 has so far been the main reason why still a number of interesting applications that run fine on MS-DOS were unimplementable as user processes under Linux. Some examples are for instance highly reliable audio recording/replay tools, control software for astronomical CCD cameras, real-time signal processing algorithms, serial port smartcard emulators, MIDI computer music tools, etc. With the recent addition of POSIX.1b memory locking and static priority scheduling functions to Linux 1.3, this starts to change now. A lot of things still have to be implemented and your contributions are very welcome! This text summarizes, what functions have not yet been implemented and which people have already started to work on some of these. Please contact them if you want to know the status of their work or if you want to contribute. Please let me know about any progress so that I can keep this text up-to-date. In order to use the first new POSIX.1b features that have recently been added to Linux, please install a recent kernel, libc-5.3.12, and man-pages-1.11 or newer as available via ftp from sunsite.unc.edu or tsx-11.mit.edu. The real-time-support is far from complete, but some interesting features are already available. POSIX.1b-1993 defines in addition to POSIX.1-1990 the following new concepts and functions: Improved Signals ---------------- POSIX.1b adds a new class of signals. These have the following new features: - there are much more user specified signals now, not only SIGUSR1 and SIGUSR2. - The additional POSIX.1b signals can now carry a little bit data (a pointer or an integer value) that can be used to transfer to the signal handler information about why the signal has been caused. - The new signals are queued, which means that if several signals of the same type arrive before the signal handler is called, all of them will be delivered. - POSIX.1b signals have a well-defined delivery order, i.e. you can work now with signal priorities. - A new function sigwaitinfo() allows to wait on signals and to continue quickly after the signal arrived with program execution without the overhead of calling a signal handler first. The new queued signals are a necessary prerequisite for the implementation of the POSIX.1b asynchronous I/O interface (see below). They might also provide a good interface for delivering hardware interrupts to user processes. New functions for signals are: sigwaitinfo(), sigtimedwait(), sigqueue(). Implementation status: not yet implemented. Kevin Tran has sent me a short note that he has started to do some work on POSIX.1b signals for Linux. Inter Process Communication (IPC) and memory mapped files --------------------------------------------------------- POSIX.1b now defines shared memory, messages, and semaphores. The functionality and design of these is similar or better than the System V IPC mechanisms which we have already in Linux. The major extensions compared to System V IPC are: - Strings (like filename paths) instead of integers are used now to identify IPC resources. This will allow to avoid IPC resource collisions much easier than in SysV IPC. The POSIX IPC name space should probably be made visible as a /proc/ipc subdirectory, such that the usual tools like ls and rm can be used to locate and remove stale persistent IPC resources. - Semaphores come in two flavors: kernel based semaphores (as in System V, which requires a system call for each P/V operation) and now also user memory based semaphores. Kernel based semaphores are sometimes necessary for security reasons, however they are a real pain if you want to build e.g. a high performance database: Suppose there are 20 server processes operating on a single huge B-tree in a memory mapped database file. Inserting a node with minimal blocking of concurrent accesses by the other 19 processes in a large B-tree can require around 100 semaphore operations, and this means currently 100 kernel calls :-(. With POSIX.1b's user memory based semaphores, you put all your semaphores in a piece of shared memory and the library accesses them with highly efficient test-and-set machine code. System calls will then only be necessary in the rare case of a blocking P operation. A high performance database programmer's dream and easy to implement! - In POSIX.1b, both memory mapped files and shared memory are done with the mmap() system call. The new functions for IPC are: mmap(), munmap(), shm_open(), shm_close(), shm_unlink(), ftruncate(), sem_init(), sem_destroy(), sem_open(), sem_close(), sem_unlink(), sem_wait(), sem_trywait(), sem_post(), sem_getvalue(), mq_open(), mq_close(), mq_mq_unlink(), mq_send(), mq_receive(), mq_notify(), mq_setattr(), mq_getattr(), mprotect(). Implementation status: POSIX IPC has not yet been implemented (although a part of the basic mechanisms is already available in the existing SysV IPC code). Since Linux 1.3, mmap() is fully implemented. Eric Dumas has written me that he has done some work on POSIX IPC, however there are no patches available, yet. Memory locking -------------- Four new functions mlock(), munlock(), mlockall() and munlockall() allow to disable paging for either specified memory regions (mlock()) or for all pages (code, stack, data, shared memory, mapped files, shared libraries) to which a process has access (mlockall()). This allows to guarantee that for instance small time-critical daemons stay in memory which can help to guarantee response time of these processes. Under Linux, this (like many other real-time related features) is only allowed for root processes in order to avoid abuse of this feature by normal users in large time-sharing systems. Another application of memory locking are cryptographic computer security programs. Using mlock(), these systems can ensure that an unencrypted secret key or a password which is temporarily stored in a small user space array will never get in contact with the swap device, where under rare circumstances, someone might find the secret bytes even many months later. For these applications, it would be desirable if Linux allowed even non-root processes a small number of mlock()ed pages (e.g. up to four locked pages per non-root process should be ok). Implementation status: Linus has now added full POSIX.1b memory locking support to Linux alpha test kernel version 1.3.43. There exist also libc support and manual pages. So you won't have to apply the POSIX.4_locking patch from Ralf Haller any more. Synchronous I/O --------------- Databases, e-mail systems, log daemons, etc. require to be sure that the written piece of data has actually reached the harddisk, because transaction protocols require that a system crash or power failure after the write command can not harm the data any more. POSIX.1b defines the fsync() and O_SYNC mechanisms which Linux 1.2 already has. In addition, there is a very useful new function fdatasync() which requires that the data block is flushed to disk, however which does NOT require that the inode with the latest access/modification time is also flushed each time. With fdatasync(), the inode has only to be written in case the file length, file owner, or permission bits have changed. In database applications with mostly constant file sizes, where you sometimes require an fsync() after each few written blocks, but where you don't care about whether the access times in the inodes on the disc are always 100% up-to-date, fdatasync() could easily double (!) the performance of your system. There is also an msync() function for flushing a range of pages from memory mapped files to the disk. Implementation status: fsync(), fdatasync(), msync(), and O_SYNC are already available. O_DSYNC has not yet been implemented. However fdatasync() in Linux 1.3.55 is currently only an alias for fsync() and therefore not yet any more efficient than fsync(). Timers ------ - Instead of the old BSD style gettimeofday()/settimeofday() calls, POSIX.1b defines clock_gettimer(), clock_settimer() and clock_getres(). They offer nanosecond resolution instead of microseconds as with the old BSD calls (at least on Pentiums, a timer resolution much than a microsecond is available). In addition, you can query now the actual resolution of the timer with clock_getres(). - A new function nanosleep() allows to sleep also for less than a second (the old sleep() had only second resolution). In addition, nanosleep won't interfere with SIGALRM and in case of EINTR, it returns the time left, so you can easily continue in a while loop. - POSIX.1b defines also itimers, however instead of what the existing BSD itimers provide, you now can deal with several timers (at least 32 per process) and you have again theoretically up to one nanosecond resolution. The old itimer functions can still easily be implemented in libc for compatibility reasons using new POSIX-style itimer system calls. Implementation status: The POSIX clock and itimers system calls have not not yet been implemented, although much of the functionality is already available in the form of the BSD timers and adding them should be quite easy. Queued signals have to be implemented first for itimers. Nanosleep() is already available in Linux, but at the moment it supports only 10 ms resolution and it can optionally perform short microsecond precision busy waits of up to 2 ms length. Scheduling ---------- Linux 1.2 has so far been optimized a lot as a time sharing system, where several people run application programs like editors, compilers, debuggers, X window servers, networking daemons, etc. and do word processing, software development, etc. However there are a lot of applications for which Linux is currently unusable and for which even die-hard Linux enthusiasts have to keep a stand-alone DOS version on their disk. For >90% of these applications, the fact that Linux is incapable of guaranteeing the response time of an application is the major problem. Software for controlling e.g. an EPROM programmer, a robot arm, or an astronomical CCD camera is not realizable under a classic Unix if there is no dedicated real-time controller present in the controlled device. A lot of commercially available hardware has been designed with the real-time "capability" of DOS in mind and has no own microcontroller for time-critical actions. A real-world example: I have myself spent a long frustrating time of trying to implement an interface to a pay-TV decoder for Linux (which emulates a chip card and allows you to watch pay-TV for free :-). In this application, you have to wait for an incoming byte on the serial port, then you have to wait for around 0.7 to 2 ms (never shorter, never longer, otherwise the TV decoder gets a timeout and stops!) before returning an answer byte. It is virtually impossible to implement a user process for this task under Linux 1.2, while it is trivial to do this under DOS. I am looking forward to the day when Linux provides enough real-time support for this application so that I can finally remove MS-DOS from my harddisk. For these and similar real-time applications, POSIX.1b specifies three different schedulers, each with static priorities: SCHED_FIFO A preemptive, priority based scheduler. Each process managed under this scheduling priority possesses the CPU as long as (a) it does not block itself and (b) there comes no interrupt which puts another process into a higher priority wait queue. There exists a FIFO queue for each priority level and every process which gets runnable again is inserted into the queue behind all other processes. This is the most popular scheduler used in typical real-time operating systems. Function sched_yield() allows the process to go to the end of the FIFO queue without blocking. SCHED_RR A preemptive, priority based round robin scheduling strategy with quanta. It is a very similar to SCHED_FIFO, however each process has a time quantum and the process becomes preempted and is inserted at the end of the FIFO for the same priority level if it runs longer than the time quantum and other processes of the same priority level are waiting in the queue. Processes of lower priorities will like in SCHED_FIFO never get the CPU as long as a higher level process is in a ready queue and if a higher priority process becomes ready to run, it also gets the CPU immediately. SCHED_OTHER This is any implementation defined scheduler. Under Linux it is the classic time-sharing scheduler with "nice" values, etc. Under Linux, all SCHED_OTHER processes share the common static priority value 0. For security reasons, only root processes should under Linux be allowed to get any static priority higher than the one for SCHED_OTHER, because if these real-time scheduling mechanisms are abused, the whole system can be blocked. If one is developing a real-time application, it is a very good idea to have a shell with a higher static priority somewhere open in order to be able to kill the tested application in case something goes wrong. You should be aware, that if you use X11, not only the shell, but also the X server, the window manager and xterm will require a higher static priority in order to stop processes blocking the rest of the system. Therefore, testing real-time software will usually better be done on the console. With this POSIX.1b functionality, it is possible to run real-time software under Linux by giving it root permissions and assigning it a SCHED_FIFO strategy and a higher static priority than all other classic SCHED_OTHER Linux processes. In addition, typical real-time application lock their pages with mlockall() into the memory in order to avoid being swapped out. This guarantees that the real-time application can react as soon as possible on any interrupts and that the response time will not be influenced by the complicated normal Linux time-sharing priority mechanisms or by paging. However, please read also the chapter on interrupt dispatch latency below! The new functions are here: sched_setparam(), sched_getparam(), sched_setscheduler(), sched_getscheduler(), sched_yield(), sched_get_priority_max(), sched_get_priority_min(), sched_rr_get_interval(). Implementation status: The sched_*() system calls are now available since Linux 1.3.55 (Markus Kuhn ). Although much testing, performance evaluation, and optimization with the real-time scheduler is necessary (especially with regard to "fast interrupt handling, see below), you can already use the sched_*() calls in order to get the processor exclusively. There are also libc and manual pages available. Some earlier work on this was done by David F. Carlson in his POSIX.4_scheduler patch against Linux 1.2 (still available on sunsite). Asynchronous I/O (aio) ---------------------- POSIX.1b defines a number of functions which allow to send a long list of read/write requests at various seek positions in various files to the kernel with one single lio_listio() system call. While the process continues to execute the next instructions, the kernel will asynchronously read or write the requested pages and will send signals when the task has been completed (if this is desired). This is very nice for a database which knows that it will require a lot of different blocks scattered on a file. It will simply pass a list of the blocks to the kernel, and the kernel can optimize the disk head movement before sending the requests to the device. In addition this minimizes the number of system calls and allows the database to do something else in the meantime (e.g. waiting for the client process sending an abort instruction in which case the database server can cancel the async i/o requests with aio_cancel()). Another important application of aio are multimedia systems like MPEG or sound file players and recorders. These programs want to preload the next few seconds of the data stream from harddisk into locked memory, but also want to continue showing the video on the screen at the same time without any interruptions caused by synchronous I/O. The POSIX.1b async I/O calls provide a nice way for such anticipatory loading. POSIX.1b also defines priorities for asynchronous I/O, i.e. there is a way to tell the kernel that the read request for the MPEG player is more important than the read request of gcc. On a future real-time Linux, you don't want to see any image distortions while watching MPEG video and compiling a kernel at the same time if you gave the MPEG player a higher static priority. New functions in this area are: aio_read(), aio_write(), lio_listio(), aio_suspend(), aio_cancel(), aio_error(), aio_return(), aio_fsync(). Implementation status: Not yet implemented. The aio functions are probably best implemented in libc using kernel threads and the normal synchronous I/O system calls. There has recently been some progress on implementing kernel threads in Linux 1.3 using the clone() system call. Adding priority I/O to Linux might be a more complicated job, because many device drivers would have to be extended by priority wait queues. Implemented options ------------------- As POSIX.1b conformance does not require the implementation of all these functions, macros have been specified for that indicate to application software which of the POSIX.1b functionality is available on this system. This way, portable software can be written that uses real-time features only when they are available. Under the latest Linux kernel and libc development versions, the following POSIX.1b macros have been defined and indicate implemented functions: _POSIX_FSYNC _POSIX_MAPPED_FILES _POSIX_MEMLOCK _POSIX_MEMLOCK_RANGE _POSIX_MEMORY_PROTECTION _POSIX_PRIORITY_SCHEDULING The POSIX.1b options indicated by the following macros have not yet been implemented under Linux: _POSIX_ASYNCHRONOUS_IO _POSIX_MESSAGE_PASSING _POSIX_PRIORITIZED_IO _POSIX_REALTIME_SIGNALS _POSIX_SEMAPHORES _POSIX_SHARED_MEMORY_OBJECTS _POSIX_SYNCHRONIZED_IO _POSIX_TIMERS General real-time problems -------------------------- Apart from implementing new POSIX.1b system calls, there are a number of other problems with the current Linux kernel that have to be solved in order to improve real-time performance. Problem 1: Interrupt dispatch latency A blocked process waiting for the end of some I/O request becomes runnable again when the CPU receives an interrupt from the I/O device that processed the request. The time that passes between the interrupt and the execution of the interrupt handler is called the "interrupt latency". The time that passes between the interrupt and the continuation of the execution of the blocked process is called the "interrupt dispatch latency". We are assuming that the blocked process is the process with the highest priority (the POSIX.1b scheduler system calls allow to ensure this). Many real-time applications require that the interrupt dispatch latency is as short as possible, some applications require even guarantees for the maximum interrupt dispatch latency (e.g. 20 microseconds). At the moment, Linux has basically two types of interrupt handlers: "fast" and "slow" ones. The "slow interrupt handlers" call the scheduler each time immediately after the interrupt has been handled. This guarantees that if the process has become runnable by handling the interrupt and has the highest priority, then it will directly get control over the CPU after the interrupt handler. This ensures a short interrupt dispatch latency, but the cost is that the scheduler is executed for each interrupt, which can cause a system performance degradation for devices with a high interrupt rate (e.g., network controllers). The slow interrupt handler do not disable other interrupts while they are being executed. This ensures a low interrupt latency, because other higher priority interrupts will not be blocked. The "slow" interrupt handler behavior is what you want for real-time applications. The "fast interrupt handlers" only mark in a bitmask that an interrupt has been handled for this device. The scheduler is NOT called after the fast interrupt handler and the process waiting on the interrupt may have to wait up to 1 s / HZ = 10 ms for the next timer interrupt which will call the scheduler again (because the timer interrupt is handled the "slow" way). "Fast" interrupt handlers are those which are installed by the driver by specifying the SA_INTERRUPT option when calling request_irq(). "Fast" interrupt handlers are the reason why for devices like the serial port, the interrupt dispatch latency can easily reach 10 ms and more. "Fast" interrupt handlers disable other interrupts while the handler is being executed. This increases interrupt latency. The Linux "fast" interrupt handlers cause very low interrupt handling overhead, but they can be the cause for a lot of headaches for real-time application developers. See linux/arch/i386/kernel/irq.c, linux/include/asm-i386/irq.h and linux/arch/i386/kernel/entry.S for implementation details of how fast and slow interrupts are handled in Linux. Examples for "slow" Linux 1.3 interrupt drivers with good interrupt dispatch latency are keyboard floppy disk timer sound cards mouse drivers some Ethernet cards Examples for "fast" interrupt drivers with bad interrupt dispatch latency are serial port some Ethernet cards most harddisk/SCSI drivers most CD-ROM drivers Especially for the serial port, it would be very nice to have a way to switch to low dispatch latency interrupt handling, such that an application can react as fast as possible on any incoming serial byte. Apart from the "fast" interrupt handler type, another important cause of latency in the serial port are the 16550 FIFO UARTs. At the moment, the UARTs are configured to trigger an interrupt (except for for bit rates < 2400 bit/s) only if the FIFO is filled with at least eight bytes or if an UART timeout has occured (this happens after four character transmission times). The 16550 chip can be configured to trigger limits 1, 4, 8, and 14, but Linux currently provides no ioctl() for this. For details about the FIFO UART, please consult the National Semiconductor PC16550D data sheet. For real-time applications, it would be very desirable to change the FIFO trigger limit to one byte so that incoming serial bytes are immediately delivered to the waiting process. Applications that depend heavily on minimum serial port interrupt dispatch latency are for example MIDI music systems, DCF77 reference time long-wave radio signal receivers in Europe, packet radio network interfaces, and ISO 7816 smartcard interfaces. Some existing work in this area: Stuart Cheshire has implemented a "smart" interrupt handler that can switch between the classic "fast" and "slow" alternatives (see for details). This way, you can get the real-time advantage of "slow" handling for the serial port without introducing a permanent system overhead even when no real-time application is active. Nick Simicich and Rik Faith have written a program called "cytune" which can change the FIFO thresholds for individual ports in the Cyclades async mux driver. Problem 2: Timer resolution On most Linux architectures, a timer interrupt occurs every 10 ms (HZ = 100). One exception is the Alpha architecture, where the timer interrupt comes every 1 ms (HZ = 1024). The macro HZ, which specifies the timer frequency, is defined in . At the moment, the kernel timer mechanism, on which the itimer and nanosleep implementation is based, checks after each timer interrupt whether a software timer has expired, makes the corresponding process runnable again, and calls the scheduler. This means that a highest priority real-time process can have to wait up to 2 * (1 s / HZ) = 20 ms longer than requested. An excellent solution would be to implement an interrupt-on-demand timer facility. The 8253 timer/counter chip in the PC would then have to be programmed such that it delivers an interrupt with microsecond precision exactly at the time when a software timer expires. This would increase the software timer precision by a factor of 10 000! As the 8253 timer would be programmed in a single shot mode, a lost interrupt might cause an accidental system halt. If this is problem, the CMOS battery clock, which can also be used to implement periodic interrupts, should be utilized to trigger periodic calls to the scheduler in order to ensure process preemption and kernel clock update. The time stamp counter (TSC) available in all Pentium processors is a 64-bit counter clocked at CPU frequency. It can be used on Pentium systems in order to get extremely precise timing information with theoretically close to nanosecond resolution and without overflow problems. Literature ---------- For those of you who have become interested in POSIX.1b, there exists a good book: Bill O. Gallmeister, POSIX.4 -- Programming for the Real World, O'Reilly & Associates, 1995, ISBN 1-56592-074-0. This book is not only a good introduction into POSIX.1b (which was originally called POSIX.4), it is also an easy reading nice way into the world of real-time operating systems for those developers who have so far been very UNIX and time-sharing oriented. You can order the POSIX.1b standard (officially called IEEE Std 1003.1b-1993; this book includes also all text of POSIX.1 and costs 114 USD) as well as the other POSIX standards directly from IEEE: phone: +1 908 981 1393 (TZ: eastern standard time) 1 800 678 4333 (from US+Canada only) fax: +1 908 981 9667 e-mail: stds.info@ieee.org Information about POSIX and other IEEE standards is also available on and , however unfortunately the full standard documents are only available as books or on CD-ROM, not on the Internet. Having access to the POSIX specs is certainly a good idea for any Linux kernel hacker. Here is a brief list of some of the POSIX standards: POSIX.1 Basic OS interface (C language) POSIX.1a Misc. extensions (symlinks, etc.) POSIX.1b Real-time and I/O extensions (was: POSIX.4) POSIX.1c Threads (was: POSIX.4a) POSIX.1d More real-time extensions (was: POSIX.4b) POSIX.1e Security extensions, ACLs (was: POSIX.6) POSIX.1f Transparent network file access (was: POSIX.8) POSIX.1g Protocol independent communication, sockets (was: POSIX.12) POSIX.1i Technical corrections to POSIX.1b POSIX.2 Shell and common utility programs (date, ln, ...) POSIX.3 Test methods POSIX.5 ADA binding to POSIX.1 POSIX.7 System administration POSIX.9 FORTRAN-77 binding to POSIX.1 POSIX.15 Supercomputing extensions (checkpoint/recovery, etc.) and a few others which are still in early draft stage. If you want to follow progress on POSIX standardization, you should read the announcements in the moderated USENET group comp.std.unix. The current status of POSIX drafts is summarized in . ISO has also published POSIX.1 as ISO/IEC 9945-1:1990. ISO and IEEE will soon publish the new revision of this standard: ISO/IEC 9945-1:1996. This will be the new 1996 revision of POSIX.1, which will contain in one single standard POSIX.1(1990), POSIX.1b(1993), POSIX.1c(1995), and POSIX.1i(1995) (perhaps also POSIX.1a(1996) if it gets ready in time). If you want to order the POSIX standard but are not in a hurry, may be it is a good idea to wait a few months until ISO/IEC 9945-1:1996 is available. All differences between POSIX.1(1990) and POSIX.1(1996) will be marked by bars at the page margins. This text just summarizes POSIX.1b and related work on Linux. Many people interested in POSIX.1b support seem also to be interested in POSIX.1c support (threads). Some information about POSIX.1c support is on and . There is now a production release package called "PCthreads (tm) POSIX threads for Linux" available from . It is maintained by Michael T. Peterson . A POSIX.1c package that provides real kernel-threads using the clone() system call is available from and has been developed by Xavier Leroy . From owner-linux@cthulhu.engr.sgi.com Mon Aug 5 17:00:31 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id RAA09688; Mon, 5 Aug 1996 17:00:30 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id XAA07267 for linux-list; Mon, 5 Aug 1996 23:51:20 GMT Received: from neteng.engr.sgi.com (neteng.engr.sgi.com [192.26.80.10]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA07262 for ; Mon, 5 Aug 1996 16:51:19 -0700 Received: (from lm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id QAA07180 for linux; Mon, 5 Aug 1996 16:51:19 -0700 Date: Mon, 5 Aug 1996 16:51:19 -0700 From: lm@neteng.engr.sgi.com (Larry McVoy) Message-Id: <199608052351.QAA07180@neteng.engr.sgi.com> To: linux@neteng.engr.sgi.com Subject: coolness Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk For an unrelated-to-Linux project that I'm working on with some other folks over here, we needed a user level NFS server. So I stole David's GCC for IRIX compilation environment and built the Linux user level NFS server code on IRIX. I uninstalled NFS (which unfortunately also removes YP, that's a bummer, I had to copy over hosts) and started up Linux's version. You can play with it right now. Try % cd /hosts/fubar.engr % ls Pretty cool, huh? --lm --- Larry McVoy lm@sgi.com http://reality.sgi.com/lm (415) 933-1804 From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 13:33:48 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA09604; Mon, 12 Aug 1996 13:33:48 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id UAA15296 for linux-list; Mon, 12 Aug 1996 20:33:42 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA15286 for ; Mon, 12 Aug 1996 13:33:40 -0700 Received: from refugee.engr.sgi.com (refugee.engr.sgi.com [150.166.61.22]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA29529 for ; Mon, 12 Aug 1996 13:31:56 -0700 Received: from refugee.engr.sgi.com by refugee.engr.sgi.com via ESMTP (950413.SGI.8.6.12/940406.SGI.AUTO) id NAA17431; Mon, 12 Aug 1996 13:32:01 -0700 Message-Id: <199608122032.NAA17431@refugee.engr.sgi.com> X-Mailer: exmh version 1.6.7 5/3/96 To: lm@gate1-neteng.engr.sgi.com (Larry McVoy) Cc: Nigel Gamble , Alistair Lambie , ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step In-reply-to: Message from lm@gate1-neteng of 2 Aug 1996 18:11:31 PDT Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 12 Aug 1996 13:32:01 -0700 From: Steve Alexander Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk lm@gate1-neteng (Larry McVoy) writes: >The fine grained locking is a lose. It costs us way too much and what I >am seeing is a trend backwards towards coarser grained locking. If this >wasn't true, why did we build lego? Why are we building Nexus? I'd like to see some hard numbers for these assertions. All of the bottlenecks in ficus (that I am aware of) have to do with context-switching due to use of mutexes rather than spinlocks. The locking overhead itself isn't even on the map as far as I can tell. However, I agree that more locking is not the answer. Eliminating as much shared, global data as possible is the answer. Nexus doesn't really do anything towards that end that I can see. >Linux already has a RT kernel. I just reviewed a fantastic Usenix paper >that added RT to Linux - fully preemptable, hard real time to < 100 usecs, >and less than 3K lines of code change. Their test case was a 100Khz clock >pulse on a parallel port; they got it every time, with less than 15usecs >skew, while the system was tar-ing one file system to another, running >netscape, and generally doing same old Unix stuff just like normal. It's >really impressive and very uninvasive. I'd like to see that when it comes out. >Finally, think about this: how many times have you had a "great idea" >for better IRIX performance, gone off an prototyped it, only to find >that it makes no difference? Linux lets you test out those ideas and >see the real performance difference, unadulterated by any surrounding >bloat. That's cool. We want that. If it makes no difference, what difference does it make what platform you try it on? If it does make a difference, you would be able to measure it despite the underlying bloat, and again the underlying platform makes no difference. I'm in favor of some amount of Linux work on SGI gear, but this is a pretty weak argument. -- Steve From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 13:53:19 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA10554; Mon, 12 Aug 1996 13:53:18 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id UAA19246 for linux-list; Mon, 12 Aug 1996 20:53:13 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA19241 for ; Mon, 12 Aug 1996 13:53:12 -0700 Received: from ratliff.engr.sgi.com (ratliff.engr.sgi.com [150.166.42.14]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id NAA29578 for ; Mon, 12 Aug 1996 13:51:27 -0700 Received: from localhost by ratliff.engr.sgi.com via SMTP (950413.SGI.8.6.12/940406.SGI.AUTO) id NAA08696; Mon, 12 Aug 1996 13:52:47 -0700 Message-Id: <199608122052.NAA08696@ratliff.engr.sgi.com> To: Steve Alexander cc: lm@gate1-neteng.engr.sgi.com (Larry McVoy), Nigel Gamble , Alistair Lambie , ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step In-reply-to: Your message of "Mon, 12 Aug 96 13:32:01 PDT." <199608122032.NAA17431@refugee.engr.sgi.com> Date: Mon, 12 Aug 96 13:52:47 -0700 From: Bob English Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk >lm@gate1-neteng (Larry McVoy) writes: >The fine grained locking is a lose. It costs us way too much and what I >am seeing is a trend backwards towards coarser grained locking... In message <199608122032.NAA17431@refugee.engr.sgi.com> you write: >I'd like to see some hard numbers for these assertions. All of the bottlenecks >in ficus (that I am aware of) have to do with context-switching due to use of >mutexes rather than spinlocks. The locking overhead itself isn't even on the >map as far as I can tell. If you look at larry's context switch and pipe benchmarks, you'll see that they run much faster on a single CPU than on two or more. A lot of the difference is due to fine-grained locking, which causes more dirty cache lines to thrash than is actually necessary for the operations involved. Pipe communication takes locks at the vnode and inode level. Context switches take locks on threads, accounting structures, and queues. It all adds up. >If it makes no difference, what difference does it make what platform you try >it on? If it does make a difference, you would be able to measure it despite >the underlying bloat, and again the underlying platform makes no difference. Back to the previous discussion. Getting rid of any one of those locks wouldn't make much difference. Getting rid of all of them would, but is a much taller order. --bob-- From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 14:10:11 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA13409; Mon, 12 Aug 1996 14:10:11 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id VAA22942 for linux-list; Mon, 12 Aug 1996 21:10:06 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA22927 for ; Mon, 12 Aug 1996 14:10:04 -0700 Received: from neteng.engr.sgi.com (gate5-neteng.engr.sgi.com [150.166.61.33]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29622 for ; Mon, 12 Aug 1996 14:08:20 -0700 Received: from localhost (lm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via SMTP id OAA13322; Mon, 12 Aug 1996 14:09:39 -0700 Message-Id: <199608122109.OAA13322@neteng.engr.sgi.com> To: Steve Alexander From: lm@gate1-neteng.engr.sgi.com (Larry McVoy) cc: lm@neteng.engr.sgi.com, Nigel Gamble , Alistair Lambie , ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step Date: Mon, 12 Aug 1996 14:09:39 -0700 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk : However, I agree that more locking is not the answer. Eliminating as much : shared, global data as possible is the answer. Nexus doesn't really do : anything towards that end that I can see. Yeah. : >Linux already has a RT kernel. I just reviewed a fantastic Usenix paper : >that added RT to Linux - fully preemptable, hard real time to < 100 usecs, : >and less than 3K lines of code change. Their test case was a 100Khz clock : >pulse on a parallel port; they got it every time, with less than 15usecs : >skew, while the system was tar-ing one file system to another, running : >netscape, and generally doing same old Unix stuff just like normal. It's : >really impressive and very uninvasive. : : I'd like to see that when it comes out. You'll have to borrow my copy, Usenix rejected it. Fuckheads. : >Finally, think about this: how many times have you had a "great idea" : >for better IRIX performance, gone off an prototyped it, only to find : >that it makes no difference? Linux lets you test out those ideas and : >see the real performance difference, unadulterated by any surrounding : >bloat. That's cool. We want that. : : If it makes no difference, what difference does it make what platform you try : it on? If it does make a difference, you would be able to measure it despite : the underlying bloat, and again the underlying platform makes no difference. Suppose you are trying to reach some performance point, X. Whatever that is. You do some work that would, in theory, get you closer to point X. It doesn't, you can't see the difference. Just for the sake of argument, suppose that that work was indeed correctly focussed on something that needed to get fixed in order to get to point X. But the effects of that work were overshadowed by all of the surrounding bloat. That doesn't mean that the work was useless or wrong, it means that the bloat dominates. If the bloat wasn't there, you would need to do the work to get to X. A more concrete example: IRIX syscall entry is something like 15-30 usecs, if I remember correctly. On Linux, it's about .5 - 5 usecs. In the Linux world, optimizations that made a 100% difference would be lost in the bloat in IRIX. Does that mean that those optimizations are pointless? Not to me, it doesn't. I want a kernel that is fast enough that dropping into to diddle something and then popping back out is about a procedure call, no more. IRIX is miles from that, Linux is approaching it. From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 14:20:22 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA13577; Mon, 12 Aug 1996 14:20:22 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id VAA24984 for linux-list; Mon, 12 Aug 1996 21:20:08 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA24960 for ; Mon, 12 Aug 1996 14:20:06 -0700 Received: from refugee.engr.sgi.com (refugee.engr.sgi.com [150.166.61.22]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29660 for ; Mon, 12 Aug 1996 14:18:21 -0700 Received: from refugee.engr.sgi.com by refugee.engr.sgi.com via ESMTP (950413.SGI.8.6.12/940406.SGI.AUTO) id OAA18053; Mon, 12 Aug 1996 14:18:18 -0700 Message-Id: <199608122118.OAA18053@refugee.engr.sgi.com> X-Mailer: exmh version 1.6.7 5/3/96 To: Bob English Cc: lm@gate1-neteng.engr.sgi.com (Larry McVoy), Nigel Gamble , Alistair Lambie , ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step In-reply-to: Message from renglish@ratliff of 12 Aug 1996 13:52:47 PDT Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 12 Aug 1996 14:18:18 -0700 From: Steve Alexander Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Bob English writes: >If you look at larry's context switch and pipe benchmarks, you'll see >that they run much faster on a single CPU than on two or more. A lot of >the difference is due to fine-grained locking, which causes more dirty >cache lines to thrash than is actually necessary for the operations >involved. Pipe communication takes locks at the vnode and inode >level. Context switches take locks on threads, accounting structures, >and queues. If I look at those benchmarks, they'll probably run faster on Linux on an Indy than they do on IRIX on an Indy, so I doubt the locking is the big problem we need to be looking at. I don't have numbers from David to quote, so maybe I'm wrong. I'd like to be wrong. -- Steve From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 14:22:32 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA13760; Mon, 12 Aug 1996 14:22:32 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id VAA25633 for linux-list; Mon, 12 Aug 1996 21:22:20 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA25618 for ; Mon, 12 Aug 1996 14:22:18 -0700 Received: from refugee.engr.sgi.com (refugee.engr.sgi.com [150.166.61.22]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29714 for ; Mon, 12 Aug 1996 14:20:33 -0700 Received: from refugee.engr.sgi.com by refugee.engr.sgi.com via ESMTP (950413.SGI.8.6.12/940406.SGI.AUTO) id OAA18080; Mon, 12 Aug 1996 14:20:39 -0700 Message-Id: <199608122120.OAA18080@refugee.engr.sgi.com> X-Mailer: exmh version 1.6.7 5/3/96 To: lm@gate1-neteng.engr.sgi.com (Larry McVoy) Cc: lm@neteng.engr.sgi.com, Nigel Gamble , Alistair Lambie , ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step In-reply-to: Message from lm@gate1-neteng of 12 Aug 1996 14:09:39 PDT Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 12 Aug 1996 14:20:39 -0700 From: Steve Alexander Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk lm@gate1-neteng (Larry McVoy) writes: >You'll have to borrow my copy, Usenix rejected it. Fuckheads. No doubt to make room for some paper on TCL scripts or something. Feh. Yeah, if you can make me a copy or convince the authors to send me one I'd appreciate it. >A more concrete example: IRIX syscall entry is something like 15-30 usecs, >if I remember correctly. On Linux, it's about .5 - 5 usecs. In the >Linux world, optimizations that made a 100% difference would be lost in >the bloat in IRIX. Does that mean that those optimizations are pointless? To some degree, yes. This is why application developers often have a lot of platform-specific code; optimizations that work on one system won't necessarily work on all systems. I'm not saying this is good, mind you. -- Steve From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 14:25:26 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA13804; Mon, 12 Aug 1996 14:25:26 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id VAA26341 for linux-list; Mon, 12 Aug 1996 21:25:22 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA26334 for ; Mon, 12 Aug 1996 14:25:20 -0700 Received: from ratliff.engr.sgi.com (ratliff.engr.sgi.com [150.166.42.14]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29751 for ; Mon, 12 Aug 1996 14:23:35 -0700 Received: from localhost by ratliff.engr.sgi.com via SMTP (950413.SGI.8.6.12/940406.SGI.AUTO) id OAA08877; Mon, 12 Aug 1996 14:24:15 -0700 Message-Id: <199608122124.OAA08877@ratliff.engr.sgi.com> To: Steve Alexander cc: Bob English , lm@gate1-neteng.engr.sgi.com (Larry McVoy), Nigel Gamble , Alistair Lambie , ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com, renglish@ratliff.engr.sgi.com Subject: Re: Linux: the next step In-reply-to: Your message of "Mon, 12 Aug 96 14:18:18 PDT." <199608122118.OAA18053@refugee.engr.sgi.com> Date: Mon, 12 Aug 96 14:24:15 -0700 From: Bob English Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk In message <199608122118.OAA18053@refugee.engr.sgi.com> you write: >If I look at those benchmarks, they'll probably run faster on Linux on an Indy >than they do on IRIX on an Indy, so I doubt the locking is the big problem we >need to be looking at. I don't have numbers from David to quote, so maybe I'm >wrong. I'd like to be wrong. I was comparing Irix MP times vs. Irix UP times, and the difference is both substantial and locking related. Of course, if Irix were as efficient on a UP as Linux, the difference might be a factor of 10 rather than a factor of 2 :-). --bob-- From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 14:36:07 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA15199; Mon, 12 Aug 1996 14:36:07 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id VAA29439 for linux-list; Mon, 12 Aug 1996 21:36:01 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29434 for ; Mon, 12 Aug 1996 14:36:00 -0700 Received: from surfdog.csd.sgi.com (surfdog.csd.sgi.com [150.166.145.79]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29781 for ; Mon, 12 Aug 1996 14:34:15 -0700 Received: by surfdog.csd.sgi.com (940816.SGI.8.6.9/911001.SGI) id OAA05211; Mon, 12 Aug 1996 14:35:34 -0700 From: "Sandeep Cariapa" Message-Id: <9608121435.ZM5209@surfdog.csd.sgi.com> Date: Mon, 12 Aug 1996 14:35:34 -0700 In-Reply-To: Steve Alexander "Re: Linux: the next step" (Aug 12, 2:20pm) References: <199608122120.OAA18080@refugee.engr.sgi.com> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: linux@yon.engr.sgi.com Subject: Re: Linux: the next step Cc: lm@neteng.engr.sgi.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk On Aug 12, 2:20pm, Steve Alexander wrote: > Subject: Re: Linux: the next step > lm@gate1-neteng (Larry McVoy) writes: > >You'll have to borrow my copy, Usenix rejected it. Fuckheads. No, tell us what you really feel Larry; don't hold back :-) > No doubt to make room for some paper on TCL scripts or something. Feh. > Yeah, if you can make me a copy or convince the authors to send me one I'd > appreciate it. Larry, you might want to make this available to the entire Linux mailing list; I suspect a lot more people would be interested in it. (I am too) Regards, Sandeep >-- End of excerpt from Steve Alexander From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 14:39:44 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA15284; Mon, 12 Aug 1996 14:39:44 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id VAA29929 for linux-list; Mon, 12 Aug 1996 21:39:40 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29924 for ; Mon, 12 Aug 1996 14:39:39 -0700 Received: from machine.engr.sgi.com (fddi-machine.engr.sgi.com [192.26.75.22]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29798 for ; Mon, 12 Aug 1996 14:37:54 -0700 Received: by machine.engr.sgi.com (950413.SGI.8.6.12/940406.SGI.AUTO) id OAA12095; Mon, 12 Aug 1996 14:39:21 -0700 From: jes@machine.engr.sgi.com (John E. Schimmel) Message-Id: <199608122139.OAA12095@machine.engr.sgi.com> Subject: Re: Linux: the next step To: sca@refugee.engr.sgi.com (Steve Alexander) Date: Mon, 12 Aug 1996 14:39:21 -0700 (PDT) Cc: lm@gate1-neteng.engr.sgi.com, lm@neteng.engr.sgi.com, nigel@cthulhu.engr.sgi.com, alambie@wellington.sgi.com, ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com In-Reply-To: <199608122120.OAA18080@refugee.engr.sgi.com> from "Steve Alexander" at Aug 12, 96 02:20:39 pm X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit Content-Length: 1012 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk > > lm@gate1-neteng (Larry McVoy) writes: > >You'll have to borrow my copy, Usenix rejected it. Fuckheads. > > No doubt to make room for some paper on TCL scripts or something. Feh. > Yeah, if you can make me a copy or convince the authors to send me one I'd > appreciate it. Now wait a minute here. Read the paper before commenting. They create a "real-time" thread under Linux. That does not mean that it does anything, that the OS has real time support, etc. There was nothing particularly new about what it did. There is now a Linux conference which should showcase this sort of thing. Usenix is for new and interesting work that would be of interest to a large audience. > > -- Steve > > ------------------------------------------------------------ John E. Schimmel jes@sgi.com Silicon Graphics Inc. Voice: (415)933-4116 http://reality.sgi.com/employees/jes Fax: (415)967-8496 ------------------------------------------------------------ From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 14:50:22 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA15492; Mon, 12 Aug 1996 14:50:22 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id VAA02665 for linux-list; Mon, 12 Aug 1996 21:50:07 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA02659 for ; Mon, 12 Aug 1996 14:50:05 -0700 Received: from neteng.engr.sgi.com (gate5-neteng.engr.sgi.com [150.166.61.33]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id OAA29836 for ; Mon, 12 Aug 1996 14:48:17 -0700 Received: from localhost (lm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via SMTP id OAA15461; Mon, 12 Aug 1996 14:49:45 -0700 Message-Id: <199608122149.OAA15461@neteng.engr.sgi.com> To: jes@machine.engr.sgi.com (John E. Schimmel) From: lm@gate1-neteng.engr.sgi.com (Larry McVoy) cc: sca@refugee.engr.sgi.com (Steve Alexander), lm@neteng.engr.sgi.com, nigel@cthulhu.engr.sgi.com, alambie@wellington.sgi.com, ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step Date: Mon, 12 Aug 1996 14:49:45 -0700 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk : > No doubt to make room for some paper on TCL scripts or something. Feh. : > Yeah, if you can make me a copy or convince the authors to send me one I'd : > appreciate it. : : Now wait a minute here. Read the paper before commenting. : They create a "real-time" thread under Linux. That does not : mean that it does anything, that the OS has real time support, : etc. There was nothing particularly new about what it did. Here's what they did: they took over control of interrupt disable/enable and dispatch. They run Linux as a thread and allow you to have 0 or more real time threads. If all you are doing is running Linux, it looked to me like there was minimal impact to the performance of the system. Real time threads run on the bare hardware, they have little integration with the OS, other than a pipe like communications device. The programming paradigm is you run two processes, one real time that gathers events and stuffs them in the "pipe", and one normal that reads events out of the pipe and does time _un_critical processing. They can guarentee real time response of 10Khz, with less than 15 usec variation. That was with Linux running a tar f - /usr | (cd /newusr; tar xf -) a web browser, X, etc, etc. Good luck doing that on IRIX up. Maybe MP. I thought the new work was the idea of *not* buggering up your entire OS to support real time. They give the real time guys ownership of the CPU with minimal impact on the rest of the system. And they had to do next to nothing to make this work, they just use Unix for everything except the lowest level of RT work. I dunno, maybe I'm projecting some pipe dream, but I think this is really cool work. Points for orthogonal thinking on their part. From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 16:00:31 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA01324; Mon, 12 Aug 1996 16:00:30 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id WAA16841 for linux-list; Mon, 12 Aug 1996 22:59:25 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id PAA16829 for ; Mon, 12 Aug 1996 15:59:23 -0700 Received: from aa5b.engr.sgi.com (aa5b.engr.sgi.com [192.102.117.24]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id PAA29987 for ; Mon, 12 Aug 1996 15:57:39 -0700 Received: (from nigel@localhost) by aa5b.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id PAA28318; Mon, 12 Aug 1996 15:59:04 -0700 From: nigel@aa5b.engr.sgi.com (Nigel Gamble) Message-Id: <199608122259.PAA28318@aa5b.engr.sgi.com> Subject: Re: Linux: the next step To: lm@gate1-neteng.engr.sgi.com (Larry McVoy) Date: Mon, 12 Aug 1996 15:59:04 -0700 (PDT) Cc: jes@machine.engr.sgi.com, sca@refugee.engr.sgi.com, lm@neteng.engr.sgi.com, nigel@cthulhu.engr.sgi.com, alambie@wellington.sgi.com, ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com In-Reply-To: <199608122149.OAA15461@neteng.engr.sgi.com> from "Larry McVoy" at Aug 12, 96 02:49:45 pm X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit Content-Length: 2161 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk > : Now wait a minute here. Read the paper before commenting. > : They create a "real-time" thread under Linux. That does not > : mean that it does anything, that the OS has real time support, > : etc. There was nothing particularly new about what it did. > > Here's what they did: they took over control of interrupt disable/enable > and dispatch. They run Linux as a thread and allow you to have 0 or more > real time threads. If all you are doing is running Linux, it looked > to me like there was minimal impact to the performance of the system. > > Real time threads run on the bare hardware, they have little > integration with the OS, other than a pipe like communications device. > The programming paradigm is you run two processes, one real time that > gathers events and stuffs them in the "pipe", and one normal that reads > events out of the pipe and does time _un_critical processing. That's fine for one class of real-time problems, but it's not going to help make the RealAudio player (raplayer) useable on Linux. RealAudio only has to produce AM radio quality audio from a 28.8kbps input stream, yet I only have to move my mouse into a new window to get audio drop-out. The RealAudio application, together with all the support that it is getting from the kernel in audio drivers, networking code etc., needs to run at a higher priority than most of the other stuff that might be going on. This is also a "real time problem", although a different one from the "get the O/S out of my way" hard real time crowd. How will Linux solve this type of problem? > They can guarentee real time response of 10Khz, with less than 15 usec > variation. That was with Linux running a > > tar f - /usr | (cd /newusr; tar xf -) > > a web browser, X, etc, etc. Good luck doing that on IRIX up. Maybe MP. We can already do this on MP with User Level Interrupts. As for UP, stay tuned :-) (Since this mailing list goes outside of SGI, I'm not going to say any more.) -- Nigel Gamble "Are we going to push the edge of the envelope, Brain?" Silicon Graphics "No, Pinky, but we may get to the sticky part." nigel@sgi.com (415) 933-3109 From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 16:32:58 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA03109; Mon, 12 Aug 1996 16:32:58 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id XAA23375 for linux-list; Mon, 12 Aug 1996 23:31:52 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA23367 for ; Mon, 12 Aug 1996 16:31:51 -0700 Received: from ratliff.engr.sgi.com (ratliff.engr.sgi.com [150.166.42.14]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA00094 for ; Mon, 12 Aug 1996 16:30:06 -0700 Received: from localhost by ratliff.engr.sgi.com via SMTP (950413.SGI.8.6.12/940406.SGI.AUTO) id QAA09379; Mon, 12 Aug 1996 16:31:31 -0700 Message-Id: <199608122331.QAA09379@ratliff.engr.sgi.com> To: nigel@aa5b.engr.sgi.com (Nigel Gamble) cc: lm@gate1-neteng.engr.sgi.com (Larry McVoy), jes@machine.engr.sgi.com, sca@refugee.engr.sgi.com, lm@neteng.engr.sgi.com, alambie@wellington.sgi.com, ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com Subject: Re: Linux: the next step In-reply-to: Your message of "Mon, 12 Aug 96 15:59:04 PDT." <199608122259.PAA28318@aa5b.engr.sgi.com> Date: Mon, 12 Aug 96 16:31:30 -0700 From: Bob English Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk In message <199608122259.PAA28318@aa5b.engr.sgi.com> you write: >That's fine for one class of real-time problems, but it's not going >to help make the RealAudio player (raplayer) useable on Linux... >The RealAudio application, together with all the support that it is >getting from the kernel in audio drivers, networking code etc., needs >to run at a higher priority than most of the other stuff that might be >going on... > >How will Linux solve this type of problem? With mirrors. You run another copy of Linux on a high-priority RT thread. Full API, fast preemption, minimal performance and code impact. You work on linux-slow and run rt apps on linux-fast. You could even reboot linux-slow without losing your audio signal :-). --bob-- From owner-linux@cthulhu.engr.sgi.com Mon Aug 12 16:56:47 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA03662; Mon, 12 Aug 1996 16:56:47 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id XAA27436 for linux-list; Mon, 12 Aug 1996 23:55:53 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA27431 for ; Mon, 12 Aug 1996 16:55:51 -0700 Received: (from ariel@localhost) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id QAA00152 for linux; Mon, 12 Aug 1996 16:54:07 -0700 From: ariel@yon.engr.sgi.com (Ariel Faigon) Message-Id: <199608122354.QAA00152@yon.engr.sgi.com> Subject: Linux external vs. internal lists To: linux@yon.engr.sgi.com Date: Mon, 12 Aug 1996 16:54:06 -0700 (PDT) Reply-To: ariel@cthulhu.engr.sgi.com Organization: Silicon Graphics Inc. X-Mailer: ELM [version 2.4 PL24 ME5a] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Length: 750 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk [This is being sent to linux@engr.sgi.com only] Just to follow up on Nigel's last comment. Please note: linux@engr is strictly internal to sgi.com while: linuxmips@corp is unrestricted and subscribed to by Red Hat, Miguel de Icaza, ex SGI employees, and a few other interested parties. I once believed I'm allowing external addresses on linux@engr but it turned out that majordomo@engr silently rejects them. Feel free, at your discretion, to discuss internal stuff on linux@engr. Also: please continue to keep linuxmips@corp low-volume (it was originally intended only for significant announcements like "Linux now boots single user on Indy") I believe there was only one announcement on it since it was created. -- Peace, Ariel From owner-linux@cthulhu.engr.sgi.com Tue Aug 13 04:46:11 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id EAA01457; Tue, 13 Aug 1996 04:46:11 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id LAA02414 for linux-list; Tue, 13 Aug 1996 11:46:02 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id EAA02407 for ; Tue, 13 Aug 1996 04:45:59 -0700 Received: from neteng.engr.sgi.com (gate5-neteng.engr.sgi.com [150.166.61.33]) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id EAA00933 for ; Tue, 13 Aug 1996 04:44:15 -0700 Received: (from dm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id EAA01442; Tue, 13 Aug 1996 04:45:56 -0700 Date: Tue, 13 Aug 1996 04:45:56 -0700 Message-Id: <199608131145.EAA01442@neteng.engr.sgi.com> From: "David S. Miller" To: nigel@aa5b.engr.sgi.com CC: lm@neteng.engr.sgi.com, jes@machine.engr.sgi.com, sca@refugee.engr.sgi.com, lm@neteng.engr.sgi.com, nigel@cthulhu.engr.sgi.com, alambie@wellington.sgi.com, ariel@cthulhu.engr.sgi.com, linux@yon.engr.sgi.com In-reply-to: <199608122259.PAA28318@aa5b.engr.sgi.com> (nigel@aa5b) Subject: Re: Linux: the next step Reply-to: dm@sgi.com Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk From: nigel@aa5b (Nigel Gamble) Date: Mon, 12 Aug 1996 15:59:04 -0700 (PDT) > They can guarentee real time response of 10Khz, with less than 15 usec > variation. That was with Linux running a > > tar f - /usr | (cd /newusr; tar xf -) > > a web browser, X, etc, etc. Good luck doing that on IRIX up. Maybe MP. We can already do this on MP with User Level Interrupts. I cannot wait to recompile all of my applications so that they can take advantage of ULI... The whole idea behind whatever approach linux will take to anything (using clone() for threads under Linux is a good example) is that you will not need to teach all of your "dumb" old programs (even ps!) about these special processes and or things a process can do. dm@engr.sgi.com 'Ooohh.. "FreeBSD is faster over loopback, when compared to Linux over the wire". Film at 11.' -Linus From owner-linux@cthulhu.engr.sgi.com Tue Aug 13 08:37:04 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id IAA05128; Tue, 13 Aug 1996 08:37:04 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id PAA21898 for linux-list; Tue, 13 Aug 1996 15:36:38 GMT Received: from aa5b.engr.sgi.com (aa5b.engr.sgi.com [192.102.117.24]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id IAA21887 for ; Tue, 13 Aug 1996 08:36:36 -0700 Received: (from nigel@localhost) by aa5b.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id IAA29651; Tue, 13 Aug 1996 08:36:36 -0700 From: nigel@aa5b.engr.sgi.com (Nigel Gamble) Message-Id: <199608131536.IAA29651@aa5b.engr.sgi.com> Subject: Re: Linux: the next step To: dm@sgi.com Date: Tue, 13 Aug 1996 08:36:35 -0700 (PDT) Cc: linux@aa5b.engr.sgi.com In-Reply-To: <199608131145.EAA01442@neteng.engr.sgi.com> from "David S. Miller" at Aug 13, 96 04:45:56 am X-Mailer: ELM [version 2.4 PL23] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit Content-Length: 2160 Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk > From: nigel@aa5b (Nigel Gamble) > Date: Mon, 12 Aug 1996 15:59:04 -0700 (PDT) > > > They can guarentee real time response of 10Khz, with less than 15 usec > > variation. That was with Linux running a > > > > tar f - /usr | (cd /newusr; tar xf -) > > > > a web browser, X, etc, etc. Good luck doing that on IRIX up. Maybe MP. > > We can already do this on MP with User Level Interrupts. > > I cannot wait to recompile all of my applications so that they can > take advantage of ULI... I think you missed my point. The real time approach that Larry described will only suit the hard real time programmers who want only to get the O/S out of the way and program the bare metal. The real time application code will have to be custom written and will get no support from Linux itself - you can't call Linux system calls from one of these real time threads. Forget POSIX real time APIs! IRIX's ULI mechanism already provides a similar facility for people who are willing to trade O/S support for performance, (since you can't execute IRIX system calls from a ULI, either). > The whole idea behind whatever approach linux will take to anything > (using clone() for threads under Linux is a good example) is that you > will not need to teach all of your "dumb" old programs (even ps!) > about these special processes and or things a process can do. The next release of IRIX will have real time support for user applications, even on a UP, with guarantees sufficient for digital media (audio and video). You won't need to recompile your application to take advantage of this environment. We already know what we need to do to achieve this, and have already implemented most of it. Our approach is based on a fully preemptible kernel, interrupts that have a thread context that can block, and making sure that interrupts are never disabled for more than 100us. I'm still waiting to hear "whatever approach linux will take" to solve this problem. -- Nigel Gamble "Are we going to push the edge of the envelope, Brain?" Silicon Graphics "No, Pinky, but we may get to the sticky part." nigel@sgi.com (415) 933-3109 From owner-linux@cthulhu.engr.sgi.com Tue Aug 13 09:46:17 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id JAA08170; Tue, 13 Aug 1996 09:46:17 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id QAA03191 for linux-list; Tue, 13 Aug 1996 16:45:59 GMT Received: from aa5b.engr.sgi.com (aa5b.engr.sgi.com [192.102.117.24]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id JAA03117 for ; Tue, 13 Aug 1996 09:45:50 -0700 Received: from xtp.engr.sgi.com (xtp.engr.sgi.com [150.166.75.34]) by aa5b.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id JAA29778; Tue, 13 Aug 1996 09:45:46 -0700 Received: by xtp.engr.sgi.com (940816.SGI.8.6.9/911001.SGI) id JAA07240; Tue, 13 Aug 1996 09:45:46 -0700 From: "Greg Chesson" Message-Id: <9608130945.ZM7238@xtp.engr.sgi.com> Date: Tue, 13 Aug 1996 09:45:44 -0700 In-Reply-To: nigel@aa5b (Nigel Gamble) "Re: Linux: the next step" (Aug 13, 8:36am) References: <199608131536.IAA29651@aa5b.engr.sgi.com> X-Mailer: Z-Mail (3.2.0 26oct94 MediaMail) To: nigel@aa5b.engr.sgi.com (Nigel Gamble), dm@sgi.com Subject: Re: Linux: the next step Cc: linux@aa5b.engr.sgi.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk The new real-time facilities in Irix (Nawaf's scheduler) are superior to what has been available before in any Unix-like system that I know of. There are two areas of shortfall, but these are probably unavoidable given the design and its operating environment: 1. priority-based scheduling is often a poor substitute for deadline scheduling, although it is useful for many rt applications. 2. once a process gets scheduled via the rt facility, if it does any system calls you get Irix system call performance. Sometimes this will be an issue (by limiting the number of syscalls and rt apps), and other times it will not be so important. I believe that efforts to incorporate hard real-time scheduling into a full-function OS will satisfy the large majority of customers who have real-time applications. These customers do not want to run on the bare iron - they want their real-time and they want their OS also. The Linux-based hack that Larry described is indeed elegant - I had a look at the paper. However, if that hack were enough to support a wide range of real-time applications I think system hacks would have implemented it a long time ago. g From owner-linux@cthulhu.engr.sgi.com Wed Aug 14 15:58:52 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id PAA01761; Wed, 14 Aug 1996 15:58:52 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id WAA03869 for linux-list; Wed, 14 Aug 1996 22:58:41 GMT Received: from neteng.engr.sgi.com (neteng.engr.sgi.com [192.26.80.10]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id PAA03860 for ; Wed, 14 Aug 1996 15:58:39 -0700 Received: (from dm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id PAA01744; Wed, 14 Aug 1996 15:58:39 -0700 Date: Wed, 14 Aug 1996 15:58:39 -0700 Message-Id: <199608142258.PAA01744@neteng.engr.sgi.com> From: "David S. Miller" To: linux@cthulhu.engr.sgi.com Subject: bug in IRIX tftpd? Reply-to: dm@sgi.com Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Wonder if someone can help me fix this problem: Seems that the IRIX tftpd daemon will not respond correctly to requests in binary more. I can reproduce the problem at will and it happens every time. I can do the transfer just fine using ascii mode, but any attempt to use binary mode fails. What is funny is that TFTPD places a syslog entry that says: Aug 14 15:57:13 6D:tanya tftpd[1373]: sandra.engr.sgi.com: read request for /tftpboot/96A64B0F.SUN4C: success Yet tftpd does not send one packet back. It does not matter what machine I try to do this tftp transfer from. ASCII mode always works, binary mode always fails. dm@engr.sgi.com 'Ooohh.. "FreeBSD is faster over loopback, when compared to Linux over the wire". Film at 11.' -Linus From owner-linux@cthulhu.engr.sgi.com Wed Aug 14 16:46:28 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA02556; Wed, 14 Aug 1996 16:46:28 -0700 Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id XAA14812 for linux-list; Wed, 14 Aug 1996 23:46:18 GMT Received: from neteng.engr.sgi.com (neteng.engr.sgi.com [192.26.80.10]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA14807 for ; Wed, 14 Aug 1996 16:46:17 -0700 Received: (from dm@localhost) by neteng.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id QAA02549; Wed, 14 Aug 1996 16:46:17 -0700 Date: Wed, 14 Aug 1996 16:46:17 -0700 Message-Id: <199608142346.QAA02549@neteng.engr.sgi.com> From: "David S. Miller" To: linux@neteng.engr.sgi.com Subject: ignore tftp bug report Reply-to: dm@sgi.com Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk Duh, no one was answering tanya's ARP requests for sandra.engr dm@engr.sgi.com 'Ooohh.. "FreeBSD is faster over loopback, when compared to Linux over the wire". Film at 11.' -Linus From owner-linux@cthulhu.engr.sgi.com Sat Aug 24 16:29:57 1996 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by neteng.engr.sgi.com (8.7.5/960327.SGI.AUTOCF) via SMTP id QAA26392; Sat, 24 Aug 1996 16:29:57 -0700 (PDT) Return-Path: Received: (from daemon@localhost) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id XAA18942 for linux-list; Sat, 24 Aug 1996 23:29:31 GMT Received: from yon.engr.sgi.com (yon.engr.sgi.com [150.166.61.32]) by cthulhu.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) via ESMTP id QAA18931 for ; Sat, 24 Aug 1996 16:29:30 -0700 Received: (from ariel@localhost) by yon.engr.sgi.com (950413.SGI.8.6.12/960327.SGI.AUTOCF) id QAA19238; Sat, 24 Aug 1996 16:27:35 -0700 From: ariel@yon.engr.sgi.com (Ariel Faigon) Message-Id: <199608242327.QAA19238@yon.engr.sgi.com> Subject: Linux/MIPS: short update To: linux@yon.engr.sgi.com Date: Sat, 24 Aug 1996 16:27:35 -0700 (PDT) Cc: jcl@cthulhu.engr.sgi.com (John Loveall) Reply-To: ariel@cthulhu.engr.sgi.com Organization: Silicon Graphics Inc. X-Mailer: ELM [version 2.4 PL24 ME5a] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux@cthulhu.engr.sgi.com Precedence: bulk a short update, As some of you know. David is going back to school (Aug 25). We wish the summer vacation was a little longer so he could really complete and stabilize the port but a lot of time was invested in major aggressive operations like replacing the Linux libc with the GNU libc and keep up with the latest GNU development tools. This should be seen as a good investment in a better Linux foundation for the future that will benefit all ports, not just the MIPS one. We are going to try and loan the two Indys to David so he could complete the port while at Rutgers. (The assets called stacey and tanya except for one monitor which was mine belong to John Loveall who generously loaned them to us for the summer.) David plans to: 1) Merge his work with Ralf (Germany MIPS port) so there will be only one Linux/MIPS port. 2) Clean rough edges, Stabilize the port, on both hardware families. 3) Try to roll the work into Linux before Linus releases 2.1 so that everything will be a part of 2.1 and available on the net. If this can happen we've achieved our first major goal: get Linux to run on SGI hardware and be widely available. I hope David can pull it off. Lastly, I'd like to thank David for an impressive well done job and strongly hope to see him at SGI next summer too. and to all of you who helped this happen (you know who you are), thanks as well. -- Peace, Ariel