Subject: get_mmu_context()

From: [email-masked]
Date: Tue, 13 Oct 1998 21:59:27 +0200

Ok, here is a draft version of an agressively optimized version of
get_mmu_context().  I just didn't like the idea of referencing
global variables in get_mmu_context() if avoidable.  The code below
will work on both R3000 and R4000 with no performance penalty for
being generic.  The trick is to patch the operands of two machine
instructions at runtime, shoot me.

The code below should be integrated into <asm/mmu_context.h>.  The
CPU specific code in arch/mips/mm/... will then just have to include
that header file and call r3000_asid_setup rsp. r4xx0_asid_setup.

Have fun,

  Ralf

#define ASID_VERSION_SHIFT 16
#define ASID_VERSION_MASK  ((~0UL) << ASID_VERSION_SHIFT)
#define ASID_FIRST_VERSION (1UL << ASID_VERSION_SHIFT)

unsigned long asid_cache = ASID_FIRST_VERSION;

/* The next two macros know that they will only be assembled once
   per kernel.  */
#define ASID_VERSION_INC					\
 ({ unsigned long __asid_inc;					\
   __asm__(".globl\tasid_inc\n"					\
           "asid_inc:\n\t"					\
           "li\t%0,0\t\t\t#patched\n\t"				\
           :"=r" (__asid_inc));					\
   __asid_inc; })

#define ASID_OVERFLOW(asid)					\
 ({ unsigned long __res;					\
   __asm__(".global\tasid_overflow\n"				\
           "asid_overflow:\n\t"					\
           "sltu\t%0,%1,0\t\t\t#patched\n\t"			\
           :"=r" (__res)					\
           :"r" (asid));					\
   __res; })

extern inline void get_new_mmu_context(struct mm_struct *mm, unsigned long asid)
{
	/* check if it's legal.. */
	if (ASID_OVERFLOW(asid & ~ASID_VERSION_MASK)) {
		/* start a new version, invalidate all old asid's */
		flush_tlb_all();
		asid = (asid & ASID_VERSION_MASK) + ASID_FIRST_VERSION;
		if (!asid)
			asid = ASID_FIRST_VERSION;
	}
	asid_cache = asid + ASID_VERSION_INC;
	mm->context = asid;			 /* full version + asid */
}

extern void get_mmu_context(struct task_struct *p)
{
	struct mm_struct *mm = p->mm;

	if (mm) {
		unsigned long asid = asid_cache;
		/* Check if our ASID is of an older version and thus invalid */
		if ((mm->context ^ asid) & ASID_VERSION_MASK)
			get_new_mmu_context(mm, asid);
	}
}

extern inline void __asid_setup(unsigned long asid_inc, unsigned long asid_cmp)
{
	extern u32 asid_inc;
	extern u32 asid_overflow;

	asid_inc = (asid_inc & 0xffff0000) | asid_inc;
	flush_icache_range(&asid_inc, 4);
	asid_overflow = (asid_overflow & 0xffff0000) | asid_cmp;
	flush_icache_range(&asid_overflow, 4);
}

extern inline void r3000_asid_setup(void)
{
	__asid_setup(0x40, 0xfc1);
}

extern inline void r4xx0_asid_setup(void)
{
	__asid_setup(1, 0x100);
}

---

From: [email-masked]
Date: Thu, 15 Oct 1998 01:49:55 +0200

On Wed, Oct 14, 1998 at 02:45:45PM +0000, Rik van Riel wrote:

> ROFL! No offense to the code, I'm sure it works, but
> this just _has_ to be the funniest piece of source
> code to appear on the lists this month...
> 
> Self-modifying code -- this is so much fun :)

Lame example of self modifying code, it doesn't even modify the instructions
and only on bootup.

> Very nice Ralf... My compliments on this piece
> of code -- I can only imagine the amount of
> phantasy and inspiration that was needed to
> create it...

Guess why I had to post it :-)  Seriously, I didn't like the proposed
alternative and some of the older machines which have main memory latencies
like 1400ns for shure win each time when _not_ using memory.  Even in
case of a l1 hit the amount of time wasted for the load should still be
visible in context switching times.  So, no mercy.  Patch as patch can.
And maintenance isn't the issue.  I messed it up, I maintain it :-)

  Ralf

---

From: [email-masked]
Date: Thu, 15 Oct 1998 22:21:00 +0200

> It does not add performance I can see.

Your code is actually a bit shorter, so I'll mix something based on your
version and my code patching thing.  It must be good, Rik sent me fan
email ;-)

The performance hit will come into the game as soon as one tries to
build a generic kernel for both R3000 and R4000.  The R3000 PID and the
R4000 ASID mechanism are slightly different and the simple approach
to fix that is to use global variables.  Which affects both your version
and the current version.

Which I want to avoid because in worst case that means we'll have to eat
a latency of over 1400ns per cache line to fetch from memory.  This number
is measured on a R4000SC with 100Mhz.  That is enough to show up right
away on the context switch times which now look fairly nice as long as we
don't have to swallow cache faults.

(Let me dig out a cite from somebody ``Some systems seem to think a system
call without a cache miss is a day without sunshine.  :-)'')

Talking about cache faults & context switching times.  Especially on the
R4000 with it's 8kb direct mapped cache _every_ context switch will
result in cache misses which explain the fairly lousy context switch times
on R4000.  R4400 isn't much better.  For the case when we have the perfect
most cache friendly layout we can switch between two processes without
taking page faults.

Things got worse when we switched to 8kb, 8kb aligned, stacks as delivered
by __get_free_pages(1,...).  It's somewhat better on R4600 and R5000 which
have set associative caches but still they could need some optimization.
Also R3000 with it's typically 2 x 64kb per cache direct mapped caches will
looks relativly better in that aspect.

> So, I don't understand why idea of using all free upper bits as asid
> extension is bad  -- same time it increases security it allows to
> increment
> version automatically when asid overflow occurs.

No.  It gives us a view bits more, but way not enough.  The mmu context code
we have has been copied almost unmodified from the Alpha which from EV5 on
also supports ASNs.  The big difference between the Alpha and the MIPS
implementation is that on the Alpha the type ``unsigned long'' which is
used for counting ASN and ASN version number has 64 bit.  Assuming UP and
the worst case which is 2^7 ASNs they still have 2^57 versions before a
version overflow will happen.  In other words, assuming 1,000,000 context
switches per second they still have almost 600,000 years before an ASN
collission may occur.

The MIPS implementation does worse, especially for the R3000 where waste the
lowest 6 bits which effectivly truncates our ASID (or PID) to 26 bits.
Assuming just 100,000 context switches we'll wrap around after just a bit
more than 11 minutes.  I think we can do more than 100,000 context switches
per second on a reasonable R3000 system.

A possible actual solution could be to extend the ASID / PID counter to
64 bit.  This adds some overhead which I'd get rid of if possible at all.
Ideas for that wanted.

Btw, is any of the R3000 machines stable enough to run lmbench?  I'm
interested to get results, best the raw result file from
lmbench/results/mips-linux/.  Reminds me to run lmbench on my RS3230.

  Ralf

---

From: [email-masked]
Date: Sat, 17 Oct 1998 00:44:08 +0200

Ok, here is the first working and tested version of my runtime patched
get_mmu_context thing.  While writing the code I noticed that the code
patching stuff is actually already a part of what I was intending to implement
since long time like for example getting rid of function pointers and can
be used to optimize many more things throughout the kernel.  So while some
parts already look somewhat generic I'd like to make them really generic.
The thing is probably going to look somewhat similar to the Sparc btfixup
code which I'm going to read now.

  Ralf
Index: linux/arch/mips/mm/andes.c
diff -u linux/arch/mips/mm/andes.c:1.5 linux/arch/mips/mm/andes.c:1.6
--- linux/arch/mips/mm/andes.c:1.5	Fri May  1 08:33:13 1998
+++ linux/arch/mips/mm/andes.c	Fri Oct 16 21:22:42 1998
@@ -1,9 +1,8 @@
-/*
+/* $Id: andes.c,v 1.6 1998/10/16 19:22:42 ralf Exp $
+ *
  * andes.c: MMU and cache operations for the R10000 (ANDES).
  *
  * Copyright (C) 1996 David S. Miller ([email-masked])
- *
- * $Id: andes.c,v 1.5 1998/05/01 06:33:13 ralf Exp $
  */
 #include <linux/init.h>
 #include <linux/kernel.h>
@@ -13,6 +12,7 @@
 #include <asm/pgtable.h>
 #include <asm/system.h>
 #include <asm/sgialib.h>
+#include <asm/mmu_context.h>
 
 extern unsigned long mips_tlb_entries;
 
@@ -104,6 +104,7 @@
 	flush_tlb_mm = andes_flush_tlb_mm;
 	flush_tlb_range = andes_flush_tlb_range;
 	flush_tlb_page = andes_flush_tlb_page;
+	andes_asid_setup();
     
         add_wired_entry = andes_add_wired_entry;
 
Index: linux/arch/mips/mm/fault.c
diff -u linux/arch/mips/mm/fault.c:1.10 linux/arch/mips/mm/fault.c:1.11
--- linux/arch/mips/mm/fault.c:1.10	Thu Sep 17 00:50:44 1998
+++ linux/arch/mips/mm/fault.c	Fri Oct 16 21:22:42 1998
@@ -1,4 +1,4 @@
-/* $Id: fault.c,v 1.10 1998/09/16 22:50:44 ralf Exp $
+/* $Id: fault.c,v 1.11 1998/10/16 19:22:42 ralf Exp $
  *
  *  arch/mips/mm/fault.c
  *
@@ -26,7 +26,7 @@
 
 extern void die(char *, struct pt_regs *, unsigned long write);
 
-unsigned long asid_cache = ASID_FIRST_VERSION;
+unsigned long asid_cache;
 
 /*
  * Macro for exception fixup code to access integer registers.
Index: linux/arch/mips/mm/init.c
diff -u linux/arch/mips/mm/init.c:1.12 linux/arch/mips/mm/init.c:1.13
--- linux/arch/mips/mm/init.c:1.12	Thu Sep 17 00:50:44 1998
+++ linux/arch/mips/mm/init.c	Fri Oct 16 21:22:42 1998
@@ -1,4 +1,4 @@
-/* $Id: init.c,v 1.12 1998/09/16 22:50:44 ralf Exp $
+/* $Id: init.c,v 1.13 1998/10/16 19:22:42 ralf Exp $
  *
  * This file is subject to the terms and conditions of the GNU General Public
  * License.  See the file "COPYING" in the main directory of this archive
@@ -33,6 +33,7 @@
 #ifdef CONFIG_SGI
 #include <asm/sgialib.h>
 #endif
+#include <asm/mmu_context.h>
 
 /*
  * Define this to effectivly disable the userpage colouring shit.
@@ -463,4 +464,36 @@
 	val->totalram <<= PAGE_SHIFT;
 	val->sharedram <<= PAGE_SHIFT;
 	return;
+}
+
+/* Fixup an immediate instruction  */
+__initfunc(static void __i_insn_fixup(unsigned int **start, unsigned int **stop,
+                         unsigned int i_const))
+{
+	unsigned int **p, *ip;
+
+	for (p = start;p < stop; p++) {
+		ip = *p;
+		*ip = (*ip & 0xffff0000) | i_const;
+	}
+}
+
+#define i_insn_fixup(section, const)					  \
+do {									  \
+	extern unsigned int *__start_ ## section;			  \
+	extern unsigned int *__stop_ ## section;			  \
+	__i_insn_fixup(&__start_ ## section, &__stop_ ## section, const); \
+} while(0)
+
+/* Caller is assumed to flush the caches before the first context switch.  */
+__initfunc(void __asid_setup(unsigned int inc, unsigned int mask,
+                             unsigned int version_mask,
+                             unsigned int first_version))
+{
+	i_insn_fixup(__asid_inc, inc);
+	i_insn_fixup(__asid_mask, mask);
+	i_insn_fixup(__asid_version_mask, version_mask);
+	i_insn_fixup(__asid_first_version, first_version);
+
+	asid_cache = first_version;
 }
Index: linux/arch/mips/mm/r2300.c
diff -u linux/arch/mips/mm/r2300.c:1.6 linux/arch/mips/mm/r2300.c:1.7
--- linux/arch/mips/mm/r2300.c:1.6	Tue Aug 18 22:45:08 1998
+++ linux/arch/mips/mm/r2300.c	Fri Oct 16 21:22:43 1998
@@ -1,9 +1,8 @@
-/*
+/* $Id: r2300.c,v 1.7 1998/10/16 19:22:43 ralf Exp $
+ *
  * r2300.c: R2000 and R3000 specific mmu/cache code.
  *
  * Copyright (C) 1996 David S. Miller ([email-masked])
- *
- * $Id: r2300.c,v 1.6 1998/08/18 20:45:08 ralf Exp $
  */
 #include <linux/init.h>
 #include <linux/kernel.h>
@@ -14,6 +13,7 @@
 #include <asm/pgtable.h>
 #include <asm/system.h>
 #include <asm/sgialib.h>
+#include <asm/mmu_context.h>
 
 extern unsigned long mips_tlb_entries;
 
@@ -274,6 +274,7 @@
 	flush_tlb_mm = r2300_flush_tlb_mm;
 	flush_tlb_range = r2300_flush_tlb_range;
 	flush_tlb_page = r2300_flush_tlb_page;
+	r3000_asid_setup();
 
 	load_pgd = r2300_load_pgd;
 	pgd_init = r2300_pgd_init;
Index: linux/arch/mips/mm/r4xx0.c
diff -u linux/arch/mips/mm/r4xx0.c:1.29 linux/arch/mips/mm/r4xx0.c:1.30
--- linux/arch/mips/mm/r4xx0.c:1.29	Wed Oct 14 22:29:59 1998
+++ linux/arch/mips/mm/r4xx0.c	Fri Oct 16 21:22:43 1998
@@ -1,4 +1,4 @@
-/* $Id: r4xx0.c,v 1.29 1998/10/14 20:29:59 ralf Exp $
+/* $Id: r4xx0.c,v 1.30 1998/10/16 19:22:43 ralf Exp $
  *
  * This file is subject to the terms and conditions of the GNU General Public
  * License.  See the file "COPYING" in the main directory of this archive
@@ -2791,6 +2791,7 @@
 	flush_tlb_mm = r4k_flush_tlb_mm;
 	flush_tlb_range = r4k_flush_tlb_range;
 	flush_tlb_page = r4k_flush_tlb_page;
+	r4xx0_asid_setup();
 
 	load_pgd = r4k_load_pgd;
 	pgd_init = r4k_pgd_init;
Index: linux/arch/mips/mm/r6000.c
diff -u linux/arch/mips/mm/r6000.c:1.5 linux/arch/mips/mm/r6000.c:1.6
--- linux/arch/mips/mm/r6000.c:1.5	Tue Aug 18 22:45:09 1998
+++ linux/arch/mips/mm/r6000.c	Fri Oct 16 21:22:44 1998
@@ -1,4 +1,5 @@
-/* $Id: r6000.c,v 1.5 1998/08/18 20:45:09 ralf Exp $
+/* $Id: r6000.c,v 1.6 1998/10/16 19:22:44 ralf Exp $
+ *
  * r6000.c: MMU and cache routines for the R6000 processors.
  *
  * Copyright (C) 1996 David S. Miller ([email-masked])
@@ -13,6 +14,7 @@
 #include <asm/pgtable.h>
 #include <asm/system.h>
 #include <asm/sgialib.h>
+#include <asm/mmu_context.h>
 
 __asm__(".set mips3"); /* because we know... */
 
@@ -180,6 +182,7 @@
 	flush_tlb_mm = r6000_flush_tlb_mm;
 	flush_tlb_range = r6000_flush_tlb_range;
 	flush_tlb_page = r6000_flush_tlb_page;
+	r6000_asid_setup();
 
 	load_pgd = r6000_load_pgd;
 	pgd_init = r6000_pgd_init;
Index: linux/arch/mips/mm/tfp.c
diff -u linux/arch/mips/mm/tfp.c:1.5 linux/arch/mips/mm/tfp.c:1.6
--- linux/arch/mips/mm/tfp.c:1.5	Fri May  1 08:31:34 1998
+++ linux/arch/mips/mm/tfp.c	Fri Oct 16 21:22:44 1998
@@ -1,4 +1,5 @@
-/*
+/* $Id: tfp.c,v 1.6 1998/10/16 19:22:44 ralf Exp $
+ *
  * tfp.c: MMU and cache routines specific to the r8000 (TFP).
  *
  * Copyright (C) 1996 David S. Miller ([email-masked])
@@ -13,6 +14,7 @@
 #include <asm/pgtable.h>
 #include <asm/system.h>
 #include <asm/sgialib.h>
+#include <asm/mmu_context.h>
 
 extern unsigned long mips_tlb_entries;
 
@@ -104,6 +106,7 @@
 	flush_tlb_mm = tfp_flush_tlb_mm;
 	flush_tlb_range = tfp_flush_tlb_range;
 	flush_tlb_page = tfp_flush_tlb_page;
+	tfp_asid_setup();
 
         add_wired_entry = tfp_add_wired_entry;
 
Index: linux/include/asm-mips/mmu_context.h
diff -u linux/include/asm-mips/mmu_context.h:1.2 linux/include/asm-mips/mmu_context.h:1.3
--- linux/include/asm-mips/mmu_context.h:1.2	Tue May  5 12:22:01 1998
+++ linux/include/asm-mips/mmu_context.h	Fri Oct 16 21:22:54 1998
@@ -1,4 +1,4 @@
-/* $Id: mmu_context.h,v 1.2 1998/05/05 10:22:01 ralf Exp $
+/* $Id: mmu_context.h,v 1.3 1998/10/16 19:22:54 ralf Exp $
  *
  * Switch a MMU context.
  *
@@ -11,29 +11,61 @@
 #ifndef __ASM_MIPS_MMU_CONTEXT_H
 #define __ASM_MIPS_MMU_CONTEXT_H
 
-#define MAX_ASID 255
-
+/* Fuck.  The f-word is here so you can grep for it :-)  */
 extern unsigned long asid_cache;
 
-#define ASID_VERSION_SHIFT 16
-#define ASID_VERSION_MASK  ((~0UL) << ASID_VERSION_SHIFT)
-#define ASID_FIRST_VERSION (1UL << ASID_VERSION_SHIFT)
-
-extern inline void get_new_mmu_context(struct mm_struct *mm, unsigned long asid)
-{
-	/* check if it's legal.. */
-	if ((asid & ~ASID_VERSION_MASK) > MAX_ASID) {
-		/* start a new version, invalidate all old asid's */
-		flush_tlb_all();
-		asid = (asid & ASID_VERSION_MASK) + ASID_FIRST_VERSION;
-		if (!asid)
+/* I patch, therefore I am ...  */
+#define ASID_INC(asid)						\
+ ({ unsigned long __asid = asid;				\
+   __asm__("1:\taddiu\t%0,0\t\t\t\t# patched\n\t"		\
+           ".section\t__asid_inc,\"a\"\n\t"			\
+           ".word\t1b\n\t"					\
+           ".previous"						\
+           :"=r" (__asid)					\
+           :"0" (__asid));					\
+   __asid; })
+#define ASID_MASK(asid)						\
+ ({ unsigned long __asid = asid;				\
+   __asm__("1:\tandi\t%0,%1,0\t\t\t# patched\n\t"			\
+           ".section\t__asid_mask,\"a\"\n\t"			\
+           ".word\t1b\n\t"					\
+           ".previous"						\
+           :"=r" (__asid)					\
+           :"r" (__asid));					\
+   __asid; })
+#define ASID_VERSION_MASK					\
+ ({ unsigned long __asid;					\
+   __asm__("1:\tli\t%0,0\t\t\t\t# patched\n\t"			\
+           ".section\t__asid_version_mask,\"a\"\n\t"		\
+           ".word\t1b\n\t"					\
+           ".previous"						\
+           :"=r" (__asid));					\
+   __asid; })
+#define ASID_FIRST_VERSION					\
+ ({ unsigned long __asid = asid;				\
+   __asm__("1:\tli\t%0,0\t\t\t\t# patched\n\t"			\
+           ".section\t__asid_first_version,\"a\"\n\t"		\
+           ".word\t1b\n\t"					\
+           ".previous"						\
+           :"=r" (__asid));					\
+   __asid; })
+
+#define ASID_FIRST_VERSION_R3000 0x1000
+#define ASID_FIRST_VERSION_R4000 0x100
+
+extern inline void
+get_new_mmu_context(struct mm_struct *mm, unsigned long asid)
+{
+	if (!ASID_MASK((asid = ASID_INC(asid)))) {
+		flush_tlb_all(); /* start new asid cycle */
+		if (!asid)      /* fix version if needed */
 			asid = ASID_FIRST_VERSION;
 	}
-	asid_cache = asid + 1;
-	mm->context = asid;			 /* full version + asid */
+	mm->context = asid_cache = asid;
 }
 
-extern inline void get_mmu_context(struct task_struct *p)
+extern inline void
+get_mmu_context(struct task_struct *p)
 {
 	struct mm_struct *mm = p->mm;
 
@@ -72,5 +104,31 @@
 	get_mmu_context(tsk);
 	set_entryhi(tsk->mm->context);
 }
+
+extern void __asid_setup(unsigned int inc, unsigned int mask,
+                         unsigned int version_mask, unsigned int first_version);
+
+extern inline void r3000_asid_setup(void)
+{
+	__asid_setup(0x40, 0xfc0, 0xf000, ASID_FIRST_VERSION_R3000);
+}
+
+extern inline void r6000_asid_setup(void)
+{
+	panic("r6000_asid_setup: implement me");	/* No idea ...  */
+}
+
+extern inline void tfp_asid_setup(void)
+{
+	panic("tfp_asid_setup: implement me");	/* No idea ...  */
+}
+
+extern inline void r4xx0_asid_setup(void)
+{
+	__asid_setup(1, 0xff, 0xff00, ASID_FIRST_VERSION_R4000);
+}
+
+/* R10000 has the same ASID mechanism as the R4000.  */
+#define andes_asid_setup r4xx0_asid_setup
 
 #endif /* __ASM_MIPS_MMU_CONTEXT_H */

---

From: Harald Koerfgen <[email-masked]>
Date: Sun, 18 Oct 1998 21:53:18 +0200 (MEST)

On 16-Oct-98 Gleb O. Rajko wrote:

> On Thu, 15 Oct 1998 [email-masked] wrote:
> 
>> 
> We are going to provide support for building generic kernels in the near 
> future if somebody who holds r4k will help us and will try our patches 
> on a r4k box. I think the best time to start is when r3k will be synced 
> with the main branch. Perhaps, Harald may answer when it might occur.

Hmmm, I'm not shure about this. If we really want to support generic kernels,
that'll shurely needs some more work than a little patching in get_mmu_context.
Remember, Vladimir, we needed to make changes in extremely performance critical
parts of the kernel which Ralf propably won't like to have for the R4000 case. Most
of them are actually compiled conditionally.

To make those changes generic needs either a reasonable amount of hacking or
avoidable code duplication. In fact, if we really don't care about self modifying
code we should be able to remove some code duplication, for example in the fpu
stuff.

I'd personally prefer to have a few things sorted out before I check the R3000 stuff
in. What's common opinion on this?

---
Regards,
Harald

---

From: [email-masked]
Date: Mon, 19 Oct 1998 03:29:46 +0200

> with the main branch. Perhaps, Harald may answer when it might occur.

If you are in doubt about any R4k related changes feel free to send them
to me.

> Perhaps, we may use global vars in most cases and patch code at bootup 
> for critical ones.

Yes.  We can actually do even more in that direction like for example
eleminating calls via pointers as used throughout the mm.  We insert jal
instructions and patch the address as needed.  Or we even overwrite the
call by nops if the routine is emtpy.

> As I understand the only place where change the version is in 
> get_new_mmu_context(). Thus, if the version overflows we may create a new 
> process which will share its address space with other due to the same 
> ASID and varsion pair. Am I right ?  

Correct.

> Ok. I will try to run lmbench while Vladimir will think about your ideas :-)

:-)

Btw, lmbench has in the past helped me to kill several interesting bugs.

  Ralf

---

From: [email-masked]
Date: Mon, 19 Oct 1998 12:18:04 +0200

On Sun, Oct 18, 1998 at 09:53:18PM +0200, Harald Koerfgen wrote:

> that'll shurely needs some more work than a little patching in

There is a number of machines, most popular some DECstation, which are
available with both CPU architectures.

> get_mmu_context.  Remember, Vladimir, we needed to make changes in extremely
> performance critical parts of the kernel which Ralf propably won't like to
> have for the R4000 case. Most of them are actually compiled conditionally.

The other location for which patching might be useful are primarily the
differences between the R3000 and R4000 status register, especially the
KU rsp. KSU bits.  What other places do you have in mind?

> avoidable code duplication. In fact, if we really don't care about self
> modifying code we should be able to remove some code duplication, for
> example in the fpu stuff.

People should consider that we'll be able to hide the self modifications
in C code very nicely.  I would not go for anything which I consider a
maintenance problem or major uglyness.

Let me explain how the approach from my recent patch works:

#define ASID_INC(asid)                                         \
 ({ unsigned long __asid = asid;                               \
   __asm__("1:\taddiu\t%0,0\t\t\t\t# patched\n\t"              \
           ".section\t__asid_inc,\"a\"\n\t"                    \
           ".word\t1b\n\t"                                     \
           ".previous"                                         \
           :"=r" (__asid)                                      \
           :"0" (__asid));                                     \
   __asid; })

This macro will be used in a place where the C compiler is known to produce
an addiu instruction.  The trick is now that we collect the instruction's
address in a special section, __asid_inc in that case.  As a special
service the GNU linker will generate the symbols __start_<name> and
__stop_<name> if the section name <name> is a valid C symbol name.  So
after boot we can easily insert the right operands into the instruction:

/* Fixup an immediate instruction  */
static void __i_insn_fixup(unsigned int **start, unsigned int **stop,
                           unsigned int i_const)
{
       unsigned int **p, *ip;

       for (p = start;p < stop; p++) {
               ip = *p;
               *ip = (*ip & 0xffff0000) | i_const;
       }
}

#define i_insn_fixup(section, const)                                     \
do {                                                                     \
       extern unsigned int *__start_ ## section;                         \
       extern unsigned int *__stop_ ## section;                          \
       __i_insn_fixup(&__start_ ## section, &__stop_ ## section, const); \
} while(0)

void __asid_setup(unsigned int inc, unsigned int mask,
                             unsigned int version_mask,
                             unsigned int first_version)
{
       i_insn_fixup(__asid_inc, inc);
[...]

The technique itself can be used in quite a number more of places.  All
variables which are computed at boot time and never again changed later on
are good candidates.  Think for example of the variables dcache_size,
icache_size, scache_size, dcache_lsize, icache_lsize, scache_lsize.

Todo: we don't want a separate section per patched instruction.  Easy to
fix.  We also want to get rid of the special section just like other
initfunc stuff.

The next class of things to patch are the zillion of function calls using
function pointers.  We can insert a jal instruction and patch it's
destination address.  Candidates for that include calls to flush_cache_all,
flush_cache_mm, flush_cache_range, flush_cache_page, flush_cache_sigtramp,
flush_tlb_all, flush_tlb_mm, flush_tlb_range, flush_tlb_page, add_wired_entry,
clear_page, copy_page.

Both parts, patching immediate opperands and function calls can be dealt
with similar to the approach in the userland dynamic linker, that initially
the function called is the dynamic linker's kernel equivalent which will
choose the right thing to do.

Extra goodie for the R3000 fraction: Many functions called via above
mentioned pointers are empty, calls to them may be eleminated by overwriting
the calling jal with nops.  Say goodbye to another hand full of cycles.

  Ralf

---

From: Warner Losh <[email-masked]>
Date: Mon, 19 Oct 1998 22:07:26 -0600

Wouldn't it be easier to have ld to have variant fixup records?  That
way you could patch all instances at run time, much like we do when we
load stuff now...

You are basically duplicating the functionality of the linker here for
no good reason.  Actually, besides the non-standard aspect of it,
there is a good reason: it is easier to hack like this than to do
battle with the bfd and/or boot blocks to get this to happen. :-)

It is a way cool hack, don't get me wrong, but it just seems that it
would be more useful to be generic like that.

Warner

---

From: [email-masked]
Date: Tue, 20 Oct 1998 19:31:16 +0200

> would be more useful to be generic like that.

Neither compiler, assembler nor linker can solve the problem for us
because it requires knowledge of facts which we usually don't know before
running on the target machine.  On the other side as soon as we run on
the machine we know these details.  They're constants, so why not making
optimal use of them?

As alternative to achieve the same level of efficience imagine the
following configuration dialogue:

#
# Cache configuration
#
Does your machine have a second level cache (CONFIG_L2_CACHE) [N/y] y
  Is your CPU an Indy R4600SC or R5000SC (CONFIG_L2_INDY) [N/y]
  l2 linesize (16, 32, 64, 128) [128]
  Split Scache (CONFIG_L2_SPLIT) [N/y] 
  Do your VCE exceptions work (CONFIG_R4000SC_V2) [N/y]
Primary instruction cache linesize (16, 32) [16]
Primary data cache linesize (16, 32) [16]

Oh, there is more fun we'd need to ask for like DRAM controller versions
for Magnums etc.  Cool vomit bag, isn't it ;-)

Things look different for fixed configuration systems like for example
the Cobalt Qube.  But we have to deal with zillion of system variants.

You're right in that things should become more generic and I have ideas
to make them more generic.  For the moment being I don't want to continue
on that since 2.2 is coming soon and more important things are still to do.
That's should however be an interesting 2.3 project.

  Ralf

---

From: Warner Losh <[email-masked]>
Date: Tue, 20 Oct 1998 17:36:53 -0600

In message <[email-masked]> [email-masked] writes:
: running on the target machine.  On the other side as soon as we run on
: the machine we know these details.  They're constants, so why not making
: optimal use of them?

I'm confused then...

: You're right in that things should become more generic and I have ideas
: to make them more generic.  For the moment being I don't want to continue
: on that since 2.2 is coming soon and more important things are still to do.
: That's should however be an interesting 2.3 project.

Yes.  Basically something has load the kernel, and that something
could do the fixups.  Basically it would be a relocation record with a
"tag" on it that said what systems to do it on.

However, I do see your point that when you want to have the varients
based on cache size, memory controller, cache line size, etc it gets
really ugly....

It would be a cool project, however...

Warner

---

From: Harald Koerfgen <[email-masked]>
Date: Wed, 21 Oct 1998 21:31:40 +0200 (MEST)

[...]
> available with both CPU architectures.

Yup.
 
> KU rsp. KSU bits.  What other places do you have in mind?

Agreed. There are indeed other places where code could be reintegrated,
r2300_switch.S and r4k_switch.S come to mind.

On the other hand, there is code which should be seperated for performance reasons,
mainly the RESTORE_ALL macros thus affecting entry.S and scall_o32.S. Actually these
problems are solved by conditional compiling.
 
> maintenance problem or major uglyness.

Good.
 
> Let me explain how the approach from my recent patch works:

[explanation snipped]
 
> fix.  We also want to get rid of the special section just like other

That sounds great. We could even put all ISA dependant code into seperate
sections and get rid of all the ISA sections, which aren't needed.
 
> the calling jal with nops.  Say goodbye to another hand full of cycles.

That sounds even better :-).
---
Regards,
Harald