Subject: Re: CVS [email-masked]: linux

From: Ralf Baechle <[email-masked]>
Date: Tue, 14 Oct 2003 15:28:50 +0200

On Tue, Oct 14, 2003 at 11:47:59AM +0200, Maciej W. Rozycki wrote:

> > Log message:
> > 	Config bits of support for TLB page sizes other than 4k.  So not
> > 	functional yet but this is a fairly large project and everybody should
> > 	finally get rid of the assumption that PAGE_SIZE is always 4k.
> 
>  I've already thought we might unconditionally switch to a larger page
> size, e.g. 16kB, for the 64-bit kernel -- I suppose everything that
> supports 64-bit operation also supports larger page sizes, doesn't it?

True.  At 16k pagesize we can also start harvesting other advantages, for
example we'd then know that cpu_has_dc_aliases is never true which will
make the compiler simply throw away large parts of the cache code.

As the result of living in an alias-free universe we could use the nice
recursive page fault technique from very old Linux kernels again,
something that allowed blindigly fast TLB refill handlers.

An additional bonus is the increased coverage of a pagetables with 16k
page size.  A 16k page has space for 2048 pointers.  So in case of a
two level pagetable tree that makes the total capacity 16k*2k*2k = 64GB.
So that means one level of the pagetable tree less than we currently
have, yet enough capacity for everything but the largest jobs.

Still want more?  A 3 level tree would then cover 128TB of virtual
address space already exceedin the hardware limits of all processors but
the R8000.

64k pagesize stretches the limits even further.   Here a two level
pagetable tree would cover 4TB, 3-level could cover 32PB exceeding
the capacity of every MIPS processor ever made - and probably sufficient
for the coming decade :-)

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Wed, 15 Oct 2003 16:23:06 +0200 (MET DST)

> address space already exceedin the hardware limits of all processors but

 Well, the MIPS64 ISA spec allows up to 8EB of user memory to be supported
by an implementation, IIRC; probably nothing supports that much yet,
though. ;-)  BTW, is an R8000 spec available online anywhere?

> for the coming decade :-)

 Further increasing of the page size should result in better performance
due to fewer TLB misses and reduce the memory footprint of page tables,
but the drawback is more memory is wasted for maps.  Whether the end
result is a gain or a loss depends on the actual application of a system,
so I guess we should either leave the size configurable (with a sane
default for those who might have troubles judging what would suit them
best) or only decide on a given size after lots of benchmarking.

  Maciej

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: Ralf Baechle <[email-masked]>
Date: Wed, 15 Oct 2003 16:59:48 +0200

> though. ;-)  BTW, is an R8000 spec available online anywhere?

There used to be a few papers published by SGI online and various other
bits of information I found through google.

(I happen to have a paper copy of the R8000 manual but since the responsible
people still haven't informed me if I can legally use it, this book is
closed and will stay closed until this happens - if ever ...  Pitty, I
still receive mails from various R8000 users ...)

> best) or only decide on a given size after lots of benchmarking.

Unless somebody yells I almost feel like ditching 3-level pagetable
support; 2-levels with a decent pagesize should suffice for a few years
to come ...

  Ralf

---

From: Jun Sun <[email-masked]>
Date: Wed, 15 Oct 2003 10:19:02 -0700

> support; 2-levels with a decent pagesize should suffice for a few years

Isn't ia64 still using 3-level page tables?  Any performance data we
can infer from theirs?

I feel a little uneasy about ditching 3-level pagetable altogether.
Leaving all the parameters configurable, including the possiblity of
nullifying the second level and changing page size, seems to be a more 
comforting thought.

Jun

---

From: Ralf Baechle <[email-masked]>
Date: Wed, 15 Oct 2003 19:42:16 +0200

> Isn't ia64 still using 3-level page tables?  Any performance data we

Very different MMU, IA64 data is hardly an indicator.  And anyway, the
result should be MIPS TLB refill handlers suck big chunky rocks through
a straw so should be rewritten ;-)

> nullifying the second level and changing page size, seems to be a more 

3 levels are only needed if you can seriously say you're need more than
64GB of vmalloc space or processes larger than 64GB.  Little need for
that in the current universe though I know one institution which broke
the 1TB process size limit > 5 years ago.

  Ralf

---

From: Nick <[email-masked]>
Date: Sun, 19 Oct 2003 16:09:09 -0400 (EDT)

If anyone wants to poke said folks into handing in the OK I'll ship
someone an R8k I2 to work on.
	Nick