Subject: Re: sysmips call and glibc atomic set

From: Ralf Baechle <[email-masked]>
Date: Tue, 26 Dec 2000 14:02:04 -0200

On Mon, Dec 25, 2000 at 01:18:48AM -0600, Joe deBlaquiere wrote:

> I'm working with a vr4181 target and started digging into the atomic 
> test and set stuff in the kernel and glibc. The first problem I had was 
> that the glibc code assumes that all mips III targets implement the mips 
> III ISA (funny assumption, no?) but the vr4181 doesn't include the 
> miltiprocessor oriented LL/SC operations for atomic test and set.

Ok, but since the kernel disables MIPS III you're limited to MIPS II anyway ...

> So I started looking at the glibc code (yes, I know this is the kernel 
> list... I'm getting there I promise) and notice the following operations:
> 
>    __asm__ __volatile__
>      (".set	mips2\n\t"
>       "/* Inline spinlock test & set */\n\t"
>       "1:\n\t"
>       "ll	%0,%3\n\t"
>       ".set	push\n\t"
>       ".set	noreorder\n\t"
>       "bnez	%0,2f\n\t"
>       " li	%1,1\n\t"
>       ".set	pop\n\t"
>       "sc	%1,%2\n\t"
>       "beqz	%1,1b\n"
>       "2:\n\t"
>       "/* End spinlock test & set */"
>       : "=&r" (ret), "=&r" (temp), "=m" (*spinlock)
>       : "m" (*spinlock)
>       : "memory");
> 
> The significant code here being the 'll' and 'sc' operations which are 
> supposed to ensure that the operation is atomic.
> 
> QUESTION 1) Will this _ALWAYS_ work from user land? I realize the 
> operations are temporally close, but isn't there the possibility that an 
> interrupt occurs in the meantime?

Read the ISA manual; sc will fail if the LL-bit in c0_status is cleared
which will be cleared when the interrupt returns using the eret instruction.

> Of course none of this code applies to my case anyway, since the vr4181 
> doesn't implement these ops. So once I hack^H^H^H^Hadjust glibc to use 
> the 'mips1' implementation, it uses the sysmips system call. regard :
> 
> _test_and_set (int *p, int v) __THROW
> {
>    return sysmips (MIPS_ATOMIC_SET, (int) p, v, 0);
> }
> 
> So then I looked at the kernel and find the code below. The system I'm 
> working with is expressedly uniprocessor and doesn't have any swap, so 
> it looks like the initial caveats are met, but it looks to me like there 
> could be some confusion if the value of *arg1 at entry looks like 
> -ENOSYS or something like that.

Not having swap doesn't mean you're safe.  Think of any kind of previously
unmapped page.

> QUESTION 2) Wouldn't it be better to pass back the initial value of 
> *arg1 in *arg3 and return zero or negative error code?

The semantics of this syscall were previously defined by Risc/OS and later
on continued to be used by IRIX.

> 	case MIPS_ATOMIC_SET: {
> 		/* This is broken in case of page faults and SMP ...
> 		    Risc/OS faults after maximum 20 tries with EAGAIN.  */
> 		unsigned int tmp;
> 
> 		p = (int *) arg1;
> 		errno = verify_area(VERIFY_WRITE, p, sizeof(*p));
> 		if (errno)
> 			return errno;
> 		errno = 0;
> 		save_and_cli(flags);
> 		errno |= __get_user(tmp, p);
> 		errno |= __put_user(arg2, p);
> 		restore_flags(flags);
> 
> 		if (errno)
> 			return tmp;
> 
> 		return tmp;             /* This is broken ...  */
>          }
> 
> QUESTION 3) I notice that the code for this particular case of sysmips 
> has changed recently. The old code looked more like the 'll/sc' version 
> of glibc above. I would think that the 'll/sc' code would be better on 
> SMP systems.

Don't think about SMP without ll/sc.  There's algorithems available for
that but their complexity leaves them a unpractical, theoretical construct.

> Is there a good reason why this reverted?

Above code will break if the old content of memory has bit 31 set or you take
pagefaults.  The latter problem is a problem even on UP - think multi-
threading.

Finally, post such things to one of the MIPS-related mailing lists.  If
you're unlucky nobody of the MIPS'ers might see your posting on l-k.

  Ralf

---

From: Joe deBlaquiere <[email-masked]>
Date: Tue, 26 Dec 2000 10:49:33 -0600

Ralf, firstly, thank you for the answers :)

> Ok, but since the kernel disables MIPS III you're limited to MIPS II anyway ...

This makes sense...

> which will be cleared when the interrupt returns using the eret instruction.

I tried to find a MIPSIII manual from mips.com but all I could find was 
mips32 and mips64 (which are not the same as MIPSII/MIPSIII/MIPSIV).

> Not having swap doesn't mean you're safe.  Think of any kind of previously

Is there a reason why it doesn't just force that page to be mapped first?

>> Is there a good reason why this reverted?

Looking at 2.4.0-test5 I see the ll/sc code, but -test12 doesn't use it. 
I was just curious at why it was taken out.

-- 
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839

---

From: Pavel Machek <[email-masked]>
Date: Tue, 26 Dec 2000 21:53:17 +0100

Hi!

> Is there a reason why it doesn't just force that page to be mapped

You can map it in... But background daemon can map it out in the
meantime :-). You'd have to map in and pagelock.
								Pavel

-- 
I'm [email-masked]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email-masked]

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Thu, 28 Dec 2000 13:06:59 +0100 (MET)

> on continued to be used by IRIX.

 Ralf, could you please provide me a copy of a man page for the call?  I
don't have access to either of the systems and a search of the Net
returned nothing. 

> that but their complexity leaves them a unpractical, theoretical construct.

 For SMP there is a simple kernel solution available.  It suitable for a
syscall or a ll/sc emulation.  There is no easy userland-only solution
AFAIK.

> pagefaults.  The latter problem is a problem even on UP - think multi-

 If the code is written carefully you don't ever get a pagefault that
would break consistency.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Thu, 28 Dec 2000 13:25:56 +0100 (MET)

> mips32 and mips64 (which are not the same as MIPSII/MIPSIII/MIPSIV).

 Get "IDT MIPS Microprocessor Software Reference Manual" from
http://decstation.unix-ag.org/docs/ic_docs/3715.pdf (the original is no
longer available from IDT servers).