Subject: sti() does not work.

From: "Steven Liu" <[email-masked]>
Date: Tue, 3 Jul 2001 15:48:04 -0700

Hi All:

I am working on the porting Linux to mips R3000 and  have a problem
about sti( ) which is called in start_kernel( ).

As we know, sti() is defined as __sti( ) in the
include/asm-mips/system.h:
 
extern __inline__ void  __sti(void)
{
	__asm__ __volatile__(
		".set\tnoreorder\n\t"
		".set\tnoat\n\t"
		"mfc0\t$1,$12\n\t"
		"ori\t$1,0x1f\n\t"
		"xori\t$1,0x1e\n\t"
		"mtc0\t$1,$12\n\t"               /* <----- problem  here
! */
		".set\tat\n\t"
		".set\treorder"
		: /* no outputs */
		: /* no inputs */
		: "$1", "memory");
}

Before calling this function, status_register = 0x1000fc00 and
cause_register=0x00008000. 
Clearly, this is an interrupt of the CPU timer. 

When mtc0 instruction above is executed, the system hangs and the
control does not go to the timer handler.

Any help is greatly appreciated.

Thank you.

Steven liu

---

From: Thiemo Seufer <[email-masked]>
Date: Wed, 4 Jul 2001 12:23:29 +0200

> 		"mtc0\t$1,$12\n\t"               /* <----- problem  here

Here should follow some nop's on a MIPS I system to make sure $12
is written (why is noreorder used here?).

> 		: "$1", "memory");

HTH,
Thiemo

---

From: "Gleb O. Raiko" <[email-masked]>
Date: Wed, 04 Jul 2001 16:23:34 +0400

> is written (why is noreorder used here?).

Support for r3k in 2.2 is broken for a long time. Use 2.4 instead.

Regards,
Gleb.

---

From: Ralf Baechle <[email-masked]>
Date: Wed, 4 Jul 2001 15:29:56 +0200

> Clearly, this is an interrupt of the CPU timer. 

R3000 doesn't have a CPU timer, so either you're porting to something else
than the R3000 or you don't have a CPU timer.

> control does not go to the timer handler.

When the mtc0 gets executed you take the pending interrupt which goes to
the general exception vector at 0x80000180.  That's magic done in hardware.
So it looks like your interrupt handler is buggy.

  Ralf

---

From: Ralf Baechle <[email-masked]>
Date: Wed, 4 Jul 2001 15:26:19 +0200

On Wed, Jul 04, 2001 at 12:23:29PM +0200, Thiemo Seufer wrote:

> > 		"mtc0\t$1,$12\n\t"               /* <----- problem  here! */
> 
> is written

There are no nops there since we simply don't care how how many cycles
after the mtc0 the interrupts actually get enabled.  Worst case is the
R4000's 8 stage pipeline where we have a latency of 3 cycles, clearly
nothing that justifies wasting memory and cycles for nops.

> (why is noreorder used here?).

Without the .set noreorder the assembler would be free to do arbitrary
reordering of the object code generated.  Gas doesn't do that but there
are other assemblers that do flow analysis and may generate object code
that doesn't look very much like the source they were fed with.

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Thu, 5 Jul 2001 13:35:11 +0200 (MET DST)

> nothing that justifies wasting memory and cycles for nops.

 Still there is a nop missing after mfc0 if this is to be executed on a
MIPS I CPU.  The 2.4.x code is fine, though, so nothing to worry about. 

> that doesn't look very much like the source they were fed with.

 Hmm, I would consider that a bug in such an assembler.  The mtc0 and
possibly the mfc0 opcode should be treated as reordering barriers as they
may involve side effects an assembler might not be aware of. 

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email-masked], PGP key available        +

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 13 Jul 2001 13:35:18 +0200

> may involve side effects an assembler might not be aware of. 

Assembler is the art of using sideeffects so things are fairly explicit.
Optimizations are controlled using

  .set noreorder / reorder
  .set volatile / novolatile
  .set nomove / nomove
  .set nobopt / bopt

  Ralf

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Fri, 13 Jul 2001 16:01:29 +0200 (MET DST)

>   .set volatile / novolatile

 Sure, but sometimes ".set reorder" allows you to achieve better
optimization across various ISAs without a need to resort to the
preprocessor.  Consider the following code: 

	lw	$1,($2)
	addu	$3,$1

You need an instruction between the two for a MIPS I CPU but MIPS II+ CPUs
interlock here if no instruction is placed.  Assuming no real instruction
can be reordered here, a nop must be inserted if the code gets compiled
for a MIPS I CPU but no instruction is preferred otherwise.  The assembler
does it automatically if the ".set reorder" directive is active, but you
need to decide yourself if it is not.

 Actually with mfc0 there is no problem -- you need a nop in the case like
the above one as coprocessor transfers never interlock; at least docs
state so.  But who believes docs without a grain of salt, so please
correct me if I am wrong (I don't have appropriate hardware to perform a
test).

---

From: Ralf Baechle <[email-masked]>
Date: Sat, 14 Jul 2001 13:04:48 +0200

> correct me if I am wrong (I don't have appropriate hardware to perform a

Real wild pig hackers on R3000 were writing code which knows that in the
load delay slot they still have the old register value available.  So you
can implement var1++; var2++ as:

	.set	noreorder
	lw	$reg, var1($gp)
	nop
	addiu	$reg, $reg, 1
	lw	$reg, var2($gp)
	sw	$reg, var1($gp)
	addiu	$reg, $reg, 1
	sw	$reg, var2($gp)

	.common	var1, 4, 4
	.common	var2, 4, 4

Of course only safe with interrupts disabled.  So in a sense introducing
the load interlock broke semantics of MIPS machine code ;-)

  Ralf

---

From: "Kevin D. Kissell" <[email-masked]>
Date: Sat, 14 Jul 2001 13:39:58 +0200

> the load interlock broke semantics of MIPS machine code ;-)

Architecturally, the target register value is UNDEFINED during
the load delay slot on a MIPS I CPU.  Anyone who coded to any
particular assumption regarding its value was coding to a 
specific CPU implementation.  Introducing the load interlock
in later versions of the ISA and later implementations did not
reach backward in time and break the old hardware.  The
implementation-specific code still works for its specific 
implementation.  Refining the spec did not break the code for later
implementations - it was *always* broken for later implementations! ;-)

In a less pedantic tone, there actually is an architecturally
legal case where an assembly coder can justify the use of
noreorder for something other than CP0 pipeline hazards.
If what I want to do is to test a value, branch on the result,
and modify that value regardless of whether the branch is
taken, I can code something like:

    .set noreorder
    bltz    t0,foo
    sra    t0,t0,2
    .set reorder
    <other code>
foo:

Whereas otherwise I need to either consume another
register or replicate the shift both after the branch and
after foo.  If I'm very very lucky, the assembler will "hoist"
such a replicated instruction into the delay slot - a  good
compiler back-end optimiser certainly would.  But I'm not 
aware of any MIPS assembler that would perform that
optimisation - certainly the GNU assembler does not.

            Kevin K.

---

From: "Maciej W. Rozycki" <[email-masked]>
Date: Mon, 16 Jul 2001 14:46:49 +0200 (MET DST)

> can implement var1++; var2++ as:

 That's crazy...

> the load interlock broke semantics of MIPS machine code ;-)

 That broke the MIPS' virtue as well, as MIPS stands for "Microprocessor
without Interlocked Pipeline Stages" (actually mfhi/mflo broke that in the
first place, but it was less significant due to the multiplier being a
separate unit).