Subject: FW: Db1500 PCI Auto Scan Question

From: "Tom Cernius" <[email-masked]>
Date: Wed, 11 Jun 2003 15:12:18 -0700

Hello,

   I am porting my first PCI driver for a PCI card hosted by the AMD Db1500
"Zinfandel" development board.
   This driver had been previously working on another host, where
CONFIG_PCI_AUTO was not enabled.

    My PCI card REQUIRES 0xFF000000 and 0x90000000 be programmed into BAR0
and BAR1 respectively.
    My PCI card has nothing programmed into BAR0 and BAR1 at power-up.
    My host Linux kernel was built with CONFIG_PCI, CONFIG_NEW_PCI, and
CONFIG_PCI_AUTO turned on in the .config file.

     I noticed that during boot, the kernel tickles my devices BAR's and
then writes these BARs with addresses in the range of
     4000 0000 thru 43FF FFFF

     I have tried everything and although I am able to write the proper
(0xFF00 0000 and 0x9000 0000) addresses into the BAR's,
     I have been unable to successfully read anything from my PCI cards CPU
Register/Sdram space.

     I suspect that it is NOT possible to use hardcoded PCI BAR addresses
with the MIPS processor AND CONFIG_PCI_AUTO turned on,
     as the kernel expects (and configures the PCI BARS of) PCI devices to
reside in the address space 0x4000 0000 thru 0x43FF FFFF ??

     I tried disabling the device, updating my BARS, reenabling in the
driver code (a loadable module).
     I tried writing the BARS just prior to tickling in the
linux/arch/mips/kernel/pci_auto.c code.
     I tried writing the BARS as soon as my device/vendor id are detected
also in the linux/arch/mips/kernel/pci_auto.c code.

Thanks,
Tom

---

From: Pete Popov <[email-masked]>
Date: 11 Jun 2003 16:22:01 -0700

> CONFIG_PCI_AUTO turned on in the .config file.

Why the 0xFF000000 and, in general, the hardcoded addresses?

Which kernel are you using?

> then writes these BARs with addresses in the range of

Right.

>      I have been unable to successfully read anything from my PCI cards CPU

No, that's not going to work. I'll elaborate below.

> reside in the address space 0x4000 0000 thru 0x43FF FFFF ??

Just to be clear, there's nothing MIPS generic about these addresses.
They are Au1500 specific with a doze of arbitrary chose on my part. The
addresses can be changed, carefully :)

> also in the linux/arch/mips/kernel/pci_auto.c code.

OK, none of that will work the way you're doing it. Here's the problem:

The PCI space on the Au1500 SOC is addresses with 36bit physical
addresses. The PCI noncachable memory space available for PCI devices is
0x4 0000 0000 to 0x5 0000 0000.  But, you can't write those addresses in
a BAR and device drivers read the address and try to do an ioremap on
it.  What I did was this:

- set pci memory start address to 0x4 4000 0000 
- set pci memory end address to   0x4 43FF FFFF

These variables are in include/asm-mips/au1000.h. If you take a look at
arch/mips/au1000/common/pci_ops.c, you'll see that in the
pci_mem_resource structure these addresses are truncated to 32 bit:
- 0x4000 0000 and 0x43FF FFFF. These are the addresses that the pci auto
code sees and programs in the BARs. 

When a device driver reads the resource value, 0x4000 0000 for example,
and calls ioremap on that address, the "fixup_bigphys_addr()" function
will get called in __ioremap() in arch/mips/mm/ioremap.c. That function
will detect that 0x4000 0000 falls within the pci memory window, and
"fix it up" to the proper 36 bit physical address. In other words, it
will return 0x4 4000 0000 so that 0x 4 4000 0000 can be remapped instead
of 0x4000 0000 so you can access the pci memory window.  The actual
physical pci address the SOC will put on the bus will be 0x4000 0000 and
since that's what's programmed in the BAR, the device will respond to
it.

The way you're approaching the problem is bypassing the entire
"fixup_bigphys_addr()" (the function is in setup.c) mechanism and it has
no chance or working.

In theory I think you should be able to do what you need. Suppose you
bypassed pci_auto altogether, for example, and you hardcoded the BARs in
setup.c.  One of your bars will be 0xFF000000 the other 0x90000000 (are
both of these supposed to be pci memory addresses?). You need to fixup
those addresses so that when ioremap is called with such an address, the
32 bit address will be fixed up to a 36 bit address and it's that
address that will get remapped. For example, if a device driver calls
ioremap with 0x9000 0000, you have to detect that address in the
fixup_bigphys_addr() function, and return 0x4 9000 0000. Then when you
use the returned virtual address to access the memory window, the CPU
will understand this virtual address to translate to a PCI mem address
and in turn it will put 0x9000 0000 on the bus. That should give you
access to your device.

Pete

---

From: fpga dsp <[email-masked]>
Date: Thu, 12 Jun 2003 12:15:17 +1000 (EST)

Hi,

I am using kernel 2.4.20-pre6 version on db1500 and
having problem with PCI card as bus master. Basicly,
the kernel can recognize the card and assign into
0x40000000 address region, irq is 1. Now that is
really strange, stty1 using irq 1 as well. Are they
the same or different. However ,the problem is that
after setup the device and trigger it, it should go
and fetch the descriptor and will fetch the content
pointed by that descriptor afterward but it only fetch
the descriptor and quiet. I am even try to trigger it
again by write into it register mapped on PCI memory
region but after the first trigger, the second trigger
doesn't appear on pci bus analyzer at all. 

Another issues, it when I look at au1000_eth.c device
driver , dma_alloc() function allocate a DMAable
buffer in KSEG0 region but pci_alloc_consistent return
in KSEG1 region. So which one is right?

Many thanks

 --- Pete Popov <[email-masked]> wrote: > On Wed,
> and in turn it will put 0x9000 0000 on the bus. That

http://mobile.yahoo.com.au - Yahoo! Mobile
- Check & compose your email via SMS on your Telstra or Vodafone mobile.

---

From: fpga dsp <[email-masked]>
Date: Thu, 12 Jun 2003 14:22:13 +1000 (EST)

Hi again,

I think I found the problem, it is the virt_to_bus()
function mapping virtual memory to DMA address and it
return 0x2xxxxxxx, and it could be the reason the PCI
bus master refuse to read those address. Is there
another replacement function call for virt_to_bus()
since I don't want to use pci_alloc_consistent()

Many thanks

 --- fpga dsp <[email-masked]> wrote: > 
> > >    I am porting my first PCI driver for a PCI
> card
> > hosted by the AMD Db1500
> > >     My PCI card REQUIRES 0xFF000000 and
> 0x90000000
> > be programmed into BAR0
> > > and BAR1 respectively.
> > > then writes these BARs with addresses in the
> range
> > of
> > >      4000 0000 thru 43FF FFFF
> > 
> > Right.
> > 
> > > with the MIPS processor AND CONFIG_PCI_AUTO
> turned
> > on,
> > >      as the kernel expects (and configures the
> PCI
> > BARS of) PCI devices to
> > Just to be clear, there's nothing MIPS generic
> about
> > these addresses.
> > OK, none of that will work the way you're doing
> it.
> > Here's the problem:
> > 
> > a BAR and device drivers read the address and try
> to
> > do an ioremap on
> > it.  What I did was this:
> > 
> > These variables are in include/asm-mips/au1000.h.
> If
> > you take a look at
> > will return 0x4 4000 0000 so that 0x 4 4000 0000
> can
> > be remapped instead
> > The way you're approaching the problem is
> bypassing
> > the entire
> > "fixup_bigphys_addr()" (the function is in
> setup.c)
> > mechanism and it has
> > no chance or working.
> > 
> > In theory I think you should be able to do what
> you
> > need. Suppose you
> > both of these supposed to be pci memory
> addresses?).
> > You need to fixup
> > those addresses so that when ioremap is called
> with
> > such an address, the
> > 32 bit address will be fixed up to a 36 bit
> address
> > and it's that
> > will understand this virtual address to translate
> to
> > a PCI mem address
> > and in turn it will put 0x9000 0000 on the bus.
> That
> > should give you
> > access to your device.
> > 
> > Pete
> > 
> > 
> >  
> 
=== message truncated ===

---

From: Pete Popov <[email-masked]>
Date: 12 Jun 2003 10:12:01 -0700

On Wed, 2003-06-11 at 19:15, fpga dsp wrote:
> the same or different. 

There's no UART1 on the Au1500 so you don't have to worry about that
interrupt, even though it's defined in the .h file for the Au1000.

> However ,the problem is that

> in KSEG1 region. So which one is right?

Right for what? KSEG0 works because the cache is coherent. But due to a
pci coherency bug, I think for a new device driver that's a pci bus
master you need to use kseg1 for now.

Pete