Subject: troubles writing to a mmapped PCI BAR

From: Clem Taylor <[email-masked]>
Date: Thu, 21 Apr 2005 15:31:21 -0400

I'm working on a Au1550 driver for a PCI based co-processor. The
driver provides mmap() that allows the mapping of a PCI BAR. From
userspace I can happily read from the mmaped region, but writes just
hang the user space program. gdb shows that the user program is
sitting at the write statement. The read() and write() system calls
work just fine as well.

In the driver mmap() does:
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
return io_remap_page_range(vma, vma->vm_start,
    barStart + (vma->vm_pgoff << PAGE_SHIFT),
    vma->vm_end - vma->vm_start, vma->vm_page_prot);

The user space program does:
u32 *mem;
fd = open("/dev/moo-0", O_RDWR);
mem = mmap(NULL, 4*1024*1024. PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
...
fprintf(stderr, "mem[42]=0x%08X\n", mem[42]);
mem[42] = 0xDEADBEEF;

When I run the test, it will print the current value of mem[42], then
hang on the write to mem[42]. /proc/pid/maps shows the mmap with rw-s
permissions:
2abd4000-2afd4000 rw-s 00000000 00:09 134        /dev/moo-0
top shows that the test process is consuming ~100% of the CPU.

I'm really at a loss as to what is happening. I'd imagine that the
userspace program is page faulting (not sure how to verify that) and
the fault handler is not returning. Any ideas what I might be missing?

                       Thanks,
                       Clem

---

From: Clem Taylor <[email-masked]>
Date: Fri, 22 Apr 2005 16:22:53 -0400

> sitting at the write statement. The read() and write() system calls

I saw in drivers/video/epson1356fb.c it was doing:
        // FIXME: shouldn't have to do this. If the pages are marked writeable,
        // the TLB fault handlers should set these.
        pgprot_val(vma->vm_page_prot) |= (_PAGE_DIRTY | _PAGE_VALID);

So I tried adding this before my io_remap_page_range() and it seems to
have fixed my problem. My mmap() write to a PCI device is working now.
I tried setting just the _PAGE_DIRTY bit and that seems to be the
trick. Any ideas why I would need do to this or what is going on?

                        --Clem

---

From: Ralf Baechle <[email-masked]>
Date: Fri, 22 Apr 2005 21:27:40 +0100

> trick. Any ideas why I would need do to this or what is going on?

Without the process simply won't have access priviledges to that page, see
the TLB chapter of the CPU manual.

(I'm simplifying things alot here.)

  Ralf