Subject: RE: iptables/vmalloc issues on alchemy

From: "Christian Gan" <[email-masked]>
Date: Thu, 28 Apr 2005 15:11:27 -0500

Could this also related to this problem I posted a long time ago?  I
haven't had a chance to revisit things for a while...

-----Original Message-----
Just a little update on this:

I turned off 36 bit address support for the MIPS32
(CONFIG_64BIT_PHYS_ADDR) and found that my tests for vmalloc work.

Can anyone who knows better point me in a direction of where I should
start?

Thanks!

Christian

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Christian Gan
Sent: Tuesday, January 04, 2005 3:25 PM
To: [email-masked]
Subject: vmalloc memory corruption

Hey all,

I'm running into a strange memory corruption problem while trying to get
a driver to load firmware using hotplug support on a DBAU1550 eval
board.  This board has a single AU1550 MIPS32 processor.  I'm running
kernel version 2.6.10-rc3.

I've attached a code snippet below that I'd like somebody to try to
verify for me so I can determine whether or not it is a problem with my
kernel/hardware.  

The test code is a simplified version of the interaction between the
driver and the firmware/hotplug support of the kernel.  The firmware
hotplug support (linux/drivers/base/firmware_class.c) if fed data from a
file and places it into a buffer, this buffer gets reallocated when
required and grows as more data is read in.  Old data is memcopied into
the new reallocated buffer and newly read data is appended to the end.
What I'm finding is that eventually this data is corrupted.

Compile the test snippet below as a module and insmod it.  If things are
successful you should see results like this:

# insmod vmalloctest.ko
Using vmalloctest.ko
testing vmalloc
0x00 0x01 0x02 0x03 0x04 0x05 0x06 0x07 0x08 0x09 0x00 0x01 0x02 0x03
0x04 0x05 0x06 0x07 0x08 0x09 0x00 0x01 0x02 0x03 0x04 0x05 0x06 0x07
0x08 0x09 ...
0x00 0x01 0x02 0x03 0x04 0x05 0x06 0x07 0x08 0x09

But if data changes at any time, you've got problems like me.  When I
run this on my platform eventually the data becomes all 0x00.

Notice that if you use kmalloc instead of vmalloc (define USE_KMALLOC),
everything works.

Thanks!

Christian Gan

// Start vmalloctest.c

#ifndef __KERNEL__
#define __KERNEL__
#endif

#include <linux/config.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/vmalloc.h>

#include <asm/io.h>

//#define USE_KMALLOC

void fillbuf(char * buf, unsigned long len) {
	int i = 0;
	for (i = 0; i < len; i++)
	{
		buf[i] = 0xFF & i;
	}
}

int __init vmalloctest_init(void)
{
    char * buf = NULL;
	int i = 0;

	printk("testing vmalloc\n");
	
	for (i = 0; i < 50; i++)
	{
		int j = 0;
#ifndef USE_KMALLOC		
		char * newbuf = vmalloc( (i+1)*PAGE_SIZE ); // allocate
a new buffer that grows in size #else
		char * newbuf = kmalloc( (i+1)*PAGE_SIZE, GFP_KERNEL );
// allocate a new buffer that grows in size
#endif		
		if (!newbuf)
		{
			printk("Could not allocate memory: %ld bytes\n",
(i+1)*PAGE_SIZE);
			break;
		}
				
		if (i==0)
		{
			fillbuf(newbuf, PAGE_SIZE );		// fill
the original buffer with some data... any will suffice
		}
		else
		{
			memcpy(newbuf, buf, i*PAGE_SIZE);	// copy
the contents of the old buffer to the new	
		}
		
#ifndef USE_KMALLOC
		vfree(buf);		// free the old buffer
#else
		kfree(buf);		// free the old buffer
#endif		

		buf = newbuf;
		
		// print out the first few bytes
		for (j=0; j < 10; j++)
		{
			printk("0x%02X ", 0xff & buf[j]);
		}
		printk("\n");
	}

#ifndef USE_KMALLOC
	vfree(buf);
#else
	kfree(buf);
#endif		

	return 0;
}
module_init(vmalloctest_init);

MODULE_LICENSE("GPL");

---

From: Dan Malek <[email-masked]>
Date: Thu, 28 Apr 2005 16:56:39 -0400

> haven't had a chance to revisit things for a while...

I've just been talking about 2.6, so "long time ago" can't be
that long :-)  I have the updates to the exception handler so
the PTEs get loaded correctly, that's on the way.  I think 2.4
should be OK if anyone is using that.

The one that bothers me is this patch I've been sitting on for
quite some time.  It's not specific to Alchemy, it's a generic
MIPS page table issue.  The problem started when someone
loaded very large modules which caused a new kernel page
table to be allocated.  For some reason I don't remember,
the vmalloc_fault fixup didn't handle this, and the applications
would crash the system because their pgd page  didn't
get updated to reflect this.  The address must have been in
the vmalloc space, but I no longer have the system for testing
(but the code is running in a several thousand real products).
The only way I could make this work is to test the fault address,
the most significant bit anyway, in the TLB miss handler to
see if it was a user or kernel address.  If it was a kernel address,
I loaded the init_mm pgd instead of the thread pgd.  Just one
test and a couple of instructions, but it was necessary.  I'm
still puzzling, but will remember :-)

Thanks.

	-- Dan