Subject: Does e2fsprogs-1.26 work on mips?

From: "H . J . Lu" <[email-masked]>
Date: Sat, 23 Mar 2002 14:07:28 -0800

I got

[root@localhost e2fsprogs-1.26]# ./e2fsck/e2fsck -f /dev/hda1
e2fsck 1.26 (3-Feb-2002)
Pass 1: Checking inodes, blocks, and sizes
File size limit exceeded

on Linux/mipsel. /dev/hda1 is a 7GB ext3 partition. e2fsprogs-1.23
works fine. Strace

open("/dev/hda1", O_RDWR|O_LARGEFILE)   = 4
fstat(4, {st_mode=S_IFBLK|0660, st_rdev=makedev(3, 1), ...}) = 0
setrlimit(RLIMIT_FSIZE, {rlim_cur=-1, rlim_max=-1}) = 0
getrlimit(RLIMIT_FSIZE, {rlim_cur=-1, rlim_max=-1}) = 0
lseek(4, 1024, SEEK_SET)                = 1024
read(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
102
4
lseek(4, 4096, SEEK_SET)                = 4096
read(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
40
96
time(NULL)                              = 1016919001
lseek(4, 1072, SEEK_SET)                = 1072
write(4, "\331\363", 2)                 = 2
lseek(4, 1120, SEEK_SET)                = 1120
write(4, "\2\0", 2)                     = 2
lseek(4, 4096, SEEK_SET)                = 4096
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 134217728, SEEK_SET)           = 134217728
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 134221824, SEEK_SET)           = 134221824
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 402653184, SEEK_SET)           = 402653184
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 402657280, SEEK_SET)           = 402657280
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 671088640, SEEK_SET)           = 671088640
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 671092736, SEEK_SET)           = 671092736
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 939524096, SEEK_SET)           = 939524096
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 939528192, SEEK_SET)           = 939528192
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
lseek(4, 1207959552, SEEK_SET)          = 1207959552
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
lseek(4, 1207963648, SEEK_SET)          = 1207963648
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
_llseek(4, 18446744072770027520, [3355443200], SEEK_SET) = 0
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
_llseek(4, 18446744072770031616, [3355447296], SEEK_SET) = 0
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
_llseek(4, 18446744073038462976, [3623878656], SEEK_SET) = 0
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
10
24
_llseek(4, 18446744073038467072, [3623882752], SEEK_SET) = 0
write(4, "\2\0\0\0\3\0\0\0\4\0\0\0\0\0\365?\2\0\0\0\0\0\0\0\0\0\0"..., 4096) =
4
096
_llseek(4, 18446744071696285696, [6576668672], SEEK_SET) = 0
write(4, "\0\200\22\0b\357$\0\304\330\1\0\230\211\36\0\35\322\21"..., 1024) =
-1
 EFBIG (File too large)
--- SIGXFSZ (File size limit exceeded) ---
+++ killed by SIGXFSZ +++

Any ideas?

H.J.

---

From: Andrew Morton <[email-masked]>
Date: Sat, 23 Mar 2002 16:21:50 -0800

> on Linux/mipsel. /dev/hda1 is a 7GB ext3 partition. e2fsprogs-1.23

Common problem - it's due to internal API changes in resource
limits.  You need to ensure that your maximum file size
is set to `unlimited'.  Kernel is currently applying file
size limits to block devices (which is broken) and I
think e2fsprogs' attempt to set sile size limits to
RLIM_INFINITY gets broken by a consipiracy between the
kernel change and glibc headers.

---

From: "H . J . Lu" <[email-masked]>
Date: Sun, 24 Mar 2002 01:28:19 -0800

On Sat, Mar 23, 2002 at 11:03:30PM -0800, Andrew Morton wrote:
> "H . J . Lu" wrote:
> > 
> > ...
> > RLIM_INFINITY is not ((unsigned long)(~0UL)). Also you can't assume
> > the type of rlim.rlim_cur.
> > 
> > Here is a patch.
> > 
> 
> I suspect it's not right.
> 
> I don't pretend to understand the details, but they're
> messy.   See Ted's recent words at
> 
> 	http://www.uwsg.iu.edu/hypermail/linux/kernel/0203.2/0846.html
> 

I look at the glibc code. It uses a constant RLIM_INFINITY for a given
arch. The user always passes (~0UL) to glibc on x86. glibc will check
if the kernel supports the new getrlimit at the run time. If it
doesn't, glibc will adjust the RLIM_INFINITY for setrlimit. I don't see
how glibc 2.2.5 compiled under kernel 2.2 will fail under 2.4 due to
this unless glibc is misconfigureed or miscompiled.

> (Sorry - I should have dug that message out earlier).

The problem is not all arches use (~0UL) for RLIM_INFINITY.

# cd linux/include
# grep RLIM_INFINITY asm-*/resource.h | grep define
asm-alpha/resource.h:#define RLIM_INFINITY      0x7ffffffffffffffful
asm-arm/resource.h:#define RLIM_INFINITY        (~0UL)
asm-cris/resource.h:#define RLIM_INFINITY       (~0UL)
asm-i386/resource.h:#define RLIM_INFINITY       (~0UL)
asm-ia64/resource.h:#define RLIM_INFINITY  (~0UL)
asm-m68k/resource.h:#define RLIM_INFINITY       (~0UL)
asm-mips64/resource.h:#define RLIM_INFINITY  (~0UL)
asm-mips/resource.h:#define RLIM_INFINITY       0x7fffffffUL
asm-parisc/resource.h:#define RLIM_INFINITY   (~0UL)
asm-ppc/resource.h:#define RLIM_INFINITY        (~0UL)
asm-s390/resource.h:#define RLIM_INFINITY   (~0UL)
asm-s390x/resource.h:#define RLIM_INFINITY   (~0UL)
asm-sh/resource.h:#define RLIM_INFINITY (~0UL)
asm-sparc64/resource.h:#define RLIM_INFINITY    (~0UL)
asm-sparc/resource.h:#define RLIM_INFINITY      0x7fffffff

What should we do about it? I know e2fsprogs-1.26 doesn't work on mips
nor alpha because of this. I don't think it works on sparc.

BTW, mips has

/*
 * SuS says limits have to be unsigned.
 * Which makes a ton more sense anyway.
 */
#define RLIM_INFINITY   0x7fffffffUL

It doesn't make any senes.

H.J.

---

From: Theodore Tso <[email-masked]>
Date: Mon, 25 Mar 2002 00:31:59 -0500

> nor alpha because of this. I don't think it works on sparc.

Yeah, I forced the release of e2fsprogs 1.27 because of this, back on
March 8th.  That was my fault, and I fixed it as soon as I discovered
it.  (1.26 was released on Feb 3, and I released 1.27 on March 8th).

In e2fsprogs 1.27, I do the following:

#ifdef __linux__
#undef RLIM_INFINITY
#if (defined(__alpha__) || ((defined(__sparc__) || defined(__mips__)) && (SIZEOF_LONG == 4)))
#define RLIM_INFINITY	((unsigned long)(~0UL>>1))
#else
#define RLIM_INFINITY  (~0UL)
#endif

Basically because I can't depend on the RLIM_INFINITY being "right".
(Remember, I'm trying to make sure that e2fsprogs can compile on any
arbitrary glibc, and then run on any other-not-necessarily-the-same
glibc, which gets "challenging".)

The problem now is that some older glibcs are capping RLIM_INFINITY
with the older value, and so a combination of a new kernel, new
e2fsprogs, and old glibc results in problems.  My current feeling
about how to fix this is to bypass glibc altogether, and simply call
the setrlimit system call directly.  This is ugly-ugly-ugly, but I
can't see another way around this.

And just to be clear ---- although in the past I've been really
annoyed when glibc has made what I've considered to be arbitrary
changes which have screwed ABI, compile-time, or link-time
compatibility, and have spoken out against it --- in this particular
case, I consider the fault to be purely the fault of the kernel
developers, so there's no need having the glibc folks get all
defensive....

						- Ted

---

From: Andrew Morton <[email-masked]>
Date: Sun, 24 Mar 2002 21:43:18 -0800

> developers, so there's no need having the glibc folks get all

So... Does the kernel need fixing? If so, what would you
recommend?

-

---

From: Theodore Tso <[email-masked]>
Date: Tue, 26 Mar 2002 01:54:40 -0500

> So... Does the kernel need fixing? If so, what would you

1) Created a new syscall for the unsinged setrlimit, not just for
getrlimit.  This should have been done from the very beginning, IMHO.

2) If the old value of RLIM_INFINITY is passed to the old setrlimit,
translate it to the new value of RLIM_INFINITY.  (This would not have
been strictly necessary of glibc wasn't playing RLIM_INIFITY capping
games; as it turns out, if you pass the "new" version of RLIM_INIFITY
to an "old" 2.2 kernel, the right thing happens.  So there really is
no need for glibc to cap the limit of RLIM_INFINITY to the old value.)

3)  RLIMIT_FILESIZE should not apply to block devices!!!

							- Ted

---

From: Paul Mackerras <[email-masked]>
Date: Tue, 26 Mar 2002 21:51:56 +1100 (EST)

> 3)  RLIMIT_FILESIZE should not apply to block devices!!!

Absolutely.

I would go further and say that it should only apply to writes to a
regular file that would extend the file past the filesize limit.  At
the moment the check in generic_file_write is simply whether the file
offset is greater than the limit, or would be greater than the limit
after the write.  This doesn't seem right to me.  If, for example, my
RLIMIT_FILESIZE is 1MB, and I have write access to an existing 100MB
file, I think I should be able to write anywhere in that file as long
as I don't try to extend it.

If we did that then the block device case would fall out, since you
can't extend block devices (not by writing past the end of them
anyway).

Paul.