Re: ext2fs 2GiB file size limit

Previous thread: Sugestões de Natal da Loja21 by Loja21 - Os gadgets mais incriveis on Tuesday, December 7, 2010 - 10:30 am. (1 message)

Next thread: Hello Misc by gdr@gno.org on Wednesday, December 8, 2010 - 1:09 am. (1 message)
From: LEVAI Daniel
Date: Wednesday, December 8, 2010 - 1:23 am

Hi!


I'm having this issue with an ext2fs filesystem on an external USB
drive.
I've created the partition/disklabel/filesystem with OpenBSD tools, and
after mounting, and dd'ing /dev/zero onto the fs, it bails out at 2GiB:

$ dd if=/dev/zero of=/mnt/hotplug/sd3i_FUJITSU/Downloads/testfile bs=4096
dd: /mnt/hotplug/sd3i_FUJITSU/Downloads/testfile: File too large
524288+0 records in
524287+0 records out
2147479552 bytes transferred in 399.042 secs (5381579 bytes/sec)


I don't remember having a 2GiB filesize limit anywhere near, but the old
extfs. What am I missing here?


Thanks,
Daniel

-- 
LIVAI Daniel
PGP key ID = 0x83B63A8F
Key fingerprint = DBEC C66B A47A DFA2 792D  650C C69B BE4C 83B6 3A8F

From: Guillaume Dualé
Date: Wednesday, December 8, 2010 - 1:35 am

On Wed, 8 Dec 2010 09:23:49 +0100, LEVAI Daniel <leva@ecentrum.hu>
http://en.wikipedia.org/wiki/Ext2
 Max file size 	16 GB - 2 TB

So, check if your partition is really in ext2 and the size of this
partition.
And what is the physical size of your external drive ? :)


From: LEVAI Daniel
Date: Wednesday, December 8, 2010 - 1:59 am

$ mount
/dev/sd3i on /mnt/hotplug/sd3i_FUJITSU type ext2fs (local, nodev, noexec, nosuid)


$ df -h
/dev/sd3i     73.3G   26.4G   46.9G    36%    /mnt/hotplug/sd3i_FUJITSU


$ fdisk sd3
Disk: sd3       geometry: 9729/255/63 [156301488 Sectors]
Offset: 0       Signature: 0xAA55
            Starting         Ending         LBA Info:
 #: id      C   H   S -      C   H   S [       start:        size ]
-------------------------------------------------------------------------------
 0: 00      0   0   0 -      0   0   0 [           0:           0 ] unused
 1: 00      0   0   0 -      0   0   0 [           0:           0 ] unused
 2: 00      0   0   0 -      0   0   0 [           0:           0 ] unused
*3: A6      0   1   1 -   9728 254  63 [          63:   156296322 ] OpenBSD


$ disklabel sd3
# /dev/rsd3c:
type: SCSI
disk: SCSI disk
label: FUJITSU MHV2080B
duid: b122005bf43a432c
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 9729
total sectors: 156301488
boundstart: 63
boundend: 156296385
drivedata: 0

16 partitions:
#                size           offset  fstype [fsize bsize  cpg]
  c:        156301488                0  unused
  i:        156296321               63  ext2fs


dmesg sd3 part:
umass1 at uhub0 port 1 configuration 1 interface 0 "Initio MHV2080BH" rev 2.00/3.01 addr 5
umass1: using SCSI over Bulk-Only
scsibus5 at umass1: 2 targets, initiator 0
sd3 at scsibus5 targ 1 lun 0: <Initio, MHV2080BH, 3.01> SCSI0 0/direct fixed
sd3: 76319MB, 512 bytes/sec, 156301488 sec total



dmesg:
OpenBSD 4.8-current (GENERIC.MP) #545: Mon Dec  6 16:24:15 MST 2010
    deraadt@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Genuine Intel(R) CPU T2400 @ 1.83GHz ("GenuineIntel" 686-class) 1.83 GHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,VMX,EST,TM2,xTPR,PDCM
real mem  = 2145808384 (2046MB)
avail mem = 2100625408 ...
From: Guillaume Dualé
Date: Wednesday, December 8, 2010 - 2:42 am

On Wed, 8 Dec 2010 09:59:42 +0100, LEVAI Daniel <leva@ecentrum.hu>

Hi,
you could try it :
http://www.openbsd.org/cgi-bin/man.cgi?query=newfs_ext2fs&sektion=8&apropos=0&...

Guillaume.

From: LEVAI Daniel
Date: Wednesday, December 8, 2010 - 3:09 am

On Wed, Dec 08, 2010 at 10:42:09 +0100, Guillaume Duali wrote:

So, you think recreating the filesystem would solve the problem? I'm
sorry, but I'm not used to the "reboot if it's not working" scheme.


Daniel

-- 
LIVAI Daniel
PGP key ID = 0x83B63A8F
Key fingerprint = DBEC C66B A47A DFA2 792D  650C C69B BE4C 83B6 3A8F

From: Peter J. Philipp
Date: Wednesday, December 8, 2010 - 4:47 am

Hi,

I think it is enforced here (in the file /sys/ufs/ext2fs/ext2fs_inode.c:

--
int
ext2fs_setsize(struct inode *ip, u_int64_t size)
{
        if ((ip->i_e2fs_mode & IFMT) == IFREG ||
            ip->i_e2fs_mode == 0) {
                ip->i_e2fs_dacl = size >> 32;
                if (size >= 0x80000000U) {
		^^^^^^^^^^^^^^^^^^^^^^^^
--

Which leaves probably this i_e2fs_mode which is probably a superblock flag.

I checked the newfs_ext2fs manpage and it says the following:

--
     -O filesystem-format
                 Select the filesystem-format.

                       0    `GOOD_OLD_REV'; this option is primarily used to
                            build root file systems that can be understood by
                            old or dumb firmwares for bootstrap.  (default)
                       1    `DYNAMIC_REV'; various extended (and sometimes
                            incompatible) features are enabled (though not all
                            features are supported on OpenBSD).  Currently
                            only the following features are supported:
...<some cut>...
                                  LARGEFILE    Enable files larger than 2G
                                               bytes.
--

so perhaps you need to turn on this "LARGEFILE" feature at newfs time somehow...

Good luck,

-peter

From: LEVAI Daniel
Date: Wednesday, December 8, 2010 - 4:57 am

Indeed.

It seems my ignorance and impudent style didn't get me anywhere (again).

Thanks to You and Guillaume for the pointers and the info!


Daniel

-- 
LIVAI Daniel
PGP key ID = 0x83B63A8F
Key fingerprint = DBEC C66B A47A DFA2 792D  650C C69B BE4C 83B6 3A8F

From: frantisek holop
Date: Friday, December 10, 2010 - 9:11 am

you left out the interesting bit :]

                        if (fs->e2fs.e2fs_rev <= E2FS_REV0) {
                                /* Linux automagically upgrades to REV1 here! */
                                return (EFBIG);
                        }

there was a funny chicken-egg problem with this.
the e2fsprogs' mkfs.ext2 does not set 'large_file'
by default either, but as the linux kernel "automagically
upgrades to REV1" whenever a process tries to write a bigger
file for the first time, it is not really a problem.

but naturally, people with linux background trying to use
ext2fs on different systems get to know about this flag after
lot of hair pulling.

i have asked Theodore in the past to enable this feature
by default, but looks like it never happened.

i think it could save a lot of time and sanity if the
openbsd kernel emitted a dmesg when someone tries
to write a bigger file..

Index: ext2fs_inode.c
===================================================================
RCS file: /cvs/src/sys/ufs/ext2fs/ext2fs_inode.c,v
retrieving revision 1.43
diff -u -r1.43 ext2fs_inode.c
--- ext2fs_inode.c      23 Nov 2008 23:52:35 -0000      1.43
+++ ext2fs_inode.c      10 Dec 2010 16:09:33 -0000
@@ -83,6 +83,8 @@

                         if (fs->e2fs.e2fs_rev <= E2FS_REV0) {
                                 /* Linux automagically upgrades to REV1 here! */
+                               log(LOG_NOTICE,
+                                   "ext2fs revision does not support large files\n");
                                 return (EFBIG);
                         }
                         if (!(fs->e2fs.e2fs_features_rocompat

-f
-- 
life is lived forwards, but understood backwards.

Previous thread: Sugestões de Natal da Loja21 by Loja21 - Os gadgets mais incriveis on Tuesday, December 7, 2010 - 10:30 am. (1 message)

Next thread: Hello Misc by gdr@gno.org on Wednesday, December 8, 2010 - 1:09 am. (1 message)