Re: [SLUG] 100k files in one directory

From: Logan (logan1304@home.net)
Date: Wed Dec 12 2001 - 19:56:40 EST


I am using ext2. How many files can reiserFS handle in one directory?
Or does it matter since it writes a journal?

On Wed, 2001-12-12 at 12:11, Derek Glidden wrote:
> On Tue, 2001-12-11 at 19:46, Logan wrote:
> > Various folks in my office dumped just shy of 100K files to a linux
> > file server I made for back ups. But, they dumped them all into one
> > directory. There are so many files there that ls times out.
> > I think the cure would be to recompile a kernel, editing
> > /usr/src/linux/include/linux/fs.h to change the NR_FILE variable from
> > 8192 to 8192*10 or even 8192*100. I think this would allow things like
> > ls and rm to work. Or am I barking up the wrong tree?
> > Has anyone else had any experience like this? I know I can also cure
> > it by making more subdirectories and force them to put there files in
> > the subdirectories. But chiding from the Windoze users stirs me to try
> > to let them leave their crap all in one big heap.
>
> What filesystem are you using?
>
> If ext2, there are some patches floating around that dramatically
> increase the speed of file I/O against directories with lots of files.
> You'll have to either ask on lkml or check Kernel-Traffic to find the
> source of them, though, because I don't remember where they are.
>
> ReiserFS really should be able to handle it without much trouble.
>
> XFS and JFS I'm not so sure of, although I use XFS virtually everywhere
> nowadays and have no problems with it, and have a server at home with
> some "junk" directories that get thousands of files dumped into them and
> have never had issues.
>
> Also, how much RAM and what CPU are in that server? Adding more RAM
> will help the system cache filesystem ops better.
>
> If your Windows people are hassling you, you can just come back with
> "So, what do you do if NTFS falls over on something? What other
> filesystems can you use? FAT32?" and then laugh.
>
> --
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> #!/usr/bin/perl -w
> $_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map
> {$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110;
> $t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z)
> [$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join
> "",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d=
> unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d
> >>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q*
> 8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]}
> print+x"C*",@a}';s/x/pack+/g;eval
>
> usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \
> | extract_mpeg2 | mpeg2dec -
>
> http://www.cs.cmu.edu/~dst/DeCSS/Gallery/
> http://www.eff.org/ http://www.anti-dmca.org/

-- 
7:50pm up 6 days, 12:33, 1 user, load average: 0.79, 0.41, 0.42
Frisbeetarianism, n.: The belief that when you die, your soul goes up
the on roof and gets stuck.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 20:11:35 EDT