[PLUG] Database layout vs. performance

Andrew Puch aapuch at attbi.com
Sat May 18 02:46:18 UTC 2002


On Fri, 2002-05-17 at 14:42, Alex Daniloff wrote:
> Hello Brian,
> Thanks for your response.
> This MySQL with InnoDB tables database is intended for collecting real
> time test/process data from the running equipment log files and
> providing raw data for data analysis software to display results over
> the FCGI web interface. Web server, data analysis software and
> database should be residing on the same box to reduce network traffic.

MySQL, is not the greatest database. 

It does not pass Mr Dodd ACID Test.

Look into postgres, in some cases it was faster then MySql. 

MySQL has the habbit of not gracefully failing. 

here read these links. 

http://www.phpbuilder.com/columns/tim20000705.php3


> 
> We have two el'cheapo SideBus soft Raid0 EIDE controllers and four
> ATA100 EIDE 40GB hard drives. Sorry we can't afford SCSI. Budget is
> tight.
> 
> There are two possible solutions: 
> We can set up two Raid0 arrays and mirror one another simultaneously
> or syncronizing them once a day.
> Or we can just leave these hard drives where they are and spread data
> directories across all four of them.
> 
> My major question is which method is better for best performance and
> what FS (EXT2, EXT3, ReiserFS, JFS) is better to use for this task.

EXT3 or XFS. 

I know the Redhat developers they swore by EXT3. but they pay for the
development of ext3. Aka the own dogfood theory. Steven Tweedy of Tweedy
disk farm, works for the hat. but I would look into.

I love ext3 for my home box, I have 2 40G /home. So Fdisk is nice and
fast. 

I used to work for IBM setting up Linux for benchmarking SAP/Oracle. 

We found that we got higher threw-put via software raid, on 4-8 ways
Zeons. And 4 way PPros.

Working at IBM we had all the nice raid cards and they were always
slower. Then software RAID. Buy a nice SMP motherboard, will out
preform, UNI motherboard with the raid controller that money could by.
If the drives remain the same, and the # of channels in the raid array. 

XFS. 

IMHO is the best journaling file system for linux. 

It has been design from the ground up to be a true journaling file
system. 

EXT3 is a hack, to ext2 to add journaling. 

JFS is not the JFS on AIX. 

It is from the source tree of Stale OS/2 JFS. So the development is not
the to the same level as the enterprise level as IBM AIX. 

http://oss.sgi.com/projects/xfs/features.html

2^63 = 9 x 1018 = 9 exabytes.  or 9000 Terabytes I think :) 
That is alot of MP3. 

POSIX Access Control Lists (ACLs)

The Linux XFS filesystem supports the ACL semantics and interfaces
described in the draft POSIX 1003.1e standard.

Maximum File Size

For Linux 2.4, the maximum accessible file offset is 16TB on 4K page
size and 64TB on 16K page size. As Linux moves to 64 bit on block
devices layer, file size limit will increase to 9 million terabytes (or
the system drive limits).



Hardware 

I would buy a new MPX Athlon with a dual 2, load up the ram. 

Buy a few Promise ATA 133 cards. 

For Best performance, no chaining the ide. aka 1 drive per each ide
cable.  

Loadup the ide, there is a new Wester Digital Driver with 2 or 4 megs of
cache. I think. 

Also I would do a nice raid 5. 

The hardware above is what I priced out for myself. 

was around 2,000 for 1/4 Terabyte. 


Also you want to see if the LVM, stuff works now. Aka not buggy. 





More information about the PLUG mailing list