BBS水木清华站∶精华区

发信人: saka (机器猫), 信区: Linux
标  题: NT与Linux大比拼的真实情况
发信站: BBS 水木清华站 (Mon Apr 19 07:36:04 1999) WWW-POST

Mindcraft Reality Check

                      E.L. Green, Linux Hardware Solutions

I. Executive Summary:

Recently Mindcraft released a study comparing network performance of Windows
NT 4.0 and Red Hat 5.2. The results
of such performance benchmarks are of great interest to us and to the Linux
community as a whole. Past benchmarks of
Linux performance vs. its competitors have allowed us to perform many
enhancements and improvements to the
performance of the Linux kernel and associated programs, such as improved
network performance (a result of
benchmarks against FreeBSD), improved thread switch performance (a result of
benchmarks against Windows NT),
and improved performance of the kernel open() call (a result of benchmarks
against Windows NT).

Fair comparisons are welcomed by those of us in the Linux community.
Unfortunately, this Mindcraft comparison
appears to be fatally flawed by misconfigurations of Linux, and thus is not
useful for comparing the true performance of
Red Hat 5.2 versus Windows NT.

II. Personnel Problems with Study

Mindcraft has proven themselves in the past to be masters of expertly tuning
Windows NT. See, for example, their
comparison testing of Windows NT versus Novell Netware 5.0, which showed
Mindcraft's skill at expertly tuning
Windows NT to its best. In this study, Mindcraft continues to display their
expertise in tuning Windows NT.
Unfortunately, they also demonstrate their lack of expertise in tuning Linux.

Tuning Linux, and any Unix in general, requires a certain level of expertise.
This level of expertise cannot be gained
within the three-day time period during which this study took place. This
expertise can be gained via coursework from
companies such as Learning Tree International or Eklektix, via concerted
querying of the USENET newsgroups via
DejaNews over a period of months, and by other such mechanisms over a period
of months. This level of expertise is
similar to the level of expertise needed to tune Windows NT, which generally
requires MCSE coursework in order to
fully master.

A useful comparison test between Windows NT and Red Hat Linux 5.2 would
require that a Linux engineer with similar
expertise in tuning high-end systems be on hand in order to properly tune the
Linux system. When ZD Labs
benchmarked Samba versus NT, they invited Jeremy Allison of the Samba team to
assist in setting up their Samba
configuration. A comparison test of properly tuned Linux and Windows NT
systems would be quite constructive.

Unfortunately, from looking at the large number of technical mistakes (listed
below) made while configuring the Linux
system, it is quite obvious that such Linux expertise was not on hand. This
violation of basic testing protocol means that
the results of the test are not useful for valid comparison purposes.

III. Technical Problems with Study

A. Hardware Problems with Study

1) RAID controller Problems: The AMI RAID controller used in the study is
known to have difficulties running under
Linux. Linux Hardware Solutions was approached several months ago by AMI
regarding using their RAID controllers
on our systems, and after thorough evaluation, we concluded that the drivers
for those controllers were still of
beta-quality and not appropriate for a production server. AMI themselves to a
certain extent admits this -- the version
of the AMI RAID driver used by Mindcraft was version 0.92, i.e., it wasn't
even mature enough to be awarded a 1.0
version number.

A fair test would use a RAID controller equally well supported by both Linux
and Windows NT, such as the ICP-Vortex
GDT line of RAID controllers used by Linux Hardware Solutions, or the Mylex
line of RAID controllers used by VA
Research Linux Solutions. In reality, a company buying a server of this size
for use with Linux is going to buy one
optimized for Linux with an appropriate RAID controller, rather than buy an
off-the-shelf DELL server with a poorly
supported RAID controller.

2) Disk Configuration Problems with Study: Both Windows NT and Linux were
installed simultaneously on the
tested system. The drives were partitioned such that Windows NT occupied half
the drive, and Linux occupied the other
half. Unfortunately, it is not stated within the study whether Linux was on
the inner half or outer half of the drive. Note
that the outermost tracks of a hard drive have approximately 1.5 to two times
the data transfer rate of the innermost
tracks, due to the larger number of sectors spinning past the head in a
second's time.

B. Linux Configuration Problems with Study

1) Failure to configure Linux with large memory support A quick search of
DejaNews with the parameters "2
gigabyte memory linux" swiftly turned up a message on how to configure Linux
to access up to 2 gigabytes of physical
memory. In their test, Mindcraft had Linux accessing only 96% of the memory
that they had allocated for NT. This
would be an immediate 4% performance hit in most file serving situations.

2) Failure to tune buffer allocation parameters Like all Unix-type operating
systems, the Linux operating system has
various tuning parameters to control the allocation of buffer memory, number
of file handles available, and number of
inode entries allowed. The default Linux 2.2 kernel will use up to 60% of
physical memory as disk buffer cache, will allow
up to 4096 open files, and will allow up to 12288 inodes. This is optimal for
applications serving, but is completely
inappropriate for file or web serving on a 1gb machine. Tuning the file
buffer size so that more than 60% of memory can
be used (90% in this example) can be accomplished by issuing the following
command:
                          echo "2 10 90" >/proc/sys/vm/buffermem"

This is documented in the file /usr/src/linux/Documentation/sysctl/vm.txt
along with many other tuning parameters,
such as the 'bdflush' parameter. Jeremy Allison of the Samba team claims a
factor of two improvement in Samba
performance when these parameters are set properly.

Please note that queries of the DejaNews archives looking for queries by
MindCraft researchers regarding tuning the
Linux 2.2 kernel during the relevant time period turned up only one possible
(but not probable) message, which asked a
question about network card tuning. During the time span of January 1, 1999
through April 12, 1999, there were no
messages posted to the Usenet asking about memory or buffer tuning for large
memory machines.

3) Use of inappropriate kernel version The Linux 2.2.2 kernel's network stack
had severe performance problems
interacting with Microsoft clients. For more information, see the Kernel
Traffic newsletter for March 4, 1999. Given
the known problems with the 2.2.2 kernel, either the 2.2.1 kernel should have
been used, or the 2.2.3 kernel should have
been used.

4) Allocation of Swap not Specified Linux requires a swap partition for best
operation. If the swap space is allocated on
multiple drives (one swap partition per drive), the Linux VM system will
utilize the drives in parallel, effectively using
them as a RAID0 (striping) array. Unfortunately, we don't know how swap space
was allocated in the Mindcraft test, and
thus cannot duplicate their configuration.

C. Samba Configuration Problems

1) Recompilation of Samba with inappropriate settings: Red Hat Software has
the Samba 2.0.x server available
precompiled on their web site as a pre-packaged upgrade to go along with the
2.2 kernel. For some reason Mindcraft
chose to use their own custom compiled package which was not compiled with
the optimizations that Red Hat compiles
into their own packages. In particular, the Mindcraft package was recompiled
with the -O option rather than the -O2
option that Red Hat uses. In other words, the compiled binary was compiled
with options that would generally make it
noticably slower than the one shipped by Red Hat Software. To quote the GNU
"gcc" documentation regarding the
"-O2" option:
    As compared to `-O', this option increases both compilation time and the
performance of the generated code.

For a fair test, either the Red Hat provided Samba should have been used, or
if a 'performance tweak' was needed to
insure that both systems were optimized, the "egcs" C compiler (a standard
part of Red Hat Linux) should have been
used with the -mpentiumpro option to optimize the resulting code to take
advantage of Pentium II features.

2) Inappropropriate use of the widelinks=no option: Andrew Tridgell of the
Samba team reports that setting
"widelinks=no" option will noticably slow performance of the Samba server.
The "widelinks=no" option is a security
feature intended for mixed-use systems, rather than for dedicated server use.
For dedicated server use it is unnecessary,
since Windows clients are unable to set filesystem links on a Samba server
(only interactive Linux users can do so).

Please note that Jeremy Allison of the Samba team reports no queries on the
Samba mailing lists or at the Samba home
page that could have been about tuning Samba for use on large file servers
during the time period in question.

D. Apache Configuration Problems

1) Inappropriate compilation:Again, for some unknown reason Mindcraft chose
to re-compile Apache rather than
use the default Red Hat version. This makes it difficult to compare "out of
box" performance. If the purpose was to
improve performance, using "egcs" to recompile it with the -mpentiumpro
optimizations would have accomplished
much more. Some Apache experts claim as much as 30% improvement by compiling
Apache with the Pentium Pro
optimizations.

2) Inappropriate settings for StartServers and SpareServers: If you are
running an enterprise server expecting 150
or more simultaneous connections, you would set StartServers to 150 or
greater. It's not as if there is a shortage of
memory, after all.

3) Inadequate information to duplicate the test:Access to copies of the httpd
configuration files is needed to further
duplicate the test and diagnose the problems found there. For example, the
"collapse" of the Apache server described
therein is counter to what every other performance survey of Apache has
found, which is that Apache reaches a
maximum and then mildly degrades as overload continues.

A search of the DejaNews archives shows that a "will@whistlingfish.net"
posted a query about this "collapse" problem.
Unfortunately, it does not give enough information to make any sort of
diagnosis possible (probably why it recieved no
meaningful responses).

III. Non-Technical Problems with Study

The overall tone of the report was rather unfortunate. For example:

We posted notices on various Linux or Apache newsgroups and received no
relevant responses

This implies that they made multiple requests for help ("notices") and that
the Linux community, in violation of its
reputation for excellence in technical support, dropped the ball. In
actuality, exhaustive searching of the DejaNews
archives by a team of Linux users has identified one single message as a
possible candidate for these "notices". It was
cross-posted under a different title between a Linux newsgroup and the Apache
newsgroup, and recieved a reply which
basically requested more information (albeit in a rather impatient manner).
If the requested information was ever
supplied, it was not ever posted to the newsgroup where other members of the
Linux community could add their own
comments.

The report is full of similar statements which could be interpreted as
showing partiality or being inappropriate. Given
the fact that this study was funded by Microsoft Corporation, Mindcraft
should have taken special care to protect their
reputation by appearing as impartial as possible.

IV. Conclusions

We welcome the opportunity to work with Mindcraft to resolve these glaring
issues and produce a quality analysis. The
Linux community is extremely interested in a fair, impartial analysis of the
performance of Linux as compared with
competitors. Such analysis has been very valuable in the past, and promises
to be equally valuable to the technical
advancement of Linux in the future.

Unfortunately, given the severe technical shortcomings of this report, we
must respectfully request that Mindcraft
withdraw it until such issues are resolved. A report which is perceived to be
biased is not in the best interests of either
Mindcraft or its client, and may in fact be embarrasing to both.

Eric Lee Green
Last modified: Fri Apr 16 12:52:04 EDT 1999

--
※ 来源:·BBS 水木清华站 bbs.net.tsinghua.edu.cn·[FROM: 202.118.66.88]