Return to LivecamServer.com
John Sokol's Home Page - Inventor of AfterBurner web server

Afterburner benchmarks

The setup for this series of tests is as follows

• Server. The server machine is a 450MHz Pentium II. It has 1GB of memory in it (we maxxed it out for a different test). It has four 100Mbps NICs (Intel EtherExpress PRO/100+).
• Clients. Four different client machines, each with one 100Mbps NIC:
  • A 350MHz K6-2.
  • A 200MHz Pentium Pro.
  • A 166MHz Pentium.
  • A 100MHz Pentium.
• Network. The four clients are connected to the server in a star configuration, using crossover UTP cables. Each client uses a different IP network number.
• OS. All machines are running FreeBSD, either 2.2.7 or 3.0. maxusers has been raised to 256, otherwise the kernels are pretty stock.
• Server software. We compare four web server programs:
  • Apache 1.3.3, the current version of the world's most popular web server.
  • thttpd 2.04, a small and fast server I wrote a few years ago.
  • Zeus 3.1.7, a high performance commercial server.
  • Afterburner, the new defunct DVBS's product, now is opensource or SourceForge, which is similar in structure to thttpd and Zeus but runs even faster.
• Client software. The requests are generated by http_load.

• Small Files
• Pseudo-SPEC
• Large Files
• Large Files Throttled

Small Files

In this test we use 1000 512 bytes files, and we look at the maximum rate each program can serve them. The idea is to get a look at the per-connection overhead.

	Apache	thttpd	Zeus	Afterburner
max ops/sec	890	2030	3560	5200
max bits/sec	3,600,000	8,300,000	14,600,000	21,300,000

Pseudo-SPEC

This test uses a data set similar to the standard SPECweb96 set. It's a mix of file sizes from 102 bytes to 900KB. The average works out to about 14KB. The main difference between real SPECweb96 and this pseudo-SPEC load is that in real SPEC, the number of files is required to grow with bandwidth. This is meant to model a large ISP's usage. For the bandwidths supportable by modern CPUs, the data set is required to be a couple of gigabytes. Thus SPECweb96 has become mainly a test of how much memory you can cram into a machine and how fast a disk subsystem you can build, and the performance of the web server software itself is secondary. In this pseudo-SPEC, on the other hand, the data set size remains constant at about 5MB, so we can compare server programs more easily.

The linear behavior of thttpd is interesting, especially since it didn't show up in the small-files test. Its slope indicates a latency of 1/7 second per request. A previous benchmark showing a similar latency of 1/5 second per request resulted in a bug fix that improved performance tremendously. Since Afterburner and thttpd are similar in structure, it's probably worth debugging this just to make sure that Afterburner won't suffer from this problem under some other circumstances.

	Apache	thttpd	Zeus	Afterburner
max ops/sec	660	1140	1260	1290
max bits/sec	79,000,000	137,000,000	151,000,000	155,000,000

Large Files

In this test we use ten 5MB files. The idea is to see just how much raw bandwidth each program can pump out. To do this we just start up fetches on all four client machines, and measure the bandwidth using "netstat -p tcp". The maximum possible bandwidth with this network is 400,000,000 bits/sec; how close each server comes to this mark shows how efficient it is at bit-pushing. Well, almost - there's also the issue of whether the four clients have the CPU power to suck down their full 100,000,000 bits/sec. It looks like the 350MHz and 200MHz machines do, the 166MHz almost does, and the 100MHz can do about half its channel. So the actual maximum bandwidth with this setup is more like 340Mbps.

	Apache	thttpd	Zeus	Afterburner
bits/sec	297,000,000	250,000,000	253,000,000	321,000,000

Large Files Throttled

In this test we again use ten 5MB files, however each connection is throttled to the speed of a 33.6Kbps modem. This is meant to model a site used for large downloads. We look at the maximum number of simultaneous connections each program can support.

This is a difficult test for most server programs. The ones like Apache, which use a separate process for each connection, crash pretty quickly due to too many processes. Thttpd and Zeus avoid that problem because they handle many connections in a single process; but they're limited by the maximum number of file handles in a single process. Afterburner, on the other hand, handles many connections in a single process, and uses multiple processes if necessary. Only this last model can handle the many thousands of simultaneous connections needed by a busy download site.

	Apache	thttpd	Zeus	Afterburner
max connections	90	250	1000	>3000

Note: thttpd should have gone up to 1000, like Zeus. This may indicate it has a bug. Also, I've heard that Zeus can be configured to use multiple processes like Afterburner, but I didn't find that setting.