Whenever you read a PC review or a component review, benchmark results typically accompany it. Such results are most often in the form of numbers, such as a score or a frames-per-second total. Sometimes they're relative, in that the result is posted as a multiple of some reference system or number.
Because benchmark scores mostly appear in product reviews, that's how people frequently view them--simply as a way of comparing products. But benchmarking is useful for more than just figuring out what graphics card to buy.
- Benchmark software can help to stress-test new systems (particularly important for PCs you assemble yourself).
- Benchmarks can test for sudden performance issues. If your system has apparently slowed down in some way, you can use before-and-after benchmarks to confirm your suspicions.
- You can run some benchmarks to check whether certain tweaks you've implemented on a system actually speed things up or not.
There's more to benchmarking than simply installing the benchmark software, firing it up, and running it at whatever default settings exist. I'll walk through the reasons you may want to benchmark in more detail, and then I'll discuss how to properly run a benchmark.
Before diving into the whys and wherefores of benchmarking, let's start with my Golden Rule.
My Golden Rule of Benchmarking--
By the way, there is a corollary to My Rule: If your PC runs games very well, it will run just about anything well.
You'll find some narrow exceptions to that rule. For example, if you want to use an architectural CAD application, a system that runs games smoothly will likely do fairly well with that CAD app--but it would probably perform even better with a professional graphics card. At the same time, if you use a pro graphics card, game performance will likely decrease a bit.
Also, it's true that a great gaming system will run office applications very well, but if you're using office apps exclusively, you probably don't need a high-performance graphics card--and certainly not two of them.
Finally, keep in mind a general exception to Loyd's Golden Rule of Benchmarking: If you're a PC-performance geek, you'll be driven to dissect the performance of every aspect of your PC.
Most people don't care if the primary hard drive is a little slower than a newer model. PC-performance geeks are obsessed with knowing just how much slower their current drives are in comparison with new models. It's just the way their brains work--I should know.
Stress-Testing Your System
Whenever I burned in a system this way, I ran both applications simultaneously for 2 to 4 hours.
It's my belief that system burn-in isn't as necessary today as it once was. One of the reasons to burn in a system was to check for early component failure: If all the components survived the burn-in process, they'd likely run for years without issues. Today, components are generally more reliable. If you've built your own system, though, you'll probably want to run a few stress tests to confirm that it's stable at a high performance level before you get started with any serious gaming or other taxing tasks.
Benchmarks for Troubleshooting
If you want to use benchmarks as problem-solving tools, first you need to run some tests while your system is healthy, so that you have a baseline for comparison.
Remember the Golden Rule of Benchmarking--use tests that are applicable to what you want to do with your system. For example, you may find that altering some memory timings could have a huge impact in a synthetic memory benchmark, but video-rendering time remains the same.
Now that you've looked at the whys of benchmarking, it's time to dive into how to run a benchmark effectively.
How to Benchmark
Remember a few rules of thumb:
- Prep for benchmarking (including cleaning out the system).
- Let the system settle down for a bit.
- Understand the benchmark.
- Check for repeatable results.
- Read the results carefully.
- Don't draw conclusions that are too broad.
- Document, document, document!
Prep for BenchmarkingBefore you fire up a benchmarking app and start running performance tests, you need to prepare the PC. In a perfect world, you want repeatable results, but the architecture of modern operating systems sometimes conflicts with that laudable goal.
Ideally, you want a clean system for running performance tests. But Windows runs a lot of stuff in the background. You should leave normal Windows services running; usually they don't have a major impact on benchmark results (unless something like a scheduled defrag or backup is occurring). However, third-party software often installs services and tray applications that use up CPU cycles, ping the network, or occasionally write to storage.
You can review your machine's background items by running msconfig.exe: Open the Start menu, type Run, and then type msconfig in the Run box.
In the example shown here, I'd uncheck Microsoft Office, Adobe Updater, all the Logitech crud, the Acronis backup helper apps, and other similar stuff. If this were a new PC on which I'd want to run baseline performance tests, I'd probably uninstall any antivirus or security programs as well (remembering to reinstall them when the machine was back in actual use).
One other major headache is making sure that all needed updates--including Windows patches--are installed. After you install the latest Windows fixes, it's best to disable the Windows Update service. (You'll find the control panel link to change Windows Update settings under the 'System and Security' heading in Windows 7.)
Make sure all your drivers are up-to-date as well--particularly the driver for your graphics card, but also the audio driver.
Once you have a clean, updated system, you're almost ready to start.
Let the System Settle
Understand the Benchmark
Take particular care if you're running game benchmarks. Some games will reset their detail settings if you install a new graphics card--or even install an updated driver. If you run the test with different detail settings, it's impossible to get a valid comparison.
Knowing the ins and outs of your performance tests is vital to ensuring repeatable results. Which brings us to our next topic...
Check for Repeatable Results
Or, perhaps the benchmark settings have changed. Altering default settings can bite you if you're not careful. For example, when I run the Unigine Heaven benchmark, I change the default settings, but the test resets itself every time it runs. So I have to take special care to ensure that I reenable the settings.
Benchmarks that allow you to create batch files with custom command-line settings are terrific, because you can always be sure that the settings are what you want, even if the benchmark resets itself upon exit.
The flip side of the coin is that if you consistently get results within 3 percent of one another, that's good enough. People tend to agonize over a sub-1 percent difference in benchmark results. Even if that sub-1 percent number is repeatable, it's generally irrelevant in terms of real-world performance.
Remember, if you want repeatable results, make sure that all things are equal, except the one thing you're changing. For instance, don't test different hard drives in different systems--always use the same system for testing the same type of component. That gives you a frame of reference for your results. If you're changing multiple items of hardware at the same time, it's hard to figure out exactly which one was responsible for any change in performance.
Read the Results Carefully
For example, here's a results presentation from a popular game benchmark, FRAPS:
Average FPS: 39
Minimum FPS: 13
Maximum FPS: 79
It's easy to look at this result and write down "79." Whoa, that new graphics card really is fast. But if you've been recording the average frame rate, such a quick glance will mean you've just skewed the results unfairly--and completely unintentionally. So read the results with care.
Don't Draw Overly Broad Conclusions
Let's say you run a popular CPU benchmark--something like Prime95--and you find out that your machine's score is relatively low compared with those of other processors. This depresses you somehow, because your system will now be forever inadequate in all applications.
Actually, that may not be true. For example, your PC may run some games just fine. Or, if you have ample DRAM installed, your system's performance in photo-editing applications will be pretty good.
It's easy to draw a sweeping conclusion from a couple of tests. Resist the urge and remember: Run the benchmarks that matter for your own system usage pattern.
Document, Document, Document!
You did record your earlier results, right?
I keep master spreadsheets of most of my performance tests, usually by component type. I can fire up my graphics-card spreadsheet and look at results going back several years. Such information is really useful, because we want to believe that newer hardware runs applications faster, but that isn't always true. Unless you document past results, any actual gain is hard to determine.
Go Forth and Benchmark
Now that you understand some of the basics of performance testing, download a few benchmarks and give it a try. Whether you want to compare different components, troubleshoot a performance issue, or see if your system tweaks have had any effect, benchmarking will help you better understand your PC and how it performs.