Pickup truck test - How we configure vehicles

We're frequently asked about how we decide to equip the test vehicles we buy. Since we buy every vehicle we test, we have more control over trim level and equipment than car magazines and Web sites who borrow vehicles from a manufacturer. These press fleets aren't limitless. In many cases, their size has dwindled in recent years as manufacturers look to reduce costs, making it even harder for the media to set up apples-to-apples comparisons. Since car companies want to do well in magazine tests, those press vehicles often have every performance-oriented feature available, are high trim line, and sometimes omit weight-adding options.

Choosing the right pickup truck variation to test is a difficult challenge--one that took a lot of consideration on our part. Pickups are unique in that they offer seemingly millions of combinations of body styles, trims, engines, and axle ratios. According to recent sales patterns, we knew that half-ton crew cab 4x4 trucks in a mid-level trim are the popular choice for most consumers, so we focused on those for our recent test.

Beyond those considerations, when we buy a test vehicle, choosing its equipment requires balancing several goals. The process is complicated since manufacturers don't make "ease of magazine comparison test construction" part of a vehicle's design brief, nor do dealers necessarily stock similar truck packages from brand to brand. In general, we want to test a representative vehicle that is comparable to other vehicles in the test group (and previously tested peer vehicles). We also typically test the version--powertrain and trim level--that most regular consumers will buy. And, even though price is not a factor in our scoring, it's nice to keep the as-tested prices in the same ballpark.

So, based on those principles, we tested a Silverado 2LT with the 5.3-liter V8 and a Tundra SR5 with its 5.7-liter V8. These are the most popular powertrains for the 4x4 crew cab trucks we tested, according to manufacturers' sales data. But given the Tundra's horsepower advantage, we considered testing a Tundra with the smaller 4.7-liter V8 or, on the flip side, a Silverado with the 6.0-liter V8.

The problem is: maintaining parity with the smaller engine in the Tundra would then suggest opting for less-powerful V8 engines in the other trucks. The result would mean a 4.7-liter Dodge, a 4.8-liter Chevy, and a 4.6-liter Ford. That test would be evenly matched in displacement, but given that those engines aren't really bought by most consumers in the crew cab 4x4 trucks we were testing, it wouldn't be very useful.

So what about the 6.0-liter Chevy? This up-level powertrain would have improved acceleration times, but revised 2008 EPA fuel economy estimates show the K1500 4x4 6.0-liter gives up two mpg overall to the 5.3-liter.

Pickup trucks are unique in that you can typically select the rear axle ratio. Again, like most buyers, we stuck with the standard axle ratios. Manufacturers fit these because they are a good all-around compromise of fuel economy and towing capacity. We could have opted for a different axle ratio in the Silverado and the others that would have improved towing capacity and acceleration--but that would have adversely affected fuel economy.

Indeed, when you read advertisements touting the maximum towing capacity claims for most trucks, they are for an optimized combination of body type, drivetrain, and axle ratio--regardless of if anyone buys them that way or if you'd find that configuration at your local dealer. In contrast, the 5.7-liter Tundra Double Cab 4x4 comes in only one configuration, with a relatively aggressive rear axle ratio that helps provide a high standard towing capacity of 10,300 pounds.

Even though the axle ratio subtracts from overall fuel economy, the Tundra boasts the best fuel economy (by a small margin) and the best acceleration (by a big margin) of this group--an impressive feat. GM has done a lot of work to optimize their 5.3-liter pushrod engine, even adding a system that shuts down four cylinders in some conditions, but it can't quite match the combination of the Tundra's overhead-cam engine and six-speed automatic. (For now, the Silverado 1500 has only a four-speed automatic.)

Of course, many choices beyond powertrain and driveline decisions can affect the scoring. When we went to buy a Tundra SR5 with desired equipment (like a power seat), we found that Toyota's limited build configurations meant we couldn't get a Tundra without the TRD suspension package. Given our druthers, we would have avoided the serious off-roading package, as we did on the other trucks. With the Silverado, for example, we bought the Chevy's Z85 towing suspension instead of the stiffer Z71 off-road set-up. That Z85 suspension gave our Silverado a better ride than the Tundra.

To sum up, keep in mind that our Ratings reward well-rounded vehicles. Clearly, we use different scoring criteria for pickup trucks than for other vehicles since pickup truck buyers care about things like payload and towing capacity. But there's more than that; we look at trucks from the point of view of private owners who use the truck to commute during the week and then haul mulch or tow a boat, horse, or other trailer on weekends. Many other factors like fuel economy, acceleration, interior ergonomics, and fit and finish all play a role in daily use. (This is reflected in our top-rated truck, the Chevrolet Avalanche, a truck that tows less and is slower than the Tundra.) In other words, while towing capacity matters, it's certainly not the end-all of the scoring.

To clarify how we set up the pickup truck test (added 8/13):

1) This test was not a comparison of maximum towing capabilities for each truck platform. Towing capacity is just one of many test findings discussed in the report; we perform more than 50 tests on each vehicle. Remember that the Chevrolet Avalanche, which is slower and tows less than the Tundra, handily outscores the Toyota.

2) Even if we picked a Silverado that towed more, either via a different axle ratio or the 6.0-liter engine, it is no guarantee that it would result in a higher Overall Score. We weigh fuel economy more than towing capacity, and either change in the truck's equipment would adversely affect the fuel economy score.

3) Why weigh fuel economy more than towing capacity? Because you have to have gas in the truck every day, but most owners only tow occasionally. At 14-15 mpg overall, another 1-2 mpg matters a lot. (Both EPA and other comparison tests show that the 6.0-liter Silverado gets less economy than the 5.3-liter or the Tundra.) Also, even the 7500-pound towing capacity of the tested Silverado is plenty for most owners.

4) Sure, you can custom-order a truck that is optimized for towing. But we spoke with truck engineers from each of the three domestic competitors before purchasing the trucks. Their insights helped us determine the most representative trucks to test. Note: the take rate for the 6.0-liter in the Silverado 1500 Crew Cab 4x4  is all of 9 percent.

5) The Tundra wasn't optimized for this test because it was the only truck in the test with an off-road suspension. Saddled with the TRD package, the Tundra had a stiff ride that hurt its final score.

6) Our subscriber data, based on millions of vehicles over the years, has shown consistently high reliability for Toyota vehicles. By contrast, even though the first-year reliability of the Tahoe (which shares the Silverado's platform) has been good, year after year the Silverado has been less reliable than the Tahoe.  Because of the Silverado's past below-average performance, we are unable to predict its future reliability, whereas the strong historical performance for both Toyota and Tundra enable us to predict its future performance.

--Tom Mutchler

Read the complete full-sized pickup truck test.
Watch the pickup truck videos.
From the blog:"Big trucks = big bucks"

When commenting, please indicate if you have read the entire story, either online or in print.

Comments

[derf]

A few thoughts for you to consider:

If you're going to compare standard offerings at a certain trim level between all manufacturers, that is fine. But say so up front, and DON'T focus on towing. If the review had been well rounded and dipped into all features of the trucks an even amount, then most people would not be crying foul.

Problem is, you focused on towing and fuel economy, which sound funny anyways. Are these two things not opposites? Most consumers who want big towing capacity really don't care as much about fuel economy, even with the high prices. If you think I'm wrong, visit central CA or the mid-west and ask around.

If you're going to write an article which focuses on towing, that's fine. But compare apples to apples, irregardless of trim level. Because in a towing test, your goal should be to pit each manufacturers best offerings against each other and make notes regarding which towing options are going to cost consumers extra.

But again, that is not what you did which is why you are seeing this reaction.

-Derf

[Kent Young]

I appreciate the difficulty in getting the vehicles you want for a test, however, you stated that you chose those configurations. If you couldn't get what you wanted, why did you not just say this?

Also, you said that the Toyota has very good reliability, but the Chevy is undetermined. They are both new trucks, so how can one be rated and the other not? Don't you remember the broken camshaft problems Toyota has/had?

Lastly, if you had poorly matched trucks, wouldn't you want to discuss this in your article?

In the end, this comparison, in addition to the child seat debacle, is tanking your credibility. You should keep this in mind for your future articles...

[Brian]

How can you rate the Tundra as "very good" in reliability, and the Silverado as "new"? The Tundra has been out for a much shorter period of time than the Silverado. How then, can the Tundra have a rating for reliability but not the Silverado? In addition, the Tundra has had a problem of having the camshafts snap on it's 5.7L engine. This is "very good" reliability? This is also why you can recommend the Tundra but not the Avalanche, which scored higher. You say the Avalanche is "new" as well, but has been out over a year longer than the Tundra.

[Phillip Quinn]

I'm sorry, but that's a b.s. reason not to test truly comparable vehicles. I've been following the auto world, and personally believe that Consumer Reports is the most biased of the reviewers. Keep in mind, I am not that biased. Every make has its car that is better than the competition, and I will give that Japanese small cars and sedans are generally better. However, when you see a test like this that favors the import, especially when the Chevy can be ordered with a truly equivalent setup, and still be in the same price range, there is some favoritism apparent. This is why I won't use CR as a resource when it comes to my next vehicle purchase. Whoever approves tests and articles like this is not giving every competitor a fair shake, and is defending themselves with a weak statement, knowing that they have excessively loyal readers who will believe whatever they say. All I can say to those readers is this: read more than one publication if you want to make a truly intelligent decision, be sure to test drive all the vehicles in the class you are looking at, and DO YOUR RESEARCH!!

[Don]

The statement that "maintaining parity with the smaller engine in the Tundra would then suggest opting for less-powerful V8 engines in the other trucks" is a weak arguement. The Toyota has two engine choices - you chose the BIGGEST engine choice. Parity would have been to pick the BIGGEST engine choice for each of the tested vehicles. When consumers compare vehicles they they attempt to compare apples-to-apples. If I was in the market for a light duty tow vehicle I would look for the optimal engine and gearing paring. And that wouldn't have been the 5.3 Chevy. You have done yourselves a disservice by publishing this "comparison".

[bosco]

Having read your publication for several years, I simply cannot understand your choice of trim levels for the truck comparison. Pick-up trucks are work trucks, period. True some people buy them for other reasons, but most buy b/c they need the payload capacity or towing capacity. If you are going to compare towing capacity, you have to compare each manufactures best towing option. You did not. I suggest you stick to toasters.

[Dave B.]

You cite that the 5.3L is the most popular choice of engine configuration for the Silverado. But have you stopped to think that most people don't tow. And that the people who do tow tend to select the more powerful engine option.

By your own admission you say that yiou picked the 5.7L which is a upgrade over the standard engine on the Tundra, yet you choose not to select that option on the Silverado.

Your justifications leaves a lot to be desired.

If I am pulling a 28ft boat I am not going to settle for the most popular engine choice. I am going to select the most powerful engine and the lowest gear set I can select to get the job done. And maybe by having said 6.0L V8 while towing my mileage might not be as bad as the engine does not have to work as hard.

Behold the power of common sense, something which you guys have been missing for quite a while.

Apples to apples people.. not apples to oranges to bananas to grapes.

[mike w]

This evaluation and comarison seems ill conceived. Nice to know that you are so biased in favor of Toyota that you set up each of the competitors to fail. You should have matched comparable products! The strongest point of the competitors is they can offer many different setups to suit customers needs and that does not mean they would pick ill suited vehicles to compare against. You are obviously over your heads and need some tunning up. You should stick to vacuums!!

[Jim C]

I wish that they would have used the 6.0L in the Chevy. The Toyota still out performs the 6.0L. I think a fair comparison would have been to buy one of each truck, the companys smallest V8 and their biggest V8, then no one would be able to complain about the results.

[CV]

Not that I think you did the right thing, but you make a point when you say you want to test the most often purchased combination for each truck. I think it should have been apples to apples. That would have been the FAIR way to do it. Also, I don't see an explanation for the reliability ratings issue.

[CJ]

Oh Boo Hoo, listen to all the Chevy owners mad that the Tundra has trumped them, yet again, in another comparison test. Except this time CR has slipped up and chose the wrong engine (for the right reason). Oh well, this will just give them an excuse to do a heads up comparo with the 6.0L Chevy and watch the Chevy super-fans get embarrassed again. VortecMax, DuraMax, doesn't matter, as published in the test Tundra outpulls them all..

[Reader]

Reliable Tundra?

Broken camshafts by the score on this truck....

Hmmmmmm. CR, you can do better.

[Jim D]

This explanation is unconvincing. It sounds like a rationalization for bad decision making. Over the years, I've become less and less enthralled with and more and more skeptical of Consumer Reports. My family has been subscribing to your magazine for 30+ years, but I find myself not relying on many of your product evaluations because your feature/function weighting often doesn't match my needs. Furthermore, unrelated to this vehicle test, your persistent big government, anti-individual agenda is, in my opinion, anti-consumer.

[Tundra Headquarters]

The amazing thing about reading your response to the complaints raised by some journalists about the test being "unfair" is that your methodology makes perfect sense. Why compare a 6.0L Silverado to anything when fewer than 10% of buyers opt for that engine? To say that your test should have included the 6.0L Chevy, even though few people will every buy one, in the interest of "fairness" is ridiculous. That logic can be extended to justify comparing trucks only manufactured in the US (thus excluding the Dodge), or comparing trucks only of the same color. While these test conditions would have been more "apples to apples", they would have been silly. The fact is that at some point, the vehicles are inherently un-equal. Choosing NOT to test the 6.0L Chevy is an example of making a choice that will raise the overall usefulness of your comparison to 90% of consumers.

Also, to anyone who argues about your emphasis on towing AND fuel economy, we submit that they're probably not a truck owner.

[Ryan S]

I agree C.V. The Tundra completly smokes the Chevy with any engine. It can out tow both the 5.3 and 6.0 litre engines. I don't know what everyone is talking about with these camshaft problems, myself I have a Tundra and I drive it to the max day in and day out. It dosen't lose any power like my last Chevy did. I get 22 mpg with the 5.7 and I'm usally drive over 140 km/hr. The CHEVY has a hard keeping that speed let alone getting good mpg. Keep whinning Chevy lovers cause I used to be just like you, tell I borke down and bought a Tundra. Hopefully well see them change things to keep up with the Tundra. If anyone watchs the Craftman Truck series then you'll know which Truck is the most reliable with the most horsepower. I love seeing a Chevy on the highway cause then I know he'll be easy to pass.

[Ben French]

The 5.7L Toyota can't truly match the Chevrolet Duramax diesel V8, even when you consider the fact that the Tundra beat the Duramax in this comparison when towing a trailer. The Tundra has 381 hp and 401 lb.-ft. of torque, while the Duramax has 365 hp and an amazing 650 lb.-ft. of torque. Comparing the Tundra to the big boys would be like comparing a Buggati Veyron to a Kenworth T2000. Even though the Kenworh has a lot more torque (2050 compared to 900+ for the Veyron), the Veyron will easily beat the Kenworth to the end of the quarter mile because it is lighter, and has more horsepower. However, if you attach a 40,000 pound trailer that the Kenworth is desgined to pull, the Veyron won't even get off the starting line. You can say the same with this comparison. The Tundra is not hauling a trailer that can even compete with the towing capacities of the big boys (10,300 versus 13,600 and higher, 9800 pounds for the Silverado would be true, if it was equipped with a V8 engine, but the one used for this comparison was a diesel). Next time, if CR insists on comparing very dissimilar trucks, they should use 13,000 pound trailers.

[John Hutchins]

I had exclusively owned Toyotas and a few Mazdas since 1974. I went out on a limb in 05 and bought a Dodge RT Magnum and then a 05 QC 4X4 Hemi Ram. I have been pleasantly surprised, 110,000 troublefree miles on the Magnum and 75,000 troublefree miles on the Ram. Just replaced the tires at 75k on the Ram. I will still look at Toyotas, tried outa Tundra, like my Ram better!

[gary]

i don't know about other places but down south the chevy vehicles perform a lot better than posted in the magazines and the forums.I don't pay any attention to the comparo's, I go with what I know.CHEVY RULES-alway's has alway's will.