New & Improved ULTIMATE REVIEW System!

[Tony Covey MGS]

Time For An Upgrade

When we created the “ULTIMATE REVIEW SYSTEM!” a year and a half ago we started with a single goal in mind: create the most complete, unbiased, and most importantly, objective review system we possibly could. As you are no doubt aware, the overwhelming majority of golf equipment reviews found online and in print are almost entirely subjective, and contain little or no actual data. While we believe aspects of those types of reviews have value, we also believe the greatest value comes in being able to backup perceptions with actual data. As we've proven with past incarnations of our review process, perceptions (the stuff subjective reviews are made of) – especially those related to distance and accuracy – often bear little correlation to the reality proven by the numbers.

 

We view the original incarnation of the “ULTIMATE REVIEW SYSTEM” as a great start. Since the time that process launched we've learned a great deal about how golf clubs perform, how golfers perceive performance, and how to sum up the correlations and contradictions into an informative and hopefully interesting package. Almost from the beginning we've made minor tweaks to the system. As we found things that didn't work (SpecCheck for example) we took them out. When we found things we're missing, we added them in. Over time those modifications have added complexity to our scoring equations and the process itself. We realized we'd reached a point where two of our core goals: transparency, and simplicity were falling by the wayside.

 

Those problems, along with our enhanced knowledge, and our strong desire to improve on the overall detail and quality of our reviews made it abundantly clear it was time to release version 2.0 of the “ULTIMATE REVIEW SYSTEM!”

 

READ ABOUT THE UPGRADE AND CHANGES

[Matt Saternus]

Sounds like a winner. I'm definitely eager to see greater disparity between the "good clubs" and the "bad clubs" in terms of the final score.

[Tony Covey MGS]

Sounds like a winner. I'm definitely eager to see greater disparity between the "good clubs" and the "bad clubs" in terms of the final score.

 

Thanks Matt. The first driver review using the new system will post tomorrow. It should be more interesting when we get into fairways, hybrids, and especially irons. Based on what I've seen from iron tests so far, our scoring assumptions (thanks to a suggestion from ddelise) are looking pretty solid, and numbers are consistent for short, middle, and long irons. What we hope to identify over time is clubs for which longer irons are measurably more difficult to hit.

 

As much as I'd like to promise a tremendous variance in scores, the reality is most of what we test is going to shake out along fairly similar lines. Our scoring system has always done a good job representing exactly how close performance usually is from one club to the next. What we hope the new system will be better at identifying are the true standouts, and the absolute clunkers.

 

One other cosmetic change is that that we've moved the interactive data into the review itself (instead of having it linked to a separate page). We had to scale it down a bit (size not content), but we expect more readers will take notice of the data, which really is at the core of our review system.

[Moecat]

Are there any plans to possibly redo at least a few of the past reviews to see how some of those clubs would fare in the new review system - and thus see from that standpoint if there would be any increased separation among clubs of the same category?

[Tony Covey MGS]

Are there any plans to possibly redo at least a few of the past reviews to see how some of those clubs would fare in the new review system - and thus see from that standpoint if there would be any increased separation among clubs of the same category?

 

We initially looked at the idea of applying the new formula to the older reviews, but decided against it. When you consider Fairway Woods, Hybrids, and Irons, in order to re-factor, we'd require data we simply don't have (with fairways and hybrids we never tested to a target before, with irons, we only tested to 150, so lots of data is "missing" there as well).

 

We did actually actually re-crunch the numbers on previous driver reviews for the 3 testers who have been a part of every review. We used those as a proof of concept to validate the new system. Overall the majority of the final numbers were similar (within a couple of points either way of the old system). That said, I believe one of the greatest strengths of the new system is the continuity of golfers. With the same 6 guys now taking part in every test, I wasn't comfortable backtracking and basing previous scores on the data from just 3 guys.

 

The one part of all of this that completely blows is that our older reviews don't necessarily translate under the new system. Yeah...we could go back and potentially re-review some of that stuff, but the reality of the golf industry is that the OEMs are always moving forward. There's already more gear being released than we could ever hope to thoroughly review. We simply have to keep pace as best we can and that means spending our time on the 2012 gear as well as some interesting lab articles we have planned.