The "Accuracy" score on club tests

[Trip]

I did a search for this and couldn't find anything relevant, by all means point me to previous answers if I've missed them.

I'm curious to know how the accuracy score is calculated on MGS club tests. I've seen it described as using the Strokes Gained metric. I know how SG is calculated and it requires a lot of information from a set of players on (at least) a hole. I can't see how this can be done when you're not actually playing a hole but hitting (say) three irons from a set with a LM. 

[GolfSpy_APH]

Accuracy

For most golfers, distance is the priority. However, we firmly believe accuracy’s importance needs to be emphasized. The purpose of our accuracy category is to help you identify the golf clubs that fly straight, helping you avoid penalty strokes.

Straight Shot Percentage – Think of this as an adjusted Fairway Percentage. However, shots aren’t penalized for missing the fairway simply because they were a few yards longer. The target area widens the further a shot is hit.

Playable Shot Percentage – Obviously, a playable shot is ideal. We identify a playable shot as one that is in or within 10 yards of the edge of a 35-yard-wide fairway.

 

Q: How is the “Most Accurate Driver” determined?

A: The metrics that determine the Most Forgiving Driver are straight shot percentage, playable shot percentage and Strokes Gained (see Most Wanted Scoring section above for more detail).

 

https://mygolfspy.com/buyers-guides/drivers/best-golf-drivers-2023/

 

https://mygolfspy.com/buyers-guides/all/how-we-test-most-wanted/

 

ACCURACY

Accuracy evaluates if a player’s shots with each club are above or below what’s expected based on that player’s skill level.

 

https://mygolfspy.com/buyers-guides/irons/best-players-distance-irons-2023/

 

hope that helps a bit!

[Trip]

Thanks for the prompt reply.

Quote

Straight Shot Percentage – Think of this as an adjusted Fairway Percentage. However, shots aren’t penalized for missing the fairway simply because they were a few yards longer. The target area widens the further a shot is hit.

Playable Shot Percentage – Obviously, a playable shot is ideal. We identify a playable shot as one that is in or within 10 yards of the edge of a 35-yard-wide fairway.

 

This doesn't seem to bear any relation to Strokes Gained. It looks like an attempt to measure something similar using a completely different calculation. I think it's slightly misleading to say you're using Strokes Gained. 

Quote

Accuracy evaluates if a player’s shots with each club are above or below what’s expected based on that player’s skill level.

This is very imprecise language. How do you determine what is "expected" and whether the shot is "above" or "below" that using data from the LM?

 

[GolfSpy_APH]

12 minutes ago, Trip said:

Thanks for the prompt reply.

This doesn't seem to bear any relation to Strokes Gained. It looks like an attempt to measure something similar using a completely different calculation. I think it's slightly misleading to say you're using Strokes Gained. 

This is very imprecise language. How do you determine what is "expected" and whether the shot is "above" or "below" that using data from the LM?

 

I'm simply copying the language from the main site. All of that is easily found in any of the most wanted articles. 

Also I am not involved in MW Testing so truly can't speak to the ins and outs. 

Best thing that you could do if you want more concrete info or guidance (other than what is found in the articles) is to reach out to Phillip who runs the most wanted testing. 

Also keep in mind there are different measurements for accuracy when it comes to drivers vs irons vs wedges vs putters.

[cnosil]

1 hour ago, Trip said:

Thanks for the prompt reply.

This doesn't seem to bear any relation to Strokes Gained. It looks like an attempt to measure something similar using a completely different calculation. I think it's slightly misleading to say you're using Strokes Gained. 

This is very imprecise language. How do you determine what is "expected" and whether the shot is "above" or "below" that using data from the LM?

 

For several of the clubs being tested, the test is conducted on a simulated hole.  There is a fairway, trees, and rough.   

for irons and wedges, we are hitting approaches into a green.   
 

While I don’t know the exact calculations MGS uses, Expectations are driven by mark Brodie’s stroke gained metrics.  There is a huge population of data associated with stroke gained that can be used for comparisons.  It works for most wanted testIng just like it would it you hit a tee shot, irons shot, or wedge shot on a golf course.  
 

 

[Dweed]

Oh Wow!  You really have to be into the science and the math involved in establishing accurate testing parameters. That's why I'm not an engineer nor physics researcher.  I trust the base recommendations of MGS folks when they test. Just like I did in selecting my financial adviser and my tax adviser. Do research within my understanding and select whom I "feel" is best for the job.  Can I afford the recommendations that I'm being given. One size certainly doesn't fit everyone. Same here. Thanks MGS for your efforts and getting logical data posted for us.

[Trip]

10 hours ago, Dweed said:

Oh Wow!  You really have to be into the science and the math involved in establishing accurate testing parameters. That's why I'm not an engineer nor physics researcher.  I trust the base recommendations of MGS folks when they test. Just like I did in selecting my financial adviser and my tax adviser. Do research within my understanding and select whom I "feel" is best for the job.  Can I afford the recommendations that I'm being given. One size certainly doesn't fit everyone. Same here. Thanks MGS for your efforts and getting logical data posted for us.

I have no doubt that MGS knows more about golf equipment than I do. I've been reading the tests for years and find them very useful.

But I am a physicist, I understand numbers, and I know how the Strokes Gained calculation works. It requires a corpus of data from a group of players who play the hole to completion. FSX can record shot data but does not record the shot location or marry it to scoring outcomes in the way required. So to have that data MSG would need to gather it themselves. Maybe they have done this. Maybe there is some other way I haven't thought of.

Since asking the question I have seen a description on the site that says they use a "proprietary algorithm" to calculate Strokes Gained. That sounds like they have come up with their own mechanism for this calculation. In which case, I think using the term Strokes Gained is a bit misleading because people - including one respondent to my question - will naturally assume they are using Mark Broadie's algorithm which relies on PGA Shotlink data. Those data are available but they are not relevant to a simulator hole. 

[Trip]

On 6/23/2023 at 2:56 PM, cnosil said:

While I don’t know the exact calculations MGS uses, Expectations are driven by mark Brodie’s stroke gained metrics.  There is a huge population of data associated with stroke gained that can be used for comparisons.  

I think this is unlikely because Mark Broadie's algorithm relies on PGA data. How could those data predict expectations of an 18 handicapper?

You could certainly compare the shots of an 18 handicapper with those of PGA professionals but I don't think you can set expectations that way.

[cnosil]

10 hours ago, Trip said:

I think this is unlikely because Mark Broadie's algorithm relies on PGA data. How could those data predict expectations of an 18 handicapper?

You could certainly compare the shots of an 18 handicapper with those of PGA professionals but I don't think you can set expectations that way.

The strokes gained metrics could be based on how the testers performed; handicaps are known, and how the clubs compare against each other.   ShotScope and Arccos have data for all handicap ranges.  You can set expectations however you want;  it is just a measurement.  An 18 handicapper can hit the ball 300 yards and be in the fairway every time;  there is more to handicap that a single shot.     Here is an article that shows a way you can calculate strokes gained for an individual shot:  https://www.golfstatlab.com/stats/total+strokes+gained

 

[Tony Covey MGS]

Let me see if I can clear this up:

For driver, fairway woods, and utilities, the accuracy score is derived from the Playable shot percentage and Straight Shot percentage. I believe those are explained in the relevant posts - although we did have the strokes gained reference in the post longer than we should have. We used SG in the past but moved away from it.

Anticipating the follow-up questions - we got rid of strokes gained because, after looking at several years of data, we concluded that it just doesn't work as a primary metric in a large driver test. While we worked with Lou Stagner to enhance SG with additional penalties as the ball travels farther offline, in practice, there is a strong correlation with fairway percentage almost without regard for distance.

With respect to Trip's comment there are a couple of points I should make. First, we don't use the PGA Tour tables. Instead, we use tables developed for scratch golfers. As it relates to higher handicap golfers - it's not a big deal. Effectively SG just sets a baseline for values, so while higher handicap golfers will produce lower SG scores, in our implementation, the best-performing clubs still produce higher scores. In that respect, the scale is almost arbitrary. It gives us a reasonable starting point. We've gone to great lengths in our methodology to neutralize the SG advantage of better/longer players. 

I'm not going to get into the details, but the simple explanation is that Strokes Gained is typically used to compare performance difference between golfers. We've adapted it to compare differences between clubs for the individual golfer.

While I think SG is incredibly useful in a driver demo/fitting scenario, in driver and fairway wood tests, you need additional insights.

For our target-based tests (hybrids, irons, and wedges, we use Strokes Gained as our accuracy metric. In this scenario, there is a correlation with greens hit, however, SG is more effective because it also considers pin proximity and lie condition. We've looked at other metrics in isolation (pin proximity, greens hit) and they don't work nearly as well as SG in this scenario.