Is testing really testing?

[PaBr]

Relatively new member here, but pretty obvious that testing golf gear is core business here. A big reason for me being here. 
 

but is this testing really testing? Or better said, to me testing is prior release validating of product to allow feedback to result in product improvement. 

 

without trying to be funny, disrespectful or whatever, what is labelled as testing to me seems a lot like product comparison. The product is a given. one could even argue testing in this set up is part of the OEM marketing mix. 
 

reason for me asking is that I think pre-release testing would be very interesting. I know that supposedly happens with your players but that is not a reflection of the target market for the majority of products. 
 
wouldn’t it be cool to have mgs/ consumer pre-release influence in product rather than after the fact. Possibly all of that already exists and I’m simply not aware of it. 

Edited July 12 by PaBr

[NM01]

Well testing post release is to see if what the marketing says is true. So there is a valid reason to test after a release.

I’ve done testing for this site and it’s yes in some ways a comparison of what one has already to what one is testing. Also a valid reason to test. Is there an improvement or not. But that also ties into the seeing in the marketing materials holds up to real life experience.

What I don’t like about testing and it’s not just what’s done here but on all the forums and one of the reasons I haven’t applied for any in awhile, is that each persons results are just that their results. How one gets along with equipment, whether it’s shaft, clubbed, the combo of the two has no bearing on how it will perform for someone one else. Even if the two parties have same handicap or swing speed. Each of us has our own swings, our own biases, preferences and influences along with feels and sensitivity to change in weight and balance.

Also why I rarely read any of the testing done here. 
 

Overall it’s for the oem to get more exposure to their products in an effort to drive sales

[cnosil]

39 minutes ago, PaBr said:

but is this testing really testing? Or better said, to me testing is prior release validating of product to allow feedback to result in product improvement. 

without trying to be funny, disrespectful or whatever, what is labelled as testing to me seems a lot like product comparison. The product is a given. one could even argue testing in this set up is part of the OEM marketing mix. 
 

There are multiple ways to “test” and the testers choose how and what they do for testing.  When I am testing a product one of the things I do is validate the OEM claims.  Also the OEMs are monitoring or engaging in the testing and they do take the feedback provided by the testers to improve their future products.  Call it testing call it product evaluation it is what is done on all golf forums.  

sure it is part of the OEM marketing.  Feedback from testers has been used on some of the companies website to show real world feedback on their product.   Additionally since people read the reviews there is more exposure and potentially more sales.   
 

I don’t know what you buy, but pretty much every company I have bought a product from requests product feedback….that is the essence of the forum testing.  Get a product, test it how to see how you like it an provide honest feedback.  

[Hacker60521]

“Testing” is an interesting term here. As @PaBr and @RickyBobby_PR point out, testing and test results are more of a group report on their personal experiences. This is not a place for scientific method types of testing. At first, I didn’t place a lot of value in these tests. But as I continued to follow along I learned a couple of things. First, I try to find testers with similar handicaps or profiles to me. I use their outcomes and opinions as indicia of what I might experience. Second, just like in other parts of life, personal opinion will always bring in bias to facts and data. For example, I don’t care how great a mallet putter might be. I like blades. Hah.

 To @cnosil point, I remember how testers were gushing over time they got to spend meeting with Sasho MacKenzie regarding speed training. So if there’s dialog behind the scenes that’s great 

[PaBr]

To be clear, I’m not knocking on any of the post release activities. I think it makes sense, is fun and one the perks of this day and age, it is very public y it sing a forum like this. 
 

but would it not be great if pre-release testing would become similarly public? I sympathise with possible additional complications it would bring regarding secrecy etc. But all stuff that can be worked around I would think. All this to say, how relevant are JT’s, spieth and Wyndham Clark test results for the product success with the consumer market. 
 

possibly there is an entire dark web test setup that I’m not aware off. If so, I’m applying:)

[Golfspy_CG2]

26 minutes ago, PaBr said:

To be clear, I’m not knocking on any of the post release activities. I think it makes sense, is fun and one the perks of this day and age, it is very public y it sing a forum like this. 
 

but would it not be great if pre-release testing would become similarly public? I sympathise with possible additional complications it would bring regarding secrecy etc. But all stuff that can be worked around I would think. All this to say, how relevant are JT’s, spieth and Wyndham Clark test results for the product success with the consumer market. 
 

possibly there is an entire dark web test setup that I’m not aware off. If so, I’m applying:)

I’m quoting your latest post but referencing what you said in your first. Most OEM I am aware of have a new product “tested” by regular golfers from 5 to 20 handicaps at their facilities.  
 

I have been able to witness this in person at both PING and Titleist.   They gather feedback from a rather large group of players like that.  Some are employees who work in accounting or shipping etc and some are local residents who they have trusted to NDA’s to come in and test. 

[cnosil]

38 minutes ago, PaBr said:

but would it not be great if pre-release testing would become similarly public?

sure it would be great but this is what we are given the opportunity to do.    As mentioned OEMs probably do this already and the clubs being tested are probably multiple years out from being released.   The moderators do a great job working with the OEMs and something like you are looking for could become a reality for a company.    
 

I just remembered it does happen.  I was sent a Sacks Parente pre release putter to test because during my fitting with the OEM he thought I would be a good candidate to review the product.  

[Rob Person]

There have been several products here that have been tested pre-publici release. Some were also tested within a month of the release to the public, and some even longer. 

Odyssey offered mgs the opportunity on the new AI-ONE putters more than a month prior to sales went live on their site. Titleist offered a blank Test ball months before it's release as well. 

Other companies offer not yet released training aids, accessory products, etc too

Those are just a few I can name off the top. 

 

 

 

 

 

[GolfSpy_APH]

I think the only "issue" with pre-release testing is that the experience and information cannot be shared in real time with embargos and that. 

It would make for a great experience, but less activity or even acknowledgement from the group. So certainly less value from a community perspective. 

[Indy_Oz]

Just adding my 2c regarding the use of 'test'/ 'testing' here and where and when it applies. Testing covers a broad spectrum of activities.

I'm an engineer in a high tech industry and, for us, testing of a product continues from initial development through the entire lifecycle. At the start it's to determine if the design meets the specifications placed on it, then once you've proven a product does what was intended you test to confirm it can meet reliability and lifecycle demands and is safe for the end user. Once released/ entered service, there is continued testing  to get ahead of any issues consumers may need (hence product recalls or alerts to consumers) and the design can - and is- improved upon, but essentially the through-life testing is to confirm that general use doesn't unearth issues you hadn't thought of or expected (deterioration or wear quicker than experienced during design and development phase).

It's all testing, it's just the purpose and methods that can change somewhat through the product lifecycle. Much the same with golf equipment- after initial design there is testing in controlled environments to ensure it does what you want it to do, but post-release consumer-testing can be important to gather more information as well as help market the product by showing the general public what Joe Bloggs found when testing it against OEM claims.

Behind the scenes there will be changes to products that are not visible in the stores or when purchased. I'd imagine Taylormade took customer feedback and warranty claims and made changes to the way the carbon face was bonded to the club head on the Stealth drivers. Changing bonding agent and method of application or method of attaching the carbon face would be invisible to the consumer, but is a product improvement based on real-life 'testing' and customer feedback.

Edited July 13 by Indy_Oz

[buckpillar]

6 hours ago, Rob Person said:

There have been several products here that have been tested pre-publici release. Some were also tested within a month of the release to the public, and some even longer. 

Odyssey offered mgs the opportunity on the new AI-ONE putters more than a month prior to sales went live on their site. Titleist offered a blank Test ball months before it's release as well. 

Other companies offer not yet released training aids, accessory products, etc too

Those are just a few I can name off the top. 

 

 

 

 

 

I too have been in the same boat as @Rob Person, we completed the Odyssey AI putter test together, as well as the the Titleist white box testing. That being said, I was an unofficial tester of the Bridgestone Mindset golf balls, The were official tested also and if I'm not mistaken there were multiple rounds of testing's on these balls of up to 500 golfers each cycle as long as you were able to answer a couple questions after watching their video. 

I would say that the majority is pre launch and very very few during or after launch

[Mike10487]

The testing that is done at MGS is more of a consumer safety/protection test that anything else. In my opinion of course. As testers we are trying to determine if the product does what it is advertised to do. I recently tested Galway Bay, I and everyone else knew it was a rain suit. My first order of business was to research what GWB claimed the suit would do.  After knowing what the manufacturer claimed, it was up to the "testers" to determine in their opinion, if the product matched up to the claims. In the case of GWB it far exceeded the claims. 

So, to follow up. Yes, it is a test. I want to know if the Hammer driver is the longest driver in golf before I buy it. MGS testing ensures that I am buying a product for the right reason, and it is as advertised.  I look at the testers and find the closest handicap to mine and start with their review first. Yes, a 30 handicap with have a different result than that from a 3 handicap with a club. As long as the test and reviews are honest, it helps people who are on the fence about a purchase and I believe that is the entire precedence behind the MGS testing.

Edited July 13 by Mike10487

[Shapotomous]

The word "review" is used at least as much as "test" in describing what members do with the products.  Maybe that is a better word to focus on in describing what members do with the provided products?  I can see both words being accurate with respect to how i use a product when I have been selected. 

If I am collecting shot data either from a launch monitor or video / hand measuring results for comparison i view that as a "test".  If I am giving my opinion of playability or feel or looks of a product and how it works for me that falls into the "review" category in my mind.  Both can be used to verify or refute a mfg claim.

I think both are valuable and I look for both types of info when reading through the "test reviews" that are done.  As mentioned I look at players with a similar swing and 'cap to mine first and make that a starting point when doing my own search for a new club or ball.  

[Rob Person]

4 minutes ago, Shapotomous said:

The word "review" is used at least as much as "test" in describing what members do with the products.  Maybe that is a better word to focus on in describing what members do with the provided products?  I can see both words being accurate with respect to how i use a product when I have been selected. 

If I am collecting shot data either from a launch monitor or video / hand measuring results for comparison i view that as a "test".  If I am giving my opinion of playability or feel or looks of a product and how it works for me that falls into the "review" category in my mind.  Both can be used to verify or refute a mfg claim.

I think both are valuable and I look for both types of info when reading through the "test reviews" that are done.  As mentioned I look at players with a similar swing and 'cap to mine first and make that a starting point when doing my own search for a new club or ball.  

Well said, and very accurate!

[Erin B]

22 hours ago, Golfspy_CG2 said:

I’m quoting your latest post but referencing what you said in your first. Most OEM I am aware of have a new product “tested” by regular golfers from 5 to 20 handicaps at their facilities.  
 

I have been able to witness this in person at both PING and Titleist.   They gather feedback from a rather large group of players like that.  Some are employees who work in accounting or shipping etc and some are local residents who they have trusted to NDA’s to come in and test. 

I know the equipment I tested was tested. For instance, the Callaway ball. It was tested with a trackman, tested for durability, spin, and distance. I also enlisted another spy to help with the testing to widen the scope a little bit and see if the numbers I was getting compared to his numbers, which they did. While testing it was also compared to other balls in the same format. There are some solid reviews and results from the 30 of us who tested those balls. When I tested the Motocaddy I ran it through its paces, how it handled on hills and rough, charging capacity, ease of use, and durability. I also compared it to another motorized cart system. But I can see how some would call it a review as well.

[golfr]

5 hours ago, Erin B said:

I know the equipment I tested was tested. For instance, the Callaway ball. It was tested with a trackman, tested for durability, spin, and distance. I also enlisted another spy to help with the testing to widen the scope a little bit and see if the numbers I was getting compared to his numbers, which they did. While testing it was also compared to other balls in the same format. There are some solid reviews and results from the 30 of us who tested those balls. When I tested the Motocaddy I ran it through its paces, how it handled on hills and rough, charging capacity, ease of use, and durability. I also compared it to another motorized cart system. But I can see how some would call it a review as well.

 

 

Yeah, I think that's the OP's point.  It's not "testing" if the feedback isn't going anywhere to make it better; it's just a review.  What's more interesting, is it "really" a review or an elaborate form of alternate marketing?  

It's an astute question, and we're starting to learn some makers may use it that way. For example, I'll pick on one of the influencer "over-exposed" brands right now.  Are LAB putters really being "tested and reviewed", or is the company just giving away $800 putters to a select group of influencers (yes, testers are influencers too)?  In hopes of getting positive reviews? More clever,  since it's so expensive and they got it free, will some influencers give a more positive story and entice others to buy? It's certainly a cheaper approach than some other marketing channels. 

So the OP is probably right to some extent.  

However, I guess if you want to look at the big picture, manufacturers would argue the feedback goes back into the development cycle for the next iteration?  If it really does or not is probably a different discussion.  

[GolfSpy_APH]

11 minutes ago, golfr said:

 

 

Yeah, I think that's the OP's point.  It's not "testing" if the feedback isn't going anywhere to make it better; it's just a review.  What's more interesting, is it "really" a review or an elaborate form of alternate marketing?  

It's an astute question, and we're starting to learn some makers may use it that way. For example, I'll pick on one of the influencer "over-exposed" brands right now.  Are LAB putters really being "tested and reviewed", or is the company just giving away $800 putters to a select group of influencers (yes, testers are influencers too)?  In hopes of getting positive reviews? More clever,  since it's so expensive and they got it free, will some influencers give a more positive story and entice others to buy? It's certainly a cheaper approach than some other marketing channels. 

So the OP is probably right to some extent.  

However, I guess if you want to look at the big picture, manufacturers would argue the feedback goes back into the development cycle for the next iteration?  If it really does or not is probably a different discussion.  

For what it's worth we have had products that were reviewed very poorly and those companies were not happy, but we stand by our testing platform and the members we select. If a product doesn't perform so be it! At least more people will know the how and why it doesn't. 

All we ask from our testers are thorough, open and honest reviews on the product.

Is it good marketing for these brands, of course. We have a wide reach and can help showcase brands and products. It's why we dont offer up all the products we are presented with. 

Now more than ever we are getting more selective and looking to have a further purpose with the products we are bringing to member testing.

I will say they are and always will be a handful that HQ provide us with which can be great or smaller items, but those are still very useful and he'll us build connections.

As for our testers being influencers... On the forum sure. I don't know if I would go much further outside of that tho.

[Donn lost in San Diego]

You can test balls in a lab but most everything else is use and review.  A robot IronByron can test and compare clubs, but no one is a swing robot.  So it can guide you but it can't guarantee that the #1 club will work with your swing dynamics.  Especially, our tempos and speed of transition at the top can be wildly different, so the same shaft is bending differently even among the same swing speed as measured at impact.

Laser and watches can be tested but actual use in real weather can bend the results.  Metal and plastic expand and contract at temperature swings, so these devices can suffer if you leave them in cold or hot location when not using it. 

[golfr]

4 minutes ago, GolfSpy_APH said:

For what it's worth we have had products that were reviewed very poorly and those companies were not happy, but we stand by our testing platform and the members we select. If a product doesn't perform so be it! At least more people will know the how and why it doesn't. 

All we ask from our testers are thorough, open and honest reviews on the product.

Is it good marketing for these brands, of course. We have a wide reach and can help showcase brands and products. It's why we dont offer up all the products we are presented with. 

Now more than ever we are getting more selective and looking to have a further purpose with the products we are bringing to member testing.

I will say they are and always will be a handful that HQ provide us with which can be great or smaller items, but those are still very useful and he'll us build connections.

As for our testers being influencers... On the forum sure. I don't know if I would go much further outside of that tho.

 

 

It's not a forum criticism; it's simply the nature of our consumer market.  

The influencer thing for testers here goes way further than the forum, though.

Anyone who types in "product X reviews" into Google will likely end up directed to one of the testing threads here on the forum for that product. The larger the testing group and conversation, the more likely it is to pop up in direct search results (funny enough, making a product seem even more popular while researching). 

You don't need to be a forum member or even read this forum to be influenced by the testing content; Google makes sure this content goes to everyone interested. 

Manufacturers (and their marketing consultants) are getting wiser and playing the game. Consumers must stay aware that much of what we see is because someone wants us to see it. 

[NM01]

28 minutes ago, Donn lost in San Diego said:

You can test balls in a lab but most everything else is use and review.  A robot IronByron can test and compare clubs, but no one is a swing robot.  So it can guide you but it can't guarantee that the #1 club will work with your swing dynamics.  Especially, our tempos and speed of transition at the top can be wildly different, so the same shaft is bending differently even among the same swing speed as measured at impact.

Laser and watches can be tested but actual use in real weather can bend the results.  Metal and plastic expand and contract at temperature swings, so these devices can suffer if you leave them in cold or hot location when not using it. 

The same can be said for the balls you mentioned that can be tested in a lab. There are things like sound and feel that influence people’s decisions. There is also a performance factor from the human aspects. If there wasn’t Sergio wouldn’t have violated his contest with Callaway to go back to a TM ball.

If there wasn’t a performance factor guys playing Titleist would play another ball because they could make up the loss of sponsorship money in their winnings because they would have better performing ball thus better results.

 

[NM01]

39 minutes ago, GolfSpy_APH said:

If a product doesn't perform so be it! At least more people will know the how and why it doesn't.

This gets to my post the other day. Doesn’t work for who? The golfer testing? That has no indication of how it will perform for anyone else even a golfer of the same handicap level or swing speed.

The bad result could be from a bad fit, a member having a rough patch of playing, or some other reason.

the testing is much closer to a user product review than a test. 
 

There is value in the companies participating in the “testing” as it gives exposure to the brands, but there’s definitely a downside for them when there’s a poor test for whatever reason.

Edited July 13 by RickyBobby_PR

[GolfSpy_APH]

Just now, RickyBobby_PR said:

This gets to my post the other day. Doesn’t work for who? The golfer testing? That has no indication of how it will perform for anyone else even a golfer of the same handicap level or swing speed.

The bad result could be from a bad fit, a member having a rough patch of playing, or some other reason.

the testing is much closer to a user product review than a test. 

[Donn lost in San Diego]

1 hour ago, RickyBobby_PR said:

The same can be said for the balls you mentioned that can be tested in a lab. There are things like sound and feel that influence people’s decisions. There is also a performance factor from the human aspects. If there wasn’t Sergio wouldn’t have violated his contest with Callaway to go back to a TM ball.

If there wasn’t a performance factor guys playing Titleist would play another ball because they could make up the loss of sponsorship money in their winnings because they would have better performing ball thus better results.

 

Can o worms, there RB: you suggest that Titleist is not the top o the heap for balls?  Or you just saying there's horses for courses, that some players favor other well designed balls?

[NM01]

13 minutes ago, Donn lost in San Diego said:

Can o worms, there RB: you suggest that Titleist is not the top o the heap for balls?  Or you just saying there's horses for courses, that some players favor other well designed balls?

At no point in my post did i say that titleist want the top of the heap.

What I said was that like the clubs balls have performance factors that are influenced by the human using them. So they too have to be tested outside a lab. Which is why Titlest has done the white box testing for a very long time and now others are jumping on that bandwagon. And also what I said is that if there wasn’t that performance factor then guys would switch balls away from titleist because they coulda make make in winnings than their sponsorship pays them. But titleist performs well in the lab and with golfers.

With that said there are some golfers who have had better performance from non titleist balls. John rahm being one. The TaylorMade ball played better for him in the wind and overall. There’s a YouTube video of him testing balls when he switched to TM.

Sergio showed that despite great lab testing the Callaway balls were not good for him.

Long story short every piece of equipment we use needs testing outside of a lab. That testing is on the individual because we have to see what it does for our swing.  Because all of our swings are different how anything performs for one golfer is no indication how it will perform for anyone else. Its why anything like putter, any club or shaft testing that’s done anywhere has no interest to me.

Things like range finders, gps watches or other devices would have more of my attention than anything equipment related.  

 

[PaBr]

In no way was this topic intended to be an attack on MGS, the testing procedures or anything. More than anything I think it is cool that producers and consumers are getting closer. And post release testing makes sense and are of benefit to consumers, at the very least to go from a very long list to a short(er) list to take into account when testing yourself. 

 

My intend was to merely get a better understanding of the pre-release testing procedures in play. But also, given the fact that producers and consumers are getting closer, connections between MSG and OEM's getting stronger, isn't there an opportunity for pre release testing?

 

At the early ages of youtube golf, I enjoyed product reviews. THey have become pretty nauseating over time. You get a 10, you get a 10, you get a 10. And oh yeah get fitted. Close to useless for the consumer. I see an opportunity for more involvement from real life consumers. 

[Donn lost in San Diego]

1 hour ago, RickyBobby_PR said:

At no point in my post did i say that titleist want the top of the heap.

What I said was that like the clubs balls have performance factors that are influenced by the human using them. So they too have to be tested outside a lab. Which is why Titlest has done the white box testing for a very long time and now others are jumping on that bandwagon. And also what I said is that if there wasn’t that performance factor then guys would switch balls away from titleist because they coulda make make in winnings than their sponsorship pays them. But titleist performs well in the lab and with golfers.

With that said there are some golfers who have had better performance from non titleist balls. John rahm being one. The TaylorMade ball played better for him in the wind and overall. There’s a YouTube video of him testing balls when he switched to TM.

Sergio showed that despite great lab testing the Callaway balls were not good for him.

Long story short every piece of equipment we use needs testing outside of a lab. That testing is on the individual because we have to see what it does for our swing.  Because all of our swings are different how anything performs for one golfer is no indication how it will perform for anyone else. Its why anything like putter, any club or shaft testing that’s done anywhere has no interest to me.

Things like range finders, gps watches or other devices would have more of my attention than anything equipment related.  

 

Yep.  So I am considering 3 watches"  Voice Caddy T9, Bushnell Ion elite, and shotscope V5.  I posted a new thread asking opioins earlier today.  Do you have  a recommend based on your use?

[NM01]

1 minute ago, Donn lost in San Diego said:

Yep.  So I am considering 3 watches"  Voice Caddy T9, Bushnell Ion elite, and shotscope V5.  I posted a new thread asking opioins earlier today.  Do you have  a recommend based on your use?

I have only ever used bushnell watches. Once I started using the bushnell geo ghost I stopped wearing watches

i am a fan of bushnell products 

[cnosil]

2 hours ago, PaBr said:

In no way was this topic intended to be an attack on MGS, 

My intend was to merely get a better understanding of the pre-release testing procedures in play. But also, given the fact that producers and consumers are getting closer, connections between MSG and OEM's getting stronger, isn't there an opportunity for pre release testing?

I don't think anyone took it as an attack; it is an interesting question.  The relationship between MGS (headquarters) and OEMs is let's just say interesting as it varies based on the results of most wanted testing.   The relationship between MGS (forum) and OEMs is strong but I personally doubt there is an opportunity for pre-release testing.  I think the big drawback would be that if people were doing the kind of testing you are describing,  they wouldn't be able to say anything for probably a year or more.  

[bens197]

8 hours ago, GolfSpy_APH said:

For what it's worth we have had products that were reviewed very poorly and those companies were not happy, but we stand by our testing platform and the members we select. If a product doesn't perform so be it! At least more people will know the how and why it doesn't. 

I never considered this beyond what Jamie describes here; regarding the aftermath of a test.

This is big deal stuff. A real commitment and the repercussions of a poorly executed test fall on the staff, not necessarily the members. 
 

[NubianRugby]

On 7/12/2024 at 7:24 AM, PaBr said:

Relatively new member here, but pretty obvious that testing golf gear is core business here. A big reason for me being here. 
 

but is this testing really testing? Or better said, to me testing is prior release validating of product to allow feedback to result in product improvement. 

 

without trying to be funny, disrespectful or whatever, what is labelled as testing to me seems a lot like product comparison. The product is a given. one could even argue testing in this set up is part of the OEM marketing mix. 
 

reason for me asking is that I think pre-release testing would be very interesting. I know that supposedly happens with your players but that is not a reflection of the target market for the majority of products. 
 
wouldn’t it be cool to have mgs/ consumer pre-release influence in product rather than after the fact. Possibly all of that already exists and I’m simply not aware of it. 

I'm relatively new to the site and part of the attraction was real golfers talking about the stuff being sold.

Is what we do here part of marketing for golf companies? Sure but there's a degree of transparency about how all this is going down that inspires confidence. No one is trying to trick or bull$hit you.