Has MyGolfSpy Lost Its Edge?

[ChetlovesMer]

I used to really like MyGolfSpy. And I still do to an extent. 

But lately they just seem kind of bitter. They also just seem like we should all believe them for everything they say. Maybe I'm mistaken, but it seems like they used to publish all the data from all their tests. Now, it seems like they keep using the phrase "Our data shows this is the best performing … " or "Our data says this" or "Our data shows that"... 

They say that, but then the don't publish the actual data anymore. 

They also kind of seem really bitter toward anything Taylormade or Callaway do. Similarly, it seems like they are basically calling anyone who buys Taylormade or Callaway product stupid people who "Just buy into the marketing". 

Again, I'll be the first to admit I'm wrong, maybe it's just because the PGA show happened recently. Perhaps they are burned out from travelling down to hear a lot of marketing speak. It sure seems to me like a lot of the stuff I've seen from them lately has just been venom toward the bigger golf companies and they keep claiming to be data driven, but it seems like I no longer can find said data. 

 

Is it just me? Let's discuss.

[iacas]

They've been lousy for a few years now.

• They don't show their data.
• They get basic facts about products wrong (their personal launch monitor test was horrible).
• They don't disclose their actual relationships — Snell, EVNROLL, etc. And as everyone here can attest I love Snell.
• They email constantly, and have click-bait headlines.

The lack of data is the worst, though, really.

[Darkfrog]

They never had an edge to lose, at least since I have been aware of the site.

Just about every article reads like a PR piece. Their tests / data are meaningless. Sure they come across as "science-y", but there is no scientific approach to any of it. Embedding a table with data and then throwing out some gibberish about using a "proprietary methodology that considers the statistical reliability of a Strokes Gained metric" doesn't make their conclusions credible or trustworthy. The fact that they don't disclose relationships with manufacturers is a big cherry on top. 

I used to occasionally check out the site for the latest news on equipment releases, but their virtual driver fitting app / algorithm you had to pay to see the results for was the last straw for me.

[ChetlovesMer]

4 hours ago, iacas said:

• They don't show their data.

The lack of data is the worst, though, really.

I seem to remember back 7, 8 or maybe 10 years ago they would publish their tests, how they did them, what data they gathered and then their would be an open discussion. The discussion would be not just on the results but also on how the tests were performed. Back then they seemed legitimately interested on how they could improve their testing. Now it "feels" like (I know feel ain't always real) they just state what's best but give no real reasoning why. Something like "We rolled 36,000 putts and our data shows this putter performs the best." 

It's hard to even find what other equipment was tested and really hard to find anything on how the tests were performed, what the equipment scored, how that score was determined. Perhaps they've gotten so much bigger now that they just don't have time to include all of that anymore. 

3 hours ago, Darkfrog said:

The fact that they don't disclose relationships with manufacturers is a big cherry on top. 

Agreed.

3 hours ago, Darkfrog said:

I used to occasionally check out the site for the latest news on equipment releases, but their virtual driver fitting app / algorithm you had to pay to see the results for was the last straw for me.

I never saw that app. Interesting. 

I still know a bunch of golfers who take what ever MyGolfspy says as Gospel. Usually followed by something like "Don't mindlessly buy into the marketing, listen to MyGolfSpy." 

"Aren't you mindlessly buying into MyGolfSpy?" 

[Darkfrog]

14 hours ago, ChetlovesMer said:

I seem to remember back 7, 8 or maybe 10 years ago they would publish their tests, how they did them, what data they gathered and then their would be an open discussion.

Even though I have been golfing for 20+ years, I only got heavily invested in golf as a serious pastime/hobby in the last 2-3 years, so my frame of reference for MGS doesn't go back that far. It would be interesting to contrast their original work to what they do currently.

That said, I don't believe it is possible to test what they are trying to test (isolate the variable of the club, from all the other variables).

14 hours ago, ChetlovesMer said:

I still know a bunch of golfers who take what ever MyGolfspy says as Gospel.

I know a few MGS disciples too, and honestly I can't say their equipment choice makes any difference in their game. I also know some brand loyalists who believe the marketing hype. I have a buddy who only plays Cobra gear because of Greg Norman strangely enough, and another who only plays TaylorMade because of Tiger Woods.

[lastings]

Their ball test was pretty good last year.

[DeadMan]

I didn't really follow them or hear about them much, somehow, until that ball test last year. That ball test was utter garbage. They didn't publish their data, and they didn't go into details about what their test actually consisted of. Then they concluded that Snell was one of the best balls, and had Dean Snell do an interview to that effect. 

So now I'm pretty much ignoring everything they come up with.

[lastings]

21 minutes ago, DeadMan said:

I didn't really follow them or hear about them much, somehow, until that ball test last year. That ball test was utter garbage. They didn't publish their data, and they didn't go into details about what their test actually consisted of. Then they concluded that Snell was one of the best balls, and had Dean Snell do an interview to that effect. 

So now I'm pretty much ignoring everything they come up with.

They actually did describe all of their testing methods and published data, via a video podcast on YouTube.   There is a link to the video in the article.  

Edited February 14, 2020 by lastings

[Darkfrog]

21 minutes ago, DeadMan said:

They didn't publish their data, and they didn't go into details about what their test actually consisted of.

I agree that their tests are poorly done, including the ball test. But it's probably a good move by MGS to not publish any specifics about how they perform their testing and data analysis (which makes them even shadier).

If I were "Big Golf Company X" and my product was not rated as most wanted, if I knew their exact test methods, I would simply repeat the test, likely get different results, and prove that the MGS experiments are poorly designed. Or I would hire some real subject matter experts on experiment design (i.e, scientists, not MGS staff), and have them pick apart everything MGS did wrong. Either way, MGS loses. When they don't say what they did, the worst they get is internet dissent, and their most wanted stamp of approval retains it's significance (i.e., monetary value to their brand).

[ncates00]

30 minutes ago, DeadMan said:

They didn't publish their data

Yes they did.

30 minutes ago, DeadMan said:

they didn't go into details about what their test actually consisted of

They did offer some insight.  I'll concede that whether it was enough disclosure is the better question.

 

31 minutes ago, DeadMan said:

hen they concluded that Snell was one of the best balls, and had Dean Snell do an interview to that effect.

I suppose you're attempting to demonstrate bias (post-testing) here?

[lastings]

2019 Golf Ball Buyer's Guide | MyGolfSpy

---

This is the most impactful test we have ever published. Over the past 10 years, MyGolfSpy has conducted hundreds of tests, and published thousands of articles that have influenced millions[...]

 

 

 

[ncates00]

15 hours ago, ChetlovesMer said:

what ever MyGolfspy says as Gospel

And I don't do this, mind you.  Just trying to be fair about with my statements.

[klineka]

1 hour ago, lastings said:

and published data, 

1 hour ago, ncates00 said:

Yes they did.

The charts they post online of their results are averages, not the raw underlying data of every shot. So that means we don't know the sample size for each, how many outliers they included/didn't include, etc. which can obviously drastically alter results. 

On the 2019 Driver test, they said they took 10-12 "good shots" over multiple sessions. What a "good" shot is for a 15 handicap is different than a "good" shot for a +1 handicap. 

I'm not really sure what their reasoning is as to why they wouldn't make the underlying raw data available to the public.

 

[lastings]

Yeah, the driver tests and club tests are imperfect, because of human testing. Is what it is. 
 

the ball tests are robotic testing.  If you watch the video they tell you how many hits with each ball.  I can’t really remember.   In the data table at the bottom, they include not only the averages, but standard deviation as well.  

[klineka]

9 minutes ago, lastings said:

the ball tests are robotic testing.  If you watch the video they tell you how many hits with each ball.  I can’t really remember.   In the data table at the bottom, they include not only the averages, but standard deviation as well.  

IMO I shouldn't have to watch a separate video just to figure out how many hits they did with each ball. It just seems like a common sense thing to include when describing your testing methodology. How do we know that they did an equal number with each ball? 

And if they are going to go through the effort to put both the averages and standard deviations in a tab why not just publish the raw data as well? 

I use the same exact program they use (Tableau) on a daily basis for my career and know exactly how easy it is to show views that contain all of the results not just averages. 

[iacas]

2 hours ago, lastings said:

Their ball test was pretty good last year.

The ball test was garbage.

2 hours ago, lastings said:

They actually did describe all of their testing methods and published data, via a video podcast on YouTube.   There is a link to the video in the article.  

The last thing I saw was that they published the results.

Please link to the video here and tell us the time codes when they tell us how many balls were hit, what control measures were taken, etc.

The ball test was shit, and for many, was the last straw.

0:15 - "Each hit multiple times with three clubs" - okay so we know they hit each model ball at least six times.

So, where exactly did they tell us whether they hit the golf balls six times or sixty?

[lastings]

Ok.  I thought they were a bit more specific, been a while since I watched it.  
 

maybe this is a bad test.  Any idea where I can find a better test? 

[Darkfrog]

1 minute ago, lastings said:

the ball tests are robotic testing.

I'm not sure using a robot makes the test any better.

Who are the experts they spoke with?

What are the tolerances of the robot, and how was this verified for before/during/after testing?

What is the expected variance in spin, distance, and dispersion based on the swing robot tolerances?

What are the tolerances of the Trackman used to generate the spin, distance, and dispersion data?

Was any data generated to show that the robot delivered the club the same way, within tolerance for every swing? 

Was strike location for every single ball hit verified to be the exact same spot on the club face? How was strike location accounted for  in the data if it varies?

Were Trackman data for face/path/angle of attack accounted for when looking at the spin, distance, and dispersion data from each ball struck?

How many balls of each type to you need to hit to establish statistical significance of the data set, and did they hit this many balls?

Did they prove linearity of performance between swing speeds or is that an assumption?

Did they perform testing with different club designs to eliminate the club type as a variable (e.g., blade vs. hollow body vs. cavity back)?

Was the test done indoors or outdoors? What was the temperature, wind, humidity when each shot was struck? How were environmental conditions controlled and normalized?