problem with handicap system

[iacas]

Originally Posted by shredfit

I think I understand. The graph posted(by iacas) basically shows the lines converging at a zero handicap index. However, the lines would also start to diverge when getting into + handicaps. Thus, showing that these plus indexes are indeed effected by slope(but it will not be a lot). Is this correct? Will? reid?

I realize we're getting into the nitty gritty here, but I don't think that's right, no, and I said as much earlier.

The graph related to the handing out of handicap strokes based on the index. Our main point of discussion has been the calculation of the handicap index to begin with (per round). One comes before the other, and if we're going to change the first item (calculation of the handicap index), the latter will necessarily change as well. In other words, the major point of discussion is now how many strokes a +4.1 index guy should give to a guy on a course with 135 slope, but rather whether he should be a +4.1 index or a 5.3 index to begin with (or whatever the numbers might be).

[shredfit]

I realize we're getting into the nitty gritty here, but I don't think that's right, no, and I said as much earlier.

Respectfully iacas, I understand what you are saying. The graph however, is still related to the calulation(it's a graphic representation of the formula). If one extrapolates the line below zero(for a better than scratch golfer). Figures will get skewed with respect to the slope(ie the slope mathematically, starts to matter again, although not by much). I think this is what these guys are saying...

FWIW: I honestly do not think that this matter too much.

[iacas]

Respectfully iacas, I understand what you are saying. The graph however, is still related to the calulation(it's a graphic representation of the formula). If one extrapolates the line below zero(for a better than scratch golfer). Figures will get skewed with respect to the slope(ie the slope mathematically, starts to matter again, although not by much). I think this is what these guys are saying...

No, and I've already explained why. I'll try one more time:

The graph displays handicap strokes given to a player with handicap index X. We're talking about how one even arrives at the proper value of X to begin with. It's subtle, but I don't think it's so subtle that we can just say "awww, shucks, it's the same thing."

[Will]

Big Lex,

Ok, I've reviewed the USGA letter. The author seems to be entirely wrong, due to the fact that slope is derived from the bogey rating, and should only apply to scores above the course rating. (slope = bogey rating - course rating * 5.381)

The author doesn't seem to understand that simple fact. His discussion regarding the slopes intersecting at the beginning of the article makes that clear. It is he who is confused by the negative differential. He also makes other mistakes, like using handicap index several times in place of differential. The system would work if there were a reverse slope which was calculated based on a "pro-rating". So, reverse slope = pro rating - course rating * 5.381 (or some other number). Then, the differentials for scores lower than course rating would be calculated using the reverse slope, and things would work fine. I do think it might be necessary to implement a pro-rating, so that plus-handicappers can compete fairly with other plus-handicappers in net tournaments. Now, they can't. So, I've modified "my system." (I can't recall, was it mine, or someone else's? Oh, well, doesn't matter.) Ok, here it is my (or someone else's) New System: The current slope is derived using the formula (bogey rating - course rating) * 5.381. Slope is used to determine the differential, which is basically your score adjusted for the difficulty of the course. This works for scores worse than the course rating. For scores better than the course rating, there is currently no valid slope to adjust those scores for the difficulty of the course. So I (or someone else) propose the SuperPro Rating (tm, all rights reserved, copyright, copyleft and patent pending). SuperPro Rating: A “SuperPro golfer” is a player who can play to a Course Handicap of +10 on any and all rated golf courses. A male SuperPro golfer, for rating purposes, can hit tee shots an average of 350 yards and can reach a 570-yard hole in two shots at sea level. A female SuperPro golfer, for rating purposes, can hit tee shots an average of 300 yards and can reach a 500-yard hole in two shots at sea level. The "SuperPro slope" is then calculated by (Course Rating - SuperPro rating) * 11.3 (which i estimate from 113/(72-62)). The differential is then (course rating - score * 113 / SuperPro Slope). That's the general idea. Should work.

[shredfit]

No, and I've already explained why. I'll try one more time:

Yes I understand, but the graph can still be sited as a trend(ie it should be mathematically relative to other index values). If not, I'm totally missing something here.

[Will]

Yes I understand, but the graph can still be sited as a trend(ie it should be mathematically relative to other index values). If not, I'm totally missing something here.

Again, you are correct.

[Big Lex]

Will, Reid, Erik, Shred, et al:

I read the article last night, and I understand now what the USGA is saying.

The crux of our complaint has been "how can a golfer who scores 2 below the rating on a 120 slope course be _better_ than someone who scores 2 below the rating on a 155 slope course? The guy who played on the harder course has a higher hcp, which makes no sense!"

In short, we're wrong (according to Knuth/USGA) because, as slope is defined and conceptualized, the guy who shot 2 below the rating on the 120 slope course IS the better golfer. The higher the slope rating, the easier it is to shoot below the course rating. This seems counterintuitive, but I understand it in the statistical sense.

I'm not sure it's completely valid, but I understand why the score "differential" adjustments are made as they are, including for plus-hcps with scores below the course rating...Here's a long discussion why.

Slope does not equal "difficulty;" slope is a number which is proportional to the spread between the expected scores of their defined bogey golfer and defined scratch golfer. These two points--the bogey score, and the scratch score--describe a straight line, and the Handicap System (HS) presumes that the line is valid when extended beyond these points in both directions .

The key part is that it is a straight line, by definition . IT CONTINUES ON BOTH SIDES OF THE POINTS WHICH DEFINE IT. Slope is a mathematical construct intended to represent the distribution of scores at a given course, it is not a direct measurement of anything, or a natural law, or a first principle. It's something Knuth invented to try to accurately predict the spread of scores between golfers of various abilities.

A course with a high slope has a big spread between a scratch and a bogey golfer. Let's explore this idea.

Most people are familiar with the normal distribution or "bell curve." Let's assume that golf scores on any course, if played by a typical population of golfers of all abilities, will plot out in a bell curve shape. I think the HS (Handicap System) presumes this.

Forgive me if you guys all know these basic stats principles, but in case you don't, I'll mention them briefly. For any bell curve distribution, whether it's golf scores, height, or SAT scores, a score which is 2 standard deviations higher or lower than the mean (average) is considered exceptional. If the average score on a math test is 50 and the standard deviation is 7.5, someone who scores 65 is really good at math, because they were 2 standard deviations better than average...someone who scored 80 is really, really, really smart, because they were 4 SD's better than average. For normally distributed data, a 95% of scores will be within 2 SDs of either side of the mean. The guy who scored 65 on the math test would be better than 95% of the people who took the test, the guy who scored 80 would be better than probably over 99%, for data that is normally distributed (bell curve).

Getting back to golf, let's say that the USGA-defined bogey golfer is an average golfer. His scores will represent the average at any golf course in the world. The scratch guy is much, much better, and his scores will in general be 2 standard deviations better than the bogey guy. The Pro will be maybe 3 SDs better. Stated another way, the scratch guy is better than 95% of all golfers, and the pro is better than maybe 99% of all golfers.

In the USGA Handicap System (HS), slope is a surrogate for SD.

The higher the SD, the greater the difference will be between the bogey score and scratch. This says nothing about the overall difficulty. The system allows for the fact that two courses may be of equal difficulty to one class of golfer, but very different for others.

Let's assume a course where a billion golfers of all abilities play a billion rounds, and we get an average score of 92, and SD of 10. On this course, the scratch player has a potential score of 2 SDs below the mean, which equals 20 strokes below 92, or 72. The pro is even better,and has a potential score of 3 SDs below the mean, or 62.

Now assume there's a course which is easier for the average golfer, maybe 88 is the average score, but it is equally difficult for the scratch guy. We can conceptualize this, right? This course is easier for the chopper because there are fewer forced carries of 180 yards, say. Since the scratch guy has no problem with such a carry, the course isn't any easier for him, but it is easier for the chopper.

So, 88 is the mean. But let's say the SD for the million golfers playing a million rounds was 8, not 10. The scores were more bunched up. In this example, the scratch players will shoot 72 again (remember it's no more difficult for them than the last course was), and this is in fact 2 SDs below the mean (88 minus 8x2 = 72).

In handicapping a match between an average golfer and a scratch golfer on these two courses, the average golfer should get fewer strokes on the lower SD (slope) course. Why? Because even though the statistical spread between their scores is the same on the two courses (2 SDs), because the SD is smaller on one course, less actual strokes must be given.

Now can you see why they give fewer strokes on the plus side of handicaps? As the SD goes up, the difference between expected scores of various levels of golfers also goes up. So, on the high slope course, the pro should have an easier time outpacing the scratch guy. If the SD of the scores on a course is very high (high slope), it will be reflected not only in the difference between the scratch and the chop, but also between the scratch and the pro.

In other words, a really low score below the rating is _more_ impressive on a low slope course than a high one, because it represents a greater relative distance from the scratch player.

So in calculating handicap on high slope courses (relative to lower slope courses), just as you give more weight to scores closer to scratch for scores over the rating (because it was harder for the golfer to get close to scratch), you must give less weight to scores further from scratch (because it was easier, statistically speaking, to attain that spread).

The confusion we are suffering (according to the USGA) has to do with what the HS "should" do in handicapping matches. If scores are spread way out on a course it is both harder for the chopper to beat the scratch player, while at the same time easier for the pro to do so. Given this, the score differential used for handicapping on higher slope courses should be lowered for the scores better than scratch.

Really, all golfers are being compared to a theoretical minimum score, say 54. If the distribution of scores on a course is very narrow (low SD, low slope), it would take millions and millions of rounds before someone happened to shoot 54, because that 54 is way, way out on the tail of the distribution, maybe as much as 8 SDs below the mean. On a course of very flat distribution, it would take fewer scores to get one as low as 54, because this might only be 6 SDs below the mean.

Similarly, if a 54 is less likely, a 64 will also be less likely, assuming a normal distribution of scores. This is counterintuitive, but this is why the HS "sees" a score of 4 below rating on a 113 slope as better than a score of 4 below rating on a 135 slope.

In other words, the "difficulty" of a course is relative, depending on your perspective. Handicaps and slope ratings relate to the spread of ability and score. Under this system, on a course of high slope, it can be expected that the high handicapper will find it relatively harder to beat the scratch guy, while the plus-handicapper will find it relatively easier.

That's the best I can do to explain the USGA's position.

In the end, I'm still not sure the system is correct, because I'm not sure the assumption that scores are randomly distributed is correct. My sense is that golf scores would be quite skewed toward the high end, and that assumptions based on a bell curve distribution wouldn't be valid.

But here the math gets too hard for me, and I don't know.

Sorry for using up so much bandwidth guys.

[iacas]

Again, you are correct.

The graph is correct for what it shows, yes. However, the graph has

nearly nothing to do with what we're talking about for two reasons: 1) It only goes down to scratch 2) It uses established handicaps. For the last time: we're talking about something that would change handicaps, which are determined before that graph would ever come into play (and which would change that graph). I've attached a graph that better illustrates the problem. The same scores relative to the course rating (+18, +12, +6, 0, -6, -12) are plotted for two courses: one sloped 150, the other sloped 76 (113 +/- 37). 76 is blue, 150 is green. At scratch, currently, the guy playing the easier course (blue) suddenly begins getting the lower handicap index. This differs from the previous graph, which went the other way: given a handicap, how many strokes do you get? This one answers the question: "given a score, course rating, and slope, what's your handicap?"

The system would work if there were a reverse slope which was calculated based on a "pro-rating". So, reverse slope = pro rating - course rating * 5.381 (or some other number). Then, the differentials for scores lower than course rating would be calculated using the reverse slope, and things would work fine.

I still don't like that method because I'm not in favor of over-complicating it. And at this point, you're guessing just as much as I am as to what would be the most effective, so there's not much more to discuss.

And you used "differential" exactly the same way here for which you (correctly) pointed out that I'd mis-used it earlier.

[shredfit]

Big Lex,

Wow! very good post... I think I know more about the "system" then I ever have(or wanted to know).

The USGA is using the Standard Deviation from the Mean for the calulation? Wouldn't they have to assume the Mean HAS to fall in a realatively unskewed bell curve?

I too, do not think this is quite right... but I did get a C in college statistics...

[iacas]

The USGA is using the Standard Deviation from the Mean for the calulation? Wouldn't they have to assume the Mean HAS to fall in a realatively unskewed bell curve?

I'm not sure what you're suggesting uses standard deviation or a bell curve, but no, I don't think any handicap calculations use the bell curve. I don't even think courses are distributed in a bell curve re: their slope.

http://www.usga.org/playing/handicap..._handicap.html

[Big Lex]

Shred:

As to your question, the one iacas quoted:

I'm not sure Knuth had "bell curve" in his mind when he developed his system, but the best way I can explain the system is to use the bell curve concept.

Thinking about it this way, all scores for a course describe a normal distribution. On the bell curve, the x-axis is actual scores, and the y-axis is frequency. In a million rounds, you'd have most of them in the center of the bell, and in a perfect distribution, the average (mean), median, and mode would all coincide exactly in the center. If you mark an x-intercept at points 2 SDs on either side of the mean, you would have 95% of the scores. 3 SDs would give you 98% of the scores.

My concept of how the handicap system works is that the course rating is an x-intercept of the score they think the scratch golfer is capable of shooting. Another x-intercept is the bogey rating, or the score they think their 20-hcp, bogey golfer is capable of shooting. In terms of the normal distribution, I assume that the course rating/scratch score is 2 SD's better than the bogey score.

So if you know the mean and standard deviation, you can draw a normal bell curve that fits those numbers. Then, you can interpret any given score in terms of how it relates to the scratch golfer.

I think in creating the slope system, Knuth's ratings (rating and slope)estimate/establish the mean and SD. Then, any individual score on that particular course can be "translated" into a statistical value--the number of SD's from the mean, or whatever. Then, when you go to another golf course, your SD stat (or whatever variance number they are using) is plugged into _that_ course's bell curve (which itself is determined by it's own mean and SD).

Similarly, in determining a player's handicap index, it is necessary to convert their scores into a universal scale that is applicable to all courses (Reid--sort of like the INR for clotting times for coumadinized patients). This is what the slope-adjusted differential is.

And if you're assuming a bell curve, then the slope adjustments are correct just the way they are.

Now, if scores are not normally distributed, I'm not sure this system is correct. But as I say, that's more stats than I can handle.

So, finally...to answer our original conundrum, I think this is what the Handicap System says:

Two courses, 1: 72/120, and 2: 72/140. Two players, one tour pro, and the other highly ranked amateur, club champ level guy.

One golfer plays course 1 exclusively, the other course 2 exclusively.

Both average 3 below the course rating. Which player plays which course?

The answer is that the tour pro plays course 1, and the club champ course 2. If not, then either one guy is sandbagging, or the ratings/slopes are wrong.

In a million rounds on course 2 (higher slope) there will be _more_ scores of 69 than on course 1. Why? Because more low scores equals higher spread, and higher spread is what is needed to result in a higher slope calculation. If there are more scores of 69, that means higher frequency, which means that point on the bell curve is closer to the mean...which means that more golfers of lesser ability will be able to attain those scores, capturing the scratch guy.

Ok, Reid, have at me!!

[Will]

So, finally...to answer our original conundrum, I think this is what the Handicap System says:

Pretty impressive amount of bandwidth. Two lengthy posts, lots of math. Only, can you link me to some USGA publication that references bell-curves and standard deviations? Ones which specifically address our issue? I'd really like to know, can you give me a link?

You seem to be trying to justify the handicap error by postulating some sort of known distribution. Then you say you can't substantiate said distribution. In fact, the USGA never considers the issue in that light (to my knowledge). The issue remains: There is a bogey rating for all golf courses which establishes the expected score for a player of certain abilities. There is a course rating for all golf courses which establishes the expected score for a player of certain abilities. Both of those ratings expect the score to be between scratch and bogey. The formula that calculates the (diff, index, and course handicap) for scores better than the course rating uses this thing called slope, which has nothing to do with the better-than-scratch golfer. The slope doesn't extend beyond the course rating because it has no meaning beyond the course rating. You can postulate some mathematical relevance, but it really doesn't exist. I found your posts interesting reading, but not relevant to the issue at hand. I have no dog in this fight, I would be happy to be convinced otherwise. The USGA will look pretty stupid if they have to change their handicap system after all this time. Please use simple language and examples to refute the error that was posed in my initial post.

[Big Lex]

Pretty impressive amount of bandwidth. Two lengthy posts, lots of math. Only, can you link me to some USGA publication that references bell-curves and standard deviations?

No, Will, I can't. What I'm saying is that the Handicap System _seems_ to work perfectly if you assume it is based on these statistical concepts relating to the normal distribution. I suspect, from reading the letter you linked in a previous post as the USGA's reply to Reid's original questions, that the system is exactly set up this way.

I'm not sure I'm right about the specifics--that the bogey golfer is the mean, and a scratch is 2 SDs better than that, ,etc. But even if the specific x-intercept points are different, the concept is the same.

You seem to be trying to justify the handicap error by postulating some sort of known distribution. Then you say you can't substantiate said distribution. In fact, the USGA never considers the issue in that light (to my knowledge).

Not exactly...I'm saying there isn't any error. I originally thought there was, but now, thinking about it in terms of the normal distribution, I don't think there is any error. You are correct that there is nothing I can find in USGA writings or postings that says specifically that hcp/slope is basedon the normal distribution, etc., and I don't know for sure if in reality scores are normally distributed. But if they are, the slope system, as I'm purporting it to work, is accurate.

The issue remains: There is a bogey rating for all golf courses which establishes the expected score for a player of certain abilities. There is a course rating for all golf courses which establishes the expected score for a player of certain abilities. Both of those ratings expect the score to be between scratch and bogey. The formula that calculates the (diff, index, and course handicap) for scores better than the course rating uses this thing called slope, which has nothing to do with the better-than-scratch golfer. The slope doesn't extend beyond the course rating because it has no meaning beyond the course rating. You can postulate some mathematical relevance, but it really doesn't exist.

Strictly speaking, yes, the system rates courses between boundaries of bogey golfers and scratch golfers, so you can argue that slope should go away below the course rating. But if you do that, slope MUST also go away above the bogey score.

If there is a problem with the HCP system for plus hcp players, then by definition there is also an error for guys with hcp over 20. This isn't necessarily impossible, because many mathematical models designed to fit certain data are only relevant for a certain part of the data, and lose accuracy at the extremes. Maybe this is the case. But I don't think this is the case, on either count. I think, based on the letter you attached to your prior message, that slope DOES have meaning for the better than scratch player, and there is mathematical relevance. From what I can gather reading the USGA literature, the same factors that determine the course rating also determine the bogey rating. The slope is then calculated based on the difference. What I'm saying is that I think that slope number - the difference between the two ratings, times 5.something, is a number they arrived at through fancy math to convert actual scores into numbers with universal statistical relevance. The slope number converts any score, whether above or below the scratch score, to a standardized score that is 'k' units of SD away from the scratch player's score. The slope of the line between the bogey and scratch scores gives you a spread of scores; the slope number allows them to extrapolate the shape and spread of the expected distribution of scores at that course, and standardize actual scores accordingly. We need Dean Knuth here pronto! Look, I admit that my math ability isn't in Reid's or Erik's ballpark, but I think what I'm suggesting makes sense from the USGA letter you attached to your prior post, and it makes sense statistically. Boy, this is fun. Come on, Dean, show up!!!

[shredfit]

I think Big Lex is correct. There HAS to be some statistical relationship(component) in different handicap calculations. Regardless of what the USGA has on the subject. It's just like, or similar to using a different method to solve or explain the same problem.

[Will]

What I'm saying is that I think that slope number - the difference between the two ratings, times 5.something, is a number they arrived at through fancy math to convert actual scores into numbers with universal statistical relevance.

You're kidding. I urge everyone to read the above quote three times, without gagging. I asked for some concrete examples, and that's what I get. I think I'm beginning to understand what's going on here.

Go ahead, cloud the issue with "fancy math", as you put it. Ignore the issues that I've taken the time to carefully point out, continue to offer theoretical rebuttal to real problems. Increase the bandwidth. Have fun. I'm off to discuss the issue with the USGA.

[Big Lex]

There's nothing silly about that quote. What I'm saying is that through mathematic modeling and so forth that I can't readily figure out, the slope ratio of 113/x gives a number that can be used to convert a score on a course x to a standardized score.

The USGA writing that made me think of this is right in the very letter you attached to a post yesterday.

The author of that paper writes more than once that the slope relates to the spread of scores, and that the larger the spread, the larger the slope.

The larger the spread, the larger the SD in a normal distribution.

What is so difficult about that?

[flash]

I would like to know where I may find a good laymans' explanation of the slope index and why it all began.I want to know why it was started.May sound lame but you fellows have sparked my interest in the topic and I should know more about it than I do.Thanks.

[iacas]

What I'm saying is that the Handicap System _seems_ to work perfectly if you assume it is based on these statistical concepts relating to the normal distribution. I suspect, from reading the letter you linked in a previous post as the USGA's reply to Reid's original questions, that the system is exactly set up this way.

It's not set up that way at all, no.

I'd be willing to bet a good sum of money the bell curve has nothing to do with it. Nothing at all - and attempts to make it fit are quite senseless given the amount of information published on how it came about, how it currently works, how people rate courses, etc. In no step does "let's see what the average scores are" come into play let alone standard deviations and bell curves.

From what I can gather reading the USGA literature, the same factors that determine the course rating also determine the bogey rating. The slope is then calculated based on the difference.

That's not really true, no. Talk with a course rater. I spoke to this earlier - and in an email to you, JP. The same factors are used, but in different ratios and they're looked at differently.

For example, a narrow chute of trees around the tee is a BIG deal to the bogey golfer. So is a carry of 185 yards off the tee over a gorge. Those don't even factor in to any of the categories for the scratch golfer because they aren't "problems" for the scratch golfer.

You're kidding. I urge everyone to read the above quote three times, without gagging. I asked for some concrete examples, and that's what I get. I think I'm beginning to understand what's going on here.

I'm with Will on this one. I don't think you're even close to correct. Dean's written a few articles about how slope and handicapping came about. Have you read them? Not once is standard deviation or the bell curve mentioned. There's a reason for that........................

The author of that paper writes more than once that the slope relates to the spread of scores, and that the larger the spread, the larger the slope.

You're way overthinking this or something, and I'm not sure why you're suddenly so gung-ho about this standard deviation bit, but suddenly we're way,

way off topic.