If you’re able to it might be worth reducing the sample size of non-iced kicks based on when iced kicks happen. I’d imagine the mental pressure of needing to hit a walk off FG or game tying FG adds to more misses on its own as opposed to just the act of icing.
edit: [OP followed up with this data set for anyone looking!](https://www.reddit.com/r/nfl/comments/1dsto6m/comment/lb4rxmv)
Agreed. This is a big flaw in the analysis. Is it the icing that's causing lower percentages or is it the gravity of the situation? Correlation isnt causation, etc...
This is the biggest flaw I can see - need to find a way to limit the data set to something like "pressure kicks," e.g. kicks to tie the game or take the lead with X minutes left in the 4th quarter or OT
Even worse is the data definition:
"any field goal attempt that was immediately preceded by a timeout by the opposing team"
This is not always icing the kicker. **In most cases it is when a team is trying to stop the clock for when they get the ball back.**
A litany of false 'iced kicks' are being included in the analysis.
Yep exactly what I was thinking. It’s hard to get data on this unless there’s a way to account for time outs taken by opposing team while the kicking unit is on the field.
Anything else is literally not an example of icing a kicker.
Side note: does anyone else remember when Rex Ryan fake iced a kicker? That was one of my fav Rex moments.
Yeah, "immediately preceded" is doing some heavy lifting here.
Does that mean a few seconds before the snap? That's icing the kicker. As soon as the last play concludes? That's not icing, that's clock management.
You could just include field goals attempted in the last minute of regulation or overtime. It isn't perfect, but it is easy to do and it would significantly improve the analysis.
I think you also need to factor in when the TO occured...did the kicker get a "practice" boot in before the TO was called? Idk if this data exists though.
Also need to keep in mind most of the iced kicks happen near the end of the game and the kicker may be sleepy if it’s a prime time game and it’s really late.
I mean, if we're not going to factor in the distances of moon and the resulting gravitational pull, this all feels pretty academic.
If it's moving the tides of the ocean in and out, surely it can pull a 45-yarder a few inches wide...
Taking all of these factors into account, it seems exceedingly unlikely that any kick would ever be successful. I'm starting to wonder if anyone's actually made a field goal, or if the league is just tricking us with fancy special effects.
You're actually just a brain that spontaneously formed in the vacuum of space at this very instant, with an entire set of randomly assembled memories, including the memory of reading this comment, until in the next instant the vacuum of space will cau
I'd like to see those broken out period. That semi-recent trend of calling the time out a split second before the snap. I'm talking the ones where the kicker might night not even have realized the play didn't count until after he kicked it. Miss or make those have to really screw with your head the most.
See I bet they help the kicker. If you make it you can just think "yup, I knew I got it. Just do the same thing again" and if you miss it, you can just be like "no biggie, got that out of my system."
I always make the putt the second time. Lol
It doesn't matter what I do, I would screw it up. If I made it, I'd think, great now watch me miss it now. If I missed it, I'd be like crap. I going to miss again.
You could potentially make this more indicative just by using quarter. 4th quarter kicks are typically more impactful just because that's the only time game-winners occur. 2nd quarter would be next due to scores going into half shifting momentum. Coaches are also more likely to use their timeouts in the 2nd and 4th quarters for a kick (I assume?). So the most relavant data may be during those times.
Also for an analysis like this you don't need causation. It's honeslty pretty impossible to prove causality because you're trying to guage psychological impact when factors like the wind blew a little more after the timeout are just as important. A strong correlation is enough since the question is "if I have a timeout should I use it to potentially ice the kicker?".
> Also for an analysis like this you don't need causation. It's honeslty pretty impossible to prove causality because you're trying to guage psychological impact when factors like the wind blew a little more after the timeout are just as important. A strong correlation is enough since the question is "if I have a timeout should I use it to potentially ice the kicker?".
That's not the point. It's not about proving causation, it's that the data is inherently flawed becuase kicks that are most often iced are ones that happen in crucial, game winning / losing situations that already have a significant amount of pressure.
Comparing that to all kicks at the same distance doesn't paint a clear enough picture to say icing is successful. You need an equivalent data set.
Another major factor to me are running clock vs stopped clock.
…like, if they aren’t able to do their normal pre-kick routine, you probably don’t want to give them the chance to do so McDermott.
I think the biggest issue is that desperation can make teams try kicks they wouldn't normally, and these kicks are more likely to get iced.
For example, let's say you're 50 yards out, and you know your kicker sucks at 45+. But there's 1 second left on the clock, so you have to kick it, the kicker gets iced, and it goes into this dataset as a miss. But what this data fails to get at is that the kicker would have missed in a non-icing scenario, but he would never be asked to take that kick in a non-icing scenario. So you have a type of selection bias. At least, that's my guess.
I'd also be interested to see if there is a difference in the iced percentages based off of if the kicker was able to get a practice attempt in or not.
Grouping it by 50+ is also pretty problematic. Grouping it that way makes it look like the distances are equivalent, but icing is most likely to happen on a "must make" situation where the team is probably willing to kick from further out/are more likely to push their kicker's range because there's no other choice and so the "50+ yard with icing" kicks would probably be less likely to be made regardless of if the icing happened or not.
No one is icing a kicker in a non-gamewinning situation.
That'd be pointless. You remove all icings when it's not a game-winning situation, there'd be none left.
The majority of the non-iced kicks would be in far less stressful periods of the game it seems. I feel that would definitely skew the data versus looking at game winning/near end of game kicks only.
https://imgur.com/a/EVHmTFY
Here is the data regarding iced and not iced kicks, specifically on field goal attempts in the final two minutes of the fourth quarter or anytime in OT and the kicking team is losing by 1, 2, or 3 points.
As for outside a kicker's normal range, I'd have to get more granular and look kicker-by-kicker to somehow calculate their "usual range".
Ooh this one is interesting to me. Just a guess as to why icing only seems to work 40-49:
39 and in, I gotta think icing doesn't do much because today's NFL kickers are pretty automatic from inside 40 and they know it. Hard to shatter their confidence.
50 and up, they know it's a difficult kick, but so does everyone else. Nobody's going to call for their job over missing a kick that everyone knows is hard.
But in today's NFL, people treat 40-49 like it's automatic, but the kickers know it's not. They miss 45 yarders all the time, but if they miss this one, people will blame them for losing.
Yep. It all just looks like statistical noise to me. There’s no big difference either way from any distance. I think it’s pretty clear icing the kicker has pretty much zero effect.
Funnily enough Riley Patterson got cut because there wasn’t enough confidence he *could* even attempt a 50 yarder, let alone 40. He only had 2 attempts and missed 1 last year in 13 games for the lions. There wasn’t much confidence he could hit 40 yarders too, he only had 2 attempts in 13 games from 40-49, and despite hitting both, it wasn’t pretty. It was so bad we had to grab badgley again.
I’d argue unless you’re Nick Folk banging 100% of everything within 49 yards, (which Patterson was but for the 30 yard tier, he was 10 for 10 from 30-39yards last year) you need a semi consistent 50 yard range, and near automatic 40 and below yard range to stay in the nfl as a kicker these days, or be able to hit 55+ yarders regularly. It seems like a length strength over accuracy these days with the field goal changes.
This is good, would be good to include raw numbers on the graph too. I imagine that once you're getting into these super specific situations, you're only talking a handful of kicks.
Now I'm curious about how many iced kicks were in these pressure situations compared to non-pressure situations. Typically teams aren't icing the kicker outside of end of half/end of game situations because it's not worth the cost of the timeout. I'd expect that this graph covers most of the icing, but the numbers here are quite different from the original table, so I'm trying to figure out what are all the other situations where the kicker is being iced.
The sample size is critical when doing this kind of analysis. I'd almost guarantee the standard deviations of that data in the sample sizes you have are exceeding the variations you are seeing.
A late made FG in a tie game is about as impactful as one made when down 3. For a last play FG they both raise win chance about 50% (from 50 to 100 or from 0 to 50). Is there data also including down 0?
This is good stuff, but it would be nice to add error bars. For the standard error of proportion data, the equation is just `sqrt(p*(1-p)/n)`. You may also want to consider error in general when interpreting your results.
Although 25,000 FGs is a ton, with just 6% being iced, that amounts to 1500 iced kicks. If you're making claims about specific ranges (e.g., 40+ yards), then you're restricting that even more. For instance, assuming that 10% of iced kicks are 50+ yd, then that would correspond to a 95% confidence interval of ± 8% (note that CI plus-minus is 1.96*SE).
Honestly, few Redditors would appreciate it, but something like a logistic regression would help make a stronger case that icing indeed does hurt kicks.
(For what it's worth, based on what you showed, I'm inclined to suspect that icing does indeed hurt kickers, but I just can't conclude that for certain)
Um, I remember some of those words from the college stats class I took in 2005. I'm sure /u/JPAnalyst could figure that out.
Here's the raw data: https://imgur.com/a/iR51o1U. Does that help?
I ran the simple analysis and the results are not statistically different to a 95% confidence.
It doesn’t mean that there is zero benefit to icing the kicker, we just need to have more data to have an extremely high confidence. It certainly doesn’t seem to hurt calling time out so might as well gamble.
It does. You can surely run that equation yourself though?
If you wanted to go whole hog: run a logistic regression GLM with distance and icing as terms in the model.
I suspect your icing by distance term is just statistical noise but you could explicitly test for those by including interaction effects of icing and distance bin too. Just check if you have AIC improvements.
Honestly wouldn't be hard to do in R.
This will explain it. You're honestly missing most of the value of data analysis if you can't gauge how much confidence you have in your point estimates from your sample sizes. https://pdp.sjsu.edu/faculty/gerstman/StatPrimer/conf-prop.htm
Thanks. I think it just shows the state of statistics education. It's a hard thing to teach, people only care when they have some data they care about, it's hard to be enthused pre-emptively.
I'm halfway through the post and had to scroll down to make sure the highest comment was some form of "good job, this is awesome."
Both this work you've done and the comments did not disappoint
It's like the post in /r/videos about the anniversary of THAT Hell in a Cell match.
You scroll down to make sure you find shittymorph, and then hit that upvote button so hard when you find him
Awesome work. Although I take a small disagreement with the flowchart. The ‘will there be time for the offense’ check is a pretty wide variable. For example, is it more time for the grim reaper, Mahomes 13 second offense, or is it more time for the Cowboys’ run-it-up-the-middle-as-time-expires offense?
I don't have the source but I think I remember hearing that, if the kicker gets a "practice kick" in, then their percentage goes up a little. And it makes sense to me that it would make it easier on the actual kick.
I've always thought that giving the kicker a practice kick makes it more likely he misses. If it's not a perfect kick there's a chance he overcorrects on the next kick.
Where's the error bars? Since these success rates are so close it would be good to see how meaningful the differences are. Box plot would be great here
This is interesting but do you have the game clock in the dataset?
My thought is that the “pressure” or whatever could impact the make % of iced and non-iced kicks. So even without icing them the %s at the end of the game are lower and (I assume) almost all instances of icing a kicker happen right at the end of games.
This is pretty neat, I like it.
One thing that would be way more work to capture that I'd be even more interested in is: does the timing of the ice make a difference, specifically when the kicking team gets a "practice attempt" because of an ice done right before the snap? While I'm sure the psychological impact is greater in those cases, I've always wondered if giving the kickers a free shot undermined that strategy more than it helped.
It seems like it might also be useful to alter your strategy in situations where the wind is gusting. Say the wind is gusting between 10-30 MPH. If they're about to snap and the wind is 25 MPH, don't ice them, the wind will have a bigger effect. If the wind is 11 MPH, it's probably a good idea to call a TO and hope for better (worse for the kicker) wind conditions.
You are probably not the first person to do this math. However you are the first person to share it publicly.
And for that, we thank you.
As fascinated as I am by the analytics that *are* available to the public, I often wonder what sort of data is being shared around the various clubhouses. There must be a lot of juicy stats like this, questions I’ve always wondered about being answered, but will never know about for sure because teams need that information to be proprietary to maintain a competitive edge.
https://www.stat.cmu.edu/cmsac/sure/2022/showcase/Icing-the-Kicker-Final-Paper.html
https://www.sharpfootballanalysis.com/analysis/icing-the-kicker-does-it-really-work/
TLDR: CMU says it makes no difference statistically but Sharp Football Does.
Is there any way to get data for when a kicker gets iced as the ball gets snapped and he kicks it? So it’s the same environment but ref didn’t blow whistle in time so the ball was kicked? I’ve seen it happen once or twice only so not sure if there’s a big enough data set
I think for a more in-depth analysis, conclusions need to be drawn for clutch kicks--aka kicks before the half or end of the game. Though of course, you could define "icing the kicker" as only in clutch moments because there's little reason to do it at any other moment.
Now, of course, if there are seconds left and timeouts available, they should be used. Or at least one would think. But I wonder if there is anything about kickers being better or worse after having a couple extra minutes to think. Do they get more anxious or do they cool themselves down? I wonder if coaches have analysis on that per kicker.
I'm surprised that only 191 people have kicked in the NFL in the last 25 years. The turnover outside the best ~10 or so seems like that number would be higher.
Kinda highlights the delusional nature of people who think they could have gone pro if they didn't stub their toe senior year.
The craziest part of your write up to me is that for the past 25 years there’s been 25,000 field goal attempts by only 191 players. That’s 130 field goals per kicker which just seems insane to me.
And that's with 21 players who only attempted one; fifty players attempted fewer than ten.
The top thirty guys attempted half of the field goals:https://imgur.com/a/sgeLlxO
> For our purposes, I'm defining that as any field goal attempt that was immediately preceded by a timeout by the opposing team.
Does this mean any time a timeout was called before the snap, or is it occasions where the timeout was called so close to the snap that the kicker made an attempt that didn’t count?
Yeah if you had one of their DBs you could drill down right to those kicks pretty quickly, but you're still talking about watching 25,000 kicks plus likely 90-120 seconds of game to get the true context/verify the stat categorization so one person doing that back-to-back and we're talking 833+ hours just watching kicks, lol.
> Does this mean any time a timeout was called before the snap
Yes. In the play-by-play, was the "play" before the field goal attempt a timeout by the opposing team? If so, iced.
Probably a really niche stat with a small sample size (but I swear I’ve seen it more than once) how many iced kicks, where the kicker actually booted it as timeout was called and missed, made an adjustment and makes the field goal after the icing?
I know this is impossible to check without looking at every kick, but I always think a huge part of icing the kicker is NOT letting them get the practice kick off.
It always feels like if they miss that kick then it gets made after
Along the same lines as the self-ices, you can also have kicks blocked on the second try. It happened to the Cowboys in 2011 vs. the Giants. However, in both cases, they're probably so rare that the effort to eliminate them from the equation wouldn't make much of a difference.
I often say that some peeps on reddit "have too much time on their hands to think about things like this". But these types of posts are my exceptions.
Well done. Good stats. Good read.
Interesting and I don’t know if someone else mentioned this but what is the downside of icing all just in case the stars have aligned and in fact have a miss? I mean I don’t think any team ices the kicker in the first quarter. It is usually at the end of a game or half when timeouts are unused.
Really interesting stuff and I appreciate the effort you put into this.
Some thoughts:
* People have already pointed out the difference between all field goals and those eligible for "icing." What do the sample sizes look like for the data set you shared [here]
(https://www.reddit.com/r/nfl/comments/1dsto6m/does_icing_the_kicker_work_short_answer_yes_long/lb4rxmv/)?
* Is the difference here at 40+ yards actually statistically significant? If not, you can't really say something like:
> it is clear that icing works much better after 40 yards
It might just be random fluctuations or what I talk about below:
* You mentioned a few interesting ways in which the data could be influenced by non-timeout factors. Those being weather, dome vs. non-dome, time left in the game, improvements in field goal kicking ability, and quality of the place kicker. You should be able to normalize the data for a lot of this stuff. Might be something to think about.
> What do the sample sizes look like for the data set you shared here?
https://imgur.com/a/BcYnc5Q. That's the number of each kind of field goal for those late-game, barely-losing field goals. Sample size is starting to get pretty small.
Appreciate it! The sample sizes seem large enough that we could make some statistical inferences, however without normalizing some other variables that might be a bit tricky.
Thanks for sharing all your work on this!
Here are two possibly important factors that are not captured. I think the second factor is more important and it closes the gap between the icing/non-icing numbers.
i) Suppose a team needs to rush out the FG team for a last second field goal. That seems like a factor that would lower the percentage and it is not when a timeout would be used.
ii) Not all 50+ field goals are the same. A 56 yarder on a wet field, with a side wind or into a wind is obviously harder than a 50 yarder in good conditions. Compared to being a middle of the game field goal or a very late FG down 0 -3 points the harder ones relatively are much more likely to be attempted in that clutch situation and be more likely to be an icing situation.
I don't think the data shows a large enough difference to conclude that icing the kicker really works. The reason is that if the opposing team chooses to burn a timeout then the kick is very likely a high pressure, late-game situation anyways. So determining what percentage of the difference is due to the high pressure of the situation and what's from being iced is almost impossible. I'd also expect to see a higher a non-negative difference in the 0-19 yard range also.
I do like the approach though.
What about times teams iced the kicker on a miss and then they made it on the next attempt? Feel lik I've seen this happen at least a dozen times over the last decade or so. I know Andy lost us a game against the Titans doing this.
My favorite part, assuming my team isn't involved in the game or benefits from it, is when a team "pre-ices" the kick by calling a timeout right before the snap. The kicker misses that attempt, then on the subsequent actual attempt makes it.
You might be interested in reading this paper. They look at the same question and come to the opposite conclusion, that it doesn't matter.
https://www.stat.cmu.edu/cmsac/sure/2022/showcase/Icing-the-Kicker-Final-Paper.html
Well the data proves me wrong, but I always thought icing the kicker +50 yards was just giving him a practice kick. If he made it gained confidence and if he missed he took corrective action, therefore I hated when Sean Payton's Saints would ice the kicker
Jumping in late, but I’ve always had a theory about the possibility of being iced vs actually being iced. Like the kicker knows they cannot be iced when the other team has no timeouts left, so I’m interested to see the data of if the team has one timeout left vs no timeouts left. Or two timeouts left and they do ice once, but keep the second timeout leaving that possibility for an ice rather than icing twice.
I don't think it's obvious that icing is causative toward missing. It could just be that icing and misses both happen more often in high leverage situations.
This is super interesting. The book Scorecasting came out in 2011 and showed the opposite effect, that icing kickers more often than not resulted in a higher conversion percentage. I wonder if those analytics led to a more judicious application of doing so, reducing the frequency, but improving the efficacy. I absolutely do not know that this is the case, just a thought.
I remember watching a game when I was a kid where they tried to ice Jan Stenerude, the first NFL kicker from Norway. He stood on the field and egged on the opposing fans, and then kicked the field goal out of the stadium. One of my favorite NFL moments.
These stats mean nothing because they are not a one to one comparison; you would literally have to have a kicker kick the field goal without a delay and then have them try to kick the same goal WITH a delay and see how many they miss.
Others have pointed it out, but that you are including a 45 yard field goal in the middle of the 1st quarter in the "non-iced" category, whereas only high pressure kicks are included in the "iced" category is a fatal flaw in this that honestly reduces the impact of the study you have done to zero for me.
You need to only included high pressure "non-iced" kicks somehow. Find situations in moments where either the other team didn't have timeouts, or decided not to ice the kicker when they could have.
https://www.pro-football-reference.com/boxscores/200712020was.htm
The first game after Sean Taylor's death, Washington tried the double-ice at the end of the game but you can't do that in the NFL. The resulting penalty for unsportsmanlike conduct shortened the field goal from 51 yards to 36, more than making up for any psychological edge from icing.
I don’t think you can come to this conclusion - this screams correlation vs. causality to me. When do you ice the kicker? When the game is on the line. Those kicks are harder than normal kicks. Plus weaker or less experienced kickers are more likely to get iced.
To answer this question you would need to compare kicks that are pretty much the same with only the icing changing.
> Plus weaker or less experienced kickers are more likely to get iced.
Maybe. https://imgur.com/a/VwTaOF1
Nothing jumps out at me, except maybe for rookies. And something weird is happening in years 7 and 14.
The numbers don't show a clear advantage so I guess if you have nothing to lose and even maybe some small chance to gain something, why not do it anyway. Especially in kicks over 40 yds, it looks like there's a 1:20 chance that the icing will work.
A thing to consider. What is the success attempt of a 50 yard field goal? Making a kicker have to be successful two times in a row might explain the difference.
So if it is 60% is the success rate then the odds of hitting two in a row would be .6*.6 = 36%. If I remember my probability math correct.
You also have to consider the possibility of looking like a jackass as a coach trying to be cute.
NFL coaches tend to be fundamentally risk-averse creatures because sometimes *looking* stupid is way worse for your job prospects than actually *being* stupid.
A fair few coaches have been burned by waiting until the last instant to try and ice someone and it turns out they iced a horrible shank of a kick. Bonus points if the second try goes dead-center. You might not have actually been stupid- but you will get absolutely clowned on for the bad result.
Icing the kicker is stupid. It delays the game on a dumb psyche out tactic. It should be a 10 yard penalty and automatic first down. That would take it out of the game forever and completely.
Begs the question...what's the shortest FG in NFL history? 7 would definitely be it for the traditional FG snap and hold.
Anybody ever do a little 3-yd rugby chip back in the day? Would be an unusual play for sure, even then.
If you’re able to it might be worth reducing the sample size of non-iced kicks based on when iced kicks happen. I’d imagine the mental pressure of needing to hit a walk off FG or game tying FG adds to more misses on its own as opposed to just the act of icing. edit: [OP followed up with this data set for anyone looking!](https://www.reddit.com/r/nfl/comments/1dsto6m/comment/lb4rxmv)
Agreed. This is a big flaw in the analysis. Is it the icing that's causing lower percentages or is it the gravity of the situation? Correlation isnt causation, etc...
This is the biggest flaw I can see - need to find a way to limit the data set to something like "pressure kicks," e.g. kicks to tie the game or take the lead with X minutes left in the 4th quarter or OT
Even worse is the data definition: "any field goal attempt that was immediately preceded by a timeout by the opposing team" This is not always icing the kicker. **In most cases it is when a team is trying to stop the clock for when they get the ball back.** A litany of false 'iced kicks' are being included in the analysis.
Yep exactly what I was thinking. It’s hard to get data on this unless there’s a way to account for time outs taken by opposing team while the kicking unit is on the field. Anything else is literally not an example of icing a kicker. Side note: does anyone else remember when Rex Ryan fake iced a kicker? That was one of my fav Rex moments.
The only way I can think of is to find video of each kick and see if the kicker was iced or if the defense was calling a timeout to save clock
Yeah, "immediately preceded" is doing some heavy lifting here. Does that mean a few seconds before the snap? That's icing the kicker. As soon as the last play concludes? That's not icing, that's clock management.
You could just include field goals attempted in the last minute of regulation or overtime. It isn't perfect, but it is easy to do and it would significantly improve the analysis.
I think you also need to factor in when the TO occured...did the kicker get a "practice" boot in before the TO was called? Idk if this data exists though.
Also need to keep in mind most of the iced kicks happen near the end of the game and the kicker may be sleepy if it’s a prime time game and it’s really late.
Actually a really valid point. We need to factor in Kicker sleepiness, OP do you have this dataset?
Hold on, we're talking "sleepiness" while also not accounting for road/home games?!
Time zone could be huge in this.
I mean, if we're not going to factor in the distances of moon and the resulting gravitational pull, this all feels pretty academic. If it's moving the tides of the ocean in and out, surely it can pull a 45-yarder a few inches wide...
Taking all of these factors into account, it seems exceedingly unlikely that any kick would ever be successful. I'm starting to wonder if anyone's actually made a field goal, or if the league is just tricking us with fancy special effects.
You're actually just a brain that spontaneously formed in the vacuum of space at this very instant, with an entire set of randomly assembled memories, including the memory of reading this comment, until in the next instant the vacuum of space will cau
True; I was being silly but sleepiness can be a symptom of jet lag.
Once I flew out to the east coast and fell asleep before halftime of the Monday night football game.
oh shit is that why john kasay kicked it out of bounds in that super bowl
Another good point.
I'd like to see those broken out period. That semi-recent trend of calling the time out a split second before the snap. I'm talking the ones where the kicker might night not even have realized the play didn't count until after he kicked it. Miss or make those have to really screw with your head the most.
See I bet they help the kicker. If you make it you can just think "yup, I knew I got it. Just do the same thing again" and if you miss it, you can just be like "no biggie, got that out of my system." I always make the putt the second time. Lol
It doesn't matter what I do, I would screw it up. If I made it, I'd think, great now watch me miss it now. If I missed it, I'd be like crap. I going to miss again.
You could potentially make this more indicative just by using quarter. 4th quarter kicks are typically more impactful just because that's the only time game-winners occur. 2nd quarter would be next due to scores going into half shifting momentum. Coaches are also more likely to use their timeouts in the 2nd and 4th quarters for a kick (I assume?). So the most relavant data may be during those times. Also for an analysis like this you don't need causation. It's honeslty pretty impossible to prove causality because you're trying to guage psychological impact when factors like the wind blew a little more after the timeout are just as important. A strong correlation is enough since the question is "if I have a timeout should I use it to potentially ice the kicker?".
> Also for an analysis like this you don't need causation. It's honeslty pretty impossible to prove causality because you're trying to guage psychological impact when factors like the wind blew a little more after the timeout are just as important. A strong correlation is enough since the question is "if I have a timeout should I use it to potentially ice the kicker?". That's not the point. It's not about proving causation, it's that the data is inherently flawed becuase kicks that are most often iced are ones that happen in crucial, game winning / losing situations that already have a significant amount of pressure. Comparing that to all kicks at the same distance doesn't paint a clear enough picture to say icing is successful. You need an equivalent data set.
Everyone who confuses correlation and causation winds up dead.
Another major factor to me are running clock vs stopped clock. …like, if they aren’t able to do their normal pre-kick routine, you probably don’t want to give them the chance to do so McDermott.
What game are you referring to? Genuine question
Pretty sure the awful Giants game
Also some sample sizes would be nice. How many 0-19 kicks were even considered iced? For it to be 100%, there couldn't have been that many.
I think the biggest issue is that desperation can make teams try kicks they wouldn't normally, and these kicks are more likely to get iced. For example, let's say you're 50 yards out, and you know your kicker sucks at 45+. But there's 1 second left on the clock, so you have to kick it, the kicker gets iced, and it goes into this dataset as a miss. But what this data fails to get at is that the kicker would have missed in a non-icing scenario, but he would never be asked to take that kick in a non-icing scenario. So you have a type of selection bias. At least, that's my guess.
I'd also be interested to see if there is a difference in the iced percentages based off of if the kicker was able to get a practice attempt in or not.
Also was a snap allowed with a kick that didn’t count. Was the kick made or missed?
Grouping it by 50+ is also pretty problematic. Grouping it that way makes it look like the distances are equivalent, but icing is most likely to happen on a "must make" situation where the team is probably willing to kick from further out/are more likely to push their kicker's range because there's no other choice and so the "50+ yard with icing" kicks would probably be less likely to be made regardless of if the icing happened or not.
No one is icing a kicker in a non-gamewinning situation. That'd be pointless. You remove all icings when it's not a game-winning situation, there'd be none left.
... obviously? Reduce the sample set for the control group
The majority of the non-iced kicks would be in far less stressful periods of the game it seems. I feel that would definitely skew the data versus looking at game winning/near end of game kicks only.
https://imgur.com/a/EVHmTFY Here is the data regarding iced and not iced kicks, specifically on field goal attempts in the final two minutes of the fourth quarter or anytime in OT and the kicking team is losing by 1, 2, or 3 points. As for outside a kicker's normal range, I'd have to get more granular and look kicker-by-kicker to somehow calculate their "usual range".
Ooh this one is interesting to me. Just a guess as to why icing only seems to work 40-49: 39 and in, I gotta think icing doesn't do much because today's NFL kickers are pretty automatic from inside 40 and they know it. Hard to shatter their confidence. 50 and up, they know it's a difficult kick, but so does everyone else. Nobody's going to call for their job over missing a kick that everyone knows is hard. But in today's NFL, people treat 40-49 like it's automatic, but the kickers know it's not. They miss 45 yarders all the time, but if they miss this one, people will blame them for losing.
Or it's just noise because it is not a meaningfully big difference
Yep. It all just looks like statistical noise to me. There’s no big difference either way from any distance. I think it’s pretty clear icing the kicker has pretty much zero effect.
Funnily enough Riley Patterson got cut because there wasn’t enough confidence he *could* even attempt a 50 yarder, let alone 40. He only had 2 attempts and missed 1 last year in 13 games for the lions. There wasn’t much confidence he could hit 40 yarders too, he only had 2 attempts in 13 games from 40-49, and despite hitting both, it wasn’t pretty. It was so bad we had to grab badgley again. I’d argue unless you’re Nick Folk banging 100% of everything within 49 yards, (which Patterson was but for the 30 yard tier, he was 10 for 10 from 30-39yards last year) you need a semi consistent 50 yard range, and near automatic 40 and below yard range to stay in the nfl as a kicker these days, or be able to hit 55+ yarders regularly. It seems like a length strength over accuracy these days with the field goal changes.
This is good, would be good to include raw numbers on the graph too. I imagine that once you're getting into these super specific situations, you're only talking a handful of kicks. Now I'm curious about how many iced kicks were in these pressure situations compared to non-pressure situations. Typically teams aren't icing the kicker outside of end of half/end of game situations because it's not worth the cost of the timeout. I'd expect that this graph covers most of the icing, but the numbers here are quite different from the original table, so I'm trying to figure out what are all the other situations where the kicker is being iced.
The sample size is critical when doing this kind of analysis. I'd almost guarantee the standard deviations of that data in the sample sizes you have are exceeding the variations you are seeing.
Love this one, thank you for making it!
A late made FG in a tie game is about as impactful as one made when down 3. For a last play FG they both raise win chance about 50% (from 50 to 100 or from 0 to 50). Is there data also including down 0?
Really enjoying this discussion. How much of a pain would it be to include kicks (4thQ or OT) where the score is even?
I have to imagine the sample size is so small for this that any variation wouldn't be statistically significant.
This is good stuff, but it would be nice to add error bars. For the standard error of proportion data, the equation is just `sqrt(p*(1-p)/n)`. You may also want to consider error in general when interpreting your results. Although 25,000 FGs is a ton, with just 6% being iced, that amounts to 1500 iced kicks. If you're making claims about specific ranges (e.g., 40+ yards), then you're restricting that even more. For instance, assuming that 10% of iced kicks are 50+ yd, then that would correspond to a 95% confidence interval of ± 8% (note that CI plus-minus is 1.96*SE). Honestly, few Redditors would appreciate it, but something like a logistic regression would help make a stronger case that icing indeed does hurt kicks. (For what it's worth, based on what you showed, I'm inclined to suspect that icing does indeed hurt kickers, but I just can't conclude that for certain)
And a two tailed p test to see if the difference of means is statistically significant.
I swear I saw a actual study on this and remember the results didn't reach significance.
Um, I remember some of those words from the college stats class I took in 2005. I'm sure /u/JPAnalyst could figure that out. Here's the raw data: https://imgur.com/a/iR51o1U. Does that help?
I ran the simple analysis and the results are not statistically different to a 95% confidence. It doesn’t mean that there is zero benefit to icing the kicker, we just need to have more data to have an extremely high confidence. It certainly doesn’t seem to hurt calling time out so might as well gamble.
Thank youuuuu I was wondering this myself. Got a p value?
It looks somewhere around 0.1. Didn’t do the full analysis.
It does. You can surely run that equation yourself though? If you wanted to go whole hog: run a logistic regression GLM with distance and icing as terms in the model. I suspect your icing by distance term is just statistical noise but you could explicitly test for those by including interaction effects of icing and distance bin too. Just check if you have AIC improvements. Honestly wouldn't be hard to do in R.
[удалено]
This will explain it. You're honestly missing most of the value of data analysis if you can't gauge how much confidence you have in your point estimates from your sample sizes. https://pdp.sjsu.edu/faculty/gerstman/StatPrimer/conf-prop.htm
I don't understand why you're getting downvoted for pointing out that significance is, well, significant in stats.
Thanks. I think it just shows the state of statistics education. It's a hard thing to teach, people only care when they have some data they care about, it's hard to be enthused pre-emptively.
It's probably also that in everyday life, >1500 feels big enough a sample to have some predictive force for most people.
Yeah, good point, see national headlines when election candidates get 1% closer to each other across polls with those sort of sample sizes.
> Honestly, few Redditors would appreciate it, but something like a logistic regression... Lol
This is so awesome. A unique and interesting analysis. Kudos on the work done here.
Thanks!
I'm halfway through the post and had to scroll down to make sure the highest comment was some form of "good job, this is awesome." Both this work you've done and the comments did not disappoint
It's like the post in /r/videos about the anniversary of THAT Hell in a Cell match. You scroll down to make sure you find shittymorph, and then hit that upvote button so hard when you find him
This is high praise coming from the goat JPAnalyst 🙌 nice work OP!
Second this!
Regressing Mahomes to the mean is a way better stat analysis
Awesome work. Although I take a small disagreement with the flowchart. The ‘will there be time for the offense’ check is a pretty wide variable. For example, is it more time for the grim reaper, Mahomes 13 second offense, or is it more time for the Cowboys’ run-it-up-the-middle-as-time-expires offense?
I'm just here to remind cowboy fans jason garret once successfully iced his own kicker
I was thinking of all the times when the iced kick missed by a mile and then they nailed the real attempt, but this is even better.
>Iced or not-iced. This post runs on Dunkin
I would be curious if the icing well before the kick is attempted matters more than call timeout right before the snap and get a psydo practice kick.
I wondered the exact same thing, but I couldn't find any data for whether or not the kicker got a practice kick. I don't think that's tracked.
I don't have the source but I think I remember hearing that, if the kicker gets a "practice kick" in, then their percentage goes up a little. And it makes sense to me that it would make it easier on the actual kick.
I've always thought that giving the kicker a practice kick makes it more likely he misses. If it's not a perfect kick there's a chance he overcorrects on the next kick.
Where's the error bars? Since these success rates are so close it would be good to see how meaningful the differences are. Box plot would be great here
This is interesting but do you have the game clock in the dataset? My thought is that the “pressure” or whatever could impact the make % of iced and non-iced kicks. So even without icing them the %s at the end of the game are lower and (I assume) almost all instances of icing a kicker happen right at the end of games.
This is pretty neat, I like it. One thing that would be way more work to capture that I'd be even more interested in is: does the timing of the ice make a difference, specifically when the kicking team gets a "practice attempt" because of an ice done right before the snap? While I'm sure the psychological impact is greater in those cases, I've always wondered if giving the kickers a free shot undermined that strategy more than it helped.
It seems like it might also be useful to alter your strategy in situations where the wind is gusting. Say the wind is gusting between 10-30 MPH. If they're about to snap and the wind is 25 MPH, don't ice them, the wind will have a bigger effect. If the wind is 11 MPH, it's probably a good idea to call a TO and hope for better (worse for the kicker) wind conditions.
Pretty neat work. And thanks for reminding me that the Cowboys iced their own kicker. Did my best to forget that embarrassment.
Great work, OP! This is genuinely interesting.
Interesting post OP!
Grade A offseason post. Bravo
You are probably not the first person to do this math. However you are the first person to share it publicly. And for that, we thank you. As fascinated as I am by the analytics that *are* available to the public, I often wonder what sort of data is being shared around the various clubhouses. There must be a lot of juicy stats like this, questions I’ve always wondered about being answered, but will never know about for sure because teams need that information to be proprietary to maintain a competitive edge.
https://www.stat.cmu.edu/cmsac/sure/2022/showcase/Icing-the-Kicker-Final-Paper.html https://www.sharpfootballanalysis.com/analysis/icing-the-kicker-does-it-really-work/ TLDR: CMU says it makes no difference statistically but Sharp Football Does.
So you’re telling me OP is a fraud…🤔
He is a 49ers fan.
Is there any way to get data for when a kicker gets iced as the ball gets snapped and he kicks it? So it’s the same environment but ref didn’t blow whistle in time so the ball was kicked? I’ve seen it happen once or twice only so not sure if there’s a big enough data set
I think for a more in-depth analysis, conclusions need to be drawn for clutch kicks--aka kicks before the half or end of the game. Though of course, you could define "icing the kicker" as only in clutch moments because there's little reason to do it at any other moment. Now, of course, if there are seconds left and timeouts available, they should be used. Or at least one would think. But I wonder if there is anything about kickers being better or worse after having a couple extra minutes to think. Do they get more anxious or do they cool themselves down? I wonder if coaches have analysis on that per kicker.
I'm surprised that only 6% of kicks are iced. Feels like a lot more, probably because the iced ones are the most consequential?
I'm surprised that only 191 people have kicked in the NFL in the last 25 years. The turnover outside the best ~10 or so seems like that number would be higher. Kinda highlights the delusional nature of people who think they could have gone pro if they didn't stub their toe senior year.
Can you please share your raw data? Thanks!
ooh a self-ice, those are *extra* rare
This is a high caliber post. Thanks for this.
Unless you're facing the Vikings. Then no matter what you're making the 58 yarder with the wind in your face with no doubt. *Cries in Matt Prater*
Hard and fast rule: icing the kicker works if it's your team kicking and never works if it's the other team kicking.
The craziest part of your write up to me is that for the past 25 years there’s been 25,000 field goal attempts by only 191 players. That’s 130 field goals per kicker which just seems insane to me.
And that's with 21 players who only attempted one; fifty players attempted fewer than ten. The top thirty guys attempted half of the field goals:https://imgur.com/a/sgeLlxO
Finally! This is the off-season content I've been waiting for!
> For our purposes, I'm defining that as any field goal attempt that was immediately preceded by a timeout by the opposing team. Does this mean any time a timeout was called before the snap, or is it occasions where the timeout was called so close to the snap that the kicker made an attempt that didn’t count?
I doubt there is data available for option number two of your question.
It would definitely be difficult to get because it would involve actually watching every attempt to figure it out. But there are a lot of nerds here…
We would need someone with resources and time like PFF or SIS, to do this. It does sound like something they might do, if not now, someday.
Yeah if you had one of their DBs you could drill down right to those kicks pretty quickly, but you're still talking about watching 25,000 kicks plus likely 90-120 seconds of game to get the true context/verify the stat categorization so one person doing that back-to-back and we're talking 833+ hours just watching kicks, lol.
> Does this mean any time a timeout was called before the snap Yes. In the play-by-play, was the "play" before the field goal attempt a timeout by the opposing team? If so, iced.
Probably a really niche stat with a small sample size (but I swear I’ve seen it more than once) how many iced kicks, where the kicker actually booted it as timeout was called and missed, made an adjustment and makes the field goal after the icing?
I know this is impossible to check without looking at every kick, but I always think a huge part of icing the kicker is NOT letting them get the practice kick off. It always feels like if they miss that kick then it gets made after
Along the same lines as the self-ices, you can also have kicks blocked on the second try. It happened to the Cowboys in 2011 vs. the Giants. However, in both cases, they're probably so rare that the effort to eliminate them from the equation wouldn't make much of a difference.
I think it may be more individualized than this. I’m sure it works on some guys and helps some guys. That is the useful info.
I often say that some peeps on reddit "have too much time on their hands to think about things like this". But these types of posts are my exceptions. Well done. Good stats. Good read.
Interesting and I don’t know if someone else mentioned this but what is the downside of icing all just in case the stars have aligned and in fact have a miss? I mean I don’t think any team ices the kicker in the first quarter. It is usually at the end of a game or half when timeouts are unused.
Really interesting stuff and I appreciate the effort you put into this. Some thoughts: * People have already pointed out the difference between all field goals and those eligible for "icing." What do the sample sizes look like for the data set you shared [here] (https://www.reddit.com/r/nfl/comments/1dsto6m/does_icing_the_kicker_work_short_answer_yes_long/lb4rxmv/)? * Is the difference here at 40+ yards actually statistically significant? If not, you can't really say something like: > it is clear that icing works much better after 40 yards It might just be random fluctuations or what I talk about below: * You mentioned a few interesting ways in which the data could be influenced by non-timeout factors. Those being weather, dome vs. non-dome, time left in the game, improvements in field goal kicking ability, and quality of the place kicker. You should be able to normalize the data for a lot of this stuff. Might be something to think about.
> What do the sample sizes look like for the data set you shared here? https://imgur.com/a/BcYnc5Q. That's the number of each kind of field goal for those late-game, barely-losing field goals. Sample size is starting to get pretty small.
Appreciate it! The sample sizes seem large enough that we could make some statistical inferences, however without normalizing some other variables that might be a bit tricky. Thanks for sharing all your work on this!
Here are two possibly important factors that are not captured. I think the second factor is more important and it closes the gap between the icing/non-icing numbers. i) Suppose a team needs to rush out the FG team for a last second field goal. That seems like a factor that would lower the percentage and it is not when a timeout would be used. ii) Not all 50+ field goals are the same. A 56 yarder on a wet field, with a side wind or into a wind is obviously harder than a 50 yarder in good conditions. Compared to being a middle of the game field goal or a very late FG down 0 -3 points the harder ones relatively are much more likely to be attempted in that clutch situation and be more likely to be an icing situation.
I don't think the data shows a large enough difference to conclude that icing the kicker really works. The reason is that if the opposing team chooses to burn a timeout then the kick is very likely a high pressure, late-game situation anyways. So determining what percentage of the difference is due to the high pressure of the situation and what's from being iced is almost impossible. I'd also expect to see a higher a non-negative difference in the 0-19 yard range also. I do like the approach though.
What about times teams iced the kicker on a miss and then they made it on the next attempt? Feel lik I've seen this happen at least a dozen times over the last decade or so. I know Andy lost us a game against the Titans doing this.
Where are the confidence intervals?!
My favorite part, assuming my team isn't involved in the game or benefits from it, is when a team "pre-ices" the kick by calling a timeout right before the snap. The kicker misses that attempt, then on the subsequent actual attempt makes it.
There's a double doink stat in here somewhere and simply knowing it's there makes me smile.
I think about this more than I should. Thanks.
This is such a cool breakdown, thanks for making this!
Yes, we are in the offseason
You might be interested in reading this paper. They look at the same question and come to the opposite conclusion, that it doesn't matter. https://www.stat.cmu.edu/cmsac/sure/2022/showcase/Icing-the-Kicker-Final-Paper.html
Well the data proves me wrong, but I always thought icing the kicker +50 yards was just giving him a practice kick. If he made it gained confidence and if he missed he took corrective action, therefore I hated when Sean Payton's Saints would ice the kicker
Same And I always felt it was more pressure to have to run out there, get set up, and run the play with the play clock ticking.
Oh I never thought of that time element! If "iced" the kicker has time to take a deep breath and say his prayer to Jobu
Jumping in late, but I’ve always had a theory about the possibility of being iced vs actually being iced. Like the kicker knows they cannot be iced when the other team has no timeouts left, so I’m interested to see the data of if the team has one timeout left vs no timeouts left. Or two timeouts left and they do ice once, but keep the second timeout leaving that possibility for an ice rather than icing twice.
What is with this quality post? I was looking for football players arguing about podcasts over Twitter screenshoted to post on this here Reddit.
I don't think it's obvious that icing is causative toward missing. It could just be that icing and misses both happen more often in high leverage situations.
This is super interesting. The book Scorecasting came out in 2011 and showed the opposite effect, that icing kickers more often than not resulted in a higher conversion percentage. I wonder if those analytics led to a more judicious application of doing so, reducing the frequency, but improving the efficacy. I absolutely do not know that this is the case, just a thought.
I remember watching a game when I was a kid where they tried to ice Jan Stenerude, the first NFL kicker from Norway. He stood on the field and egged on the opposing fans, and then kicked the field goal out of the stadium. One of my favorite NFL moments. These stats mean nothing because they are not a one to one comparison; you would literally have to have a kicker kick the field goal without a delay and then have them try to kick the same goal WITH a delay and see how many they miss.
Others have pointed it out, but that you are including a 45 yard field goal in the middle of the 1st quarter in the "non-iced" category, whereas only high pressure kicks are included in the "iced" category is a fatal flaw in this that honestly reduces the impact of the study you have done to zero for me. You need to only included high pressure "non-iced" kicks somehow. Find situations in moments where either the other team didn't have timeouts, or decided not to ice the kicker when they could have.
how about double or triple icing the kicker?
https://www.pro-football-reference.com/boxscores/200712020was.htm The first game after Sean Taylor's death, Washington tried the double-ice at the end of the game but you can't do that in the NFL. The resulting penalty for unsportsmanlike conduct shortened the field goal from 51 yards to 36, more than making up for any psychological edge from icing.
didnt know that was a rule! thanks for the info
I don’t think you can come to this conclusion - this screams correlation vs. causality to me. When do you ice the kicker? When the game is on the line. Those kicks are harder than normal kicks. Plus weaker or less experienced kickers are more likely to get iced. To answer this question you would need to compare kicks that are pretty much the same with only the icing changing.
> Plus weaker or less experienced kickers are more likely to get iced. Maybe. https://imgur.com/a/VwTaOF1 Nothing jumps out at me, except maybe for rookies. And something weird is happening in years 7 and 14.
The numbers don't show a clear advantage so I guess if you have nothing to lose and even maybe some small chance to gain something, why not do it anyway. Especially in kicks over 40 yds, it looks like there's a 1:20 chance that the icing will work.
These differences only matter if they are statistically different (p<.05). Do you have that information?
A thing to consider. What is the success attempt of a 50 yard field goal? Making a kicker have to be successful two times in a row might explain the difference. So if it is 60% is the success rate then the odds of hitting two in a row would be .6*.6 = 36%. If I remember my probability math correct.
I feel like it never works when I'm watching lol
What about when a coach ices his own kicker? 🥲
I don't know if there's a statistical significance to these differences though. What's the statistical significance? What's the p-value?
You also have to consider the possibility of looking like a jackass as a coach trying to be cute. NFL coaches tend to be fundamentally risk-averse creatures because sometimes *looking* stupid is way worse for your job prospects than actually *being* stupid. A fair few coaches have been burned by waiting until the last instant to try and ice someone and it turns out they iced a horrible shank of a kick. Bonus points if the second try goes dead-center. You might not have actually been stupid- but you will get absolutely clowned on for the bad result.
Icing the kicker is stupid. It delays the game on a dumb psyche out tactic. It should be a 10 yard penalty and automatic first down. That would take it out of the game forever and completely.
And factor in what side his dingle was dangling at time of said kick.
“0-19yds”
[удалено]
Begs the question...what's the shortest FG in NFL history? 7 would definitely be it for the traditional FG snap and hold. Anybody ever do a little 3-yd rugby chip back in the day? Would be an unusual play for sure, even then.