This article previously appeared in the Society for Workforce Planning Professionals online journal, On Target. They appear here with the gracious permission of SWPP and Vicky Herrell.

Watch out for folks with PhDs

Years ago, I attended a session on forecasting with a focus on forecast error.  The speaker was a college professor with real expertise in statistics and forecasting. The session was very interesting and well done. But I asked him a question that was not meant to be a gotcha: “Does your error metric change, based on how you’ll use the forecast?”

I had to explain: “Doesn’t when your forecast is in error matter, given that you will use it to figure out when to hire, for example?”  He still didn’t understand it, and it became clear that his expertise was in generating time-series forecasts, but not actually using his forecasts. When a forecast is in error could be of extreme importance to us; missing a volume forecast at peak is much more important than missing a forecast in a valley, if we use the forecast for budgeting. A weather forecast that underestimates the temperature a few degrees every day in Summer is not a problem.  But missing the snow storm once is a big freaking deal.

But this got me thinking.  I don’t think that standard statistical methods consider too much about its practical application. Shouldn’t we contact center folks measure forecast error by how much it would overstaff or understaff our operation? Our volume forecast error should be measured in FTEs, not contacts; that is what matters to us.

What does forecasting math really try to do?

When talking about this with our forecasting whiz, Naman Doshi, he had a keen observation:  Forecast math tries to mimic what we, as humans, do quite well-- pattern match. When a forecasting algorithm searches for a trend, it steps through a bunch of math steps to try and find the slope of that trend.  If it is looking for seasonal peaks and valleys, its algorithms are trying to find the amplitude and period that matches the historical data.

We are naturally good at this; our eyes and brains are very powerful.  We can look at a graph and often know what will likely happen next.  The math, the Holt-Winters or the exponential smoothing or whatever, has no information our eyes don’t see.  It is not finding hidden patterns that aren’t available to you.  It is faster, and it certainly informs our judgement, but it isn’t necessarily better.

Look at the historical volume data below in Figure 1.  This graph represents three years of volume history.  Without saying any more, I bet that if I were to hand a pen and a printed version of Figure 1 to everyone reading this paper, we could draw out a forecast, and would get pretty dang close to the logical obvious pattern. Your kid could draw it correctly.

Figure 1.  Three years of contact volume history

We can spot problems

Figure 2 graphs the previous history but adds a forecast, derived using a form of exponential smoothing. While it appears that this forecasting technique got the trend right, it missed all the regular peaks and valleys. We know, without even doing any calculations, that this forecast is wrong.  We can see it.

Figure 2. Three years of volume history with one year of forecast

Mathematicians may still defend such a forecast, saying this or that about the error within hold-out samples and what-not. But we can still look at it and know it is off.

We need to be able to stand our ground and say, this forecast missed the normal lull in February, or it missed the hype surrounding the football season, or the Tulip festival in The Netherlands. We know stuff the math doesn’t, and we should use our pen and paper to make those adjustments.

Confidence intervals

Confidence intervals are clever things, they give you a range within which the math believes the true value (say, volume) lies.  When forecasting, it would say I am 95% sure that the actual volume will come in between X and Y for every week into the future.  It is meant to be a measure of variability to some extent.  It is clever because it can give us a false view of the range of likely outcomes.

Figure 3 shows us confidence intervals around our sample forecast. Most often, forecasted confidence intervals start very narrow, but get wider the farther out they go.  This makes sense. We are better at forecasting the next month than we are at forecasting nine months from now.

Figure 3.  Forecast with 90% confidence intervals

We often think of confidence intervals the wrong way.  It is natural to think that within an interval every data point is equally probable, meaning that the actual value that occurs could be someplace in the middle or it could be at the outer ranges, who knows?  That is not what it means. It just means that you are 90% confident the real point will be somewhere within the range.  But our natural inclination to think that all are equally probable makes showing confidence intervals less worth showing.

When looking at your forecast (say, the very first forecasted points in Figure 3), you can look at a point on the bounds of the confidence interval and definitively say, “no way that our volumes will be that high unless there is a major shift in the business”. In other words, your considered judgment says you have no confidence that the volumes will be near the edge of the confidence interval.  But your math says it is 90% confident that that edge point is in the range of possibility.  The forecasting math, by the way, doesn’t consider that there is some change coming at all, it only considers your history, it knows nothing else. 

We have no confidence, but the math implies it does.

What do you do with them?

In the absence of certainty, we still need to make decisions about hiring, contact routing, outsourcing, and the like.  How do confidence intervals inform our decision-making?

Should we pad our staffing? Should we keep an overtime budget in reserve just in case? Being risk-averse, this is what we would be tempted to do with this sort of information.

There has been a movement to overstaff our contact centers and use either technology or early leave to manage the overage in real-time.  We are still undecided about the wisdom of this, it sure feels like it would be easy to accidentally or purposely overstaff.  For highly seasonal businesses we will often be overstaffed naturally.  Padding seems like overkill.  Frankly, we are not convinced that always hitting service levels week over week is a good policy either.

One last point about confidence intervals.  If your confidence interval is very wide, like they often are, how does it help anyone? It seems to say “Look at how unconfident we are in our forecast. Look at this wide range. Forecasting is hard!”  It does not inform our decision-making.

There is a big difference between statistical confidence and our real-world judgment. We are paid for our judgment.

Why the disconnect between math and judgment?

Some things.  The math only knows what your history knows.  It has 156 data points, that’s it.  But you know that marketing is dropping a big mailer the first week of June.  You know that the first six weeks of the year is your enrollment period.  You know that Congress is considering major changes to your lending rules.  You know that the company has initiatives to drive volume to self-service.  You know management is considering new technology to reduce handling times. You know there is an election, or a home football game, or three new cruise ships coming online. You’ve seen it all before.

Any forecasting system that shows a crazy spread in confidence was programmed by math guys who do not understand your business problem, they only know the math. 

Takeaways

• In no way do we believe that you should not use algorithms to build your forecasts.  Quite the contrary, they are extremely helpful.  But we wholeheartedly believe you should look at your history with the forecast and ask, does this make sense?

• Graph everything and look at it. Don’t ever be afraid to manually change a forecast, you have more information than the math does.

• If you see with your eyes that a forecast is wrong, it is.  Don’t accede to math mumbo jumbo.  You are right and the math is wrong.

• There are data points in your forecast that are important to get right. There are points where it is unimportant that you are wrong.

• If you want to produce confidence intervals, go ahead and plot them out.  Then giggle to yourself and never show another soul.

Ric Kosiba is a charter member of SWPP. Chris and Ric are both founders of Real Numbers, a contact center modeling company.  Ric can be reached at ric@realnumbers.com  or (410) 562-1217, Chris is at chris@realnumbers.com.  Please know that we are *very* interested in learning about your business problems and challenges (and what you think of these articles). Want to improve that capacity plan? You can find Ric’s calendar and can schedule time with him at realnumbers.com.  Follow Ric on LinkedIn!  (www.linkedin.com/in/ric-kosiba/)