Forecasting excellence – without the crystal ball

Forecasting excellence – without the crystal ball

9 years ago 0 0 1774

I have been working with more than 30 business leaders from around Europe, the Middle East and Africa at a meeting in Barcelona. Our aim was to tackle some of the ‘big issues’ in strategy, marketing and sales, so that they could take back practical and powerful tools to their organisations.

When we asked them what they wanted (or more specifically: “What could you take away from this meeting that would have a significant impact on your business”), one answer shone through strongly – most of them wanted to help their key personnel within their organisations to forecast better.

I therefore created a hands-on workshop session for the attendees which focused on this topic. It reminded me that, amongst all of the important business skills that leaders and managers are taught, forecasting is often one which often gets less time, effort and attention than other topics.

Following a practical case (teams had to forecast 3 years of data based on a marketing plan), the results were collected and the variation in results was 5:1 (highest values to lowest) – with exactly the same data, this level of variation existed in the forecasting of senior, highly experienced leaders.

Imagine the impact this could have on a real business – if senior executives are missing the mark – over or under – by a factor of 5!

How are your forecasting skills?

From this session, and in terms of general best practice, forecasting is something which should be approached strategically, and with an appropriate process.
The role of forecasting:
Forecasting is a critical element in Marketing strategy and planning. Having an accurate idea of what a product or service may achieve in sales in a market allows appropriate investment decisions to be made, which together directly influences the product’s P&L.

Forecasting is reliant on the completion of an excellent marketing strategic plan – what is captured in the plan feeds directly into the process steps of forecasting to allow for the most accurate assessment of what may be possible in a market. It is critical for management level discussions around resource planning and go/no go decisions for products and product launch viability. A well-constructed forecast allows intelligent discussion and appropriate decision making on behalf of leadership.

Poor forecasting, and poor use of forecasting, has cost companies dearly. Many have been put out of business because the forecasting process failed them. Over investments are made in market entries, opportunities are missed, production and supply chains are over-utilised (or stretched to catch up), decisions about people are made and the expectations of investors and shareholders are set (and trust breached) when forecasting is performed poorly.

Forecasting is not a political or a power exercise. It is a critical business function that requires effort, intelligence and accuracy.

The problems with forecasting:
By its very nature, forecasting is about describing something which does not yet exist – sales projections that happen in the future. There is therefore no right or wrong in forecasting; only being as thoughtful and accurate as we can be. No matter what information we use, forecasting involves uncertainty and some judgements and guessing. We simply cannot know.

A big error occurs when forecasts are set but not seen in the context of the uncertainty they are created in. It is critical that the insights, assumptions and forecasting processes are chosen are clearly defined as part of the whole forecasting process to ensure this correct context is maintained. That is, a forecast is incomplete without the methodology and the assumptions made to generate it. Problems also emerge when forecasts are created to support a political or financial target, regardless of their connection to the data, insights and realistic assumptions available. This often happens when there is pressure for certain results, or ‘cut offs’ are established regarding sales projections for product launch, etc.
Understand the market situation and dynamic
To forecast what will happen in a market, we first need an understanding of what is currently going on there. To do this, we need quality information about the market that we can draw insights from, and to model the market as it is today, and its recent historical trends.

As we collect information, it is useful to break this down and accumulate it so that we can ask “What insights can I draw from this information”.

To help this process, insights (what we have learned from the information, rather than the information itself) can be categorised into 5 key areas for analysis:

  • Context – What insights do we have about the market – the context into which we will sell our product or service?
  • Customer – What insights do we have about the customer – the person or people who buy what we sell?
  • Competitor – What insights do we have about competitors, future competitors and indirect competitors and how they may affect or respond to our efforts?
  • Capability – what insights do we have about our product, our capability, experience, skills in that specific market segment, relative to the customer and the competition?
  • Consumer – What do we know about the people who consume what we sell? How and why do they consume it?

These insights provide the basis for making critical decisions for marketing the product, but also feed into expectations regarding forecasting.

Market modelling:
Besides the development of insights, it is critical to model the market. We do this to the best of our ability, with the data that is available, using proxies for data that we cannot get, and being clear when we make critical assumptions in the model. We can use a number of approaches, such as a population based model to look at overall potential, as well as a market sales model approach to understand sales performance against that potential.

If we model the market for a period of time (say, the last three years), we are able to see trends emerging which can be used as the basis of thinking about how the market will change in the future (forecasting the market). This ‘time series’ approach is the basis of most first level forecasting approaches.

The more data points that we have, and the finer the granularity of the information that we use, the more accurate our models will be. The accuracy of the historical nature of a market has NO bearing on the accuracy of your forecast, but it can significantly help.

Creating a waterfall analysis of the customer or consumer buying process allows a clear representation of the value potential of each step to be clarified. Using the waterfall analysis to describe the market, and the sales potential, assists in making strategic decisions, and also allows them to be ‘sanity checked’ later in the process. If we suggest that there are 1000 customers total available at one point in our process, and that by taking an action we will grow our share to 1500 customers, we can see that somehow the forecast doesn’t fit with reality. (If other things change, this could be possible, but would be known and included in our modelling!)

Strategy Options and decisions:
As part of our strategic planning process, we decide what potential actions we could take to change the behaviour of the market – and maximise our return on investment. For example, we could decide to directly attack a competitor product, to take share. We could target new entrants into the market, and get in first. We could grow the number of entrants to the market, and take our share of that. Each of these is a strategic option, and a strategic choice.

Strategic choices need to be taken, and the strategy that we choose determines what will be possible to achieve in that market. If we say we want to get customers to switch from product X, then if product X has 10,000 customers, then we know the limits of what may be possible. Based upon the insights, we can then make assumptions of what we could achieve within those limits (customer perceptions, capability assessments, competitive analysis, price elasticity, etc.)

Building a forecast requires a sequential appreciation of these strategic choices. (“First we will do A, then we will do B, in 12 months we will start C, etc”). This allows a forecast to be connected to the insights of the market, connected to our strategic choices, and can be progressed from one opportunity to another as the product or service evolves in the market.

There are always the opportunity for spill-over. For example, whilst you are employing a focused strategy to achieve one thing, other customers that you are not targeting may be engaged and purchase anyway. For example, if I was selling a ‘smart watch’ targeting Hipsters, I could calculate the limits of sales possibility, make some assumptions, and set a forecast. However, whilst I was busy advertising to ‘Hipsters’, some executives, empty nesters or grandmothers that ride Harleys might also notice and be attracted to the watch and want to purchase it. These are ‘spill-over’ because they were not the target of my marketing approach, but we gain their sales anyway.

It is best not to forecast for such spill-over, but rather forecast in line with your strategy, and accept this as a ‘bonus’. If it is clear that there will be a big spill-over into a different customer segment, then include this in your insights, build it into your modelling and expectations, and it can flow into your strategy and therefore into your forecast.

Selecting a forecasting approach:
There are many methods to forecasting, from simply guessing, to using algorithmic approaches on big data to develop and refine predictive modelling.
There are a couple of questions that you need to ask yourself before you choose a forecasting approach:
• What level of data do I have at my disposal? What volume, granularity and accuracy does it demonstrate?
• Who and what am I forecasting for, and what level of accuracy is required?
• What is the cost/return for the level of accuracy that I need?
• What may change significantly in the market that may make the forecast worthless? (a regulatory change, an expected power competitor, etc)

Based upon the answer to these questions, you can employ a number of different methods:

  • Qualitative, Judgemental or Naive methods: When there is little data, or low need for accuracy, you can use qualitative methods by simply making a reasoned judgement, or can take a naïve (high level) approach to providing a forecast.
  • Simple estimation, especially time series methods: The next step is to use historical data as the basis for predicting the future (time series) with an ‘estimation’ (judgement call) of things that will occur or change. The core assumption is that historic trends that can be discerned from the market will continue into the future. This may or may not be true, but it is the underlying assumption of time series forecasting.
  • Causal methods: By determining the key underlying factors that drive the market, and breaking the analysis down into the actions and responses on each of these factors, we can begin to predict cause and effect. By using time series, insights and some judgement, we can model the impact of these actions and amalgamate the result into the forecast.
  • Algorithmic methods: Uses predictive modelling of big data (which many markets do not have available or organisations do not have cost effective access to) to take causal methods to more precise application. Sophisticated programming algorithms are applied to data mined from a range of sources and used to predict behaviour. Often, these models are responsive, and adapt to new information and results. Algorithmic forecasting can be seen when you buy something on Amazon, and it offers ‘suggestions’ (really predictions) of what a customer like you may want to buy. Amazon get 30% of their sales from such approaches. These approaches can be used to model finely granularised data and create predictive outcomes, which can be modeled over time to provide a forecast.
  • Probabilistic methods: These approaches use a risk-adjustment strategy. By estimating the likelihood that an outcome will be achieved, the value of that outcome can be ‘tempered’ by that probability, which provides a result for forecasting. For example, if we set a target of getting 50 new clients at $20.00 each, but we estimate the likelihood of that occurring (based upon insights we have) at 80%, then we would forecast 50 x 20 x 80% = $800.00 (rather than 1000). This is a good way to account for risks in your forecasting process.
    Selecting a process:

By asking yourself the questions, you will know which method is right for you. If you have massive data, data mining and algorithmic programming skills, then go for an algorithmic approach. I the real world, a combination approach of causal methods, built on time series, with some judgement of assumptions, modified with probabilistic approaches serves as a good basis for deriving an intelligible and (hopefully) accurate forecast.

Where your market is likely to encounter significant change, or success depends on factors which may or may not eventuate, then a case model is also useful. Each case describes a potential path (with registration of X, without registration of Y; If new law passes, if it doesn’t; etc).

This allows review by management of the different possibilities and contingencies, and make investment and other decisions based upon this. Probabilistic methods can be used in case methods, as each case contains events that may be modelled for likelihood of occurrence.

Keys for forecasting
Forecasting excellence relies on:
• Developing appropriate insights and market models.
• Creating a consumer flow model and waterfall to describe the market
• Describing the potential strategic choices, and making assumptions about their impacts within the limits prescribed by the market model. Sequencing the strategies to create the basis for the forecast.
• Using an appropriate methodology to construct the forecast – depending on data, your needs and the cost benefit of your approach.
• Ensure that your forecast includes all of the assumptions that you have made, and why you made them. Also include your methodology, models, insights and strategy choices. This is critical, because a discussion about a forecast is really just a discussion about the assumptions, methodology and model. The number that drops out the back end is just the result. Without this, it is not a forecast, just a guess. It may as well be a horoscope.

How are the forecasting skills within your organisation?

Do you have a standardised approach? Are your key business leaders trained and engaged in forecasting excellence?

If you want to review your forecasting approaches and how this fits into your strategic marketing approaches, please drop me a line.

Leave a comment