In the introductory article of this series, we described the broad scope of factory planning problems. We also presented the key needed abilities and complementary characteristics of heuristic algorithms that are typically used by Advanced Planning Systems (APSs) in order to provide automated solutions to such problems. These abilities and characteristics are summarized in the following graph.



heuristics abilities and characteristics
In this article we focus our attention on the two key abilities that a heuristic algorithm should always deliver; the feasibility and the good solution quality.

The feasibility of the solution strongly depends on the level of accuracy that we use when we define the factory planning problem. Inaccuracies in the problem definition will result in smaller or bigger infeasibility issues. For example, if an operation lead time of 5 hours and 5 minutes is aggregated to 5 hours, a corresponding infeasibility of 5 minutes will appear in the solution in all cases where this operation is active. The functional and technical owners of the solution have the responsibility to define and fine-tune the required level of accuracy, taking into account the capabilities of the heuristic solver and the acceptable infeasibility levels.

Inaccuracies may also appear because of outdated problem descriptions. For example, if a new shop-floor constraint is left out of the factory planning description, this will result in a model inaccuracy and could possibly lead to infeasibilities in the solution. Good communication and proactive maintenance of the factory planning model will minimize such issues.

Note that the heuristics used in most modern APSs provide a sufficient level of detail to allow for the accurate description of complex industrial size planning problems. However, approximations may be necessary for scheduling requirements that fall outside or are at the boundaries of the planning scope. Such approximations lead to infeasibilities that most of the times are considered to be acceptable from a planning perspective and are typically resolved at a more detailed scheduling/sequencing phase. 

The solution quality usually refers to the ability of the heuristic to find solutions that are acceptable to the business user, keeping in mind that by definition, a heuristic is meant to provide “good” but not necessarily optimal solutions. Note the vagueness of this definition; a business user decides on what is acceptable or not based on several subjective criteria, including the following: 

A.    Quality of an existing system or process

The solution quality delivered by an existing system or process that the new APS is supposed to replace could be used as a quality indicator.  Indeed, if the existing system or process is of adequate quality, then the business users will probably have high expectations from the heuristic of the new APS. Unfortunately, in most cases, companies decide to go through the process of investing on a new APS exactly because the old system was inadequate!

B.    Company’s Key Performance Indicators (KPIs)

Broad company expectations set at a management level are sometimes used as a guide for judging the solution quality of the APS's heuristic. However, this process may be inaccurate since company-wide Key Performance targets are only achieved as a result of a gradual, coordinated effort of the business. Consequently, although the APS has a significant role to play in this process, there are several other factors which are not technology related and affect the company’s KPIs, including the people, the business processes, the market, etc. Actually, in large scale implementations of advanced APSs it is common to use different terminology when talking about the company’s KPIs and the specific performance indicators of the planning solution, sometimes referred as Plan Performance Indicators (PPIs). PPIs are measured and fine-tuned on a regular basis by the corresponding planning department, unlike KPIs which are typically measured on longer intervals and on a company-wide level.

C.    “Best” industry practices

Another category of subjective criteria could be derived from publicly available information of “best” industry practices. Most of the times, such practices cannot be directly transferable to a company’s planning process, due to the specific individual challenges that each company faces. In addition, such information is usually available only on a high level and it is difficult to decipher it on an APS process level. 

To summarize, most of the times we do not have a clear, objective indication of how good the solution quality of the APS’s heuristic really is. We may however have hints that it is underperforming where one or both of the following characteristics appear:
  • Complex, non-decomposable manufacturing processes significantly increase the complexity of the resulting planning problem. Some characteristics to look for are critical resources that are used in multiple operations, intervening production routings with re-entrant flows, low availability resource calendars, etc
  • The consideration of multiple conflicting objectives is usually problematic even by the most advanced heuristic algorithms. For example, although good performance could be expected when our only objective is to minimize order lateness, solution quality issues arise in cases where minimum capacity utilization in critical resources is also required or in cases where inventory minimization in critical locations is an additional objective.
  • Particular needed functionalities that have global optimization characteristics could be in conflict with  the nature of a heuristic algorithm. For example, some heuristic algorithms decompose the planning problem into two steps, a material allocation step, where material is allocated and manufacturing orders are created, followed by a capacity balancing step, where capacity overflow issues are resolved. Such heuristics cannot handle complex alternates selection requirements where both material and capacity need to be considered simultaneously.
  • The solution quality also tends to deteriorate as the size of the planning problem increases. This is particularly true when one or more of the previous characteristics are also present, meaning that the complexity of large sized problems scales up radically.
An interesting point that we encountered in our projects is that the three characteristics that heuristics usually possess meaning simplicity, configurability and speed typically tend to postpone the moment of truth, meaning the moment when we really conclude that we have performance issues with a heuristic solver. Because of the simplicity of the heuristic solution algorithm, people may underestimate the complexity of the underlying problem. Also, due to its high configurability and speed, it is not uncommon to go through a long process of fine-tuning. Although such a process may be beneficial in some cases, we should be careful to catch early signs of underperformance and take necessary actions before wasting too much time and before people’s confidence on the success of the APS implementation is breached. Typically, the unquestionable moment of truth arrives when we reach the threshold of achieving minimal improvements after performing an exhaustive number of tests.

At the end of this article we ask two fundamental questions:
  1. Is there a way to accurately measure the solution quality of the heuristic used by a modern APS in an industrial size application?
  2. Could we reach the ultimate goal of improving the delivered solution quality?