At EURODECISION, we increasingly design neural network-based AI models (e.g., machine learning) for our clients. These AI models are often viewed as “black boxes”. Metrics such as precision, recall, f1-score, r2-score, etc., enable the performance of these AI models to be verified. However, our clients regularly ask us to explain how they “reason”. With a model, for example, based on Linear Programming (an explicit system of mathematical equations), it is fairly straightforward to understand how the model functions, to explain why it produces a particular result, or to demonstrate that it generates solutions aligned with those of the human expert. Using a neural network-based AI model, performance metrics indicate reliable results, but it is difficult to understand how it arrives at them. Our pragmatic clients need a better understanding of how their AI model works to ensure it “reasons” correctly.
To address this need, we are therefore investigating explainability methods for neural network-based AI models. Several explainability methods exist, such as SHAP, LIME, and Integrated Gradients. These methods perform well on relatively simple prediction models, but what about industrial-scale models like those of our clients?
To answer this question, we applied the SHAP (Shapley Additive exPlanations) explainability method to an AI model that predicts picking times in a warehouse based on product slotting. Picking is an order-preparation method that involves collecting, in an orderly manner, items from different locations in the warehouse for several orders. Slotting is the process of determining each item’s location in the warehouse. Slotting optimisation reduces travel time during picking and congestion in the warehouse.
The image below shows the warehouse layout plan, with slotting represented by a colour gradient: dark for high-frequency picking items and lighter for low-frequency picking items. High-frequency items are generally placed near the dock to minimize travel, but they are also separated to avoid congestion.
The SHAP explainability method is based on computing and analyzing Shapley values. These values indicate the contribution of each explanatory variable in the model to the predicted variable. In our “Slotting” AI model, this gives us the contribution of each item-location assignment to picking travel time.
Standard graphical representations of SHAP values enable a data scientist to analyse and verify the effectiveness of this AI model. For example, in this graph, each line represents a numbered location, and each point indicates the positive or negative contribution of that location to picking travel time. Thus, when a high-frequency item is assigned to location 140, the travel time to location 43 is reduced.
Unfortunately, these graphs do not make it easy to explain to our clients how an AI model functions. It is often necessary to present these SHAP values in a more meaningful graphical format for the client. For this “Slotting” AI model, we repositioned the SHAP values on the warehouse layout plan. Locations in blue indicate those that reduce picking travel time when a high-frequency item is assigned to these locations, and vice versa. Viewing this type of business-oriented graph, the warehouse manager, who is not a data scientist, can more easily verify the proper “reasoning” of the AI model.
This example demonstrates the utility of explainability methods in making an AI model’s operation comprehensible. The standard outputs from these methods are primarily intended for data scientists. To present the results of these explainability methods to our clients, it is often necessary to translate them into graphical representations closer to their business domain. All of this facilitates the adoption of AI models in industry.