Process mining is an analysis technique that uses data from IT systems to show how a process really runs, rather than how it should run according to the procedure. Software reconstructs the process from the traces every action leaves in systems, including all detours, repetitions and waiting times.
How does process mining work?
Almost every system, such as an ERP, CRM or case management system, records what happens when. Such a record is called an event log. For process mining you need at least three pieces of data per event: a case number (for example an invoice or customer number), the activity and the timestamp.
From thousands of such lines, the software automatically draws the process, with how often each route occurs and how long each step takes. The image at the top of this article shows an example: in the purchase invoice process, 31% of invoices turn out to go back for data corrections. That rework appeared in no process description at all.
The method was largely developed in the Netherlands, by Wil van der Aalst and his research group at Eindhoven University of Technology, from the late 1990s.
Three uses
This is one of the tools you learn to apply in the Green Belt training.
- Discovery: automatically draw the actual process from the event log, with all its variants.
- Conformance: compare the actual process with the prescribed one. Where is a step skipped, or a check bypassed?
- Enhancement: add lead times, waiting times and volumes to the process model, so you can see where the bottlenecks are.
Examples
- Customer service: a telecom company can see how often customers contact it again about the same question, and through which channels, and so tackle the causes of repeat contact.
- Purchasing and finance: an organisation sees how many invoices are corrected, how long approval takes and which suppliers are paid late.
- Healthcare: a hospital can map patients’ paths through departments and see where they wait for tests or results.
Software
- Celonis: a widely used commercial platform for large organisations.
- Disco by Fluxicon: a user-friendly tool, born from the research in Eindhoven.
- ProM: an open-source framework from Eindhoven University of Technology, widely used in research.
Many ERP vendors and analytics packages now also offer their own process mining features.
Advantages
- A view of the real process: objective, based on all cases instead of a few interviews.
- Finding improvement opportunities: waste, repetition and bottlenecks become visible, and with them opportunities for process optimisation.
- Checking compliance: deviations from rules and procedures come to light.
- Continuous monitoring: after an improvement you can keep measuring whether the effect lasts.
Disadvantages and pitfalls
- Data quality: incomplete or inconsistent data from different systems gives a distorted picture. Preparing the data often takes more time than the analysis itself.
- Complex models: large processes quickly produce an unreadable “spaghetti model”. So start with a clearly scoped question.
- Cost and time: licences, integrations and training require investment.
- Privacy: event logs often contain personal data. Make sure they are processed in line with the GDPR, for example by pseudonymising them.
- Resistance: employees may feel monitored. Be open about the aim: improving the process, not judging people.
- No context: the data shows what happens, not why. For that you still need to talk to the people, at the gemba.
Process mining and Lean Six Sigma
Process mining and Lean Six Sigma complement each other well. Process mining quickly provides a factual picture of the process, which helps above all in the Measure and Analyse phases of DMAIC. Lean Six Sigma provides the approach to understand the causes, implement improvements and sustain the result. In the Control phase, process mining lets you keep checking whether the improvement holds.
Conclusion
Process mining gives an objective picture of how processes really run, and reveals improvement opportunities that would otherwise stay hidden. Make sure you have good data, a clear question and attention for the people in the process. Then you get the most out of it.