Getting event data out of a system
This page is written to be forwarded.
The request
Section titled “The request”I need an event log export for process mining. One row per event, as CSV, with at least three columns: a case identifier (the thing being processed — e.g. the order number), an activity name (what happened, as a short readable label), and a timestamp (when it happened, with time of day, in a consistent format, ideally ISO 8601 with a timezone). Please also include any attributes that are cheap to add: the user or team that performed the step, the site or region, and one or two case attributes such as order value or customer type. Please cover [DATE RANGE] and include all events for the cases in that range, even those that started earlier. No aggregation, no pivoting — raw rows are exactly right.
That paragraph is usually the whole specification.
Why “one row per event” matters
Section titled “Why “one row per event” matters”Many systems will offer you one row per case, with columns like created_at, approved_at, shipped_at. That is a status table, not an event log. It can be converted — each timestamp column becomes a row, with the activity name taken from the column it came from — but it is much cheaper for whoever writes the query to produce the long format directly.
The four problems that cost the most time
Section titled “The four problems that cost the most time”Coarse timestamps. Dates without times mean activities on the same day have no order, and a discovered model shows them as parallel. If the system stores a time, ask for it.
Batch-written timestamps. If a nightly job writes rows, every event in a day carries the same stamp, and you end up analysing the job rather than the process. Ask whether the timestamp is when the thing happened or when it was recorded.
Over-specific activity names. Order 10482 approved by J. Smith gives every case its own unique activity and makes discovery impossible. You want Order approved, with the user in a separate column.
Truncated cases. An export cut to a date range contains cases that started before it and cases that finish after it. Both look abnormally fast or slow. Ask for complete cases where possible, and filter to them if not.
Personal data
Section titled “Personal data”You almost never need names. A user identifier is enough to see that work bounces between two people, and pseudonymized identifiers work fine for that. Ask for the minimum, and note that because Promenade parses the log on your own machine rather than uploading it, the data does not leave the environment it is already allowed to be in.
Check it before you build on it
Section titled “Check it before you build on it”After import, run the Log Quality plugin. It reports missing timestamps, single-event cases, timestamps that go backwards within a case, and activity labels differing only by whitespace or capitalization. Fixing those before discovery is far cheaper than explaining a strange model later.
Then check the obvious: does the case count match what the business believes? Does the date range look right? Is the activity count plausible — dozens, not thousands?