The Digital Darkness
Trifecta:
With the ever-expanding universe of digital information,
concepts about what we do not know are also emerging. Gartner
has defined the term “Dark Data” to
describe data that are collected, processed and stored by organizations only to
be put to little or no use. As an
example, thy cite data that are captured and stored for compliance purposes as
well as analytics data. Additionally, Jim Sinur, formally a Distinguished Analyst at Gartner, has defined the concept of “Dark Processes” (jimsinur.blogspot.com). According to Sinur, “…it is a process that is
hidden or partially blind as it executes”.
In other words, it has become part of an undocumented process with rouge
tasks executing outside of the official workflow. Now, to this dynamic duo of digital darkness,
I propose an addition, “Dark Events”. Dark Events are typically part of a business process and are unknown and
unreported due the nature of the data collection technology provided by most
workflow and Enterprise Resource Planning Systems.
Dark Events: an
example
Have you ever called a taxi service and been given an
expected arrival time, only to find that after 30 minutes or more, the ride has
yet to arrive? So, frustrated, you place another call to the dispatcher and,
once again, you are told that the taxi should arrive shortly. And after yet another 10 minutes; still no
luck! Well, the last time this happened
to me, I was sitting at a restaurant in a residential district in N.W.
Washington, D.C. and decided to download Uber, set up my account and request a
car. Immediately, I was aware of a
number of things; first, my request was accepted, the driver was identified, a
mobile number with a name a picture of the diver was provided, an estimated
pick-up time was noted and maybe best of all, a map was presented that showed
my location, the location of the driver and the car’s progress toward me with
the time to arrival decremented as the driver approached. I had all the information I needed to reduce
the uncertainty of not being picked up.
In the dispatched taxi case, all of the information between the request
for a ride and my current situation was dark; I had no way of knowing anything about
the expected pick-up from when I placed the call to the taxi company to when
the driver would, if ever, arrive. In
the case with Uber, the entire event from the call to the actual pick-up was visible
to me, giving me a high level of confidence in the process as well as the
expected outcome. In both cases, what we
are talking about is a process which consists of a series of events. Once
again, the events are the same as in both cases. The difference is that in the
dispatched taxi process, most of the events are dark, meaning that I had no
visibility into what was happening, leaving me with simply the faith that the
driver would appear: or not! In data
science terms, this illustrates the concept of “opacity” whereby, much of the
phenomena are unable to be seen, hence are opaque.
Dark Events in
Organizations:
As with the case of the opaque taxi process, Dark Events are
an everyday occurrence within most organizations. Sometimes they called “dark spots”, places
where it is known that the information about the event is incomplete but there
few solutions that will help provide visibility into the situation (e.g., a known unknown). Similar to dark matter in physics, Dark Events
comprise a significant percentage of the universe of events within a typical
business process. Research has shown that Dark Events can outnumber knowingly
captured events, those captured by business process focused IT systems, by a factor of 15 leaving 60 % of the
organizations events un-illuminated.
Imagine the effect on enabling desired outcomes through the use of
digital information if actionable decisions are being taken based on
understanding what actually occurred in less than half of all the events? It does not matter how good your analytical
models are if the data are grossly incomplete!
Dark Events; next
topic:
The next post will discuss Dark Event discovery and analysis
showing a path toward illuminating the “unknowns”.
See
No comments:
Post a Comment