Course 1 Lessons
We can reliably deliver, and get things done. That’s good, but that doesn’t indicate success. If the trajectory of our impact does not oscillate upward, consistently, failure is subtly being achieved, unknowingly.
– Curtis Thompson, MBB, MSIM, PMP
Foundations for Value Creation: Dark Data & Data Products
Organizational Impacts
Dark data can have many impacts in an organization. Some of them are decision-making/missed opportunities, data products, profitability, productivity, knowledge-base, and overall direction. Dark data’s impacts are limited to this list, these are some of the more common impacts experienced.
Dark data represents untapped potential for insights that can drive business growth, identify new revenue streams, and create cost-saving efficiencies. By not analyzing this data, companies miss out on valuable opportunities to gain intelligence that could be used to drive innovation, deepening their relationship with consumers and improving their bottom line. Having large amounts of Dark Data unexplored also leads to incomplete data sets, further complicating things and leading to incomplete information. Dark Data can also lead to decision bias that can cause failure by skewing the overall directional view of decision-makers.
Flawed dashboards, incomplete data visualizations, misleading and inaccurate results, increased storage costs, increased complexity and lack of intelligence creation are all symptoms that your organization’s data products are being impacted by Dark Data. Business intelligence dashboards, analytics platforms, and machine learning models are all data products that can be leveraged for innovation, they are also some of the first places Dark Data can cause harm if not explored properly.
Organizations can expect to see an increase in the risk of data breaches and regulatory non-compliance due to unknown vulnerabilities in their systems, increasing their overall risk levels. Organizations internal systems that contain sensitive information, could be a target for cybercriminals or unauthorized access. In turn leading to costly fines, legal liabilities, and damage to the company’s reputation.
Most companies experience operational inefficiencies that are unknowingly caused by Dark Data. These impacted processes can create confusion, slow down processes, and usually result in duplicate efforts. If employees are unable to access the data they need to do their jobs, it can lead to delays and decreased productivity. It can also lead to the ever-growing problem of silo culture and territorialism in many of the corporations that we see today.
Avoiding Negative Impacts
Investigate Dark Data First
One segment of data that is critical but is not typically seen as such is organizational performance trends. Furthermore, the performance of an organization, and its relation to market, competitor, and disrupter performance trends can provide critical insight when the market becomes overly competitive. Industry graveyards are littered with the remains of enterprises that failed to effectively utilize Dark Data relating to its performance. I’d venture to say that disruptor performance trends are becoming increasingly more important in the long game than we would expect. Given the rate at which new companies are being spun up and gainfully competing against larger more established companies, it’s important to have a solid understanding of what’s needed to maintain and or grow a competitive advantage, and Dark Data is a diamond mine for this type of information. The opportunity that presents itself for Organizations in the modern business arena is around finding non-obvious connections to seemingly disparate data sets and data trends. In the intersection of these data comparisons resides the opportunity for organizations to use Type A or B Dark Data effectively to gain the competitive advantage.
When we approach a new project, or deliverable we should always begin the investigation focused on understanding the level of impact the end deliverable will have. We need to understand the impacting Dark Data and What needs to be designed for Dark Data. In most cases when we’re working on something that is high priority, it’s most probable that designing the solution for Dark Data is imperative. Typically, when large-scale deliverables are in the build process, the MVP of the solution will be simple. The subsequent releases of the deliverable will have more Dark Data integrated into them to help provide advanced capabilities and better insights. However, in development of the MVP version we should always prepare for the future iterations and have the foundation of the solution built for compatibility with the knowns and unknowns. This must be done in the most optimal manner to better position the final product for flexibility needed due to impacts from Dark Data.
