Overview
Since innovation is all about testing new ways of doing things and taking risks, it often ventures into unknown territory that is difficult to measure.
In fact, it’s not uncommon for innovators to view traditional measurement systems, success metrics, and key performance indicators (KPIs) as roadblocks to out-of-the box thinking. This can lead to innovations not being measured due to uncertainty about what should be measured. Or, such measurement systems might even hold innovators back from developing the kinds of products and services that will move an organization closer to its North Star because they don’t align with short-term metrics.
This article explores how measurement systems influence innovation and why applying a systems lens to measurement helps you to navigate systems innovations.
Why Do We Measure Innovation?
Measuring and assessing allow innovators to identify, collect, and evaluate information that informs their innovation decisions. It can help them to:
- Communicate the value and potential impact of ideas to others;
- Draw boundaries on the system they seek to impact; and,
- Evaluate progress towards pre-established goals and success metrics.
Measurement systems allow innovators to make sense of their changing business environments, assess threats and opportunities, and track progress towards their goals. What is measured often guides managers’ decision-making during the innovation process, and that which is not is at risk of being excluded from key innovation decisions.
A firm’s performance also extends to its impact on broader stakeholder groups, including society and the natural environment. As stakeholders demand more and more information, organizations must apply an even broader lens to measuring the impact of their innovations. Innovators should take a systems view to consider how innovations impact both the organization’s performance and broader stakeholder groups. Looking at systems-wide impacts might also reveal opportunities for the organization to leverage its role in the system and create additional value.
Applying A Systems Lens to Measurement
Measuring and assessing help to reorient innovators as they navigate their systems’ environments and seek to align innovations to their organizations’ North Star. But measurement systems often do not align with their systems’ aspirations. In this article, we suggest ways to measure innovation from a systems perspective. (If you’re new to systems thinking, you may want to read this article before continuing.)
Below are three ways to measure and assess innovation through a systems lens.
1. Look at what’s happening in the system.
When innovating for systems, bringing in new perspectives can highlight additional impacts in the system that may not have been intentional. Innovators should consider the broader systems of stakeholders in their value chains (both direct and indirect) to think about how they can leverage the expanded impact of their innovations.
An innovation’s success is often measured according to impact on the organization or consumers in the context of a specific problem. But the impact of many systems innovations extends beyond the initial problem innovators were trying to solve.
The Systems Impact of Drone Bees
When worker bees suddenly disappear and leave behind a queen, few bees are left to take care of her. This phenomenon, known as colony collapse disorder, is a systemic problem threatening the health of people, ecosystems, and businesses that rely either directly or indirectly on agricultural production. Approximately 75 percent of flowering plants and 35 percent of food crops rely on pollination. Although not the only pollinators, bees play a large role.
A threatened bee population puts food production at risk by impacting agricultural producers and the overall system. Agriculture chemical firms are often blamed for the declining bee population. Financiers are at risk of food producers defaulting on payments. Perfume producers and pharmaceutical companies are also affected because they use natural goods that require pollination. This broad view shows that pollination impacts business and society by disrupting food security, nutrition, and producers’ livelihoods. Innovators interested in declining pollinator populations came up with a solution: drone bees.
Powered by remote control or artificial intelligence, drone bees artificially mimic bees’ pollination function. Programmed to pick up on the contexts of their natural environments, they find and pollinate plants requiring pollination. Innovators might be tempted to turn to traditional metrics, such as number of drone bees sold or hectares of plants pollinated, to measure the impact of drone bees.
But this narrow view does not take into account the long-term implications of drone bees on the broader system. For example, it does not consider how drones are made or disposed of at the end of their lifecycles. Drones rely on electronic components, which have a harsher environmental impact than live bees.
Also, by focusing on bees as pollinators, we forget their important role in the ecosystem in supporting birds and other animals. While drone bees can help to pollinate artificially, we need regular bees to help the foodchain and biodiversity. This expanded view of impact should encourage innovators to brainstorm how they might address the colony collapse disorder in a new way – one that positively impacts their own organizations and broader social and environmental systems.
Reflect: Consider how an innovation you are working on impacts the value chain at various stages. Who else in the system might be affected? Remember, some groups, such as future generations, may be hidden or are not currently being considered by your organization. Once you’ve reflected, ask yourself and your team: Can the innovation be tweaked or amended to create additional impact in the system?
2. Expand the boundaries of the system you think you’re innovating within.
Systems overlap with other systems and often do not have clear boundaries. Although a specific problem might have inspired your innovation, exploring new angles helps you to discover untapped potential to solve a different problem or ways to repurpose your innovation for another industry. You might end up solving a problem you didn’t intentionally set out to solve!
The Surprising Discovery of Insulin
Did you know insulin was discovered by accident? Doctors who set out to study the pancreas noticed that once the pancreas was removed from dogs, the dogs’ sugar levels spiked and they died shortly after. This prompted later studies that isolated insulin in the pancreas, leading to the creation of the diabetes medicine we use today.
As a result of researchers pivoting to explore a new perspective, an accidental discovery became a transformative innovation – and solved a problem they did not initially set out to solve.
The above example shows that expanding the view of your innovation can have positive implications by solving new problems. The next example shows that expanding the view of your innovation can highlight unintended negative consequences.
Artificial Intelligence Creates Deadly Toxic Compounds
Scientists at a pharmaceutical company programmed software that used artificial intelligence to generate new molecules. They were hoping to discover potential new medicines for rare diseases.
Later, they discovered if they flipped a single number in the code, instead of filtering out toxic compounds, the computer would generate a list of them. This meant their innovation could easily be repurposed to develop new ideas for chemical warfare. In just six hours, the software could invent 40,000 potentially lethal toxins.
An expanded sense of impact allows innovators to see opportunities for impact beyond what was initially imagined. This may require innovators to expand the boundaries of the systems they believed they were innovating within to identify potential uses or harms.
Reflect: Try to deconstruct your assumptions about a current innovation project and view it from a new perspective. Discuss your innovation with people in different industries, or with those in your organization who are not involved in the project. How did they interpret your innovation and its potential uses? Could the innovation cause harm to others within or outside the organization?
3. Use multiple metrics to paint a picture of holistic impact.
Innovators are frequently asked to communicate the impact of an innovation in a narrow way, such as through pre-established KPIs or success metrics.
When innovating in systems, the impact cannot be captured through a single metric. Instead, innovators should think of measuring and assessing as telling a story of the company’s impact on a certain outcome, using multiple metrics as needed to tell the story of impact holistically.
The Deshkan Ziibii Conservation Impact Bond
The Deshkan Ziibi Conservation Impact Bond (DZCIB) is a first-of-its-kind pay-for-performance impact bond model created to mobilize capital toward reversing the trend of habitat loss and accelerating the stewardship of healthy landscapes. The bond structure depended on certain metrics being reached to trigger payment.
The community-led team that designed the bond determined that a single metric was not enough to assess the bond’s impact. The team decided the impact story of the bond should extend to five evaluation pillars with multiple metrics under each pillar.
The pillars not only included restoring ecosystem health, but also forging connections among social groups and cultural groups, creating opportunities for knowledge transfer and economic development, and strengthening relationships between people and the land.
Metrics were not only limited to quantitative measures, but also paired with qualitative measures. In other words, the impact story was not limited to numbers, but also left space for pictures and oral stories.
Reflect: What KPIs and success metrics are being used in your organization to measure and assess impact? Often, these do not tell the full story. Consider using impact assessment tools, such as a theory of change, which help innovators understand the relationships in the system driving impact to determine potential impact metrics. Remember, there is likely not a “silver bullet” or single metric of impact. Measuring and assessing the impact of systems innovations require multiple metrics working together.
Conclusion
To measure and assess for systems innovation, innovators need to challenge current approaches, which are often too narrow and focused on generating a short-term return for their organizations. At times, innovators may need to break away from current KPIs and question how they are assessing an innovation’s impact, looking at different perspectives and expanding their metrics of impact. Innovators should consider how the innovation plays a role in the system and might create value in unexpected ways.
After reading this article, innovators should not view measurement as a roadblock holding them back, but instead as a means to better understand their innovations and their impacts in the system.