Creativity: Underrated Trait for Running Experiments

Why creativity is a crucial trait for experimentation in growth, marketing, and product teams.

Why it gets overlooked, what to do about it, and how creativity manifests in experimentation within growing businesses.

In a world of work increasingly inundated with analytics, algorithms, and arithmetic - let's put equal focus towards what will make you and your company different, unique, and valuable: creativity.

In the realm of growth experimentation, creativity is often overlooked as a valuable trait. Many organizations focus solely on data-driven decision making and fail to recognize the significant role that creativity plays in driving successful experiments. Let's go on a quick journey exploring the importance of creativity in experimentation, techniques to spark it, how to nurture it within a team, and why one gets better outcomes as a result.

As fair warning, there is a deluge of quotes in this one because great writers steal and great, honest writers cite.

“Observation is a passive science, experimentation an active science.” - Claude Bernard

Why creativity matters for experimentation

Experiments are the lifeblood of growth for many organizations. These experiments allow businesses to test hypotheses, iterate on their products or services, and ultimately fuel their growth engine. However, a data-driven approach alone is not enough to drive meaningful results. Creativity plays a vital role in the experimentation process, enabling teams to explore new avenues and discover innovative solutions.

While data can provide valuable insights into user behavior and help identify patterns, it does not always offer the full picture. Creativity allows experimentation teams to think outside the box, question assumptions, and come up with novel experiment ideas that can lead to breakthrough results. By incorporating creativity into the experimentation process, teams can uncover opportunities that data alone cannot reveal.

Imagine a scenario where a company has been analyzing user data and identifying areas for improvement in their product. The data shows that users are dropping off at a specific step in the user journey. While this information is valuable, it does not provide a clear solution to the problem. This is where creativity comes into play.

“Experimentation keeps new ideas rising to the surface.” - Joseph Orr

A creative experimentation team would approach this challenge by brainstorming various ideas to address the drop-off issue. They might consider redesigning the user interface, simplifying the process, or even gamifying the experience to increase engagement. These out-of-the-box ideas may not be immediately evident from the data alone, but they have the potential to unlock significant improvements.

Furthermore, creativity allows experimentation teams to challenge assumptions and explore new possibilities. In a data-driven approach, there is a risk of becoming too focused on the numbers and overlooking alternative perspectives. By incorporating creativity, teams can question existing assumptions and explore uncharted territories.

For example, a company may have been targeting a specific demographic based on their data analysis. However, a creative approach may lead the team to consider expanding their target audience, reaching untapped markets, and discovering new customer segments that could drive additional growth.

"Experimentation is the least arrogant method of gaining knowledge. The experimenter humbly asks a question of nature" - Isaac Asimov

Moreover, creativity encourages experimentation teams to think beyond incremental improvements and pursue breakthrough ideas. While data can guide incremental optimizations, it may not always inspire radical innovation. Creativity allows teams to dream big, push boundaries, and explore ideas that may seem risky or unconventional at first glance.

Consider a company that has been steadily improving its product based on user feedback and data analysis. While these incremental changes have been effective in driving growth, a creative experimentation team might propose a radical overhaul of the product, introducing disruptive features that could revolutionize the industry.

Data-driven approaches are essential for experimentation. I posit creativity is equally crucial. By incorporating creativity into the experimentation process, teams can go beyond the limitations of data and unlock new possibilities. Creativity enables teams to think outside the box, challenge assumptions, and pursue breakthrough ideas that can drive significant growth. So, embrace creativity and let it fuel your experimentation journey.

"Any living cell carries with it the experience of a billion years of experimentation by its ancestors" - Max Delbruck

Techniques to spark creative ideas

Creativity is not a mysterious talent reserved for a select few; it can be nurtured and developed within individuals and teams. There are several techniques that can help spark creative ideas for growth experiments.

"We must conduct research and then accept the results. If they don't stand up to experimentation, Buddha`s own words must be rejected" - Dalai Lama

One technique is the old fashioned brainstorming session, where team members come together to generate a wide range of ideas. This technique promotes a free-flowing exchange of thoughts, allowing individuals to build on each other's ideas and create innovative solutions. By creating an open and non-judgmental environment, brainstorming encourages participants to think outside the box and explore unconventional ideas. It is a collaborative process that fosters creativity and encourages everyone to contribute their unique perspectives.

Another useful technique is the "SCAMPER" method, which prompts individuals to think about how they can substitute, combine, adapt, modify, put to another use, eliminate, or reverse existing elements of a problem. This method encourages individuals to challenge the status quo and think creatively about different ways to approach a problem. By systematically considering each element of a problem and exploring various possibilities, the SCAMPER method stimulates innovative thinking and helps generate fresh ideas.

"Every great idea emerges out of a process of trial-and-error experimentation" - Scott D. Anthony

Exposing oneself to diverse sources of inspiration can stimulate new ideas and perspectives. Reading books on various subjects, attending conferences and workshops, or engaging in creative hobbies can provide a wealth of inspiration. These experiences expose individuals to different viewpoints, cultures, and ideas, which can spark new connections and insights. By immersing oneself in a variety of experiences, individuals can expand their knowledge and broaden their creative horizons.

Taking breaks and allowing time for reflection can also cultivate creative thinking. Sometimes, the best ideas come when the mind is relaxed and free from distractions. Stepping away from a problem and engaging in activities like taking a walk, meditating, or simply daydreaming can give the brain a chance to process information and make new connections. These moments of relaxation and reflection provide space for ideas to incubate and evolve, leading to fresh perspectives and creative breakthroughs.

"The basis of the discovery is imagination, careful reasoning and experimentation where the use of knowledge created by those who came before is an important component" - Bengt I. Samuelsson

Creativity is not a fixed trait but a skill that can be developed. By utilizing techniques such as brainstorming, the SCAMPER method, and exposing oneself to diverse sources of inspiration, individuals and teams can enhance their creative thinking abilities. Additionally, taking breaks and allowing time for reflection can provide the necessary mental space for ideas to flourish. With these techniques, individuals can unlock their creative potential and generate innovative ideas for growth experiments.

“Doubt the conventional wisdom unless you can verify it with reason and experiment.” - Steve Albini

Why creativity leads to better experiment ideas and insights

Creativity is essential for generating experiment ideas that have the potential to drive significant growth. By thinking creatively, experimentation teams can identify new hypotheses and experiment designs that are not limited by past assumptions or preconceived notions. This enables organizations to explore uncharted territories and discover untapped opportunities.

"There is no higher or lower knowledge, but one only, flowing out of experimentation." - Leonardo da Vinci

Creative thinking also leads to deeper insights and understanding of user behavior. By asking the right questions and challenging existing beliefs, experimentation teams can gain a more nuanced understanding of user motivations and preferences. This deep understanding allows for the development of experiments that are tailored to meet user needs, resulting in more impactful outcomes.

When it comes to generating experiment ideas, creativity plays a crucial role in pushing the boundaries of what is possible. Rather than relying on tried and tested methods, creative thinkers are able to explore unconventional approaches that have the potential to yield groundbreaking results. They are not bound by the limitations of traditional thinking and are willing to take risks in order to uncover new insights.

"Experimentation is the key to 'lively up' the art." - Bob Marley

One way in which creativity can lead to better experiment ideas is by encouraging experimentation teams to think outside the box. By challenging conventional wisdom and exploring alternative perspectives, teams can come up with innovative hypotheses that have the potential to disrupt the status quo. This can open up new avenues for growth and help organizations stay ahead of the competition.

Moreover, creative thinking allows experimentation teams to consider a wider range of variables and factors that may influence the outcomes of their experiments. By considering multiple dimensions and exploring different angles, teams can develop more comprehensive experiment designs that take into account the complexities of user behavior. This holistic approach increases the chances of uncovering meaningful insights and generating impactful results.

Another benefit of creativity in the context of experimentation is its ability to foster collaboration and cross-pollination of ideas. When teams encourage and embrace creativity, they create an environment where diverse perspectives and expertise can converge. This interdisciplinary approach can lead to the synthesis of ideas from different fields, resulting in innovative experiment designs that draw inspiration from various sources.

“I am my own experiment. I am my own work of art.” - Madonna

Creativity goes beyond generating ideas. It also involves the ability to effectively communicate and present those ideas to stakeholders. Creative thinkers are skilled at crafting compelling narratives and visual representations that capture the essence of their experiment ideas. This not only helps in gaining buy-in from decision-makers but also facilitates the understanding and adoption of those ideas by the wider organization.

Creativity is a powerful catalyst for generating better experiment ideas and gaining deeper insights. By fostering a culture of creativity within experimentation teams, organizations can unlock new possibilities, challenge existing beliefs, and ultimately drive significant growth. Embracing creativity in the context of experimentation is not only a way to stay ahead of the competition but also a means to better understand and meet the needs of users.

Focusing on creative ways to test hypotheses, not just execute tests

When it comes to testing hypotheses, executing tests accurately is undeniably crucial for gathering reliable data. However, solely focusing on test execution can limit the potential for breakthrough growth. To truly push the boundaries of experimentation, teams should prioritize thinking outside the box and exploring unconventional methods and approaches.

"Nothing has such power to broaden the mind as the ability to investigate systematically and truly all that comes under thy observation in life" - Marcus Aurelius

One creative approach to testing hypotheses is to leverage existing user touchpoints in innovative ways. By identifying moments where user behavior can be influenced or nudged in a particular direction, experimentation teams can design experiments that have a tangible impact. For example, they might craft personalized messaging to resonate with users on a deeper level, optimize onboarding experiences to enhance user engagement, or even gamify user interactions to encourage desired behaviors.

Moreover, experimentation teams should not be confined to traditional metrics of success. While conversion rate or revenue are undoubtedly important, creative experimentation involves looking beyond these metrics. Teams should strive to understand the broader impact of their experiments, such as user satisfaction, brand perception, or long-term customer loyalty. By considering these alternative measures of success, teams can gain a more comprehensive understanding of the true value their experiments bring.

"Reward worthy failure - Experimentation" - Bill Gates

In addition, experimentation teams can also draw inspiration from various industries to fuel their creativity. For instance, they can look to the gaming industry, where A/B testing is a common practice to optimize user experiences and drive engagement. By studying how game developers experiment with different features, levels, or rewards, experimentation teams can uncover valuable insights that can be applied to their own testing strategies.

Another avenue for exploring creative testing methods is through collaboration with other departments or teams within the organization. By tapping into the diverse expertise and perspectives of colleagues from different areas, experimentation teams can gain fresh insights and ideas. For example, collaborating with the design team may lead to innovative visual experiments, while partnering with the customer support team may uncover valuable customer pain points to address through testing.

“The true method of knowledge is experiment.” - William Blake

Furthermore, experimentation teams should embrace a culture of continuous learning and curiosity. Encouraging team members to stay updated on the latest industry trends, attend conferences or webinars, and engage in knowledge-sharing sessions can foster an environment where creative testing methods flourish. By continuously expanding their knowledge and skillset, experimentation teams can stay ahead of the curve and uncover new and exciting ways to test hypotheses.

Focusing solely on test execution can limit the potential for breakthrough growth. By prioritizing creative ways to test hypotheses, leveraging existing user touchpoints creatively, considering alternative measures of success, drawing inspiration from other industries, collaborating with colleagues, and fostering a culture of continuous learning, experimentation teams can unlock new possibilities and drive innovation in their testing strategies.

"Survival in the demand economy depends on and requires experimentation, risk taking, and trial and error" - Kevin Allen

Examples of highly creative experiments that drove growth

Throughout the history of growth experimentation, there have been several notable examples of highly creative experiments that have driven substantial growth for organizations.

One such example is the "Wizard of Oz" experiment conducted by Zappos. In the early days of the company, Zappos did not yet have a fulfillment center and relied on drop-shipping from its suppliers. To test the viability of their business model, they created a fake software prototype that made it appear as though they had a fully automated system. When customers placed orders, Zappos manually bought the product from the supplier and shipped it to the customer. This experiment allowed Zappos to validate their business model and gain valuable insights before investing in a fulfillment center.

"There are three principal means of acquiring knowledge... observation of nature, reflection, and experimentation. Observation collects facts; reflection combines them; experimentation verifies the result of that combination." - Denis Diderot

Another notable example is the "Magic Pricing Page" experiment conducted by Dropbox. In an effort to increase conversions, Dropbox created an experiment where they tested different pricing page layouts and copy variations. Instead of relying solely on data, Dropbox embraced creativity by incorporating magic-themed elements and captivating copy. This experiment resulted in a significant increase in user sign-ups and played a crucial role in Dropbox's rapid growth.

The creative process for generating hypothesis ideas

The creative process for generating hypothesis ideas involves several stages that encourage divergent thinking and exploration of novel possibilities.

“My best work is always done… when I’m experimenting. If I stop experimenting I feel it just becomes a drudgery.” - William Dobell

The first stage is problem exploration, where experimentation teams seek to gain a deep understanding of the problem or challenge at hand. This involves conducting user research, analyzing user feedback, and identifying pain points or unmet needs. By immersing themselves in the user's world, teams can gain valuable insights that inform hypothesis generation.

The second stage is ideation, where creativity takes center stage. This stage involves brainstorming sessions, idea generation techniques, and lateral thinking exercises. Experimentation teams should encourage wild ideas and challenge themselves to think beyond conventional boundaries. By embracing unconventional or seemingly crazy ideas, teams can uncover hidden gems that have the potential to drive significant growth.

The final stage is hypothesis prioritization, where the generated ideas are evaluated and refined. This involves assessing the feasibility, potential impact, and alignment with business goals. By prioritizing hypotheses that balance creativity with practicality, experimentation teams can ensure a balanced experimentation pipeline that delivers meaningful growth outcomes.

How to build a culture that encourages creativity in experimentation

Building a culture of creativity in experimentation requires a holistic approach that encompasses leadership, processes, and team dynamics.

“I love fools’ experiments. I am always making them.” - Charles Darwin

Leadership plays a crucial role in fostering creativity by setting the tone and providing the necessary resources and support. Leaders should encourage experimentation teams to take risks, reward creativity, and create a safe environment where failure is seen as a learning opportunity rather than a setback.

Organizations should also establish processes that incentivize creative thinking and provide the necessary time and space for experimentation teams to explore innovative ideas. This might involve dedicated brainstorming sessions, regular knowledge sharing sessions, or allowing time for experimentation teams to work on passion projects that fuel their creativity.

"Conformity can be costly in a world of uncertainty which requires innovative institutional creation because no one can know the right path to survival" - Douglass North

Lastly, team dynamics should be nurtured to foster collaboration and cross-pollination of ideas. Encouraging diverse perspectives and creating a psychologically safe environment where team members feel comfortable sharing their creative ideas is crucial for unlocking the full potential of creativity in experimentation.

Creativity as a foundational element of an experimentation program

Creativity is an essential and often underrated trait for running successful growth experiments. By incorporating creativity into the experimentation process, teams can unlock new ideas, explore uncharted territories, and generate breakthrough growth. Key takeaways for making creativity a core strength of your experimentation program include:

Recognize the value of creativity in driving meaningful results
Use techniques such as brainstorming and SCAMPER to spark creative ideas
Embrace creative ways to test hypotheses beyond traditional methods
Learn from examples of highly creative experiments that drove growth
Foster a creative process for generating hypothesis ideas
Build a culture that encourages and rewards creativity in experimentation
By prioritizing creativity in experimentation, organizations can differentiate themselves, discover new growth opportunities, and ultimately thrive in an ever-evolving business landscape.

"The country needs and, unless I mistake its temper, the country demands bold, persistent experimentation. It is common sense to take a method and try it; if it fails, admit it frankly and try another. But above all, try something" - Franklin D. Roosevelt

Friday Night Reading:

https://github.com/DavisTrey/ExperimentationResources - A list of blogs, videos, and other content that provides advice on building experimentation and A/B testing platforms.
https://growth.design/ - Get smarter about UX + Psychology with practical, interactive case studies weekly.
https://jamesclear.com/five-step-creative-process - For a More Creative Brain Follow These 5 Steps
https://artifacts.reforge.com/artifacts?topics=experimentation - Access real work and real templates from the best operators in tech.