Our latest case study highlights our long term partnership with Direct Energy, part of NRG Energy, and the impact developing a culture of experimentation continues to have within their organization.
Our work with Direct Energy has grown beyond simple CRO and now helps them make informed decisions across the organization. At Direct Energy, we spoke to some of our experimentation partners and champions about the impact a culture of experimentation has had at their organization. We interviewed:
Ethan Rahman, Sr Manager UX/UI at NRG Energy – Ethan has been involved with experimentation at Direct Energy since the early days and has been a key voice bridging the gap between the voice of the customer and the business’ digital experience.
Chris McCarthy, Manager Digital Paid Media & Acquisition at Direct Energy – Chris has been a champion of experimentation since his early involvement. He has been instrumental in building the program and gaining organizational support at every step of the way.
James Hillin, Digital Experience Manager at NRG Energy – James has been a key player in running the experimentation program for years directly involved in ideation, measurement, implementation, and beyond. He is a resident experimentation expert.
1. How has a culture of experimentation informed and influenced decision making at Direct Energy, what are some of the tangible and intangible benefits?
“I think we have become a more data driven and customer centric company. Turning to customers for research and testing to verify has been great. I think beyond the obvious benefits of increased sale and customer engagement, testing has also created an almost adventurous mindset within the team to try new ideas and not be afraid to be wrong. “ James Hillin
“On initial glance, an experimentation culture feels like everything needs to be over analyzed and backed behind data before presented. While some of the more successful tests are researched and thought out, a new mentality across the organization of “just test it”, allows for quick learnings with little permanent repercussions as a test can be rolled back easily without major changes to code.” Chris McCarthy
“We as managers cannot think that we know everything when it comes to running our business. A culture of testing is centered around innovation and that’s what exactly Widerfunnel has helped Direct Energy with. Throughout the years we had tremendous success with measuring our conversion rate alongside our overall financial impacts but I think the biggest impact was the organizational mindset shift where we want to test everything before bringing it in front of our customers.” Ethan Rahman
2. What difference has a testing culture had on the employees and organization as a whole?
“In addition to the first answer I think it highlights the importance of all the different aspects of the website, what our content writers create is just as important as the design and just important as the customer flow and has created a much more collaborative mindset in attacking new journeys.” James Hillin
“By involving the extended team, we have gotten great experiment ideas from across the company. The lower stakes of temporarily modifying a web experience to a smaller sample of people, allows for more creative ideas to come to light.” Chris McCarthy
“With continuous testing, I think the overall leadership team and the organization started avoiding being afraid of taking risks. It allowed us to seek out input from your customers and learn why a product may have failed.” Ethan Rahman
3. What are some key highlights of your experimentation program that stand out to you?
“Collaboration, having representatives from different teams in the room always added a special feel to the experience of creating a test and everyone’s willingness to take a risk or be wrong made for some exciting collaboration.” James Hillin
“Getting inspiration from not just your industry but any digital experience has not just allowed us to think outside the box, but actually build those ideas and see if they are worth implementing.” Chris McCarthy
“I think experiments like “configure your own plan”, “build your own bundle concepts” are some of the highlights of our partnership. These innovations have pushed our experiment mix further into the validation and disruption phases and advanced the maturity of our experimentation program together. These experiments have extended beyond simply improving the digital conversion rate but instead helping product teams develop plans, marketing teams fine-tune promotions, and design teams build new experiences from scratch.” Ethan Rahman
4. How do you think experimentation will influence your organization as it moves forward? What impacts do you foresee it having?
“I hope it continues to shine and gain traction in areas outside of digital, there are so many customer touchpoints that expanding on them growing and testing along the way could be an exciting thing to really make the best universal experience for our customers.” James Hillin
“Experimentation allows us to grow rapidly and in the right direction. Gone are the days of complete web overhauls that take multiple years and to hope for the best upon launch. Each page, each button, each menu has been tested so that the next evolution of a website is a culmination of many new experiences, all proven.”Chris McCarthy
“Based on the partnership and the proven track record that we had for years with WF [Widerfunnel], I feel that the testing mindset is well adopted within our Direct Energy culture and I simply foresee nothing but great things out of this partnership.” Ethan Rahman
Progressing from simple CRO, to experimentation around more meaningful tests, our journey with Direct Energy is filled with customer insights that will inform user experience and strategy for years to come and are highlighted in our joint case study.
If you’d like to learn more about how building a culture of experimentation can influence decision making at your organization, reach out to talk to one of our strategists today.
