Data-Driven Decisioning Employee Experience Redesign

Objective & Scope

Our org has a significant delinquency in data-driven decisioning resulting in lost opportunities to incorporate consistent customer feedback into the decisions that we make to drive development. We have assessed the current state of the PM experience through three lenses:

  • Awareness: Most PMs do not test or even monitor the products they manage. Of the 190 FTE PMs who are expected to have Adobe Analytics accounts, only 63% do, and only 31% of those have logged in in the last 30 days

  • Process: No onboarding processes exist for PMs to learn the benefits of analytics, tagging etiquette, testing strategies, or how to execute them

  • Culture: An analytics and testing culture does not exist. PMs are task-oriented instead of results-oriented, preventing data-driven decisioning

Discovery

While Working across Product Domains and Lines of Business, we have uncovered a number of opportunities to streamline and expedite our experience:

  • Partnerships with Adobe Testing and Analytics Teams have identified multiple gaps and inefficiencies in the PM testing experience

  • Best testing practices and culture from effective Product Teams that embrace testing were used as learnings that can be embedded across all domains

  • Interviewing PMs who have come from external organizations revealed new possibilities that can recalibrate our culture

  • Establishing a benchmark among competitive institutions and technology organizations helped define and size our opportunity

Design Recommendations

After the implementation our our redesigned Employee Experience, our efforts resulted in a 50% increase of PMs who are testing::

  • Awareness: With the design and delivery of comprehensive trainings and the adoption of integral testing frameworks across domains, PMs reaped the benefits of assessing and implementing data, analytics and testing into all decisions, enabling them to work smarter, not harder

  • Process: By updating our onboarding experience and E2E product lifecycle processes with elements of monitoring and testing, we have baked in strategies into every step of product development

  • Culture: Moving from a task-oriented mindset to a results-oriented mindset ensures that DDD is a fundamental value and practice of all PMs

Data-informed decisioning optimizes our experiences, enabling the delivery of best-in-class products for our clients, and generating significant value for our business

Opportunities for optimization are abundant, with the need for PMs to up-skill to parity with industry standards being clear

While Adobe Analytics is a powerful tool in the hands of our PMs, the pain points they have surrounding it are far too many form their POVs

Key Takeaways

  • Absence of training during onboarding results in confusion about how to use Analytics

  • New PMs are not provided with tagging or reporting archives when they take over

  • PMs don’t know what key metrics they should be monitoring for their products

  • PMs are overwhelmed with their workloads and suffer from task-oriented mindsets

Current state of the A/B Testing Process has multiple opportunities to streamline

Two questions were raised about the A/B Testing Process:

  1. How long does it take to launch a test? → 5-12 weeks

  2. Do we have the right infrastructure? → Yes, we do! The challenge is in our process.

Embedding research and testing tools in our existing process helps us create disruptive, innovative features

Demand for Data-Driven Decisioning Workshops* has increased throughout the year, reaching an audience of 500+, resulting in a 50% increase in PMs who are testing

Once we redesign our practices around Adobe Analytics optimization, the PM experience evolves, and data-driven decisioning comes to life

Key Solutions

  • Comprehensive training for all PMs informs them how to effectively use Analytics

  • New PMs are provided with flow diagrams, tagging and reporting archives

  • All processes that go into identifying key metrics, building dashboards, setting up alerts, and navigating Analytics are readily available

  • PMs become aware of the benefits of monitoring and shift to a results-oriented mindset

Previous

Next Gen Payments UX Redesign