Redesigning research and product development so that the explorative nature of data science becomes a driver for innovation

Zalando leverages cutting edgemachine learning technologies to be Europe’s leading online platform for fashion and lifestyle. In order to develop these products, data scientists and product roles have to work together closely.

As the Agile Coaching Team, we went on a journey to discover ways to make our machine learning product development more effective. As a result, we created tools and environments to help data scientists and product roles work together from day one; making solutions better, testing ideas faster and simplifying collaboration. Now, people in very different roles understand each other’s backgrounds and motivations better, reducing conflicts and handover costs. Solutions are viewed early from multiple perspectives and often the best ideas come from places where you least expect them.

Discovering the data science impossibility patterns

To avoid jumping to solutions that would have no effect, we did over 20 interviews looking at multidisciplinary areas and teams who build data science/machine learning (DS/ML) heavy products. We were looking out for “pains and gains” of all involved roles, analyzing artefacts and work styles, shadowing their meetings.

We found several patterns for which we developed a solution:

• “Data science takes a long time” - was often stated as a dogma by data scientists, product roles and leads. Of course, there are technical constraints, large amounts of data to fetch and models to be tested and trained. But much more it is about psychology: as data science takes time, it costs a lot of money, and where so much money goes, impactful deliveries are expected. Therefore, expectations to deliver results piled high in big batches in their backlogs and OKRs that “focus 120% on delivery” of course take a long time! That creates the second problem:
• “Data scientists do not know which customer problem they are solving” - as there is a complete focus on delivery there is no time to do proper customer discovery and problem definition. Therefore backlog refinement, planning and demoing gets “hard.” That creates the third problem:
• “Agile does not work in data science” - as “Scrum” is often a synonym for Agile, without a clear problem to solve, Scrum ceremonies do not work.  Also, retrospectives hardly help fix the ceremonies as they are not broken. Instead, Scrum is just the “wrong” tool to discover customer problems.

One of our biggest contributions as coaches was that we created time for innovation; a five-day learning journey combining directed and self-driven workshops, an open space, coaching sessions, community work, peer-to-peer learning and teach-back-formats in an co-creative way.

On this journey we outlined a three-day workshop around two top priority topics as real cases, each tackled by one multidisciplinary initiative. The initiative pulled together experts from formerly separate teams, to deliver customer value end-to-end. In this way, we progressed while learning (did not interrupt work with trainings), using multidisciplinary collaboration while introducing it. Tackling real cases like “Personal Relevance in Browsing” and “Transparency About Personalization” was a clear requirement to get the buy-in from the leads.

We had over 40 customer interviews performed by data scientists and engineers, fostering a much deeper understanding of the customer and problem space they build solutions for. Almost as a side effect, it raised empathy for product roles, improving collaboration and lowering the cost of handovers and conflicts.

With this journey, we served and enabled a toolchain for customer discovery, problem definition and small, fast and cheap experiments of solutions “prototypes” (ideas, hypotheses, assumptions). Zalando packaged these tools in a framework that we call 4D: Discover - Define - Design - Deliver.

One key element of the learning journey was the co-creation of a concept of how we can use “hypothesis” to steer collaboration. Known from “hypothesis driven software development,” product work, as well as in data science, we developed a concept that enables working with hypotheses across the whole DS/ML workflow. Starting with “Explorative Research” managing input like time, effort, scope, as by the nature of science the output is not predictable during exploration. We set as early finish criteria the capability to formulate a “directional hypothesis.” This enables us to switch from “Explorative Research” to “Hypothesis Testing Research” gaining more transparency, predictability and control. Combined with hypothesis-driven product work and engineering, we can use this and science as central elements to streamline collaboration in DS/ML heavy products.

In a multidisciplinary setup we co-discovered the customer, and created hypotheses around their needs, pains and gains. We tested these hypotheses early and learned, refined and iterated.

With this journey, we created space for the unknown; the place where innovation is rooted. We equipped our teammates with the “right” tools and workflows, and sent them on a learning experience across all disciplines, ranks, teams and units to find truly new land.