The latest episode of the In Context podcast is a must-listen for anyone in the machine learning space. In Context’s host, Kathryn Hume from integrate.ai, always asks a perfect balance of business and technical questions to an impressive group of guests.
If you haven’t listened to In Context before, I would highly recommend subscribing! Another great episode is Episode 7 on “Understanding the marketplace for data products” with Clare Corthell and Sarah Catanzaro.
Eric Colson is currently the Chief Algorithms Officer at StitchFix, the online subscription personal shopping service who recently IPO’d. Prior to Stitch Fix, Eric was the VP of Data Science & Engineering at Netflix, another company that has machine learning deeply integrated into the customer experience. Needless to say, StitchFix is definitely in the top tier of companies leveraging machine learning in tangible ways that impact their business goals — along with Uber, Google, Airbnb, and Apple.
Although the title of the podcast focuses on hiring for autonomy, the actual episode covers so many interesting topics — from structuring a data science team to excel within an organization, “fullstack data scientists”, and StitchFix’s own machine learning projects.
In this post, I’ll provide summary notes from the episode along with my main takeaways plus open questions to consider after listening.
1:24–1:34 > The key question at hand to kick off the podcast: “How do you develop a culture that promotes accountability and autonomy? Are there challenges specific to autonomy in a data science + machine learning context?”
3:30–4:37 > Eric summarizes Stitch Fix as an “e-commerce site for apparel” where the consumer doesn’t have to browse or do any actual shopping on the site. Stitch Fix allows users to note their preferences then will populate picks for them.
4:50–7:26 > Kathryn raises the question of whether “AI will take away our capacity for serendipity?” (reducing self-determination) and Eric responds with his experience using Stitch Fix, which he claims allows for more discovery and confidence around apparel selection.
9:30–10:30 > Eric speaks about the different algorithms that the Stitch Fix machine learning team works with: they extend a classical recommendation system with judgment from human processors (stylists). He notes that machine learning has advanced, but still lack specific skills around empathizing and creating relationships.
10:31 -12:18 > There are over 85 algorithm developers at Stitch Fix and only 5 of them work on the recommendation algorithm. Inventory management, demand management, logistic, and apparel design algorithms are some other algorithms in the Stitch Fix pipeline that contribute to the company’s sustainable competitive advantage against other retailers.
12:26–12:56 > Discussion around ML algorithms as a defendable moat in light of two trends Kathryn notes: (1) rise of open-source and (2) shifting power dynamics between industry and academia.
12:58 -14:42 > Data as a lever. The organizational structure actually acts on/pulls that lever so the company can actually makes use of data and algorithms effectively. Stitch Fix’s organizational structure brings data science on-par with other key business functions.
15:34–17:55 > Eric describes how segments of the Stitch Fix algorithms team aligns with other partner business units like marketing or styling to create the sense of an ongoing initiative not just a project. The partnership model allows for a bidirectional flow of ideas.
18:07- 19:00 > Kathryn notes how that partnership allows for a clear “Horizon of Imagination” — essentially, knowledge of what’s feasible with machine learning.
19:03–21:34 > Two properties for how the data science team works: (1) framing of ideas and (2) conception of ideas. Eric argues for data scientists to be involved in (1) especially since folks still may have an unclear Horizon of Imagination.
22:00–24:18 > Kathryn’s question about whether Stitch Fix has folks dedicated to fundamental research sparks an entire discussion around the risks of the hand-off. Eric describes how the Stitch Fix algorithms team achieves “autonomy, mastery, and purpose” through owning the end-to-end process. Kathryn juxtaposes this approach with how other companies have specialized roles (business analysts, data scientists, and engineers).
24:20 -29:35 > Eric proposes the need for full stack data scientists for more iterative processes and avoid coordination cost + wait-time nightmare! 🦄
32:05–36:35 > Stitch Fix data science candidates come from a wide range of academic disciplines, but often all have a quantitative mindset and foundation and bias towards action. Eric also emphasizes the need for experience transitioning from theory to practicality.
41:29 -43:20 > Eric’s take on how organizations who are specialized already can pivot to more of a generalist, autonomous structure.
Takeaways & Questions to Consider:
The most striking part of the podcast was when Eric essentially proposes that organizations need to break away from specialized roles in favor of generalists for data science. Stitch Fix seems to have been successful with this model because their culture, relationships with other business units, and hiring process contribute to that success — couldn’t see companies who don’t have those other components easily switching out structures.
Speaking honestly, I can see the benefits of doing things under one person to make it faster. I’m very curious to see how we can improve communication and sharing around context so folks can still collaborate while having autonomy/accountability.
Some questions to reflect on —
- Are data scientists willing to be full stack data scientists? Is hiring these ‘unicorns’ as difficult as it seems?
- How does the Stitch Fix organizational structure differ from your team/company? Do you think it would be more effective for getting work done and driving business impact?
- How are data products different than other products in terms of goals and requirements?
- Are you + your team working on any tasks where the requirements are unknown?
- What are some benefits to specialization and having multiple stakeholders involved? Do these outweigh the benefits of giving one person complete autonomy over the entire process?
- Eric describes how the Stitch Fix algorithms team is optimized for “the rapid development of differentiating capabilities” — what is your team optimized for?
Cecelia Shao is the Product Lead at Comet.ml. 🚀🚀🚀
About Comet.ml — Comet.ml is doing for ML what Github did for code. Our lightweight SDK enables data science teams to automatically track their datasets, code changes, experimentation history. This way, data scientists can easily reproduce their models and collaborate on model iteration amongst their team!