Design - ML Partnership
You don’t have to understand the ins and outs of all the models you’re working with, but without a baseline understanding of how they work, you’ll be designing for a black box.
Collaborate with your ML team members to understand and contribute to the inputs and outputs that are being shaped. While it’s unlikely you’ll be able to predict every outcome a user may face, you’ll be able to better guide users towards positive outcomes.
Communicate the goals of what you’re designing, important user outcomes, and develop an understanding of what may or may not be possible, and why.
Some Questions to ask your ML collaborators
- How sure are you when a model makes a suggestion? Are there confidence level thresholds we should design around?
- What parameters can be controlled?
- What can be done to correct the model? In what ways can user input (human-in-the-loop) help?
- What data are the model(s) trained on?
- How is it learning?
- Other strengths or weaknesses?
- Learn from label disagreements: Understand differences in how labelers interpret and apply labels to prevent problems later on. Labelers can be other models, humans or a combination of both. Label disagreements can be noise but it can also be more nuanced in a situation where there is no right or wrong.