Why Your “Silly Questions” Are Product Requirements
We have a technical writer on our team. She’s sharp, curious, and exactly the kind of non-technical professional who needs to understand machine learning to do her job well. Yesterday after a team meeting, she sent me a Slack message. My boss explained that we needed to clarify how we describe model deployment in our products. Deployment is simply the step where a machine learning model goes from being built and tested to making predictions on real data. In our product, once a model is built, it’s also immediately deployed. That’s not how it works with competitor products, and that’s what gives us an advantage.
As we were talking, I could tell she had never really created a framework around model building and deployment. She didn’t know how to conceptualize how it worked, so she couldn’t understand the technical details. This is what happens when you’re trying to understand machine learning without a technical background. You’re not missing intelligence or capability, you’re missing the framework that makes the details make sense.
A note on terms: A mental model is the internal picture you have of how something works. A framework is the structure you use to build that picture. In this post, you’ll see both. They’re related but not the same. The framework comes first. The mental model is what you’re left with once it clicks.
She was asking the right questions. What does deployment mean? When does it happen? How does it happen? What makes our approach different? Then she typed: “I feel like a tech journalist, always wondering and asking questions. Sometimes silly and basic.” I stopped what I was doing because her question wasn’t silly at all. She was telling us we’re not explaining things clearly enough, or maybe we’re starting at the wrong entry point. If she’s confused, our customers are probably confused too.
This Has Always Been a Problem
I was a math major in college. I remember sitting in the library with my textbooks, trying to work through proofs. The textbook would show step 1, step 2, then: “It’s easy to see how this follows…” and jump to step 6. It wasn’t easy to see, at least not for me. I’d sit there for an hour trying to figure out the missing steps. Sometimes I’d get it, sometimes I wouldn’t, but I always felt like I was the problem. Like everyone else could “easily see” what I was missing. The textbook authors had been doing these proofs for so long that they’d internalized all the intermediate steps. To them, it really was easy to see. They’d forgotten what it was like to not have that mental model built into their brain.
There’s research on this. Researchers studying how experts teach beginners found that teachers would consistently skip multiple intermediate steps in their explanations, not because they were lazy or didn’t care, but because those steps no longer existed as separate thoughts in their brains. Expertise had changed how their brains processed information. What once took ten steps now felt like one. The experts genuinely couldn’t see the individual steps they were skipping.
Even more interesting, the study found that teachers with more expertise were actually worse at predicting what students would find difficult. The more they knew, the less they could see through a beginner’s eyes. This is called the curse of knowledge. Once you know something, it’s very hard to remember what it felt like not to know it. And that makes it difficult to explain clearly, especially if you aren’t aware it’s happening.
That experience with my math textbooks informed how I think about teaching and explaining things. If someone doesn’t understand something I’m explaining, my first thought is that I’m skipping steps they need, not that they’re not getting it.
It’s Still Happening in Machine Learning and Data Science Education
My son, a Communication Studies major in college, is taking an introductory data science course to fulfill his mathematics requirement. This is a100-level (read: beginner) course using R, a programming language for statistics and data analysis. One month in, here’s what his homework is asking him to do: create a data frame with six different columns, each with different data types. Then find which columns have the maximum anticipated grade using factor ordering. Then find which courses have the minimum anticipated hours using conditional subsetting with the which() function. Then print just those specific course numbers and names formatted with paste() and a colon separator.
Look at the code they expect him to write:
paste(
MyCourseDataFrame[which(MyCourseDataFrame$anticipatedHours ==
min(MyCourseDataFrame$anticipatedHours)), 1],
MyCourseDataFrame[which(MyCourseDataFrame$anticipatedHours ==
min(MyCourseDataFrame$anticipatedHours)), 2],
sep=":")That’s nested functions, logical subsetting, conditional extraction, and string formatting all at once. These are sneaky hard problems for someone who started learning R four weeks ago. You have to understand what a data frame is, how subsetting works, what which() does, how to combine functions, and how to format output. Each piece individually might make sense, but combining them all requires a mental model of how data flows from one function to the next.
But the code complexity is only part of the problem. Look at what the assignment questions are actually asking. After creating the data frame, question 2 asks: “Do the data frame variables retain their original classes? Formally test this using appropriate R code.” This isn’t just asking students to write code. It’s asking them to understand type systems, how R handles data conversion, and what “formally test” means in a programming context.
Question 3 introduces lists and asks: “Do the elements of the list maintain their original classes?” Now students need to understand the difference between data frames and lists, why that difference exists, and how each structure handles data types differently. The assignment doesn’t explain this mental model. It assumes students have already built it from a few lectures.
Most ML courses aren’t building the frameworks students need. They’re teaching syntax and expecting students will form the mental models on their own. I’ve been on Zoom with him multiple times a week, backing up and rebuilding the foundation. “Okay, which() returns the positions where something is true. So when you use which() inside the brackets, you’re saying ‘give me the rows at these positions.’ Now let’s build that up piece by piece.” Once he understands how the pieces fit together, the code makes sense. But the course isn’t giving him ample time to build those connections. This is exactly what I did with those math textbooks years ago, filling in the steps the material assumes you already know.
The Question Behind the Question
Back to our technical writer and her model deployment question. With most ML tools, building and deploying a model are two separate steps. You build the model, you test it, then you deploy it separately: different processes, different tools sometimes, more steps, more time, more places for things to go wrong. With our product, they happen at the same time. Build it, and it’s immediately available if you want to use it for scoring.
That’s a real advantage over our competitors, but she had no way to see it. She didn’t know that building and deploying are usually separate things. Same as my son’s data science course assuming he already knows how to think about nested function calls and data subsetting. Same as my math textbooks assuming I could see the intermediate steps. The curse of knowledge strikes again.
Normally, building an ML model and deploying it are completely different steps. You build and test your model in one environment, then you have to move it somewhere else, set up infrastructure, configure endpoints, test again. With our product, those steps happen together. Build it, and it’s immediately available for scoring, no extra setup, no separate deployment step. Once you have that framework, the advantage is obvious.
Why ML Explanations Miss the Mark
I typed back: “I’m glad you ask questions. It is very confusing for users.”
If our writer is confused about this, our customers may be confused. If she doesn’t understand why simultaneous build and deploy is a big deal, they don’t either. People want things to be easy, and if we’ve made something easier but can’t explain why, we haven’t actually helped them.
Machine learning Product Managers do this constantly. We build features that solve real problems, then we explain them using the frameworks we’ve built up over years of working in this space. “Our product integrates the build and deployment pipeline.” What does that mean? “Traditional workflows require separate orchestration for model serving infrastructure.” Why should I care? We’re assuming everyone understands that build and deploy are normally separate, time-consuming, error-prone steps. They don’t know that, so our advantage sounds like meaningless jargon.
Three Questions to Ask Before Anything Else
After we worked through this, I realized something about how to approach learning any ML product or feature, especially if you’re coming from a non-technical background. Don’t start with “how does this work?” Start with three questions instead.
What’s the mental model you need? You can’t build it without the framework first. In this case, the mental model is that build and deploy are normally separate steps that need different tools. Once you have that, everything else clicks. This is the step my math textbooks skipped. This is what my son’s data science course doesn’t give students time to build. They show him how to write which(MyCourseDataFrame$anticipatedHours == min(MyCourseDataFrame$anticipatedHours)) without explaining that he’s building a logical vector, finding true positions, and using those positions to subset rows. He’s writing the code without understanding the mental model of how data flows from one function to the next.
Why was this built? What problem does it solve? With our simultaneous build and deploy, the problem is that the traditional two-step process is slow, error-prone, and creates a pain point. The feature exists to eliminate it. If you understand the problem, you understand the value.
How does this fit with everything else? Products aren’t random collections of features, they’re systems where pieces connect. Our product handles build and deploy together for immediate scoring. Another product feature extends that by making models available for scoring through REST APIs (how different software systems talk to each other over the internet). Each feature solves a specific problem, and together they give you flexibility for different use cases.
Most documentation skips all three of these questions. It explains how features work without giving you the framework for why they exist or how they connect.
Why Non-Technical Professionals Get Left Behind
Whether you’re building ML products, evaluating them, or implementing them, this same dynamic plays out constantly. Someone explains a product feature and assumes you already have the framework to understand it. You nod along, trying to piece it together, feeling like you should already know this. You don’t feel comfortable asking “Wait, why is combining build and deploy such a big deal?” because it sounds basic. But that’s exactly the question you should ask because if you don’t have the mental model, the details will never come to light. All the features in the world don’t help if you can’t understand why they’re valuable.
She Was Right
After we worked through the deployment question, I told our writer: “This is why we need to ask questions. You’re asking what our customers are asking. If we can’t explain it clearly to you, we definitely can’t explain it clearly to them.” She seemed relieved. “I love this connection with you,” she said. “I’ve learned so much.” I told her the same thing. “You’re teaching me where our explanations fall apart.”
What I’m Doing Differently
Watching my son work through his data science course, I see the same pattern I saw in my math textbooks 30 years ago. Experienced people are creating educational content but forgetting what it’s like to not know this material. They’re skipping the steps that seem obvious to them but aren’t obvious to anyone learning it for the first time. The leaps are too big, the pace is too fast, and they’re not giving people time to build the mental models they need.
My goal isn’t just to explain ML and AI concepts. I want to give you the frameworks to think about them. Build the framework first, and the mental model follows. I’m here to fill in the steps your textbook, your course, or your Product Manager skipped.
The Questions You’re Not Asking About Machine Learning
Right now, if you work with ML or AI products, you probably have questions you’re not asking because they feel basic or because you think you should already know the answers. Ask them anyway.
What is a model? Where does it come from?
Why do we build them?
How does the model get into production?
Why is this approach better than the old way?
What problem does this solve that I didn’t know I had?
Can you explain the mental model before the technical details?
These questions aren’t silly. They reveal where the explanation is breaking down, and if you’re confused, you’re probably not alone. Our writer reminded me of that yesterday.
