The "Curse of Knowledge" in Product Development
Putting your software in front of users brings out lots of cringeworthy experiences. As developers of new products, we all have the “curse of knowledge.” We have been living and breathing our product and we know how it works, inside and out. Knowing our product so well can be a blind spot and the best way to get around that blind spot is to put your product in front of users who have never used it before.
This is exactly what I did last week with two new users and it was both enlightening and painful at the same time. When I do these user interviews I usually structure them by asking the user to complete a task with the product. I asked each participant to model out the data structures for a dummy app. After describing their goal, I did my best not to help them at all, unless they ran into an actual software bug. I have a super strong temptation when a user gets stuck on their task to help them through it, but I try to resist this urge as lonnnng as possible. After all, when someone is using your app in the real world you will not be there to guide them through it. I want to know what they will try to do to get past the snag and how long they will try before giving up. Finally, before giving any help I ask my favorite question: “How did you expect that to work?”
This question highlights a number of important things.
- It gives insight into the users mental model of the app.
- It hints at the terminology they are familiar with.
- It gives you ideas how to make the experience more intuitive in the future.
- It gets them into a problem solving mindset so they can suggest imrprovements to the user experience.
The first round of UX testing can be super painful to watch. You’ve put so much hard work into your product only to see how clunky is can actually be. I felt that way after my last update, but it motivated me to make some major improvements this week.
My biggest takeaways from that user testing were:
- Unlocking the true value of the tool is difficult. I need a way to trigger that “aha!” moment sooner in the onboarding experience.
- The idea of “Contexts” (formerly called “Packages”) was confusing. This week I worked on delaying the introduction to that concept and trying to explain it better.
- Building relationships between models was not intuitive, but each user expected it to be done in a different way. One of the users was more backend focused while the other looked at modeling through a front-end lense. The terminology they used to described relations / references / foreign keys was different. I’m still not 100% sure how to address this. I think I need to do a few more user interviews.
This week I worked on re-implementing the build and validation hooks in the plugin system as well as trying to improve the onboarding experience around “Contexts.”
New Version of the Modeling Tool
Next Week
This coming week will be a lot of family time, but I am hoping to get a little bit of work done after the holiday. On my agenda is to fix up the plugin system so that it doesn’t require npm. Right now the tool won’t work unless you have npm installed and I don’t want this to just be for JavaScript devs. I’d also like to re-implement the migration hook for the validation system.