← back to projects

IBM Watson: Learning from User Feedback

Posted on May 8th, 2015 by Graeme

IBM Watson is a technology platform that uses natural language processing and machine learning to reveal insights from large amounts of unstructured data.


For Watson to become an expert on a given domain, a business's instance of Watson has to be fed with data and information such as Word Documents, PDFs and web pages in order for it to learn.

There is also an element of manual training that must be done by a subject matter expert to make sure Watson is learning correctly from these materials. Interestingly Watson never stops learning, even after being deployed in a business and it was this area that I explored in a team of 13 designers over a short term project.

The goal was to extend Watson's training beyond the deployment phase of a Watson powered product through obtaining feedback from users in the latter stages of the product's life cycle, thereby keeping tabs on how good a job Watson is doing even after deployment.


Our outcome was not only a set of mechanisms to get feedback from users in an intuitive way, but we also came up with a patentable concept based around feedback that I unfortunately cannot show here (NDA agreements).

Ridley Scott + IBM Watson: A Conversation

Watson learns from natural language

Design Process

Understanding and exploring

Our research driven personas included:

  • End user customers who would be using Watson powered applications
  • Developers and designers external to IBM, who would be using IBM Watson services to create cognitive applications
  • Watson service developers who would be able to use customer feedback to improve their Watson service

For each, we created empathy maps to help define and understand our personas:

After understanding our users, we analysed their current scenario, highlighted opportunities to create a better experience for providing feedback using various design tools such as story boarding and scenario mapping:

Prototyping and Evaluating

Next we worked on low fidelity prototypes in the form of paper and sketches, and tested these on our sponsor users for validation and discovering new insights:

Related story

A Date With Chef Watson

  • One of our prototypes included the IBM Chef Watson concept, where we explored different ways for users to provide feedback in a web interface where Watson was suggesting cooking recipes.

From prototyping, we discovered ways in which we could retrieve feedback from end users in a way that wasn't irritating or annoying to them. This was done via the timing of the feedback, and the ease in which they could provide it. More importantly for our end product, we discovered new relationships between different the different parties involved in giving and receiving feedback, which turned out to be our main deliverable.

Go to Ruwix to learn the solution of the Rubik's Cube and other twisty puzzles like Pyraminx, Skewb and many more.