Follow-Up, Feedback, And Fine-Tuning After Product Camp Dublin
After speaking at Product Camp Dublin in 2023, I wrote down my process for it. How can we approach a speaking opportunity as a product problem? How to apply product thinking to achieve success?
The 3 stages to a speaking engagement are:
Preparation
Performance
Post-Action
I covered the first two in the previous article. Now let’s cover post-action.
Prompt For Perspectives. Post-Action.
My first step beyond the presentation was to collect feedback. More on it below.
During the rest of the conference and happy hour, I talked to other attendees. Attendees shared their interests and thoughts, which helped me in sense-making. After the conference, I collated supplementary resources for my presentation. I shared it with all subscribers to my newsletter, including the new subscribers from the conference. You can read or subscribe here.
I also am mindful of building new relationships with interesting people. So, I reconnected with everyone I met on social media platforms like Linkedin. Lastly, I wanted to retrospect on my experience. So, I debriefed. My way to debrief was to write this article to write down my process at each step.
Collecting Feedback
I did this in a few ways. At the presentation's start, I asked for a show of hands for the number of product managers, product designers, software engineers, or other roles. This was one step in my feedback process to gauge the audience's comfort with my material and the amount of jargon vs. definitions I should use. I counted about 65% product managers, 20% UI/UX designers, 10% engineers, and 5% other roles.
The second way I gathered feedback was by doing a walkthrough. When the attendees were doing the activities, I walked around the room to eavesdrop on conversations and see their trials. You can see the slides I used to conduct the interactive activities.
In the Q&A time of the discussion, I got several interesting questions. These helped me understand the confusing and insightful parts of my talk. I used the tips from Ozan Varol, a best-selling author and professor, to encourage more questions. Later, I wrote down those questions from memory.
Attendees also filled out an anonymous survey.
Lastly, when talking to other attendees at the conference, I explicitly asked for feedback. People wanted to be friendly and not say anything negative, but when I mentioned my interest to repeat this talk at another time and my interest to do it better the next time, I got valuable feedback.
Feedback Received
Attendees anonymously (a small sample size out of 90 attendees) gave my talk an average 4.25 rating out of 5. They rated the clarity of my communication at 4.5 out of 5. I thought more attendees would find the interactive parts valuable, but the survey results did not agree.
I also got feedback that it was hard to hear me or see the visuals in my slides from the back of the room. My visuals needed to be bigger and my voice needed to be louder.
At the start of the talk, I was surprised at the lack of a mic and the small size of the presentation from the projector. Yet, I did not improvise based on my observation. So, I will keep the audience feedback in mind for the next time. I can ask for a show of hands from the audience for clarity in the visuals and my voice. I should also continue reducing text and increasing the visual size of slide illustrations.
During conversations with attendees or over social media, I gathered this feedback:
The content was very advanced. You should not assume that a lot of people know the basics. This is because not everyone has used as many tools to measure product metrics.
It will be better to use one example instead of 3. Then, instead of showing some of the steps per example, you should show all the steps for one example.
It was clear you know a lot and have done a lot. There was a lot of depth behind each slide. You had respect and attention from the crowd as they could see you know your stuff.
You should have used a different icebreaker, for example, cracking a joke. You tried a product icebreaker. But when you asked for volunteers after the product icebreaker, you were met with silence.
Good slides. The slides of other presenters were very simple, just text put together. Liked the flow of your screenshots on top of each other.
Your talk was great plus the chat after made my head spin in the right direction. I find it to be a great success if such a thing happens.
Great work on session at the event.
Great. Loved it.
Best presentation.
Best session.
Best pitch out of all.
Well done on a great talk. Customer journey maps is a great topic.
Man, great speech.
Not every feedback was elaborate.
Self-Critique: Feedback I Gave Myself Before The Conference
On Delivery: let arms relax to the sides instead of interlacing fingers. Soft delivery, which makes it monotonous. Fillers: “Ah.”
I’m looking back at the screen a lot. As I’m not sure about the animations. I need to know how many clicks there are on each slide. I should reduce clicks on other slides.
On the hotel example, I should mention that I worked on that project. Explain I choose the hotel as the first example because it is a shared experience.
I need to make the text larger on the maps.
Mention the project context for the design thinking map.
Show the software screenshots next to the map as well as thumbnails.
In “When can you use this”: I speak a lot here. See how to move some of my examples to earlier in the slide. Then reiterate them on this summary slide.
I got my spouse’s feedback:
Good energy. Good anecdotal examples. Good artwork.
There are a lot of examples where I need audience interaction. I might need to put people on the spot if there is no interaction. See whether it is working.
I mentioned JTBD and funnel in the summary slide at first. But I also told it in Approach 1 and Approach 2 slides in more depth. That repetition was confusing. I should split the summary slide into a before/after the 4 approaches.
Receptionist's artwork is confusing. Find another artwork to represent the checkout process in a budget hotel.
Questions From Attendees
Q1 - When you have a B2B2C product like a merchant and end-consumers, whose journey to draw?
If this is a two-sided platform, I suggest drawing one customer journey for the user on each side of the platform.
If this is a multi-party system, such as credit card processing, consider the customer who buys from you. Or consider the customer who interacts with your support and sales team. Draw for this customer.
Do you face gaps in your journey? Then choose another user in your ecosystem and redraw.
One B2B example of a customer journey map and 2 other examples here.
Q2 - How can you learn about the customer's journey outside of your product area focus?
You will know the most about your product area focus. So you have a dilemma of knowing your product area but wanting to know the customer’s journey outside your area.There are two steps to handle this dilemma.
First, draw the customer’s journey assuming the start of your product focus area as the start of the customer's journey. Draw the end of your product focus area as the end of your customer's journey. For example, when I drew the customer journey map for a B2B billing use case, I only drew the journey of a customer discovering their invoice and acting on it.
Second, skim news from other teams. Teams include growth, onboarding, customer success, support, solution architects, other product teams, and sales. You do not need to know everything about every team. Usually, every team in an organization sends periodic (such as quarterly) updates and hosts their documents in a public space (such as Confluence). Often their data dashboards might be in a public space. Their top-level metrics might be covered in product reviews or all-hands meetings. When you need more, reach out to those teams. Ask for read-only dashboard access. For example, I got access to marketing email analytics and heap analytics because other teams used them.
Q3 - How can you use the 3-step inverted pyramid to understand parts of the journey outside of your product, such as onboarding?
You can apply a similar approach to understanding other parts of the customer's journey. But, keep lower expectations. Read more about the 3-step inverted pyramid here: Measure Customer Experience by Merging Product Analytics, Surveys, and Interviews
For example, instead of product analytics tools for quantitative data, you use a read-only dashboard, a summary write-up, or a periodic email report.
Instead of surveying customers about other parts of their journey, you read support tickets from customers about those parts for qualitative anecdotes.
Instead of interviewing customers solely to know other parts of their journey, you listen to recordings of interviews conducted by other teams to build empathy.
Q4 - How to obtain the data?
Do you have amplitude, mixpanel, heap, google analytics, segment, looker, or tableau? Use that.
Maybe not. I suggest the mindset of How to Measure Anything: Finding the Value of Intangibles in Business by Douglas W. Hubbard. Go to first principles. What task is your customer doing? What is the result of it? What data is changed or created due to it? How can you review those modifications? How does the support team answer questions from customers? How do the sales team track their prospecting?
Do you think B2B has different data points than B2C? Yes, true. Yet, data and customer insights are still available. When you find yourself scratching your head for quantitative data, skip it. Instead, find qualitative anecdotes. Build user empathy through interviews. Once you’ve done these 2 steps, you may find ways to get quantitative data.
I used a few non-standard methods to track B2B customer experience here.
Q5 - How can you synthesize or parse the data you get from support tickets and forums? Is there an easier way to do it?
What didn’t work for me? Building a dashboard to give me the answer didn’t work. Trying to get an answer within 5 minutes of looking at anecdotes didn’t work either.
I’ll share what worked for me in the past. I read hundreds of support tickets manually. I influenced my colleagues to read 1,000s of customer emails with me. My colleagues and I tagged tickets to 140+ categories, then grouped them in 3 levels of hierarchy. I walk through the experience here. I built a dashboard to show our progress. The visual progress helped influence colleagues toward the shared goal.
In recent months, I did synthesis of anecdotes using chatGPT. I did several iterations to reduce the time required to write this article. It still took 80 hrs+. It still took a long time to synthesize 200+ years of experience. But I might have saved 20 hrs by using ChatGPT. I might have gone from 100 hrs to 80 hrs. I’ll share some tips for this in an upcoming article.
Q6 - How can you acquire data from other teams? Especially teams that don't have a process to share that data with you?
First, understand the customer, assuming the start of your product focus area is the start of the customer's journey. Draw the end of your product focus area as the end of your customer's journey. For example, when I drew the customer journey map for a B2B billing use case, I only drew the journey of a customer discovering their invoice and acting on it.
Second, “do things that don’t scale,” said Paul Graham. Ask colleagues on those teams to share the information one time. Ad-hoc information sharing without process. Ask for estimates of the data without sharing any data with you. What is the lowest fidelity information you can ask for? What is the most manual way of asking and getting the information that you can do one time?
Do it once. Then do it again. Do it periodically, manually, until you can prove its value. Prove its value, then you can convince the team to change the process. Once you’ve persuaded the team of its value, see if you need to offer guest engineering services.
This is a clash of being principled to seek data but being pragmatic that not every team will give you data.
Q7 - How can you gather data on technical services rather than business-related tasks?
Although I found that there was usually more data available on technical services than business tasks, but from this question, I see it could be the other way around.
I suggest the mindset of How to Measure Anything: Finding the Value of Intangibles in Business by Douglas W. Hubbard. Go to first principles. How does the engineering team monitor the health of their services? Is it Datadog? Rollbar? Pagerduty? Elastic search? MySQL queries? Google data studio?
Maybe none of those. Jira tickets? Slack questions? The number of late-night phone calls?
I’ve used Datadog and MySQL tables to gather data on technical services.
Q8 - How do you assign the nodes in different swimlanes? How do you know the list of agents or teams?
Step 1 - Create one swimlane per tool that gives you data. Create one swimlane per team in your organization. When in doubt, create more swimlanes. Don’t have a list of teams in your organization? Look at your organizational chart in ERP/HR software or ask 5 colleagues around you.
Step 2 - Arrange the swimlanes by looking at the amount of data they provide about customer behavior. Tools that give more insights are higher on the tracking transparency Y-axis. Tools that give summarized customer behavior information are lower in tracking transparency. Teams that interact more with customers per week are higher in tracking transparency. Teams interacting less frequently with customers are lower on the tracking transparency Y-axis.
Step 3 - Merge adjacent swimlanes if the teams or tool outputs are similar. For example, Heap, Mixpanel, Amplitude, and Hotjar might have similar outputs for your product. The Android and iOS app store might have similar insights for you.
More instructions on customer journey mapping are here. I did 3 video walk-throughs of customer journey mapping. One of them is below.
Q9 - Do you have a template for interviewing customers?
I was working to templatize my customer interview scripts, then I came across Deploy Empathy by Michele Hansen. Her book is amazing, tactical, and exactly what you are looking for. I use it as my starting point for interviewing customers.