hilary cinis

Event: Tech23
Collection date: Oct 29, 2013

Objective

My aim was to gain a better understanding of the amount of customer/audience engagement done by startups, before and during development, as well as exploring the move towards creating “hackable” platforms rather that direct products or services.

My hunches

• Most are platform style business models (either on purpose to provide early entry to market or because they’re developer driven, most likely both)
• Most are heavily domain expertise driven
• Very little customer or audience investigation is done prior to development
• Most consider customers as marketing reference
• Lean startup methods are used

Survey

18 results were collected asking sex, age, platform/product and then open notes taken about the proposition, project history and user engagement activities.  These results were collected info from one-on-one coversations (4) with the demonstrators and from watching the pitches presented (14).

No notes were taken on the quality of the demonstrations or pitches, nor results of the day’s voting.

Backgrounds

• Strong technical domain expertise,
• Years in dev in other forms or in other products
• Majority male, between 25-34 [Male = 15, Female = 3]
• Many had experiences either running a previous small company or large team

Results

• 40% propositions are inspired by direct experiences, felt by the founders (naturally)
• 40% propositions are an extension of an existing business or activity
• 90% would best be described as a platform offering (contributers and consumers + api’s and/or app facilitiation)
• All were SaaS, cloud or web based solutions
• Specialist technical driven solutions (data scientist, robotics, computer vision, machine learning)
• End users were considered in reference to marketing and commercialisation (as part of the sales story)
• Propositions were rarely articulated in a neat summary sentence, but mainly communicated by attempts to immerse the audience in the situation by sharing the founder’s story of initial frustration (“I was stuck in a cab…”) or imagining an generalised problem (“who here has a mortgage…?”).
• Australia is a suitable customer testbed for global aspirations
• One company mentioned the Lean Startup model during questions from the panel

UX specific

• During the onstage product pitches, there were very limited mentions of customer engagement (far less than I would have expected). They were referenced as informal or quite specific to a particular customer as part of the development tradeoff. The exception were two medical devices (i. respirator; ii. robot assisted disability) who did extensive end user research.
• During the one-one interviews there were very limited mentions of customer engagement (far less than I would have expected). They were referenced as informal or quite specific to a particular customer as part of the development tradeoff.
• No mentions of formal usability testing except in the two medical device cases mentioned above, which would be required as part of fomalised standards to be met.
  • One medical device company engaged the target users and community groups extensively
  • One medical device is about to enter clinical trials
  • Two devices for elderly health monitoring systems were pitched with a glaring lack consideration for the wearers of the devices or any research into why the past 15 years of attempts have failed.
• Only one demonstrator mentioned “Intuitive interface” with no references to testing.
• Lots of prototyping in the wild, gathering feedback from various informal methods like feedback invitations.
• No mentions of metrix from launched offerings
• No mention of observed testing sessions
• One entertainment offering was live and downloadable during the presentation
• One construction offering overcame a large mental model hurdle using a very simple and clever demo where their project management software was used to build with a small project out of lego.
• One offering has now engaged a UX designer in short bursts to assist the development priorities

Conclusion

• The lack of UX mentions does not mean UX activities aren’t done, but it does show that it’s not considered a worthwhile topic to discuss in either a pitch or a conversation with potential investors or partners.
• People are fixing problems they identify strongly with and then find a market for it (bias).
• No one just wakes up with a good idea and makes it happen, these projects take years to reach the market – there were clear distinctions between the folks who’ve gotten something (anything) out and iterated, and the ones just pitching an idea.

My professional opinion

The offerings were very diverse and each situation has it’s own design problems and approaches.

Generally speaking, MVP’s and prototyping in the wild are valid approaches and I support them, as nothing beats getting your products into the hands of users to stress test the offering.

However I do wonder about the “failure” rate, and associated cost of effort. Without further interviews, I can’t comment.

Notes on reducing failure

Fit for purpose doesn’t equal useful. Failures can be mitigated and both of these can be brought down by at least some minimal UX activities, including:

1. proposition development – develop a clear hypothesis
2. talking to customers early – find out their real needs, do they match the hypothesis?
3. usability testing prior to launch to check for basic failings that may impact adoption despite the offering being fit for purpose. More on this here: uxmyths.com/post/3086989914/ by Zoltán Gócza

Domain expertise can account for the proposition being fit for purpose, and customer feedback direct from the released build can account for feature requests, fixes, bugs, real world use (stress testing).

But what happens if you’re sure it’s right and you’re listening to your customers, and it still fails? Maybe it’s the workflow, or content, information heirachy, discoverability, integration with other services, device design…

The very minimum that can be done to sanity check for these less obvious issues is usability testing with the appropriate customer/audience:

• UX design assistance offers holistic fixes because attending to bugs or feature builds alone will break other workflows
• Talk to customers to find out if fit for purpose is also attending to a need… how many maps apps do you need? Is your differentiator really that compelling?

It’s been about 13 months since I started at NICTA. Yesterday was our team offsite and I was reflecting about the time I’ve spent here so far. Thought I might put it down in writing.

In that time I’ve worked on 17 projects, across 4 business teams and in a range of capacities that fully reach into every corner of my career experience.

I’ve had to learn fast about machine learning, big data, the cloud, algorithms. That coding languages are fashionable and cyclical and how patents work. I’ve been exposed to amazing information and had to get up to speed on genetics, geothermal, business rules, transport infrastructure, radiology, performance assurance, forensic video and the health of structures like bridges.

I assist in defining product propositions and put a usable interface onto emerging technologies so NICTA can go to industry with working prototypes that not only work for real, but also help to showcase the technologies used, to assist and influence change in industry. These then spin-out into fully fledge products and companies. Just like a startup incubator.

Some of the projects have been full end-to-end ux, interaction design and ui design right up to visual design and review; others have been simple consultations advising and/or directing developers on best practises or assisting in proposition development. And there has been a broad range of all in-between including art direction for video clips and brand designs.

The methodologies for each project have also varied and include full agile, “agile-but”, and the deliverables up front approach – all with the same thing in common, deep collaborative engagement with the full team to produce a viable product.

I work everyday with people who are deeply passionate about their projects, there is almost no ego and there is a genuine dedication to their work that I’ve not seen before anywhere I’ve worked.

They are seriously, the smartest people I’ve ever met and I constantly oscillate between mind boggling awe of their awesomely cool work and feeling like an ant surrounded by giants.

And they really like my work. So I am humbled and honoured and have to say it’s been the best move I ever made.

All those things you hear from the mass “Why isn’t there thing that does such-and-such?” “You think someone would come up with a better whatsy”

Well these people are building those things and more.