How do people with vastly different ways of thinking about the world collaborate productively? It’s a knotty problem, and the simplest answer is that very often they don’t. Rather, even in the innovation sphere, people stick to the disciplines they know and deploy the associated methodological tools they’re familiar with. But innovation emerges from novel mixtures of ideas, techniques and mindsets. Getting people talking – and most importantly, collaborating – has many advantages.

We’ve been looking into how and why people use mixed innovation methods in practice, talking to colleagues in Nesta, people from other organisations, and looking into the literature. We’ve learnt a great deal. We’re still digesting its nuance, but for us the most important headline is that mixed methods work. Our interviewees spoke with enthusiasm about how mixing methods enabled them to approach problems in new ways and identify novel and effective solutions.

For example, the International Rescue Committee drew on qualitative interviews and ethnography, behavioural science, design and experimentation to identify ways to challenge men’s attitudes towards gender that lead to intimate partner violence. Mixing methods allowed the research team a more nuanced insight into the experiences of the communities they worked alongside, the capacity to design better interventions, and a better understanding of what worked and what didn’t.

We’ve used mixed methods in our work at Nesta too. For example, our work on artificial intelligence (AI) in education drew on data analytics, elements of design, and strategic foresight to examine how AI is used in education, what the public thinks of this, and how it might be used in the future. Mixing methods was key to the arguments made in the report, helping to think through the potential design of AI tools; modes of collaboration between AI education companies, schools and other parties; and to imagine potential opportunities and challenges that AI in education might generate.

In this blog, we want to share five of what we think are the most important things we’ve learnt about ways of working in mixed-methods teams.

In a recent Twitter thread on mixed methods, a sociology professor expressed her frustration that ‘publishing qualitative research means clarifying 10,000 times that you are aware that findings from your small sample may not apply to every human being in the universe’. In a sympathetic response, a data scientist critiqued his own discipline noting that this happens ‘while quantitative research keeps making overgeneralized inferences ignoring nuances in the data and population of interest’. ‘Very true,’ came a diplomatic reply, ‘that’s why a mixed-methods approach is often very useful’. We couldn’t agree more.

The frustrations expressed by both sides are at their heart questions about what counts as rigour. Done well, all methods are rigorous in that they capture the particular aspects of a phenomenon they are designed to capture. In this way, sample size is no more an intrinsic measure of rigour than is interview duration or fullness of observation notebook during an ethnographic visit. This echoes the assertions of Justin Parkhurst in his book, The Politics of Evidence (PDF) (see p.123), where he argues that methods should be judged by their own internal quality standards and their suitability for the task at hand.

Judging methods differently can be challenging. In one of our evidence-gathering interviews, the respondent highlighted how there can be a disconnect when executives and those carrying out the work come from different methodological backgrounds, each with their own standards of what a successful project should deliver.

All methods are underpinned by their own systems of thought and measures of value. Because there’s no such thing as a neutral method, there’s no point in going looking for methods without bias. Just as different methods have different measures of rigour, different methods address their inevitable and inherent biases in different ways. Working in a mixed-methods team entails engaging with the different imperfections that each method has, and having the confidence to recognise that often failable methods will nevertheless produce findings which are good enough to work from.

Terms like ‘mixed methods’ can mean different things to different people. Likewise, as another interviewee described, the methods people use can employ the same word for different things or different words for the same thing. This is a consequence of methodologies in long-siloed disciplines that have evolved their own vocabulary for the work they do in them.

When working in a mixed-methods domain, it’s important to clarify the meaning behind the terminology you’re using to make sure everyone in the conversation is talking about the same thing. Where there are differences in terminology or method, it’s also important to identify whether there are meaningful differences between them. This is often a question of pragmatism. Is a service safari (PDF) really substantially different from ethnography, for instance, or is data science just a fancy rebranding of what used to be called statistics? Sometimes vocabulary choices can be simply semantics, though they may too be acts of gatekeeping by insiders of a method or approach.

What was clear from all our interviewees with experience in mixed-methods approaches was that humility is essential to working in this way. Openness to the unexpected, being willing to put yourself in situations you don’t quite understand, and being generous with colleagues who ask questions by communicating answers clearly and simply are attributes that make mixing methods work well. Creating a work environment where this is possible is vital, where trust, experimentation and curiosity are valued as highly as any other project output.

Humility and good communication extend to other elements of day-to-day practice too. A strong and healthy team dynamic is vital as mixed-methods teams often encounter new challenges for which there is no clear precedent. This may engender debate and disagreement, but there are ways of making this productive not divisive. In agile working, creating this dynamic in mixed-methods teams may be done with shared ceremonies and other novel ways of creating dialogue.

One of our interviewees ran a team of data scientists and sociologists. To open the conversation between each of the methods, participants were encouraged to use the communication tools of their colleagues: the sociologists were signed up to Github and Medium while data scientists found themselves attending reading groups to discuss writing on issues like justice or equity.

They say that to a hammer, every problem looks like a nail. As such, to a devout data scientist, there is a risk that every problem could look like a series of data points, and to an expert in randomised controlled trials, every situation could look like an opportunity for an experiment. This is not to knock any particular methods specialist, but rather to illustrate one of the most common themes from our interviews: be inquisitive, start with the question and choose the best methods to answer that question.

At Nesta, we’re excited by the possibilities a mixed-methods approach can offer and we’ve built our new strategy (soon to be published) around it. In the years to come, we’ll be working on some of society’s most challenging problems in interdisciplinary teams. We’ll be drawing data science, design, behavioural science, experimentation, collective intelligence, and arts and culture methods together with strong research and analytical capacity to design, test and scale solutions that bring about change. As such, we’ll continue to be heavily involved in this space both as practitioners and contributors to the conversation.

This blog’s authors are involved in a project to understand how to work better with mixed innovation methods. In the coming months, we’ll be sharing a range of resources based on our work which we hope will be useful to others working in this domain. We’ll also be using our own work to test how these approaches work in practice, and describing the value they add to solving the world’s trickiest problems.