The art of mixed methods

The art of mixed methods

Mixed methods research is an art, and the quality of the final piece is very much dependent on the skill of the researcher to understand all the different processes in play, how they relate to one another, and what the outcome might look like.

If you allow me some creative literary license, the art metaphor neatly sums up some of the key considerations for being a mixed methods researcher.

The interplay of different parts

First, mixed methods research, just like art, is about creating something where the whole is greater than the sum of its parts. For this, it is vitally important to understand what parts play a role. This is driven by the research question(s). Understanding exactly what it is that the client wants to explore is the most important part of the process, and it is this understanding that will govern the success of the project. Too many times I have seen mixed methods shoehorned into a project as it is perceived to add value when a set of focused individual interviews or a survey would have sufficed.

Watercolour_Mixed Methods
The whole is greater than the sum of its parts.

Choosing which methods can be tricky as it is not just a case of throwing every methodological tool at your disposal at the problem. Just like an artist would not use their entire pallet, or set of paint brushes available to them, they must choose the right combination to get the best result.

Traditionally, these methods have included the familiar semi-structured interview, focus group, or surveys, but in modern times, digital/mobile ethnography, eye tracking, facial and emotion tracking, discrete choice analysis or conjoint analysis all provide a different view. By having a clear understanding of the research question(s), it is possible to choose the right combination of research techniques to appropriately address the research question(s).

Implementation

Second, once the final picture can be imagined, and the most appropriate research techniques chosen, it is about implementing them in a logical order that follows the design process. In art, this is following principles of form, shape, line, contrast, and movement (I am sure there are many more) and what happens if any of these are modified by the artist.

In mixed methods research, it is about understanding the design process, why it occurs in that order, and what happens if any part of the method is changed. For instance, it might be sensible to have qualitative interviews before a quantitative survey as the qualitative data is being used to devise metrics that go into the survey. The opposite can also be true if you want to explore quantitative trends in more detail.

Synthesise

Third, synthesising mixed methods data is a bit of a dark art (pun intended) as it is essentially combining two fundamentally different sources of data. On the surface this might seem straightforward, but it takes care and consideration to add true insight. It is not enough to simply collect and analyse qualitative and quantitative data; they need to be mixed in some way so that together they form a more complete picture than they do when standing alone.

Origami paperwork_mixed methods
Merging or converging data.

There are three ways to do this: i) merging or converging the two data sets by bringing them together, ii) connecting the two datasets by having built on the other, or iii) embedding one dataset within the other so that one type of data provides a supportive role for the other dataset. Overlaying the qualitative data to explain the why and the how then provides an insightful and nuanced interpretation of the landscape.

Desk research

Desk research can also add invaluable context here and take the insights to new heights by overlaying the external landscape onto the findings. This is important as research rarely happens in isolation so weaving the insights into the wider fabric is key (I appreciate I transitioned to a haberdashery metaphor here).

Finally, art is often described in terms of what it represents, be it a portrait, landscape, or an abstract piece. The same is true for mixed methods research – the methods used ultimately dictate the outcome.

It is important to understand and consider each dimension of the mixed methods research and to always keep an eye on validity, as it will impact the outcome, and dictate whether you have a happy client or not.

In conclusion…

Mixed methods research is a way to offset the weaknesses of the other so that you achieve an inclusive, pluralistic, and complementary understanding of the research problem.  For instance, when you use only quantitative approaches, it is difficult to capture the understanding of contexts, perspectives, and cultural influences in a real-life phenomenon. To overcome this issue, the mixed-method approach employs quantitative research to assess the magnitude of the problem and qualitative research to explore the meaning and understanding of the construct and the context.

More detailed information on how to conduct mixed methods research.

By, Tom Parkman, Senior Research Lead at ExperienceLab.