Note that their model assumes a random sample. Data curation, Over time, the broader term data saturation has become increasingly adopted, to reflect a wider application of the term and concept. Those three questions generated 55 codes. In this paper we present a way of assessing thematic saturation in inductive analysis of qualitative interviews. You can manage to achieve trustworthiness by . (2009). Pay close attention to the data to ensure that youre not picking up on things that are not there or obscuring things that are. Saturation will inevitably occur in a retrospectively-assessed, fully-analyzed, fixed-size dataset. Check out the dedicated article the Speak Ai team put together on Can Thematic Analysis Be Used In Quantitative Research? We will prospectively calculate saturation using a base size of 4 interviews and run length of 2 interviews. In reviewing these studies to inform the development of our approach to assessing saturation, we identified three limitations to the broad application of saturation assessment processes which we sought to overcome: lack of comparability of metrics, reliance on probability theory or random sampling, and retrospective assessment dependent on having a fully coded/analyzed dataset. If we encounter problems with our themes, we might split them up, combine them, discard them or create new ones: whatever makes them more useful and accurate. These codes allow us to gain a a condensed overview of the main points and common meanings that recur throughout the data. Options for each metric can be specified prior to analysis or reported after data analysis. Saturation is conceptualized as a relative measure. https://doi.org/10.1371/journal.pone.0232076.s001. The run length is the number of interviews within which we look for, and calculate, new information. Powered by Speak Ai Inc. Made in Canada with. Creswell, J. W., & Clark, V. L. P. (2018). We could increase the run length to 3 (or an even larger number), and/or we could set a more stringent new information threshold of no new information. Automatically generate transcripts, captions, insights and reports with intuitive software and APIs. Check out the dedicated article the Speak Ai team put together on What Is Back End Speech Recognition to learn more. To choose the right statistical methods and techniques, you need to consider the, Qualitative Data Analysis Methods 101: The Big 6 Methods (Including Examples), Sampling Methods & Strategies 101: What You Need To Know, Qualitative Data Coding 101: Everything You Need To Know. Next, we look over the codes weve created, identify patterns among them, and start coming up with themes. Qualitative vs Quantitative Research: Methods & Data Analysis The lower the new information threshold, the less likely an important number of themes may remain undiscovered in later interviews if data collection stops when the threshold is reached. And lastly, these descriptive statistics help. Base size refers to how we circumscribe the body of information already identified in a dataset to subsequently use as a denominator (similar to Francis et al.s initial analysis sample). The concept of saturation was first introduced into the field of qualitative research as theoretical saturation by Glaser and Strauss in their 1967 book The Discovery of Grounded Theory [10]. Many policy researchers are predisposed to use either quantitative or qualitative research methods regardless of the research questions at hand, leading to varying degrees of gaps in . That said, if . Whilst they are increasingly used and have gained greater legitimacy, much less has been written about their components parts. No, Is the Subject Area "Binomials" applicable to this article? For Dataset 1 (Table 2), at the 5% new information threshold, the median number of interviews needed to reach a drop-off in new information was consistent across all base sizes. Which type you choose depends on, among other things, whether . comparing reflexive thematic analysis and other patternbased qualitative analytic approaches. After youve decided thematic analysis is the right method for analyzing your data, and youve thought about the approach youre going to take, you can follow the six steps developed by Braun and Clarke. We offer personal insights and practical examples, while exploring issues of rigor and trustworthiness. Benefits of Using Thematic Analysis in Quantitative Research. Competing interests: The authors have declared that no competing interests exist. At the 0% new information threshold, saturation was indicated at 12+2 and 16+3, consistent across base sizes. Thousand Oaks, CA: Sage. Taken together, the concepts of base size, run length, and new information threshold allow researchers to choose how stringently they wish to apply the saturation conceptand the level of confidence they might have that data saturation was attained for a given sample (Fig 2). Empirical research to address this issue began appearing in the literature in the early 2000s. The coding manual for qualitative researchers (3rd ed.). Based on these thresholds from 10,000 resamples, for each dataset we computed the median and the 5th and 95th percentiles for number of interviews required to reach each new information threshold across different base sizes and run lengths. Across academic disciplines, and for about the past five decades, the answer to this question has usually revolved around reaching saturation [1, 59]. Hagaman and Wutich (2017) and Francis et al. Qualitative research is multimethod in focus, involving an interpretive, naturalistic approach to its . Basing assessments of saturation on probabilistic assumptions (e.g., Lowe et al. Theres also the distinction between a semantic and a latent approach: Ask yourself: Am I interested in peoples stated opinions (semantic) or in what their statements reveal about their assumptions and social context (latent)? [16] conducted a pioneer methodological study using data collected on environmental risks. Advertisement We argue that thematic analysis is a qualitative research method that can be widely used across a range of epistemologies and research questions. A central organising concept captures the essence of a theme. What Is a Research Design | Types, Guide & Examples - Scribbr Can we conduct thematic analysis of secondary data? For our analyses we provide both options for run lengths in our calculationstwo events and three eventsto afford researchers more flexibility. An important stage in planning a study is determining how large a sample size may be required, however current guidelines for thematic analysis are varied, ranging from around 2 to over 400 and it is unclear how to . Here, we return to the data set and compare our themes against it. It's a site that collects all the most frequently asked questions and answers, so you don't have to spend hours on searching anywhere else. What qualitative and quantitative data have in common with one and another? The interviews following Interview 12, though yielding four additional themes, remained at or below the 5% new information threshold. Using descriptive statistics, you can summarize your sample data in terms of: The distribution of the data (e.g., the frequency of each score . We might decide that a better name for the theme is distrust of authority or conspiracy thinking. Some types of research questions you might use thematic analysis to answer: To answer any of these questions, you would collect data from a group of relevant participants and then analyze it. thematic analysis? Can I use Thematic Analysis for Mixed-Method approach? Descriptives describe your sample, whereas inferentials make predictions about what youll find in the population. While qualitative research methodologies are now mature, there often remains a lack of fine detail in their description both at submitted peer reviewed article level and in textbooks. by Dataset 3 (Table 4) contained more variation in the sample than the others, which was reflected in a slightly higher median number of interviews and a lower degree of saturation. Thematic analysis can be used to analyze quantitative data sets in order to identify patterns and themes. Thematic analysis is one of the most important types of analysis used for qualitative data. No, Is the Subject Area "Cross-cultural studies" applicable to this article? Selecting and interpreting levels of rigor, precision, and confidence is a subjective enterprise. . 61). Transcripts were imported into NVivo [33] to facilitate coding and analysis. PLOS ONE promises fair, rigorous peer review, Using the total number of themes in the dataset retrospectively, the number of themes evident across 67 interviews corresponded with a median degree of saturation of 78% to 82%. How to Do Thematic Analysis | Step-by-Step Guide & Examples. They found that the first five to six interviews produced the majority of new information in the dataset, and that little new information was gained as the sample size approached 20 interviews. If we consider the hypothetical data set used here (see Fig 3) and kept the run length of 2, the 0% new information threshold would have been reached at interview 10+2. Jack Caulfield. Formal analysis, [The data used for each step are included in Fig 3, along with indication of the base, runs, and saturation points. The number of new themes evident across 1216 interviews corresponded with a median degree of saturation of 69% to 76%. We propose that furnishing researchers with optionsrather than a prescriptive thresholdis a more realistic, transparent and accurate practice. This is important from an efficiency perspective. Quantitative and qualitative data are both used for research and statistical analysis. Methodology, There are several benefits to using thematic analysis in quantitative research. PLoS ONE 15(5): While methods of data collection and data analysis represent the core of research methods, you have to address a range of additional elements within the scope of your research. The interview was a follow-up qualitative inquiry into womens responses on a quantitative survey. We would annotate these two extra interviews (indicative of run length) by appending a superscript +2 to the interview number, to indicate a total of eight interviews were completed. Discrepancies in code application were resolved through discussion, resulting in consensus-coded documents. Thematic analysis is a method of analyzing qualitative data. Across four datasets, approximately 80% to 92% of all concepts identified within the dataset were noted within the first 10 interviews. This renders a quotient of 11%, still not below our 5% threshold. Check out the dedicated article the Speak Ai team put together on What Is A Normal Speech Recognition Threshold to learn more. And, like power calculations, they are moot once data collection begins. Most of the time, youll combine several codes into a single theme. This method also enables researchers to select different levels of the constituent elements in the processi.e., Base Size, Run Length and New Information Thresholdbased on how confident they wish to be that their interpretations and conclusions are based on a dataset that reached thematic saturation. In the typical application of thematic analysis to systematic reviews of qualitative research, the goal is to locate themes that apply across the results of the various studies . Muhammad Jan. University of . We selected three existing qualitative datasets to which we applied the bootstrapping method. Substantially more women from the Kenya sample were married and living with their partners (63% versus 3%) and were less likely to have completed at least some secondary education. Since we had available the total number of codes identified in each dataset, we carried out one additional calculation as a way to provide another metric to understand how the median number of interviews to reach a new information threshold related to retrospectively-assessed degrees of saturation with the entire dataset. Their calculation incorporates: (1) the estimated prevalence of a theme within the population, (2) the number of desired instances of that theme, and (3) the desired power for a study. Morgan et al. For example, we might look at distrust of experts and determine exactly who we mean by experts in this theme. Are we missing anything? Thematic analysis remains a goods approach to research where you're trying to find out something about people's views, considerations, skills, adventures or values from a set to qualitative data - for example, interview transcripts, social media profiles, or survey responses. From an applied standpoint this finding is important in that researchers can feel confident that choosing a more stringent new information thresholde.g., 0%will result in a more conservative assessment of saturation, if so desired. This means that relative to the total number of unique codes identified in the first four, five, or six interviews, the amount of new information contributed by interviews 7 and 8 was less than or equal to 5% of the total. For each qualitative dataset, we generated 10,000 resamples from the original sample. While these three studies offer diverse and analytically rigorous case studies, they provide limited generalizability. The researcher closely examines the data to identify common themes topics, ideas and patterns of meaning that come up repeatedly. Yes For Dataset 3, two coders conducted this type of inter-coder reliability assessment on 20% of the interviews (a standard, more efficient approach than double-coding all interviews [2]). For example, we might decide upon looking through the data that changing terminology fits better under the uncertainty theme than under distrust of experts, since the data labelled with this code involves confusion, not necessarily distrust. The researcher closely examines the data to identify common themes topics, ideas and patterns of meaning that come up repeatedly. We tested our method on single-tier codebooks, but qualitative researchers often create hierarchical codebooks. The practical implication of this finding is that researchers can choose a longer run lengthe.g., three interviews (or more)to generate a more conservative assessment of saturation. https://doi.org/10.1371/journal.pone.0232076.g001. Thematic analysis can be used to draw conclusions about the data, as well as to identify relationships between the data and other variables. An extract from one interview looks like this: In this extract, weve highlighted various phrases in different colors corresponding to different codes. This neutralizes differences in the level of coding granularity among researchers, as the method affects both numerator and denominator. Interested in ChatGPT For 1-on-1 Interviews? (2010), for example, consider runs of three data collection events each time they (re)assess the number of new themes for the numerator, whereas Coenen et al. This is done by randomly resampling from the sample with replacement (i.e., an item may be selected more than once in a resample) many times in a way that mimics the original sampling scheme. Qualitative research, on the other hand, relies on the collection of non-numerical data, such as words and phrases, to gain insights. Methodology, Some types of research questions you might use thematic analysis to answer: Although the datasets were all generated from individual interviews analyzed using an inductive thematic analysis approach, the studies from which they were drawn differed with respect to study population, topics of inquiry, sample heterogeneity, interviewer, and structure of data collection instrument, as described below. At a run length of two interviews, the median number of interviews required before a drop in new information was observed was six. The same can be said for how a researcher chooses to report and interpret statistical findings. Have a human editor polish your writing to ensure your arguments are judged on merit, not grammar errors. Once youve decided to use thematic analysis, there are different approaches to consider. If you want to cite this source, you can copy and paste the citation or click the Cite this Scribbr article button to automatically add the citation to our free Citation Generator. Investigation, Whats the difference between thematic analysis and IPA? Thematic analysis is an important tool for researchers to consider when conducting quantitative research. Employing this same logic, Fugard and Potts [21] developed a quantitative tool to estimate sample sizes needed for thematic analyses of qualitative data. The two branches of quantitative analysis, How to choose the right quantitative methods. In addition, we randomly ordered the selected transcripts in each resample to offset any order effect on how/when new codes are discovered. For more information about PLOS Subject Areas, click Researchers have options for how they describe saturation and can also use the term with more transparency and precision. 2022 - 2023 Times Mojo - All Rights Reserved Can I use thematic analysis in case study? This method can emphasize both organization and rich description of the data set and theoretically informed interpretation of meaning. Note that in our analyses, successive runs overlap: each set of interviews shifts to the right or forward in time by one event. What is the difference between thematic analysis and framework analysis? With statistics, you can summarize your sample data, make estimates, and test hypotheses. Many existing definitions are constrained by a dichoto-mous typology that contrasts qualitative and quantitative research or assumes a particular epistemological foundation. Generate accurate APA, MLA, and Chicago citations for free with Scribbr's Citation Generator. But the next question is a purely subjective one: What level of paucity of new information should we accept as indicative of saturation? For this example, we have selected a new information threshold of 5% to indicate that we have reached adequate saturation. Since the last two interviews did not add substantially to the body of information collected, we would say that saturation was reached at interview 6 (each of the next two interviews were completed to see how much new information would be generated and whether this would fall below the set threshold). Global Health, Population, and Nutrition, FHI 360, Durham, North Carolina, United States of America. No, Is the Subject Area "Statistical data" applicable to this article? Inductive/Deductive Hybrid Thematic Analysis in Mixed Methods Research These bootstrap findings give us information on how saturation may be reached at different stopping points as new themes are discovered in new interviews and when the interviews are ordered randomly in different replications of the sample of interviews. https://doi.org/10.1371/journal.pone.0232076.t009, https://doi.org/10.1371/journal.pone.0232076.t010, https://doi.org/10.1371/journal.pone.0232076.t011. Quantitative research is the process of collecting and analyzing numerical data. Methods that calculate saturation based on the proportion of new themes relative to the overall number of themes in a dataset (e.g., Guest et al. Roles Is the Subject Area "Qualitative studies" applicable to this article? The expression of Theme A is not necessarily to the exclusion of Theme B, nor does the absence of the expression of Theme A necessarily indicate Not-A. Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories. During the past two decades, scholars have conducted empirical research and developed mathematical/statistical models designed to estimate the likely number of qualitative interviews needed to reach saturation for a given study. Qualitative data is defined as non-numerical data, such as text, video, photographs, or audio recordings. Hence, it could be concluded that evaluation is both quantitative and qualitative. The number of new themes found in the run defines the numerator in the saturation ratio. Results from this analysis indicate the method we propose to assess and report on saturation is feasible and congruent with findings from earlier studies. Rewrite and paraphrase texts instantly with our AI-powered paraphrasing tool. Click through the PLOS taxonomy to find articles in your field. There are several benefits to using thematic analysis in quantitative research. Thematic analysis helps researchers understand those aspects of a phenomenon that participants talk about frequently or in depth, and the ways in which those aspects of a phenomenon may be connected. Check out the dedicated article the Speak Ai team put together on What Is A Acoustic Model In Speech Recognition to learn more. For inductive thematic analyses this is a subjective decision that depends on the degree of coding granularity necessary for a particular analytic objective, and how the research team wants to discuss saturation when reporting study findings. What are the benefits of using both qualitative and quantitative research? Could new themes start emerging later in the data collection process? Quantitative research is the opposite of qualitative research, which involves collecting and . This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. [26], Fugard & Potts [21], Galvin [20]) ignores the fact that most qualitative research employs non-probabilistic, purposive sampling suited to the nature and objectives of qualitative inquiry [28]. Writing original draft, Other codes might become themes in their own right. However, thematic analysis is a flexible method that can be adapted to many different kinds of research. This is compounded by the fact that detailed descriptions of methods are often omitted from qualitative discussions. Themes get identified by physically sorting the examples into piles of similar meaning. Thematic analysis is a method that is often used to analyse data in primary qualitative research. In triangulation methods of research, thematic analysis (Braun & Clarke, 2006) could be used to analysed for open . Samples and populations can both be represented by a qualitative variable and/or a quantitative variable, because the definition of sample and population does not include anything about the type of variables that can be used within it. For example, Onwuegbuzie and Johnson (2021) note "data analysis in mixed methods research [can be]the most difficult step of the mixed methods research process" (p. 1) and there is a "lack of methodological guidance in the extant literature on these topics" (p. 16). Summarizing a quantitative study is relatively clear: you scored 25% better than the competition, let's say. Qualitative variables are nominal and ordinal. Abstract. No credit card is required. In practice, most researchers agree that combining quantitative and qualitative techniques (sometimes called mixed method research) produces a richer and more comprehensive understanding of a research area. In this example, were using a run length of two, so include data for the next two interviews after the base seti.e., interviews 5 and 6. What is the difference between thematic analysis and content analysis? Here again the unit of analysis is the data collection event; the items of analysis are unique codes. Can qualitative and quantitative variables be used to describe both samples and populations? He found the probability of identifying a concept (theme) among a sample of six individuals is greater than 99% if that concept is shared among 55% of the larger study population. Funding: The authors received no specific funding for this work. Your email address will not be published. Now we have to make sure that our themes are useful and accurate representations of the data. No, Is the Subject Area "Research design" applicable to this article? How to Do Thematic Analysis | Guide & Examples - Scribbr We acknowledge and agree with these assertions. Content analysis, on the other hand, can be used as a quantitative or qualitative method of data analysis.