TL;DR: Part I: The Marketing Process and Decision Making.
Abstract: Part I: The Marketing Process and Decision Making 1. Introduction to Marketing Research 2. Ethics in Marketing Research Part II: Secondary Data and Research Designs 3. Secondary Data 4. Research Designs: Exploratory and Qualitative Research 5. Research Designs: Descriptive and Causal Research Part III: Measurement, Data Collection and Sampling 6. Measurement 7. Primary Data Collection 8. Designing the Data-Gathering Instrument 9. Sampling Methods and Sample Size 10. Fielding the Data-Gathering Instrument Part IV: Data Analysis and Reporting 11. Analyzing and Interpreting Data for Decisions 12. Advanced Data Analysis 13. The Research Report Part V: Cases
TL;DR: Nominal group technique was used in the context of four focus groups involving clinical experts from the emergency department (ED) and obstetric and midwifery areas of a busy regional hospital to assess the triage and management of pregnant women in the ED.
Abstract: This paper aims to demonstrate the versatility and application of nominal group technique as a method for generating priority information. Nominal group technique was used in the context of four focus groups involving clinical experts from the emergency department (ED) and obstetric and midwifery areas of a busy regional hospital to assess the triage and management of pregnant women in the ED. The data generated were used to create a priority list of discussion triggers for the subsequent Participatory Action Research Group. This technique proved to be a productive and efficient data collection method which produced information in a hierarchy of perceived importance and identified real world problems. This information was vital in initiating the participatory action research project and is recommended as an effective and reliable data collection method, especially when undertaking research with clinical experts.
TL;DR: While the following sections are written in the context of using interviews or focus groups to collect data, the principles described for sample selection, data analysis, and quality assurance are applicable across qualitative approaches.
Abstract: This is the second of a two-part series on qualitative research. Part 1 in the December 2011 issue of Journal of Graduate Medical Education provided an introduction to the topic and compared characteristics of quantitative and qualitative research, identified common data collection approaches, and briefly described data analysis and quality assessment techniques. Part II describes in more detail specific techniques and methods used to select participants, analyze data, and ensure research quality and rigor.
If you are relatively new to qualitative research, some references you may find especially helpful are provided below. The two texts by Creswell 2008 and 2009 are clear and practical.1,2 In 2008, the British Medical Journal offered a series of short essays on qualitative research; the references provided are easily read and digested.3–,8 For those wishing to pursue qualitative research in more detail, a suggestion is to start with the appropriate chapters in Creswell 2008,1 and then move to the other texts suggested.9–,11
To summarize the previous editorial, while quantitative research focuses predominantly on the impact of an intervention and generally answers questions like “did it work?” and “what was the outcome?”, qualitative research focuses on understanding the intervention or phenomenon and exploring questions like “why was this effective or not?” and “how is this helpful for learning?” The intent of qualitative research is to contribute to understanding. Hence, the research procedures for selecting participants, analyzing data, and ensuring research rigor differ from those for quantitative research. The following sections address these approaches. table 1 provides a comparative summary of methodological approaches for quantitative and qualitative research.
TABLE 1
A Comparison of Qualitative and Quantitative Methodological Approaches
Data collection methods most commonly used in qualitative research are individual or group interviews (including focus groups), observation, and document review. They can be used alone or in combination. While the following sections are written in the context of using interviews or focus groups to collect data, the principles described for sample selection, data analysis, and quality assurance are applicable across qualitative approaches.
TL;DR: How advances in data-intensive science provide accurate estimates in species distributions at continental scales by identifying complex environmental associations is described.
Abstract: Identifying ecological patterns across broad spatial and temporal extents requires novel approaches and methods for acquiring, integrating and modeling massive quantities of diverse data. For example, a growing number of research projects engage continent-wide networks of volunteers (‘citizen-scientists') to collect species occurrence data. Although these data are information rich, they present numerous challenges in project design, implementation and analysis, which include: developing data collection tools that maximize data quantity while maintaining high standards of data quality, and applying new analytical and visualization techniques that can accurately reveal patterns in these data. Here, we describe how advances in data-intensive science provide accurate estimates in species distributions at continental scales by identifying complex environmental associations.
TL;DR: The approach is shown to outperform the state of the art fact-finding heuristics, as well as simple baselines such as majority voting, and to offer the first optimal solution to the above truth discovery problem.
Abstract: This paper addresses the challenge of truth discovery from noisy social sensing data. The work is motivated by the emergence of social sensing as a data collection paradigm of growing interest, where humans perform sensory data collection tasks. A challenge in social sensing applications lies in the noisy nature of data. Unlike the case with well-calibrated and well-tested infrastructure sensors, humans are less reliable, and the likelihood that participants' measurements are correct is often unknown a priori. Given a set of human participants of unknown reliability together with their sensory measurements, this paper poses the question of whether one can use this information alone to determine, in an analytically founded manner, the probability that a given measurement is true. The paper focuses on binary measurements. While some previous work approached the answer in a heuristic manner, we offer the first optimal solution to the above truth discovery problem. Optimality, in the sense of maximum likelihood estimation, is attained by solving an expectation maximization problem that returns the best guess regarding the correctness of each measurement. The approach is shown to outperform the state of the art fact-finding heuristics, as well as simple baselines such as majority voting.
TL;DR: In this paper, the authors discuss the philosophical assumptions of case study research and develop a case studies research strategy based on access and ethics in case study data collection and data quality in case studies.
Abstract: What Is Case Study Research? Philosophical Assumptions of Case Study Research Developing Your Case Study Research Strategy Access and Ethics in Case Study Research Data Collection Managing and Analyzing Data Quality in Case Study Research Writing and Presenting Your Research
TL;DR: This catalog provides a starting point for researchers to consider relevant and/or comparable accelerometer decision rules, derived variables, and cut point definitions for their own research questions.
Abstract: Introduction
The National Health and Nutrition Examination Survey (NHANES) included accelerometry in the 2003–2006 data collection cycles. Researchers have used these data since their release in 2007, but the data have not been consistently treated, examined, or reported. The objective of this study was to aggregate data from studies using NHANES accelerometry data and to catalogue study decision rules, derived variables, and cut point definitions to facilitate a more uniform approach to these data.
TL;DR: In this paper, the authors used data derived from three coccinellid-focused citizen science programs to examine the costs and benefits of data collection from direct citizen-science (data used without verification) and verified citizen-scientism (observations verified by trained experts) programs.
Abstract: Citizen scientists have the potential to play a crucial role in the study of rapidly changing lady beetle (Coccinellidae) populations. We used data derived from three coccinellid-focused citizen-science programs to examine the costs and benefits of data collection from direct citizen-science (data used without verification) and verified citizen-science (observations verified by trained experts) programs. Data collated through direct citizen science overestimated species richness and diversity values in comparison to verified data, thereby influencing interpretation. The use of citizen scientists to collect data also influenced research costs; our analysis shows that verified citizen science was more cost effective than traditional science (in terms of data gathered per dollar). The ability to collect a greater number of samples through direct citizen science may compensate for reduced accuracy, depending on the type of data collected and the type(s) and extent of errors committed by volunteers.
TL;DR: It can be concluded that the real strength of focus groups is not simply in exploring what participants have to say, but in providing insights into the sources of complex behaviors and motivations.
TL;DR: The Social Hotspots Database (SHDB) as mentioned in this paper is an overarching, global database that eases the data collection burden in social life cycle assessment (S-LCA), a derivative of the well-established environmental LCA technique.
Abstract: One emerging tool to measure the social-related impacts in supply chains is Social Life Cycle Assessment (S-LCA), a derivative of the well-established environmental LCA technique. LCA has recently started to gain popularity among large corporations and initiatives, such as The Sustainability Consortium or the Sustainable Apparel Coalition. Both have made the technique a cornerstone of their applied-research program. The Social Hotspots Database (SHDB) is an overarching, global database that eases the data collection burden in S-LCA studies. Proposed “hotspots” are production activities or unit processes (also defined as country-specific sectors) in the supply chain that may be at risk for social issues to be present. The SHDB enables efficient application of S-LCA by allowing users to prioritize production activities for which site-specific data collection is most desirable. Data for three criteria are used to inform prioritization: (1) labor intensity in worker hours per unit process and (2) risk for, or opportunity to affect, relevant social themes or sub-categories related to Human Rights, Labor Rights and Decent Work, Governance and Access to Community Services (3) gravity of a social issue. The Worker Hours Model was developed using a global input/output economic model and wage rate data. Nearly 200 reputable sources of statistical data have been used to develop 20 Social Theme Tables by country and sector. This paper presents an overview of the SHDB development and features, as well as results from a pilot study conducted on strawberry yogurt. This study, one of seven Social Scoping Assessments mandated by The Sustainability Consortium, identifies the potential social hotspots existing in the supply chain of strawberry yogurt. With this knowledge, companies that manufacture or sell yogurt can refine their data collection efforts in order to put their social responsibility performance in perspective and
TL;DR: In this article, the authors describe roadway map updating using vehicle performance, route, and/or location information from plural vehicles, and present disclosure describes how to update the roadway map.
Abstract: The present disclosure describes roadway map updating using vehicle performance, route, and/or location information from plural vehicles.
TL;DR: This paper aims to solve a classification problem of GPS data into different transportation modes (car, walk, cycle, underground, train and bus), and introduces a novel approach to carry out this inference using a framework based on Support Vector Machines (SVMs) classification.
TL;DR: A simple, practical data improvement intervention significantly increased the completeness and accuracy of the data used to monitor PMTCT services in South Africa.
Abstract: OBJECTIVE: To evaluate the effect of an intervention to improve the quality of data used to monitor the prevention of mother-to-child transmission (PMTCT) of the human immunodeficiency virus in South Africa. METHODS: The study involved 58 antenatal clinics and 20 delivery wards (37 urban, 21 rural and 20 semi-urban) in KwaZulu-Natal province that provided PMTCT services and reported data to the District Health Information System. The data improvement intervention, which was implemented between May 2008 and March 2009, involved training on data collection and feedback for health information personnel and programme managers, monthly data reviews and data audits at health-care facilities. Data on six data elements used to monitor PMTCT services and recorded in the information system were compared with source data from health facility registers before, during and after the intervention. Data completeness (i.e. their presence in the system) and accuracy (i.e. being within 10% of their true value) were evaluated. FINDINGS: The level of data completeness increased from 26% before to 64% after the intervention. Similarly, the proportion of data in the information system considered accurate increased from 37% to 65% (P < 0.0001). Moreover, the correlation between data in the information system and those from facility registers rose from 0.54 to 0.92. CONCLUSION: A simple, practical data improvement intervention significantly increased the completeness and accuracy of the data used to monitor PMTCT services in South Africa.
TL;DR: Seven steps will enable researchers to complete a rigorous, qualitative research study when faced with large data sets to answer complex health services research questions, and eight recommendations will help ensure rigour for studies with large qualitative data sets.
Abstract: Health services research is multifaceted and impacted by the multiple contexts and stakeholders involved. Hence, large data sets are necessary to fully understand the complex phenomena (e.g., scope of nursing practice) being studied. The management of these large data sets can lead to numerous challenges in establishing trustworthiness of the study. This article reports on strategies utilized in data collection and analysis of a large qualitative study to establish trustworthiness. Specific strategies undertaken by the research team included training of interviewers and coders, variation in participant recruitment, consistency in data collection, completion of data cleaning, development of a conceptual framework for analysis, consistency in coding through regular communication and meetings between coders and key research team members, use of N6TM software to organize data, and creation of a comprehensive audit trail with internal and external audits. Finally, we make eight recommendations that will help ensure rigour for studies with large qualitative data sets: organization of the study by a single person; thorough documentation of the data collection and analysis process; attention to timelines; the use of an iterative process for data collection and analysis; internal and external audits; regular communication among the research team; adequate resources for timely completion; and time for reflection and diversion. Following these steps will enable researchers to complete a rigorous, qualitative research study when faced with large data sets to answer complex health services research questions.
TL;DR: The efficacy of focus group discussion as a qualitative data collection methodology is put on the line by empirically comparing and contrasting data from two FGD sessions and one-on-one interviews to ascertain the consistency in terms of data retrieved from respondents using these two data collection methodologies.
Abstract: The efficacy of Focus Group Discussion as a qualitative data collection methodology is put on the line by empirically comparing and contrasting data from two FGD sessions and one-on-one interviews to ascertain the consistency in terms of data retrieved from respondents using these two data collection methodologies. The study is guided by the hypothesis that data obtained by FGD may be influenced by groupthink rather than individual respondents' perspectives. A critical scrutiny of the data that emanated from the two organized focus groups discussion departed quite significantly from the data that was elicited from the one -on-one qualitative interviews. The difference in responses confirms that FGDs are not fully insulated from the shackles of groupthink. It is recommended, among others, that though FGD can stand unilaterally as a research methodology for nonsensitive topics with no direct personal implications for respondents; researchers should be encouraged to adopt FGD in league with other methodologies in a form of triangulation or mixed methodological approach for a more quality data, bearing in mind the central role occupied by data in the scientific research process.
TL;DR: The authors examined five mainstream public administration journals over an eight-year period regarding current methodological practice, organized around the total survey error framework, and concluded that survey research in the field of public administration features mainly small-scale studies, heavy reliance on a single data collection mode, questionable sample selection procedures, and suspect sample frame quality.
Abstract: Survey research is a common tool for assessing public opinions, perceptions, attitudes, and behaviors for analyses in many social science disciplines. Yet there is little knowledge regarding how specific elements of survey research methodology are applied in practice in public administration. This article examines five mainstream public administration journals over an eight-year period regarding current methodological practice, organized around the total survey error framework. The findings show that survey research in the field of public administration features mainly small-scale studies, heavy reliance on a single data collection mode, questionable sample selection procedures, and suspect sample frame quality. Survey data largely are analyzed without careful consideration of assumptions or potential sources of error. An informed evaluation of the quality of survey data is made more difficult by the fact that many journal articles do not detail data collection procedures. This study concludes with suggestions for improving the quality and reporting of survey research in the field.
TL;DR: Journaling as a method of data collection has long been accepted as a valid method of accessing rich qualitative data and can promote constructive and valuable participation by acknowledging the common challenges associated with the process of journaling that are experienced by the participants.
Abstract: Aims To identify the challenges associated with using journaling as a method of data collection and to offer strategies for effectively managing those challenges. Background While journaling can be used for a variety of reasons, in the context of this paper, journaling refers to the process of participants sharing thoughts, ideas, feelings and experiences through writing and/or other media. Journaling is used in phenomenological research studies to record participant experiences in their natural contexts. Data sources The findings are based on the experiences of the researchers during a qualitative study that explored the experiences of lesbian mothers and used journaling as one method of data collection. Review methods This is a methodological paper. Discussion Three main challenges affect journaling as a method of data collection: poor participation, feeling exposed and staying on track. Six strategies to promote participation in journaling are: coaching participants, limiting the journaling period, providing follow-up contact, promoting comfort, ensuring safety and providing clear content expectations. Each strategy is discussed and methods of implementing the strategies are offered. Conclusion Journaling as a method of data collection has long been accepted as a valid method of accessing rich qualitative data. By acknowledging the common challenges associated with the process of journaling that are experienced by the participants, researchers employing this data collection method can promote constructive and valuable participation. Implications for future research Further research examining participants' experiences of journaling as a method of qualitative data collection would be useful in determining challenges, barriers and benefits of the method.
TL;DR: Findings support the notion that graphic elicitation techniques can be highly useful in qualitative research studies at the data collection, the data analysis, and the data reporting stages.
Abstract: Graphic elicitation techniques, which ask research participants to provide visual data representing personal understandings of concepts, experiences, beliefs, or behaviors, can be especially useful in helping participants to express complex or abstract ideas or opinions. The benefits and drawbacks of using graphic elicitation techniques for data collection, data analysis, and data display in qualitative research studies are analyzed using examples from a research study that employed data matrices and relational maps in conjunction with semi-structured interviews. Results from this analysis demonstrate that the use of these combined techniques for data collection facilitates triangulation and helps to establish internal consistency of data, thereby increasing the trustworthiness of the interpretation of that data and lending support to validity and reliability claims. Findings support the notion that graphic elicitation techniques can be highly useful in qualitative research studies at the data collection, the data analysis, and the data reporting stages. For example, this study found that graphic elicitation techniques are especially useful for eliciting data related to emotions and emotional experiences.
TL;DR: Qualitative observational data collection methods can contribute to theoretical and conceptual development and the explanation of social processes in palliative care and should improve understanding of patients’ experiences of their care journey and thus impact on care outcomes.
Abstract: Background:Observational research methods are important for understanding people’s actions, roles and behaviour. However, these techniques are underused generally in healthcare research, including research in the palliative care field.Aim:The aim in this paper is to place qualitative observational data collection methods in their methodological context and provide an overview of issues to consider when using observation as a method of data collection. This paper discusses practical considerations when conducting palliative care research using observation.Findings:Observational data collection methods span research paradigms, and qualitative approaches contribute by their focus on ‘natural’ settings which allow the explanation of social processes and phenomena. In particular, they can facilitate understanding of what people do and how these can alter in response to situations and over time, especially where people find their own practice difficult to articulate. Observational studies can be challenging to ...
TL;DR: An algorithm embodying best practice recommendations for collecting, processing, and reporting physical activity data routinely collected with accelerometry-based activity monitors is presented, proposed as a linear series of seven steps within three successive phases.
Abstract: Although the measurement of physical activity with wearable monitors may be considered objective, consensus guidelines for collecting and processing these objective data are lacking. This article presents an algorithm embodying best practice recommendations for collecting, processing, and reporting physical activity data routinely collected with accelerometry-based activity monitors. This algorithm is proposed as a linear series of seven steps within three successive phases. The Precollection Phase includes two steps. Step 1 defines the population of interest, the type and intensity of physical activity behaviors to be targeted, and the preferred outcome variables, and identifies the epoch duration. In Step 2, the activity monitor is selected, and decisions about how long and where on the body the monitor is to be worn are made. The Data Collection Phase, Step 3, consists of collecting and processing activity monitor data and is dependent on decisions made previously. The Postcollection Phase consists of four steps. Step 4 involves quality and quantity control checks of the activity monitor data. In Step 5, the raw data are transformed into physiologically meaningful units using a calibration algorithm. Step 6 involves summarizing these data according to the target behavior. In Step 7, physical activity outcome variables based on time, energy expenditure, or movement type are generated. Best practice recommendations include the full disclosure of each step within the algorithm when reporting monitor-derived physical activity outcome variables in the research literature. As such, those reading and publishing within the research literature, as well as future users, will have the best chance for understanding the interactions between study methodology and activity monitor selection, as well as the best possibility for relating their own monitor-derived physical activity outcome variables to the research literature.
TL;DR: PARE has the right to authorize third party reproduction of this article in print, electronic and database forms and has the Right of first publication to the Practical Assessment, Research & Evaluation.
Abstract: Copyright is retained by the first or sole author, who grants right of first publication to the Practical Assessment, Research & Evaluation. Permission is granted to distribute this article for nonprofit, educational purposes if it is copied in its entirety and the journal is credited. PARE has the right to authorize third party reproduction of this article in print, electronic and database forms.
TL;DR: In this article, an examination is conducted in three school districts of how data are used to improve classroom practice, and the authors explore the effects that attitudes toward data, principal leadership, and computer data systems have on how data is used to affect classroom practice.
Abstract: In the present study, an examination is conducted in three school districts of how data are used to improve classroom practice. In doing so, we explore the effects that attitudes toward data, principal leadership, and computer data systems have on how data are used to affect classroom practice. Findings indicate that educators are ambivalent about data: they see how data could support classroom practice, but their data use operates in the presence of numerous barriers. Many of these barriers are due to principal leadership and computer data systems; these barriers often have negative effects on attitudes toward data and disrupt the progression from using data to inform classroom practice. It is hypothesized that many of these barriers can be removed through effective
TL;DR: With the use of analysis methods that produce unbiased results, planned missing data designs are an efficient way to manage cost, improve data quality, and reduce participant fatigue and practice effects.
Abstract: Data collection can be the most time- and cost-intensive part of developmental research This article describes some long-proposed but little-used research designs that have the potential to maximize data quality (reliability and validity) while minimizing research cost In planned missing data designs, missing data are used strategically to improve the validity of data collection in one of two ways Multiform designs allow one to increase the number of measures assessed on each participant without increasing each participant's burden Two-method measurement designs allow one to reap the benefits of a cost-intensive gold-standard measure, using a larger sample size made possible by a rougher, cheaper measure We explain each method using examples relevant to cognitive development research With the use of analysis methods that produce unbiased results, planned missing data designs are an efficient way to manage cost, improve data quality, and reduce participant fatigue and practice effects
TL;DR: This is the first study showing that smartphones can be successfully used for household data collection on infant feeding in rural China and should be further evaluated for other surveys and on a larger scale to deliver maximum benefits in China and elsewhere.
Abstract: Background: Maternal, Newborn, and Child Health (MNCH) household survey data are collected mainly with pen-and-paper. Smartphone data collection may have advantages over pen-and-paper, but little evidence exists on how they compare. Objective: To compare smartphone data collection versus the use of pen-and-paper for infant feeding practices of the MNCH household survey. We compared the two data collection methods for differences in data quality (data recording, data entry, open-ended answers, and interrater reliability), time consumption, costs, interviewers’ perceptions, and problems encountered. Methods: We recruited mothers of infants aged 0 to 23 months in four village clinics in Zhaozhou Township, Zhao County, Hebei Province, China. We randomly assigned mothers to a smartphone or a pen-and-paper questionnaire group. A pair of interviewers simultaneously questioned mothers on infant feeding practices, each using the same method (either smartphone or pen-and-paper). Results: We enrolled 120 mothers, and all completed the study. Data recording errors were prevented in the smartphone questionnaire. In the 120 pen-and-paper questionnaires (60 mothers), we found 192 data recording errors in 55 questionnaires. There was no significant difference in recording variation between the groups for the questionnaire pairs (P = .32) or variables (P = .45). The smartphone questionnaires were automatically uploaded and no data entry errors occurred. We found that even after double data entry of the pen-and-paper questionnaires, 65.0% (78/120) of the questionnaires did not match and needed to be checked. The mean duration of an interview was 10.22 (SD 2.17) minutes for the smartphone method and 10.83 (SD 2.94) minutes for the pen-and-paper method, which was not significantly different between the methods (P = .19). The mean costs per questionnaire were higher for the smartphone questionnaire (¥143, equal to US $23 at the exchange rate on April 24, 2012) than for the pen-and-paper questionnaire (¥83, equal to US $13). The smartphone method was acceptable to interviewers, and after a pilot test we encountered only minor problems (eg, the system halted for a few seconds or it shut off), which did not result in data loss. Conclusions: This is the first study showing that smartphones can be successfully used for household data collection on infant feeding in rural China. Using smartphones for data collection, compared with pen-and-paper, eliminated data recording and entry errors, had similar interrater reliability, and took an equal amount of time per interview. While the costs for the smartphone method were higher than the pen-and-paper method in our small-scale survey, the costs for both methods would be similar for a large-scale survey. Smartphone data collection should be further evaluated for other surveys and on a larger scale to deliver maximum benefits in China and elsewhere. [J Med Internet Res 2012;14(5):e119]
TL;DR: Research strategies for reliable data collection from the patient medical record include the development of a precise data collection tool, the use of a coding manual, and ongoing communication with research staff.
TL;DR: In this paper, the authors present two pilot projects that use mobile phone interviews for data collection in Tanzania and South Sudan, where high frequency panel data have been collected on a wide range of topics in a manner that is cost effective, flexible (questions can be changed over time) and rapid.
Abstract: As mobile phone ownership rates have risen in Africa, there is increased interest in using mobile telephony as a data collection platform. This paper draws on two pilot projects that use mobile phone interviews for data collection in Tanzania and South Sudan. The experience was largely a success. High frequency panel data have been collected on a wide range of topics in a manner that is cost effective, flexible (questions can be changed over time) and rapid. And once households respond to the mobile phone interviews, they tend not to drop out: even after 33 rounds of interviews in the Tanzania survey, respondent fatigue proved not to be an issue. Attrition and non-response have been an issue in the Tanzania survey, but in ways that are related to the way this survey was originally set up and that are fixable. Data and reports from the Tanzania survey are available online and can be downloaded from: www.listeningtodar.org.