testing the validity and reliability of research instruments

This is the degree to which the questions elicit the same type of information each time we use them, under the same conditions. Reliability, on the other hand, is concerned with the consistency of our measurement. To achieve this, we need to include other brands that represent this category, otherwise, we can’t extrapolate the results to the category as a whole.

In our example, assume we want to estimate the share of preference of our product in the hairstyling product category. In other words, the questions should represent how the target population talks and think about the issue under research, which often calls for the need to conduct exploratory qualitative research. As we all know, the way we ask questions will determine the answer we get. This refers to the extent to which the results can be generalized to the target population the survey sample is representing. Here we are looking for a relationship between independent variables (e.g., hair type, hairstyle, etc.) and the dependent variable (e.g., likelihood to buy the hair styling products). In our hair styling product example, we need to ask questions that help us identify factors that influence the selection of hair styling products. This asks whether the questions we pose can really explain the outcome we want to research. Consequently, we may end up with an incomplete picture of consumers’ behavior in this category. In this case, we are likely to miss information about product usage under different weather conditions (given that humidity can give you a bad hair day in a blink of an eye). This is related to our ability to create questions that reflect the issue we are researching and make sure that key related subjects are not excluded.įor example, we may want to learn how consumers use hair styling products and only ask how they used them in the past week. There are other types of validity survey writers should strive for. The questions seem like a reasonable way to obtain the information we are looking for, but are they really?

Most surveys often have what is called face validity, which is a matter of appearances. In other words, it depends on asking questions that measure what we want to measure. Although often discussed in the context of sample representativeness, we know that survey design also affects validity. Validity is concerned with the accuracy of our measurement. Questions’ wording, order, format, structure, and visual layoutĪlthough all these issues are important, at the end of the day, what we want is to create surveys that yield results that are valid and reliable.ĭiscussions about validity and reliability are common in the field of psychometrics, but not so much in market research.We need to consider many things in order to write surveys that gather high-quality data. We expect validity and reliability in surveys, but a lot of work is required to achieve both.

Testing the validity and reliability of research instruments