Benchmarking Question: One way to check the accuracy of the survey


Acharaporn Kamornboonyarush / IEEM, Patrick Photo, Jupiter (through all the Getty)

Have you smoked at least 100 cigarettes in your life? Have you practiced yoga last year? On average, how much sleep do you have in one day?

At the expense of the face, these questions are not directly related to the subjects. Pew Research Center is most committed to the study. Yet our researchers are asking these types of questions from time to time. Why? These are examples of benchmarking questions, which the center uses as a check to make sure that our surveys are accurate.

Why and how do we use benchmarking questions

To determine the accuracy of the survey, some type of objective standard is required against which the survey can be compared. In the elections for elections and other measures to intensify voting, the result of the standard election. But for those surveys who do not ask for the election or voting intentions, researchers have to find another way to benchmark their findings. It is often done with the help of other surveys – usually large, expensive, government survey conducted with data focusing on quality.

Pew Research Center surveys sometimes include questions about economic, demographic, and lifestyle features for which government figures are available as a benchmark. This not only helps us to check the accuracy of our findings, but also helps us to study how the surveys can be improved.

For example, take a Pew Research Center study from last year, who had examined Low reaction rate – Many potential respondents are being contacted but very few of them are participating – the meaning of telephone surveillance is correct. To help answer this question, the study compared the results of a telephone survey by the federal government with high reactions, benchmark surveys, which if there was any, differences existed.

On the major demographic and lifestyle standards, the Pew Research Center Survey closely aligns with federal surveys

The report found that the Pew Research Center survey was closely linked to federal surveys on major demographic and lifestyle benchmarks. In 14 questions about personal symptoms, the average difference between government estimates and telephone survey estimates of the center were 3 percentage points. Differences on individual questions ranged from 0 to 8 points. The biggest answer was on the respondents asking about their health status: the government found that 59% of people have rated their health as very good or excellent, while 51% of the telephone survey of the center was found. is.

The other 13 items were quite close to the benchmark, most of which were the differences of 3 percent or less, which were usually within the margin of error. These questions include family income, employment status, domestic size, citizenship, health insurance, residence length at current address, marital and parental status, smoking frequency, birth place (among Hispanics) and driver's license. In other words, on these remedies, low-response telephone surveys provided comparatively results for high-yielding governmental surveys, used as benchmark.

There is more reports of citizen engagement compared to benchmark surveys

Overall, the report has shown that due to low response rates, the bias presented in the survey are limited in scope. And, critically, telephone survey estimates for party affiliation, political ideology and religious affiliation continue to keep track with estimates from the high response rate benchmark survey.

However, the center and other survey researchers have Widely discussed, Telephone surveys continue to create big bias on measures of citizens and to a lesser extent political affiliation. This discrepancy is probably because of nonresponse bias – In which the types of people agreeing to participate in the survey are systematically different from those who can not be contacted or refused to participate. As found Previous workThose who respond to the survey are likely to be the only person involved in community life – they are included, and participating in the survey is of the same kind Supportive social behavior Volunteerization is related to other types of behavior. Fortunately for voters, there is a civil union Does not have correlation Researchers study with political views or other remedies to study in the survey.

Warning about benchmark

Politically active adults represented more in the Pew Research Center survey

Although large government surveys are generally considered to be high data quality, they are not immune to some of the problems faced by each survey researcher. For example, compared to the opinion polls conducted by other surveys, government surveys have a very high response rate (on orders of 60% or more) nonresponse bias Still exists

Government surveys are still subject to careful planning and testing measurement error, The way the questions can be asked (like a question arises instantly with a particular question, whether a survey has been done on the phone or online etc.). The Pew Research Center questionnaire, which includes benchmarking questions, does not have to repeat the exact context in which the original questions were asked, especially because the center focuses on different topics from benchmark surveys. Benchmarks are generally unavailable for questions about approaches and practices which the government does not study.

All surveys may also face issues of reaction bias, including social preference bias, where respondents can modify the answers to some questions so that they can present themselves in a more favorable manner. This is especially a risk when an interviewer asks sensitive questions: For example, respondents can increase their frequency of voting.

All these factors can affect the likelihood of similar questions to be asked on various surveys including government surveys. That said, the benchmarking question is a valuable tool for investigating and evaluating the accuracy of survey researchers. They are especially important for the center Study on survey method.

Subject: vote, research methods


Please enter your comment!
Please enter your name here