Skip to main content

Transcript of Video "Roper Center Transparency Core Elements"

the roper center transparency project

aims to increase disclosure about how

polls are conducted

transparency is not a guarantee of


however transparency allows users to

make informed judgments about data

which is why best practices in polling

call for a high level of disclosure

there are certain disclosure elements

which roper considers most important

these are the core elements in our

scoring system

in this video i'll provide an overview

of these core

elements the first group of elements are


they describe who designed funded or

fielded a poll

knowing who is responsible for a poll

could help you understand why that poll

was conducted

and knowing who fielded a poll can help

you be on the lookout for house effects

that is differences in the practices of

different organizations that can affect

the data

for example if interviewers are trained

to accept a don't know response readily

that organization's polls will show a

higher don't know result

than another organization that trains

interviewers to encourage

respondents to select responses from the

given options

even if the questions are in other ways


for some topics polling can be very

responsive to emerging events

therefore the exact dates of a poll are

important to know

just a few days difference in field

dates can make a big difference in poll


the universe and geographic coverage

tell you what population the poll is

trying to represent

that could be adults in france women in


or likely voters in a particular u.s

congressional district

no sample is perfect justification for


indicates why the polling organization

believes that their sample is

representative of the target population

in the case of probability-based polls

like random digit dial telephone poles


is based on probability that is random


in the case of non-probability polls the

justification may be based on the

weighting schemes applied to the sample

the sampling information tells

researchers how many respondents were in

the poll

and how they were selected often


selection takes place in multiple stages

rover asks for a summary and a

respondent stage selection

the mode describes the way the survey

was conducted the most common methods

are by phone

online by mail or in person but posters

are always looking for new ways to reach


like text messaging or even gaming


if a poll utilizes one of these novel


rover requires a description to help

users understand the method

mode differences can have an effect on

polling results

for example some respondents may be


giving answers to questions about

sensitive topics

with an in-person interviewer

weighting is an important tool polsters

use to increase

accuracy of poles

that is they use a reliable source of

information on demographics

like the census and match their sample

data to that data

although weighting is important for all

pollsters it's particularly important

for those who are not using a

probability method to select their


the roper center asks data providers to


quite detailed information about

weighting including which demographic

variables were used in weighting

and what resource was used to set the


the response rate indicates what

proportion of those the poll tried to


actually completed the survey for

probability samples

the american association for public

opinion research has set standards

on how to calculate this rate roper asks

data organizations to provide

either the response rate or the

information required to calculate it to

those standards there is no set standard

for calculating

a completion rate for non-probability


completion or participation rates for

non-probability polls

need to be provided along with an


of how these were calculated

finally you can't overestimate the

importance of question wording

but polling results can be affected not

just by the wording of the questions and

response categories

results can also be affected by

introductory language or

visual aids if a poll is conducted

online or in person

so the representer asks for full

question wording with

all instructions and prompts

for more on roper center's transparency

project please see our website