the roper center transparency project
aims to increase disclosure about how
polls are conducted
transparency is not a guarantee of
quality
however transparency allows users to
make informed judgments about data
which is why best practices in polling
call for a high level of disclosure
there are certain disclosure elements
which roper considers most important
these are the core elements in our
scoring system
in this video i'll provide an overview
of these core
elements the first group of elements are
organizational
they describe who designed funded or
fielded a poll
knowing who is responsible for a poll
could help you understand why that poll
was conducted
and knowing who fielded a poll can help
you be on the lookout for house effects
that is differences in the practices of
different organizations that can affect
the data
for example if interviewers are trained
to accept a don't know response readily
that organization's polls will show a
higher don't know result
than another organization that trains
interviewers to encourage
respondents to select responses from the
given options
even if the questions are in other ways
identical
for some topics polling can be very
responsive to emerging events
therefore the exact dates of a poll are
important to know
just a few days difference in field
dates can make a big difference in poll
results
the universe and geographic coverage
tell you what population the poll is
trying to represent
that could be adults in france women in
texas
or likely voters in a particular u.s
congressional district
no sample is perfect justification for
representativeness
indicates why the polling organization
believes that their sample is
representative of the target population
in the case of probability-based polls
like random digit dial telephone poles
representativeness
is based on probability that is random
selection
in the case of non-probability polls the
justification may be based on the
weighting schemes applied to the sample
the sampling information tells
researchers how many respondents were in
the poll
and how they were selected often
responding
selection takes place in multiple stages
rover asks for a summary and a
respondent stage selection
the mode describes the way the survey
was conducted the most common methods
are by phone
online by mail or in person but posters
are always looking for new ways to reach
respondents
like text messaging or even gaming
systems
if a poll utilizes one of these novel
approaches
rover requires a description to help
users understand the method
mode differences can have an effect on
polling results
for example some respondents may be
uncomfortable
giving answers to questions about
sensitive topics
with an in-person interviewer
weighting is an important tool polsters
use to increase
accuracy of poles
that is they use a reliable source of
information on demographics
like the census and match their sample
data to that data
although weighting is important for all
pollsters it's particularly important
for those who are not using a
probability method to select their
sample
the roper center asks data providers to
give
quite detailed information about
weighting including which demographic
variables were used in weighting
and what resource was used to set the
benchmarks
the response rate indicates what
proportion of those the poll tried to
contact
actually completed the survey for
probability samples
the american association for public
opinion research has set standards
on how to calculate this rate roper asks
data organizations to provide
either the response rate or the
information required to calculate it to
those standards there is no set standard
for calculating
a completion rate for non-probability
polls
completion or participation rates for
non-probability polls
need to be provided along with an
explanation
of how these were calculated
finally you can't overestimate the
importance of question wording
but polling results can be affected not
just by the wording of the questions and
response categories
results can also be affected by
introductory language or
visual aids if a poll is conducted
online or in person
so the representer asks for full
question wording with
all instructions and prompts
for more on roper center's transparency
project please see our website