Skip to main content

Additional Details Regarding Transparency and Acquisitions

Core Elements: Required for Recently Developed Methods, Strongly Encouraged for All Data Providers

Survey organization

The organization that conducted the fieldwork for a survey.

Grant funding source

Funding source for academic or other grant-supported research.

External survey sponsor

When applicable, the name of the organization that commissioned the survey. If the same organization funded, designed, and fielded a poll, no sponsor is listed.

Data collection dates

The date range during which data was collected from respondents.

Universe

The population the survey results are intended to represent. Also known as "target population."

Geographic coverage

The geographic area from which data were collected.

Justification for claims of representativeness

A description of the elements of the research design intended to ensure that the survey is representative of the universe it is designed to study.

Mode

Method by which data were collected (such as telephone, in-person, online, etc.)

Mode other: Description (filtered on

previous)

Method by which data were collected, such as telephone, in-person, online, etc.

Sample size

The total unweighted number of respondents in the survey.

Sampling procedure: Summary

The method by which participants in a poll were selected.

Sampling procedure: Respondent selection stage

The method by which participants in a poll were selected; specifically, the method by which the individual respondents were chosen. In a multistage sampling process, respondent selection is the final stage of sampling.

Sampling frame

A list of the items or people forming the universe from which a sample is taken.

Weight variable

Name of the variable in the datasets used for weighting the sample. If mutiple weighting schemes were used for different analysis, the variable identified here will be the one used for reporting on the total population, and information on other weights provided in the documentation.

Weighting benchmark source

Data source for benchmarks used to weight the sample

Variables used for weighting

Specific variables used in the calculation of survey weights.

Disposition codes (OR response rate)

A set of codes or categories used by survey researchers to document the ultimate outcome of contact attempts on individual cases in a survey sample.

Response rate (OR disposition codes sufficient to calculate response rate)

Proportion of contacted respondents who completed the survey. The American Association for Public Opinion Research (AAPOR) provides definitions for six measures of response rates.

Completion or participation rate

The proportion of all cases interviewed of all eligible

units ever contacted, used if response rates calculated to AAPOR standards would be inappropriate for the survey design.

Completion or participation rate details (filtered on previous)

Method for calculation of completion/participation rates for surveys for which standard AAPOR response rates cannot be calculated

Survey language(s)

Languages in which the survey was fielded.

Full question wording with all interviewer instructions, prompts and visual aids

A complete survey questionnaire includes all questions, including any screening questions, introductory language, interviewer instructions, and, in the case of some in-person or online polls, visual aids used to illustrate questions.

 

Additional Elements: Encouraged for All Data Providers

External sample provider(s)

The organization that provided the sampling frame to the field organization, if external sample provider used.

Proportion of sample provided (filtered on previous)

The proportion of the total sample provided by the external sample provider.

Use of breakout routers or chains

Use of online survey routers that screen respondents and direct them to open surveys for which they are qualified or use of chains that direct respondents to additional surveys at the end of completed surveys.

Breakoff rate

The percent of respondents who start the survey but do not finish it.

Estimated size of the noncovered population

Proportion of universe that had a nonzero chance of participation

Use of incentives

Use of incentives provided to survey recipients to reward participation.

What incentive was provided (filtered on previous)

Specific incentives provided to survey recipients to reward participation.

Quality control: Summary

Quality control checks performed on the data from the survey. Many possible approaches can be taken for quality assurance, such as monitoring online surveys for cases of "speeding" (answering at a rate too fast to allow for adequate comprehension of questions) or "straightlining" (providing identical answers across a range of questions); reinterviewing in- person survey respondents; or random quality control monitoring of telephone interviews.

% respondents removed due to checks (filtered on above)

Percentage of respondents whose cases were removed from the survey before analysis based on quality control checks performed.

 

Required Disclosure: Longstanding Methods Collection

  1. Information about who sponsored the survey, including all funding sources
  2. Field work provider, if outsourced
  3. The exact wording of questions asked, including the text of any preceding instruction or explanation to the interviewer or respondents that might reasonably be expected to affect the response.
  4. A definition of the population under study, and a description of the sampling criteria used to identify this population.
  5. A description of the sample selection procedure that gives a clear indication of whether or not the researcher selected the respondents or they were self-selected.
  6. The size of samples and, if applicable, completion rates and information on eligibility criteria and screening procedures. For the purpose of this policy statement response rate definition and calculation will be informed by the AAPOR report entitled “Standard Definitions.” The report defines standardized measures for response rates, cooperation rates, refusal rates and contact rates. Each of these rates requires a count of the disposition of all units selected in the sample. The various disposition codes are standardized in the report.
  7. There should be a discussion of the precision of the findings, including, if appropriate, estimates of sampling error, and a description of any weighting or estimating procedures that were used.
  8. A listing of which results are based on parts of the sample, rather than on the total sample.
  9. The mode, location, and dates of data collection.

The following guidelines were approved by the Roper Center for Public Opinion Research Board of Directors on November 19, 2002, and reviewed in June, 2012.

Acquisition Categories

Two broad categories are included in the scope of the Center’s acquisition efforts.

  1. A primary effort is made to acquire respondent-level data from surveys that may or may not have been released publicly by the original data producer or sponsoring organization. This constitutes the Roper Center’s core collection. Studies in this category offer the unrivaled potential for secondary analysis. The Center was founded, principally, on the acquisition and re-analysis of survey data in this format. This acquisition category consists of two collections based on methodological approaches: Longstanding Methods and Recently Developed Methods.
  2. A second category includes interpretive survey reports, marginal data, and news releases. These can include physical or digital materials.  These collections supplement the respondent-level data referred to above, and in some cases, provide the only preserved record of the survey. For surveys conducted using Longstanding Methods, these materials are entered into the online polling database regardless of whether respondent level data is acquired.  For Recently Developed Methods, these materials are only entered into the database if individual-level data are also provided.
     

While the Center was built upon the acquisition of respondent-level data, the importance of acquiring, preserving and making accessible marginal data cannot be underestimated within the context of the Center’s mission and the diversity of its clientele. Moreover, full marginal data constitute essential documentation of the respondent-level data.

Acquisition Criteria

Subject Matter

Collections must be of current value or potential historical interest.

Sample Populations

Material is generally selected in the following descending order of priority:

  • United States national population probability samples
  • Representative subsamples of US national populations (e.g., women, teenagers)
  • US state exit polls or other state representative polls not in ‘national’ collections
  • Representative samples of other countries and regions

Additional Acquisitions Considerations

Risk of Data Loss

Material may be acquired to mitigate the risk of data loss.

Availability of Data Elsewhere

Data may be acquired because the information is not available from another reputable, accessible data archive.

Technical Criteria that May Affect Acquiring Material Include:

  • The general condition of the study/collection and whether it requires costly methods to preserve or make it accessible
  • Whether there are any specific restrictions that would not allow the material to be freely accessible to researchers (however, the Center will occasionally accept “embargoes” on certain questions for limited periods of time, and it will accept a restriction that, for a limited period of time, where data may be disseminated only with the academic membership of the Center)
  • Whether the cost of processing materials is higher than normally expected
     

What Will Not Be Acquired

There are no firm rules concerning the kinds of survey material that will not be acquired. However, the following list represents a compilation of current practices at the Center and may serve as a guide to the staff and to the Committee as it implements this Acquisition Policy. There have been and will again be exceptions to some or all of these customary practices.

  • Surveys without content that is salient to the Center’s Mission and Purpose, such as many marketing studies of single products
  • Surveys with sample sizes that are too small to represent any population
  • Surveys which are not intended to represent a broader population beyond the individuals who participated.
  • Surveys where release of the respondent-level data would pose an unreasonable and uncontrollable risk of violation of respondents’ privacy and confidentiality
  • Data resulting from “push polls” and other uses of survey methodologies to gather data for purposes other than legitimate survey research
  • Surveys for which the data collector will not provide the full questionnaire or required methodological information.

Roles and Responsibilities

The Acquisition Policy recognizes that professional judgment is central to making decisions about new acquisitions. The broad criteria described above will serve as a guide for archive staff in their continuing efforts to build unique and valuable data collections. The Center’s Executive Director will provide final approval of all new acquisition efforts to oversee appropriate interpretation of these guidelines and safeguard against over-committing Center resources. Beyond this document, the Transparency and Data Acquisitions Committee will serve in an advisory capacity to Center staff. To facilitate this role, archive staff will provide a periodic report on recent acquisitions and rejections. In specific cases where there are questions regarding the implementation of this Policy, Center staff will call upon the Acquisitions Committee for its advice and recommendations.