Pensive young black woman using tablet computer in coffee shop

Guide to Online Research Quality: Your Checklist for Standardizing Quality

 /  Guide

Online Research Quality: An Industry Challenge

Market research professionals agree that online research data is only as good as the respondents who provide it and the surveys that collect it. With your budget, your brands’/customers’ reputations, and your business’ credibility on the line, do you always feel confident about the data driving your recommendations and business decisions? Market research associations, vendors, and industry analysts acknowledge challenges with online data quality.

Online sample quality issues are well documented


The industry can’t ignore that proactive action is required; there’s too much at stake. TrueSample is continuing to lead the efforts to set guidelines for the entire industry, and we’ve recently formulated a number of requirements that we believe the industry needs to commit to, such as transparency, audits, objectivity, and reporting.”

An Industry Response: RFP-Ready Quality Guidelines

Recognizing that the quality of online survey data is a major concern for the market research industry and for the business leaders who rely on this data, the TrueSample Quality Council was formed. The Council includes a select group of market research industry leaders, from suppliers to Fortune 50 buyers, who share the common goal of markedly improving online sample and research quality.  In an effort to establish an industry standard for quality, the Council has issued a set of standard Online Consumer Research Quality Guidelines for research buyers to use as an aid in the selection of research suppliers for online consumer research, and to aid research suppliers in conducting/fielding research that meets research quality standards. The Guidelines have been formulated into an easy-to-use list of 11 conditions, all of which should be met, to easily achieve the quality standard.

The Council encourages the use of the Online Consumer Research Quality Guidelines in all outgoing Requests for Proposals, Statements of Work, and Live Projects to ensure consistency and reliability in research. By adhering to these requirements, we can all feel more confident in the accuracy and quality of research projects.

Steps to Standardizing Online Research Quality

Many companies have already adopted and implemented these online quality standards. By following these three steps, you too will be confident you get reliable research with every project:

  1. Hone your list of approved online research suppliers and/or sample suppliers
    Use the Online Research Quality Checklist (see Appendix) to determine which suppliers meet every checklist item.
  2. Create quality standards for your organization
    Review the TrueSample Quality Council’s Online Consumer Research Quality Guidelines and, if necessary, adapt them to meet your business needs.
  3. Ensure quality on every project
    Include your Online Research Quality Guidelines in all outgoing Requests for Proposals and Statements of Work for online quantitative consumer research.

Benefit from Standardizing Quality Today

Using the Council’s Guidelines, everyone now can easily define quality standards for their organizations; whether they are an end client brand, and research supplier, and/or sample supplier. Now you too can start reaping the benefits of setting research quality standards:

  • Quickly and easily identify qualified suppliers using a consistent set of criteria
  • Feel confident that you will get reliable data, regardless of which supplier you use
  • Get unprecedented visibility into suppliers’ quality criteria and practices through auditing reports
  • Reduce field times and sample costs because studies will not need to be re-fielded or re-opened as a result of post-survey data cleaning
  • Gain confidence that every project conducted by your organization is being help to the same evidence based minimum quality standard

The challenges with online research are well-known. By honing your operations, developing your own quality standards and enforcing Online Consumer Research Quality Guidelines for every online research project, you can readily address those challenges. Following these three simple steps will dramatically and consistently increase the quality of your research, enabling you to feel greater confidence in your results and recommendations.

The handy checklist below is based on the TrueSample Quality Council’s Online Consumer Research Quality Guidelines, and will help you to quickly evaluate the state of your internal practices and that of potential suppliers you might work with.

APPENDlX: Online Research Quality Checklist

Respondent Quality

To ensure questionable online survey respondents do not compromise results, select online market research suppliers that:

  • Identify survey respondents whose identities cannot be verified against publicly available third-party databases.
  • Ensure each survey respondent completes a survey only once and that they are free of major fraud markers, regardless of whether the sample is sourced from one source or multiple sources.
  • Identify survey respondents who speed and straightline through any individual survey using cross-supplier/cross-platform evidence based algorithms.
  • Identify panelists who chronically speed and straightline through surveys so they don’t get invited to take surveys.
  • Ensure survey takers meet your eligibility criteria, regardless of whether the sample is sourced from one panel or multiple panels.

Survey Instrument Quality

To reduce the impact that complex or poor survey design has on data quality, maintain a system that allows you to benchmark and track respondent perception:

  • Measure and benchmark the quality of each survey’s design to determine its overall impact on respondent engagement and data quality.

Quality Solution

To assure replicable results, efficiency, and transparency, select data quality software that:

  • Uses objective quality criteria that are predetermined, replicable, and standardized.
  • Relies on automated processes to meet quality requirements.
  • Uniformly applies quality requirements to all projects when requested, regardless of sample source, survey technology, and geography (where technically possible).
  • Delivers reports demonstrating the impact of applying the quality requirements.
  • Protects and secures all personally identifiable and confidential information collected from respondents, suppliers, or clients.

  Back to All Articles
Like this article? Share it!