Quality Checker for Survey Responses

The Quality Checker view provides a comprehensive overview of each individual response in your experiment, allowing you to review survey answers, detailed status, and a summary table of risk signals. With this information, you can assess response quality and manually include or exclude specific responses in your analysis.

Accessing the Quality Checker view

There are multiple ways to access the Quality Checker view of your launched experiment:

• Via the experiment setting page.

• Via the Review participants page. You can also click on the icon next to each participant to directly access the Quality Checker view for that specific participant.

Viewing only responses used in analysis

The Quality Checker view includes all recorded responses for your experiment, even those automatically filtered out based on automated checks and your response quality management settings, which are labeled with a red cross ❌. To view only the responses used in your analysis, tick the checkbox labeled Viewing only responses used in analysis.

Reviewing response status

The top-right corner of the Quality checker view displays detailed information about each response, including:

• Starting time and survey duration.
• Status: Opened or completed.
• Risk signals (if detected): Low quality, duplicate IP, duplicate cookie, duplicate GET variable, touch screen.
• Device type.
• Browser details.
• Location signals.

Manually exclude or include response

To ensure accurate results, you can manually assess the quality of each response and decide whether to include or exclude it from your analysis through one of the following options:

1. The 👍 or 👎 icon
   You can find these icons below Assessment of quality section. Click the 👍 icon to mark response as high-quality and include it in analysis or the 👎 icon to mark response as low-quality and exclude it from analysis. Using these icons helps train our system for more accurate quality assessments in the future. After assessing a few responses, the system will generate a prompt that allows you to mark the remaining participants as having either good or bad quality responses.
2. The Exclude button
   Conjointly recommends using the ‘Exclude’ button in the top-right corner to remove responses due to non-quality issues, such as quota fulfilment or ineligibility.

These options are also available on the Review Participants page.

Alternatively, you can include or exclude respondents from the analysis by uploading a special external variable called CONJOINTLY_ACTION.

Checking risk signals

Underneath the survey answers and response status sections, you will find a summary table of risk signals that displays the details of any detected risk signals, including:

• Datetime: The exact date and time when the risk signal was detected.
• Occasion: Indicates when the risk signal was identified:
  • at-start: Signal detected at the beginning of the survey.
  • on-response: Signal detected when saving the response.
  • on-review: Signal detected during automated checks.
• Signal: The specific type of risk signal identified.
• Value: The specific value associated with the detected risk signal.
• Description: A detailed explanation of the risk signal.

Common survey rule violations and associated risk signals

The table below provides a list of survey rule violations and the specific risk signals triggered when each violation occurs:

General information and associated signals

The table of risk signals also record the following information: