Skip to content
Peer-reviewed86% recall validated against Baymard Institute and Nielsen Norman Group›

What Is MaxDiff?

MaxDiff ranks messages, features or prices by asking people to pick best and worst from small sets. See a 12-item example and the 300-respondent rule.

MaxDiff Definition

MaxDiff, short for maximum difference scaling and also called best-worst scaling, is a survey method where people see small sets of items and pick the most appealing and the least appealing in each. Repeated across many sets, those choices produce a ranked score for every item, such as messages, features or prices.

Key takeaways

  • Each question shows 4 to 5 items and asks for the best and the worst. Sawtooth Software suggests that range for most studies.
  • Aim to show each item 3 to 4 times per respondent, so a 12-item list at four items per question needs 9 to 12 questions.
  • Sawtooth's rule of thumb is a minimum of 300 respondents, or 200 per subgroup you plan to report separately, whichever is greater.
  • Scores show how items rank against each other. They say nothing about whether an item is good on its own, which is what an anchored MaxDiff adds.
  • Articos can shorten a long list with an AI persona message test in under 30 minutes. The MaxDiff survey itself still needs real respondents.

Explained

What Is MaxDiff?

One question type, repeated. Each answer says which item wins and which loses inside a small set.

A MaxDiff survey (also called MaxDiff scaling or best-worst scaling) shows a respondent a few items at a time, for example four headlines, and asks which is the most appealing and which is the least. The software rotates the items across many sets so that every item meets every other item several times. The pattern of best and worst picks is then converted into a score for each item.

Jordan Louviere developed the method in 1987 at the University of Alberta. Because respondents choose instead of rating, there is no rating scale for them to use differently, which Sawtooth notes helps in cross-cultural samples. Each answer also carries more information than it seems to: a best and a worst pick from four items settles five of the six possible pairs.

Why it matters

Why MaxDiff Matters

A rating scale lets respondents call every item important. MaxDiff makes them choose.

Forced choice

Respondents can't rate everything highly. Each question makes them give up one item to keep another, so real differences show up.

5 of 6

A best and a worst pick from a set of four settles five of the six item pairs. In a set of five it settles seven of ten.

No scale bias

People choose instead of rating, so differences in how respondents use a 1 to 5 scale don't distort the results.

How it works

How a MaxDiff Survey Works, Step by Step

The questions are simple. The work is in the item list and the design behind it.

All visitorsrandom 50 / 50 splitGroup A · Controlsees the current versionGroup B · Variantsees one changeMeasure one metricsign-ups, purchasesSignificant lift?ship B, or keep A
  1. 1
    Plan

    Choose the Decision and Write the Items

    Pick the one decision the survey informs, such as which message to lead with. Write every item at the same level of detail, so that "Free for small teams" sits beside "Works with Slack" and not beside "Pricing".

  2. 2
    Build

    Build the Design

    Set 4 to 5 items per question and enough questions that each item appears 3 to 4 times per respondent. The software balances the sets so each item appears equally often and meets the others about equally.

  3. 3
    Run

    Field the Survey With Enough Respondents

    Each respondent answers a series of sets and picks the best and worst each time. Plan for at least 300 respondents, or 200 for each subgroup you want to report.

  4. 4
    Analyze

    Estimate the Scores

    Best-minus-worst counts give a quick read. Models such as multinomial logit or hierarchical Bayes give scores for each respondent. Many tools rescale the results so they sum to 100.

  5. 5
    Decide

    Read the Gaps Between Scores

    Two items one point apart are close to tied. Look at how far each score sits from the next before you cut anything.

  6. 6
    Segment

    Check Segments and Anchor if Needed

    Compare scores across segments, such as new and existing customers. Use an anchored MaxDiff when you also need to know whether the top items are good enough on their own.

Key elements

Key Elements of MaxDiff Scaling

ElementWhat it meansExample
Item listThe full set of things being ranked, all at one level of detail.12 homepage messages
SetOne question that shows 4 or 5 items.Four messages on one screen
Best-worst choiceThe most and least appealing item picked in each set.Best: "Free for small teams"
DesignThe balanced rotation of items across sets and respondents.Each item shown 3 to 4 times per respondent
Utility scoreThe estimated appeal of each item, calculated from all choices.19 out of 100
AnchorAn optional reference point that shows whether items clear a bar."Would you use this?" yes or no

MaxDiff Compared With Other Ways to Prioritize

MethodWhat respondents doMain weakness / Best for
Rating scaleRate each item, usually from 1 to 5.Main weakness: many items can be rated as important, and scale use varies by person. Best for: quick screening of a short list.
Full rankingOrder every item from first to last.Main weakness: the burden grows quickly as the list grows. Best for: very short lists.
MaxDiffPick the best and worst from small sets.Main weakness: needs more questions and a few hundred real respondents. Best for: ranking messages, features or benefits.
ConjointChoose between full product profiles.Main weakness: more complex design and analysis. Best for: testing price and feature bundles together.

Example

MaxDiff Example: Choosing a Homepage Message

An illustrative case: a project-management SaaS team tests 12 candidate messages with 400 respondents. Each respondent sees 12 sets of four messages and picks the best and worst in each.

RankCandidate messageScore (of 100) and share of top score
1Free for small teams19 (100% of top)
2Plan a sprint in minutes15 (79% of top)
3See every task in one place12 (63% of top)
4Fewer status meetings10 (53% of top)
5Works with Slack9 (47% of top)
6Guest access for clients8 (42% of top)
7Built-in time tracking7 (37% of top)
8Templates for every project type6 (32% of top)
9Real-time team dashboards5 (26% of top)
10Unlimited projects4 (21% of top)
11Mobile apps for iOS and Android3 (16% of top)
12SOC 2 compliant2 (11% of top)

“Free for small teams” leads with 19 points, nearly double the 10 for “Fewer status meetings”. Ranks 8 and 9 sit one point apart, so treat them as a tie. The scores say which messages beat the others. They don’t say that the leader is good enough to convert, which is the question an anchored MaxDiff or a follow-up test answers.

When to use it

When MaxDiff Applies, and When It Doesn't

Use MaxDiff When

  • You have more items than people can rate or rank sensibly
  • All items sit on one dimension, such as messages or features
  • You need a relative priority order, not a demand forecast
  • You can reach a few hundred real respondents

It Struggles When

  • The items aren't comparable, such as a mix of prices and features
  • You need absolute demand or willingness to pay
  • You need reliable results for small subgroups
  • You want to test price and features together, which conjoint fits better
Pros of MaxDiffCons of MaxDiff
Forces trade-offs, so differences showGives relative rank, not absolute appeal, unless anchored
Avoids rating-scale biasTakes more questions than a single rating grid
Handles long lists with a sparse designNeeds a few hundred respondents and analysis software
Produces a clear ranked outputRequires items that can be compared with each other

Common mistakes

Common MaxDiff Mistakes to Avoid

01

Mixing Levels of Detail

A broad theme next to a specific feature gets chosen for its scope, not its appeal.

Write every item at one level of specificity.
02

Showing Too Many Items per Question

Sawtooth's testing favors 4 to 5 items per question for most studies.

Stay in that range.
03

Showing Each Item Too Few Times

Under 3 appearances gives each respondent less data per item. Sparse designs accept that trade only for very long lists.

Aim for 3 to 4 appearances where the list allows.
04

Reading Rank as Demand

The top-ranked message can still be one that nobody wants.

Anchor the study or follow up with a test.
05

Fielding Too Few Respondents

Subgroup results get noisy fast.

300 in total, or 200 per reported subgroup.
06

Reporting Only the Average

An average can hide segments that rank the items in opposite order.

Compare segments before deciding.

How Articos helps

How to Shortlist MaxDiff Items With Articos Before You Survey

A MaxDiff study is more useful when every item in it has a fair chance. Articos helps you remove the weak ones first.

MaxDiff respondents (Sawtooth minimum)300+Articos message testUnder 30 minutes

Cut a list of 30 to 12 in the morning. Field the survey in the afternoon.

A MaxDiff survey on its own:

  • Needs at least 300 real respondents
  • Gets longer for every item you add
  • Spends respondent attention on weak items

With Articos:

Add your candidate messages or concepts. A panel of AI personas grounded in behavioral science, each with a different stance toward your idea, reviews them and returns a report with scores, reasons and direct quotes in under 30 minutes. Drop the ones the panel finds unclear or hard to trust, then run the MaxDiff on the survivors. Articos message tests are not a MaxDiff survey and don't produce MaxDiff scores, so use the survey when you need a ranking from real respondents.

Test your messages free →

Putting it into practice

Putting MaxDiff Into Practice

Cut the List First

Pre-test candidates with a persona panel before you spend respondent budget. Shorter lists mean shorter surveys and fewer wasted answers.

Keep Items Parallel

Match the length, format and level of detail of every item so that wording quirks don't decide the ranking.

Report the Gaps Between Scores

Show the distance between scores. A one-point gap is a tie, and a ten-point gap is a decision.

FAQ

Still Wondering?

Questions teams ask before running their first MaxDiff.

01What is a MaxDiff survey?

A MaxDiff survey asks respondents to pick the most and least appealing item from a series of small sets. The results are turned into a score for every item, so you can rank messages, features, benefits or price points by how much people value them.

02What is MaxDiff scaling and how are scores calculated?

MaxDiff scaling is the analysis step that converts best and worst choices into a score for each item. A simple version subtracts the number of times an item was picked worst from the number of times it was picked best. Statistical models such as multinomial logit and hierarchical Bayes produce scores down to the individual respondent, and many tools rescale them to sum to 100.

03How many items can a MaxDiff study handle?

Sawtooth Software suggests 4 to 5 items per question and aims for each item to appear 3 to 4 times per respondent. For long lists, designs may show each item fewer times to keep the questionnaire under about 20 to 30 questions. These are called sparse MaxDiff designs.

04How many respondents does a MaxDiff need?

Sawtooth's rule of thumb is at least 300 respondents, or 200 per subgroup you want to report separately, whichever is greater. MaxDiff can run on samples as small as 20, but results from small samples are less stable.

05What is the difference between MaxDiff and conjoint analysis?

MaxDiff ranks a flat list of items, such as 12 messages. Conjoint analysis shows full product profiles that combine features and prices, so it can estimate trade-offs between them. Use MaxDiff to prioritize items and conjoint to model how features and price work together.

06Can Articos run a MaxDiff survey?

No. Articos runs message, landing page, interview and A/B comparison studies with AI personas, and it doesn't produce MaxDiff scores. Use it to cut a long list to the strongest candidates in under 30 minutes, then run the MaxDiff with real respondents.

Shortlist Your Messages in Under 30 Minutes

14-day trial · 1,000 credits · No credit card