Comparative Scaling Techniques for Survey Research

Scaling emerged from the social sciences in an attempt to measure or order attributes with respect to quantitative attributes or traits.  Scaling provides a mechanism for measuring abstract concepts.

A comparative scale is an ordinal or rank order scale that can also be referred to as a non-metric scale.  Respondents evaluate two or more objects at one time and objects are directly compared with one another as part of the measuring process.

For example you could ask someone if they prefer listening to MP3s through a Zune or an iPod. You could take it a step further and add some other MP3 player brands to the comparison.  MP3 players would be scaled relative to each other and the scale position of any one player would depend on the the scale position of the remaining players.  Because they are being compared differences such as who has the click wheel are effectively forced.  Where this is limiting is evident when you find no standard of comparison outside the objects being compared. No generalizations are made outside of these objects.  Often used when physical characteristics of objects are being compared.

This can also be referred to as a cumulative scoring or scalogram analysis.  The intent of this survey is that the respondent will agree to a point and their score is measured to the point where they stop agreeing.  For this reason questions are often formatted in dichotomous yes or no responses.

The survey may start out with a question that is easy to agree with and then get increasingly sensitive to the point where the respondent starts to disagree.  You may start out with a question that asks if you like music at which point you mark yes.   Four questions later it may ask if you like music without a soul and which is produced by shady record labels only out to make money at which point you may say no. If you agreed with the first 5 questions and then started disagreeing you would be rated a 5.  The total of questions you agreed to would be added up and your final score would say something about your attitude toward music.

This probabilistic model provides a theoretical basis for obtaining interval level measurements based on counts from observations such as total scores on assessments. This analyzes individual differences in response tendencies as well as an item’s discrimination and difficulty.  It measures how respondents interact with items and then infers differences between items from responses to obtain scale values.  This model is typically used analyze data from assessments and to measure abilities, attitudes, and personality traits.

This gives the respondent a set of items and then asks the respondent to put those items in some kind of order. The "order" could be something like preference, liking, importance, effectiveness, etc.  This can be a simple ordinal structure such as A is higher than B or be done by relative position (give each letter a numerical value as in A is 10 and B is 7).   You could present five items and ask the respondent to order each one A-E in order of preference.  In Rank-Order scaling only (n-1) decisions need to be made.

With this ordinal level technique respondents are given a constant sum of units such as points, money, or credits and then asked to allocate them to various items.   For example,  you could ask a respondent to reflect on the importance of features of a product and then give them 100 points to allocate to each feature of the product based on that.   If a feature is not important then the respondent can assign it zero.   If one feature is twice as important as another then they can assign it twice as much.   When they are done all the points should add up to 100.

This is an ordinal level technique where a respondent is presented with two items at a time and asked to choose one.   This is the most widely used comparison scale technique.   If you take n brands then [n (n-1)/2] paired comparisons are required.  A classic example of when paired comparison is used is during taste tests.  For example you could have a taste test in which you have someone try both Coke and Pepsi and then ask them which one they prefer.

This is a cumulative score that is a variant of the Guttman scale, agreement with any item implies agreement with the preceding items. This scale is used to measure how close or distant people feel toward other people.  Social distance is a concern when it comes to issues related to racial integration or other forms of equality.  This is applicable to team formation in the work place for example.  Some people accept other people easily and use trustworthiness as the basis of their relationship with other people.  Other people do not accept people who are not like them and tend to keep those that are not like them at arms length.

This is a rank order procedure where respondents are asked to sort a given number of items or statements and classify them into a predetermined number of sets (usually 11) according to some criterion such as preference, attitude, or behavioral intent.  Using cards that note an item to be ranked is the most popular and simplest method to use in the sorting process.  In order to increase statistical reliability at least 60 cards should be used and no more than 140.  This is good for discriminating among a large group of items in a relatively short amount of time.