|
Article Excerpt
Polyhedral methods for choice-based conjoint analysis provide a means to adapt choice-based questions at the individual-respondent level and provide an alternative means to estimate partworths when there are relatively few questions per respondent, as in a Web-based questionnaire. However, these methods are deterministic and are susceptible to the propagation of response errors. They also assume, implicitly, a uniform prior on the partworths. In this paper we provide a probabilistic interpretation of polyhedral methods and propose improvements that incorporate response error and/or informative priors into individual-level question selection and estimation.
Monte Carlo simulations suggest that response-error modeling and informative priors improve polyhedral question-selection methods in the domains where they were previously weak. A field experiment with over 2,200 leading-edge wine consumers in the United States, Australia, and New Zealand suggests that the new question-selection methods show promise relative to existing methods.
Key words: conjoint analysis; choice models; estimation and other statistical techniques; international marketing; marketing research; new-product research; product development; Bayesian methods History: This paper was received May 26, 2006, and was with the authors 1 month for 1 revision; processed by Eric Bradlow.
**********
1. Introduction
Toubia et al. (2003) demonstrated that polyhedral methods for adaptively selecting questions in metric conjoint analysis could improve accuracy when partworths are either homogeneous or heterogeneous, and could do so whether response errors are large or small. Toubia et al. (THS 2004) extended polyhedral methods to choice-based questions, but with mixed success. Polyhedral choice-based questions improved accuracy when response errors were low, but not when they were high. Furthermore, although polyhedral methods for metric paired-comparison questions predict well for empirical data, there have been no empirical validity tests for choice-based polyhedral methods despite the growing interest among practitioners for adaptive choice-based methods.
In this paper we propose and test a generalization of THS that takes response error into account for choice-based questions and has the potential to improve accuracy in high response-error domains. We do so by recasting the polyhedral heuristic into a Bayesian framework. This framework also includes prior information in a natural, conjugate manner. After verifying the methods with simulations, we undertake a large-scale, multicountry study in which each respondent completes two separate conjoint tasks. This design enables us to compare question selection with a within-respondent design that implies greater statistical power to distinguish methods. We compare methods on the ability to predict actual choices. We examine whether the methods lead to different managerial implications by comparing forecasts of willingness to pay as well as the optimal product lines implied by each method.
This paper is organized as follows. Section 2 briefly reviews the published choice-based polyhedral methods and discusses two key limitations. Sections 3 and 4 propose solutions to these limitations. Sections 5 examines the methods with Monte Carlo simulations. Section 6 describes the methodological results of the field experiment. Section 7 concludes and offers directions for future research.
2. Review and Critique of Polyhedral Choice-Based Methods
Choice-based polyhedral question selection selects each choice question to learn as much as possible about a respondent's preferences. The conceptual idea is to recognize that the set of choice questions and their corresponding answers define a polyhedron, i.e., a set of "feasible" partworth vectors that perfectly fit previous observations. Each choice narrows the range of feasible partworths making the range smaller and smaller until it converges toward a single partworth vector. The method works well when the respondent makes no errors, but can be highly sensitive to errors, particularly in the early choices. We now provide a brief technical review to establish both notation and conceptual reasoning for the generalizations.
Answers to Choice-Based Questions Interpreted as Constraints on the Partworths
Without loss of generality, we use binary vectors in the theoretical development to simplify notation and exposition. Multilevel features are used in both the simulations and the application. Let [X.sub.qjf] indicate that the jth alternative in the qth choice set contains the fth feature, and let [[??].sub.qj] be the binary row vector describing the jth alternative in the qth choice set. Define [[??].sub.qk] similarly for the kth profile. Let [??] be the l-dimensional vector of partworths for a given respondent. Let [[epsilon].sub.qj] and [[epsilon].sub.qk] be error terms such that the respondent's utility for profile j in choice set q is [[??].sub.qj] + [[epsilon].sub.qj]. The utility-maximizing respondent will choose profile [j.sup.*] over profile k if and only if ([[??].sub.qj*]-[[??].sub.qk])[??] + ([[epsilon].sub.qj*] - [[epsilon].sub.qk]) [greater than or equal to] O. Each choice among J alternatives implies J - 1 such inequality constraints, indicating that the utility of the chosen profile is higher than that of the other J - 1 alternatives in the choice set. Let [X.sub.{i1, ..., q}] be the matrix of the ([[??].sub.qj*] - [[??].sub.qk])S for all J - 1 inequality constraints stacked for the first q questions. Note that the respondent's q choices are coded in [X.sub.{1, ..., q}] by the selection of [j.sub.*] for each question. Let [??] be the corresponding vector of error differences and, without loss of generality, scale all partworths to be nonnegative and normalize the partworths so that they sum to 100.(1) Then, if [??] is a vector of 1s and [??] is a vector of 0s (of length l), the answers to the choice-based questions imply the following constraints on the respondent's partworths:
[MATHEMATICAL EXPRESSION NOT CONVERTIBLE IN ASCII].
[FIGURE 1 OMITTED]
Question Selection
For any given vector [??], the set of vectors [??] satisfying the constraints in (P1) is a mathematical object called a polyhedron. THS select questions such that the polyhedron corresponding to [??] = [??] never becomes empty, and effectively assume that [??] = [??]. Let [[OMEGA].sub.{1,...,q}] be the polyhedron obtained after q questions. The q + 1st question imposes new constraints on the partworths, giving rise to a new polyhedron [[OMEGA].sub.{1, ..., q+1}] that is a subset of the previous polyhedron [[OMEGA].sub.{1, ..., q}] For a linear compensatory utility model, each point in [[OMEGA].sub.{1, ..., q}] is consistent with the respondent choosing one and only one of the alternatives in choice set (q + 1) (except for a set of points of measure for which the respondent is indifferent between at least two profiles). Hence, the q + 1st question divides [[OMEGA].sub.{1, ..., q}] into J collectively exhaustive (smaller) polyhedra that are of roughly equal size. The region corresponding to the respondent's choice becomes the starting polyhedron for the next question. See Figure 1 for a choice set of two alternatives. If there were no response errors, the sequence of polyhedra would shrink toward the respondent's true partworth vector.
Question selection (choice set selection) obeys two principles: (1) choice balance and (2) postchoice symmetry. Choice balance minimizes the expected size of [[OMEGA].sub.{1, ..., q+1}] and is implemented by ensuring that a respondent who uses the working estimate of the partworths, [[??].sub.q], would be approximately indifferent between all the alternatives in the choice set. Choice balance is common in the literature and, for choice questions, typically increases the efficiency of the questions (Arora and Huber 2001, Hauser and Toubia 2005, Huber and Zwerina 1996, Kanninen 2002). (2) Postchoice symmetry minimizes the maximum uncertainty on any combination of partworths, and is implemented by constructing choice sets that divide the polyhedron [[OMEGA].sub.{1, ..., q}] perpendicularly to its longest axes.
Estimation
Because choice questions are chosen such that the polyhedron [[OMEGA].sub.{1, ..., q}] never becomes empty, all points in [[OMEGA].sub.{1, ..., q}] are consistent with all of the respondent's choices. Thus, THS use the analytic center of ][OMEGA].sub.{1, ..., q}], [[??].sub.q], as the working estimate of [??] after q questions.
Critique
Choice-based polyhedral question selection and estimation are promising. Empirically, choice balance is achieved and the polyhedra shrink rapidly (although there is no published data on the ability to predict actual choices). Compared to randomly generated questions, orthogonal designs, and aggregate customization (Arora and Huber 2001, Huber and Zwerina 1996), deterministic choice-based polyhedral questions improve accuracy when response error is low, but not when response error is high.
The poor performance for high response errors is likely due to response-error propagation, as illustrated in Figure 2. In this example, the respondent's true partworth values are as indicated by a star (*). With no response error, the respondent would choose Profile 2, corresponding to the lower polyhedron, and the set of feasible partworths (new polyhedron) would converge toward the true value. However, with response errors the respondent might choose Profile 1, corresponding to the upper polyhedron. Once such a choice is made, the partworths can never converge to the true value. The closest estimate would be on the border, as indicated by the small diamond (#). Moreover, without a formal probabilistic structure, there is no easy way to incorporate prior information on the likely distribution of partworths. We next address both response error and prior information with a Bayesian interpretation of choice-based polyhedral methods.
[FIGURE 2 OMITTED]
3. Bayesian Interpretation for Choice-Based Polyhedral Methods
We can interpret the analytic center as a working estimate if we assume a prior distribution on the partworth vector [??] that is uniformly distributed on the initial polyhedron, [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] = 100}. Denote this distribution as [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]: defined by [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] if [??] [member of] [[OMEGA].sub.0]....
|
