Structured Decision Making

Contents

• How will this publication benefit me?
• Introduction
• Overview of structured decision making
• Conclusions and resources
• References
• Other publications in this series
• Acknowledgments

How will this publication benefit me?

Structured decision making? Decision research? Decision making isn't that hard -
I don't need research to tell me how to do it; I do it successfully all the time.

Many of us think of decision making the same way we think of child rearing: it should come naturally. However, as anyone who has ever raised a child can attest, even if it is "natural," it certainly isn't easy. The same goes for decision making - especially group decision making.

Group decision making usually begins with a specific problem to solve. Often, however, the problem has been pre-identified for the group rather than by the group. Even worse, the solution to the problem may have been predetermined - making the group nothing more than an obligatory exercise, an attempt to make its members feel like they're participating in the outcome.

Participatory decision making, by contrast, invites group members not only to solve a problem, but to identify the problem to be solved - giving them a vested interest in the process from beginning to end. Thus, they truly become "stakeholders" in the outcome, each effectively holding a share of the responsibility for implementing the solution and making it work.

How does participatory decision making work? Is there a proven approach or set of guidelines to organize and conduct such a group process - efficiently and effectively?

That's what the authors of this publication are about to share with you.
- Rick Cooper, co-editor

Introduction

Stakeholder(s): an ungainly term that has become routinely used to label any individual or group that has a stake in the outcome of some planning or management process.

Stakeholders may be involved in a range of business and government decisions, but often their participation is treated as an afterthought in decision processes typically viewed as better left to government officials or technical experts.

An exception to this norm is an approach called structured decision making (SDM). SDM includes opportunities for stakeholders to access information about a particular issue and to express their views and concerns through public meetings, workshops, or other means that both highlight their concerns and explicitly incorporate them into the decision process. Examples include the Water Use Planning process in British Columbia (Arvai et al. 2001, Gregory et al. 2001b), a pilot project for the U.S. Department of Energy on the cleanup of contaminated sites (Arvai and Gregory 2003b), and several ongoing deliberative processes in Canada, the U.S., and the U.K. However, failures to involve stakeholders in a meaningful way appear to far outnumber the successes.

In our view, a primary reason for the failure of most stakeholder processes is the absence of formal methods that effectively merge scientific and values-based concerns and then use this information in the creation of options that address the problem at hand (Arvai 2007, Arvai and Gregory 2003a, Wilson and Arvai 2006). Efforts that fail to address these diverse concerns often leave participants believing that the process is not responsive to their interests, and that the opinions of technical experts dominate those of participating community members and other stakeholders. Ultimately such processes give the impression that opportunities for input are simply a diversion to draw attention away from where the "real" decisions are being made. Many of these problems in the participatory decision-making process stem from the absence of an approach that helps diverse stakeholders understand the problem, express and clarify their issue-specific values and concerns, and carefully weigh the pros and cons of different actions or options.

This publication applies insights from the decision sciences and from behavioral decision research to address problems with stakeholder participation. The focus is on practical tools that can be used during the initial phases of the participatory process, where all stakeholders work together to clarify the relevant values and identify potential solutions to the problem at hand.

Sidebar: Disagreement, trust, and decision making

"Too much agreement kills a chat," the activist Eldridge Cleaver observed. The intent of structured decision making is not to manufacture consensus or "too much agreement." Indeed, the process can help diverse views come forward and be heard, discussed, and accounted for.

Even so, group facilitators are often wary of intense disagreement within a group and what to do about that. It may help to remember that during any group discussion, differences of views are more or less continuously being considered by members of the group as they listen to others. Disagreements, therefore, are being thought, if not voiced, and they are completely normal. In fact, some scholars of collaboration argue that openly highlighting and discussing areas of disagreement can increase a group's perspective on an issue, provide understanding of its complexities, and lead to better collaboration.

The issue for group decision making is: how to encourage those disagreements in a constructive way, to keep them from becoming disagreeable or destructive. Some suggestions for facilitators:

• Create specific times and space for exploring disagreements.
• Explore disagreements by getting beneath the surface of the "punch-lines" through asking for individuals' explanations of the thinking that led them to that view.
• Consider processes such as concept mapping to make thinking visible.
• Establish a space or context of "psychological safety" in which group members are not made to feel embarrassed, rejected, or punished for expressing doubt, differences, or disagreement.
• Discuss conditions for trust. Because psychological safety is grounded in trust, it can be useful to highlight what behaviors are critical. In group decision making, trust in others is usually one of two kinds: trust in competence, and trust in commitment to the shared endeavor.
  - Joe Cone, co-editor

(References

van Rossum, Z., and D. Soule. 2010. Sources of Collaborative Capacity. Cambridge, Mass.: Harvard University Learning Innovations Laboratory.

Novak, J. D., and A. J. Ca–as. 2006, 2008. The Theory Underlying Concept Maps and How to Construct and Use Them. (Online; last accessed February 17, 2011: http://cmap.ihmc.us/publications/researchpapers/theorycmaps/theoryunderlyingconceptmaps.htm)

Overview of structured decision making (SDM)

An SDM approach is best viewed as a decision-focused process that helps people understand a problem and overcome common human errors in judgment as they evaluate potential solutions to the problem. Errors in judgment may arise from the use of mental shortcuts (heuristics) that often occur when people are faced with complex choices (see sidebar). Errors may also arise from an imbalance between emotional responses to the problem and more reasoned or deliberative analyses; and also from too-simple characterizations of the problem that may lead to overly specific solutions. Each of these issues - and others - can be addressed through decision structuring tools that help people more fully define their values, identify or understand the available options that are sensitive to these values, and then address the often difficult tradeoffs inherent in choosing among alternative solutions to the problem (Clemen 1996, Hammond et al. 1999, Kleindorfer et al. 1993).

Clarifying objectives and identifying alternatives

A critical early step in the SDM process is to engage participants in identifying their values (for example, caring about sustainability) and expressing them as objectives (for example, taking actions that promote sustainability). A second component of this step is to help people distinguish between means and fundamental objectives (Keeney 1992).

For example, a stakeholder objective for a resource-management decision may be to improve water quality. An SDM process will push this discussion a step further by asking participants to think about why this objective is important to them. An SDM facilitator will ask people to recognize and think about the difference between means and fundamental objectives. In this case, "improving water quality" may be a means to a more fundamental - or end - objective of "restoring ecosystem health." This link is discovered when a participant identifies that improving water quality is important because it would ultimately help restore ecosystem health. If restoring ecosystem health is fundamentally important, meaning that no additional explanation is necessary as to why, then this becomes the fundamental objective.

Focusing discussion and analysis on fundamental objectives helps bring to the forefront other potential means objectives that are also worthy of consideration (for example, providing habitat for wildlife, restoring vegetation, or limiting public access as a means to restoring ecosystem health). Note that differentiating means from fundamental objectives does not prevent decision makers from, for example, choosing an alternative that focuses on improving water quality. It does, however, help people realize that a single option is not a panacea and that it may be combined with other options to address a more-fundamental (and often shared) objective.

Beyond widening the range of options that decision makers might consider, helping people identify and clarify fundamental objectives and the alternatives derived from the means objectives serves two other functions. First, a thorough exploration of objectives helps achieve a balance between traditionally scientific or technical concerns (such as restoring or maintaining environmental health) and those that are more values-based (such as providing recreational opportunities). Second, exploring a comprehensive set of objectives helps avoid many of the problems associated with unstructured decision making. For example, considering a wider range of objectives helps participants realize that focusing on only one dimension cannot solve a problem. It is the perception that one dimension is most important that often leads to conflict in participatory decision making. Likewise, helping an individual or group understand what they want to achieve with a given decision places the focus on site-specific objectives and weakens the appeal of business-as-usual decision making.

Sidebar:
An objective simply reflects what one wants to achieve with the decision.

A means objective is a secondary objective that may be more solution-based and often provides one potential path to achieving a more fundamentally important objective.

A fundamental objective is a primary (or end) objective that reflects what is most important to the stakeholder, and may often be achieved through multiple means objectives.

Sidebar: The Decision Cycle

Analyze decision context

1. Identify group definition of the problem.
2. Elicit diverse stakeholder values in the form of problem-relevant objectives.
3. Separate the means from ends (fundamental) objectives.

Evaluate potential solutions

4. Create list of potential alternatives.
5. Select practical performance measures to evaluate the alternatives.
6. Use performance measures to assess the consequences of each alternative for each end objective.

Make decision

7. Identify and conduct tradeoffs to reach best-possible alternative.
8. Select preferred alternatives.
9. Implement, and evaluate success.

Attaching performance measures to objectives

Objectives need to be measurable, but making them so is often a challenge. It is not very helpful to express an objective - such as improving the health of the environment - without knowing how that particular objective will be affected by each solution under consideration. Decision makers must identify appropriate performance measures, such as which aspects of the environment will be used to estimate improved health.

This process is critical because

• the results of associated social, economic, or technical analyses meant to predict how a solution might perform will be more relevant if framed in terms of measures that make the most sense to and are most desired by stakeholders; this also makes it easier to follow and respond to changes within a managed system over time;
• doing so fosters openness and trust in the decision-making and management process; and
• it fosters more-defensible and thus, higher-quality decisions that are specific to a well-defined problem, responsive to stakeholder values, and informed by decision-relevant science.

Performance measures that predict how an objective will be affected by a particular solution or decision generally fall into one of three categories:

1. Natural measures - direct measures of the conditions in a system. For example, if one objective of a management or planning decision is to minimize the costs of long-term monitoring, then the specific performance measure can be expressed directly in dollars, or more specifically, the expected cost of long-term monitoring.
2. Proxy measures - used when it is not possible to directly measure an objective of interest. For example, there is no single direct measure of environmental health, but analysts and researchers may develop a comprehensive list of proxy measures including, for example, measures of water quality, productivity, and species diversity.
3. Constructed measures - most oftenÊused when neither a reasonable natural measureÊnor an accepted proxy measureÊexists for the particular decision. These measures must often be developed forÊobjectives thatÊare psychophysical in nature (for example,Êincreasing community pride), and may be obtained through survey-based studies of the local community or other stakeholders.

Making tradeoffs and deciding

Engaging people in identifying what matters to them and what they want to achieve with a decision begs another question: How can people choose which potential solution is "best"? In some cases - such as when only one objective matters - a single best-management option can be clearly identified. Often, however, many conflicting objectives (for example, minimizing costs, maximizing safety, and protecting the environment) are in play, and decision makers must consider tradeoffs - or giving up some measure of performance on one objective to gain a measure of performance on another.

The tradeoffs inherent in choosing one solution or option over another are difficult for most decision makers because of the psychological conflict they evoke (Gregory et al. 2001a). For example, sacrificing environmental health to save limited funds can be uncomfortable. SDM approaches can help in some cases simply by reminding people of the need to address tradeoffs. In more complex cases, SDM efforts can be designed to provide guidance about how to conduct more-formal tradeoff analyses with tradeoff support tools. In their most basic form, these tradeoff tools involve the ranking and weighting of objectives as they relate to expectations about how different options are expected to perform.

With each of these methods, the weighting of objectives should be undertaken only in a comparative framework. All too often, stakeholders will state that a certain objective - for example, minimizing the financial costs of implementing a management plan - is paramount. This kind of comparison-free weighting ignores the important concept of relative benefit. While it's tempting to focus on just one objective, the SDM process helps stakeholders see that potentially large increases in performance on one objective may be accompanied by relatively small decreases in performance on another (for example, a great increase in environmental protection may be worth the relatively small increase in cost). Therefore, a good starting point during tradeoff analysis can be the construction of a matrix of objectives, measures, and alternatives or options (see Figure 1 (omitted)). The expected performance - or consequence - of each alternative is then modeled (see Costanza and Voinov 2004) or predicted (see Failing et al. 2004, Keeney and von Windterfeldt 1989) and displayed in the individual cells of the matrix. This "decision matrix," showing how well the different options satisfy each objective, can help illuminate the necessary tradeoffs across objectives.

After constructing a decision matrix, decision makers must determine the relative weight to place on each objective when comparing options. This is critical to an SDM approach because it helps clarify what different tradeoffs will mean in terms of selecting one option over another.

In swing weighting, one useful tradeoff technique, stakeholders are presented with only the best and the worst projected consequences of each objective and told to assume the option they are evaluating possesses all the worst consequences (costs the most, performs poorly in terms of environmental protection, etc.). They are then asked which objective they would prefer to "swing" from worst condition to best possible condition to make the largest improvement (Figure 2 (omitted)). Decision makers repeat this procedure for all objectives in the set; after assigning a rank of one to the objective they most want to improve from worst to best, they are asked to think about the next objective they would most want to improve from worst to best and rank that as a two, and so on until all are ranked accordingly.

Once all the objectives have been so ordered, decision makers assign 100 points to the highest-ranking objective and a relative percentage of this weight to the others. A weight of zero may be assigned to swings on objectives from worst to best that are judged to be irrelevant (Baron 2000, Clemen 1996). For example, stakeholders should assign a weight of zero where there is no difference in real or perceived value between the worst and best performance, essentially canceling that objective and removing it from further discussion. Assigning weights and ranks helps participants identify objectives that are critically important versus those that may be no more or less important than others.

Have each participant conduct his or her own weighting exercise and identify his/her top alternative(s), which are often based on identifying the option that maximizes utility (multiplying the weight by the consequence measure for each alternative and seeing what scores highest). Then, use that information to identify the top options for the group (often, a few alternatives rise to the top). Discuss how those options may differ and what the "indifference" point might be for different participants.ÊIt may be that the top few options are indistinguishable and people can lend their support to more than one, or that minor tweaks could be made to the majority-supported option, so that those who identified it as their second or third choice can lend their supportÊbased on a few key tradeoffs.

After completing the swing weighting exercise, participants should review, compare, and evaluate the options. Each option should be accompanied by a "report card" that depicts its expected level of performance across all the objectives. That way, participants can quickly and easily cross-reference their own ranks and weights (determined during the swing weighting procedure) with the available options. In other words, a participant's ranks and weights should help direct him or her to his/her ideal option.

Conclusions and resources

Ultimately, SDM approaches are meant to provide a framework for incorporating diverse stakeholder values and objectives into the decision-making process. These approaches focus on identifying the problem, clarifying decision-relevant objectives that explicitly include diverse values at stake, identifying potential alternatives or solutions, measuring the consequences, and making the necessary tradeoffs. These five critical steps are purposefully designed to avoid potential pitfalls in human decision making, such as

• errors in judgment caused by the use of mental shortcuts when faced with complex choices,
  ¥ an imbalance between emotional responses to the problem and more reasoned or deliberative analyses, and
• too-simple characterizations of the problem that may lead to overly specific solutions.

Following such a structured approach has been shown to increase the quality of the decision, both in the short term (through increased participant comfort and satisfaction) and in the long term (through increased cost-effectiveness and long-term support). For additional resources on using such an approach, see "Smart Choices: A guide to better decision making" by Hammond, Keeney, and Raiffa; or explore the Web site "Structured Decision Making," developed by Compass Resource Management (http://www.structureddecisionmaking.org).

Sidebar: A handful of mental shortcuts and biases

Psychologists refer to a number of mental shortcuts (heuristics) and cognitive biases that people tend to employ in making decisions. Knowing about them helps someone guiding decision-making processes to avoid being blindsided by their inappropriate use. The seminal and influential early research (including items 1-3 below) helped bring its authors(1) a Nobel prize; thus practitioners can feel good about recognizing these shortcuts and biases.

• Availability heuristic: People will judge that events that can be more easily brought to mind or imagined are more likely than events that could not easily be imagined.
• Anchoring and adjustment: People will take one piece of known information and adjust from it to estimate an unknown quantity (e.g., a risk). But the adjustment usually will not be big enough.
• Risk aversion: People generally prefer avoiding losses to making gains.
• Confirmation bias: People tend to favor information that confirms their preconceptions; this can lead to overconfidence in personal beliefs and expected outcomes.(2)
• Optimism bias: The demonstrated systematic tendency for people to be overly optimistic about the outcome of planned actions.(3) This may be the sunny side of another tendency - discounting the future - in which people tend to take benefits now and leave others to pay later.(4)

1 Tversky, A., and D. Kahneman. 1974. Judgment under Uncertainty: Heuristics and Biases. Science, New Series, Vol. 185, No. 4157, pp. 1124-1131.
2 Oswald, Margit E., and Stefan Grosjean. 2004. "Confirmation Bias." In Pohl, RŸdiger F., Cognitive Illusions: A Handbook on Fallacies and Biases in Thinking, Judgement and Memory, Hove, UK: Psychology Press, pp.Ê79-96.
3 Hardman, David. 2009. Judgment and decision making: psychological perspectives. Wiley-Blackwell.
4 Bazerman, M. H., and M. D. Watkins. 2004. Predictable Surprises - The Disasters You Should Have Seen Coming and How to Prevent Them. Boston: Harvard Business School Press.

References

Arvai, J. L. 2007. Rethinking risk communication: Lessons from the decision sciences. Tree Genetics and Genomes 3:173-185.
Arvai, J. L., and R. Gregory. 2003a. A decision focused approach for identifying cleanup priorities at contaminated sites. Environmental Science & Technology 37:1469-1476.
- - - . 2003b. Testing alternative decision approaches for identifying cleanup priorities at contaminated sites. Environmental Science & Technology 37:1469-1476.
Arvai, J. L., R. Gregory, and T. McDaniels. 2001. Testing a structured decision approach: Value-focused thinking for deliberative risk communication. Risk Analysis 21:1065-1076.
Baron, J. 2000. Measuring value tradeoffs: Problems and some solutions. In E. U. Weber, J. Baron, and G. Loomes, editors, Conflict and Tradeoffs in Decision Making: Essays in Honor of Jane Beattie. Cambridge University Press, New York, NY.
Clemen, R. T. 1996. Making Hard Decisions: An Introduction to Decision Analysis. PWS-Kent Publishing Co., Boston, MA.
Costanza, R., and A. Voinov 2004. Landscape Simulation Modeling: A Spatially Explicit, Dynamic Approach. Springer-Verlag, New York, NY.
Failing, L., G. Horn, and P. Higgins. 2004. Using expert judgment and stakeholder values to evaluate adaptive management options. Ecology and Society 9:13-32.
Gregory, R., J. L. Arvai, and T. McDaniels. 2001a. Value-focused thinking for environmental risk consultations. Research in Social Problems and Public Policy 9:249-275.
Gregory, R., T. McDaniels, and D. Fields. 2001b. Decision aiding, not dispute resolution: Creating insights through structured environmental decisions. Journal of Policy Analysis and Management 20:415-432.
Hammond, J., R. L. Keeney, and H. Raiffa 1999. Smart Choices: A Practical Guide to Making Better Decisions. Harvard Business School Press, Cambridge, MA.
Holling, C. S. 1996. Surprise for science, resilience for ecosystems, and incentives for people. Ecological Applications 6:733-735.
Keeney, R. L. 1982. Decision Analysis: An overview. Operations Research 30:101-135.
- - - . 1992. Value-focused Thinking. A Path to Creative Decision Making. Harvard University Press, Cambridge, MA.
Keeney, R. L., and H. Raiffa 1993. Decisions With Multiple Objectives: Preferences and Value Tradeoffs. Cambridge University Press, Cambridge, UK.
Keeney, R. L., and D. von Windterfeldt. 1989. On the uses of expert judgment on complex technical problems. IEEE Transactions on Engineering Management 33:83-86.
Kleindorfer, P. R., H. C. Kunreuther, and P. J. H. Shoemaker 1993. Decision Sciences: An Integrative Perspective. Cambridge University Press, New York, NY.
Knutsson, J. 1997. Restoring Public Order in a City Park. Pages 133-151 in R. Homel, editor. Crime Prevention Studies. Criminal Justice Press, Monsey, NY.
Raffensperger, C., and J. Tickner 1999. Protecting Public Health and the Environment: Implementing the Precautionary Principle. Island Press, Washington, D.C.
von Winterfeldt, D., and W. Edwards 1986. Decision Analysis and Behavioral Research. Cambridge University Press, Cambridge, UK.
Walters, C. J. 1986. Adaptive Management of Renewable Resources. The Blackburn Press, Caldwell, NJ.
Wilson, R. S., and J. L. Arvai. 2006. Evaluating the quality of structured environmental management decisions. Environmental Science and Technology 40:4831-4837.

Other publications in this series

This publication and those following are part of Oregon Sea Grant's "Public Science Communication Research & Practice" series. All are available online at http://seagrant.oregonstate.edu/sgpubs or by calling 541-737-4849 or e-mailing sea.grant.communications@oregonstate.edu.

Borberg, Jenna, Joe Cone, Laura Jodice, Michael Harte, and Patrick Corcoran. 2009. An Analysis of a Survey of Oregon Coast Decision Makers Regarding Climate Change. Corvallis, OR: Oregon Sea Grant, Oregon State University. ORESU-S-09-001.

Cone, Joe. 2008. Expand Your View: Insights for public communicators from behavioral research. Corvallis, OR: Oregon Sea Grant, Oregon State University. ORESU-H-08-006.

Cone, Joe. 2008. Hold that Thought! Questioning five common assumptions about communicating with the public. Corvallis, OR: Oregon Sea Grant, Oregon State University. ORESU-H-08-005.

Kubeck, Gwenn. 2007. Public Outreach and Behavior Change: An annotated reference guide for outreach practitioners. Corvallis, OR: Oregon Sea Grant, Oregon State University. ORESU-L-07-002.

Acknowledgments

Text by Robyn S. Wilson and Joseph L. Arvai; editing by Rick Cooper and Joe Cone; design by Patricia Andersson. All photos © Oregon Sea Grant, except p. 4, © iStockphoto.com/Dave Bolton

© 2011 by Oregon State University.
This publication may be photocopied or reprinted in its entirety for noncommercial purposes. To order additional copies of this publication, call 541-737-4849. This publication is available in an accessible format on our Web site at http://seagrant.oregonstate.edu/sgpubs/onlinepubs.html
For a complete list of Oregon Sea Grant publications, visit http://seagrant.oregonstate.edu/sgpubs

This report was prepared by Oregon Sea Grant under award number NA06OAR4170010 (project number M/A-21) from the National Oceanic and Atmospheric Administration (NOAA), U.S. Department of Commerce, and by appropriations made by the Oregon State Legislature. The statements, findings, conclusions, and recommendations are those of the authors and do not necessarily reflect the views of these funders.