12 March 2015
“Knee deep in the big muddy”: Escalation of Commitment
Professor Helga Drummond
In 1961 a remarkable event occurred in the Libyan Desert. For seven fruitless years an entrepreneur named Bunker Nelson Hunt and BP drilled for oil. Finally, operatives were instructed to stop drilling and come home. “Then, just for luck, the rig superintendent drilled another ten feet into the sand before withdrawing the bit from the third hole, and, in doing so, uncovered Bunker’s ace - one of the world’s largest oil fields.
Almost any decision involving uncertainty runs the risk of failing. When an important venture falters, individuals and organizations may have a difficult decision to make about whether to quit or continue. Whether it is a decision to remain “on hold” on the telephone, repair an old car, or a multi-billion pound mega-project like London’s Cross-rail, or Brandenburg’s long delayed airport, the dilemma is the same. Do we persist and risk compounding our difficulties? Or do we cut our losses?
If at first you don’t succeed, says the proverb, try, try again. To a point this is good advice. But only to a point! Supposing those wells in the Libyan dessert had been dry? Persistence would only have made matters worse.
Economics offers clear guidance on what to do when an important venture begins to falter. That is, decision-makers should review the project (or line of activity) and persist only if the marginal benefits of completing the project promise to outweigh the marginal costs of doing so. It is good advice, but research by psychologists suggests we don’t always follow it. Instead of giving up when the odds are plainly against us, we may well re-invest only to become caught up in a spiral of escalating commitment.
Escalation is thought to be a costly and ubiquitous problem. Chicago’s Deep Tunnel sewer project was derided as ‘money down the drain’. More recently, the refurbishment of London’s Savoy Hotel cost almost double what was forecast and took almost twice as long as expected. Likewise, the cost of Amsterdam’s new underground metro almost doubled during construction. The UK’s M6 Toll Motorway has been open for over a decade. Yet traffic levels are still well below what was anticipated. Edinburgh’s tram system finally opened in 2014, three years late and £375 million over-budget. Traffic through the Channel Tunnel has been significantly less than forecast, leaving UK taxpayers to pay bills of over £10 bn. Berlin’s Brandenburg airport was due to open in 2011. Then it became late 2014. At the time of writing the date has been put back to 2016 – at the earliest. Meanwhile, costs have spiralled from an initial forecast of €2.5 billion to over €4.3 billion. Many more examples could be cited. Why do never seem to learn? In this lecture I discuss how escalation starts, what drives it and what we can do to protect ourselves and our organizations from becoming embroiled in a potentially ruinous course of action.
How Does Escalation Start?
Escalation is defined as persistence with a course of action beyond an economically defensible point. More specifically:
1. significant resources are invested,
2. feedback increasingly suggests important expectations are unlikely to be met;
3. there is an opportunity to quit or persist, and,
4. the consequences of either quitting or persisting are unknown.
It is thought that the seeds of escalation and ultimate failure are often sewn early on thanks to decision-makers’ unrealistic expectations about what can be achieved – a combination of deceit and delusion. Since I deal with this in my November 2014 lecture entitled “Predictable Surprises” I will mention the main traps very briefly.
One of the biggest problems is over-confidence. One reason why almost 80% of new business start-ups fail with two years is that entrepreneurs tend to over-estimate projected sales and under-estimate – or fail to investigate expenses. They forget that bills roll in from day one, but customer bases can take years to build.
In theory, complex organizations are shielded from human foibles because they employ rigorous project planning and sophisticated forecasting tools. Yet in practice those tools may foster overconfidence because they tend to be inward looking, focused on the organization’s capabilities, how executives propose to surmount challenges and address perceived risks, blind to the competition, and market realities. Moreover, because comparable projects are frequently ignored, forecasted costs and benefits may be rooted in fantasy. Tesco assumed that their US venture would be profitable within two years when industry standards said twenty years and Tata’s ill-judged hopes for the Nano motor car. Another problem is that politically adroit project planners may conceal the true costs and over-emphasize the benefits of a project. Remember the extravagant promises that were made about the long term benefits of the London Olympic Games. How many of those have actually materialised?
What Drives Escalation?
Eventually, chickens return to roost. Costs start to over-run, deadlines slip and doubts emerge about whether promised benefits will materialize after all. In a nutshell, what drives persistence is the sheer cost of quitting.
Some of those costs are likely to be psychological. Beleaguered decision-makers may feel bound to justify all the time, money and effort invested in the venture. Although these sunk costs are irrelevant because they cannot influence outcomes, decision-makers may feel they simply have “too much invested to quit”. For example, imagine trying to halt London’s Crossrail project now – all that investment and nothing to show for it except for a big hole in the ground!
The prospect of huge loss may tempt decision-makers to take bigger risks than objective conditions warrant. More specifically, choices are likely to be expressed (framed) negatively. Abandonment will definitely mean a loss. Persistence offers the chance of avoiding that loss if the project is ultimately successful, but risks seriously compounding it. Caught between a rock and a hard place, decision-makers may prefer to take that risk, particularly if they are liable to be held personally responsible for failure.
Impending failure can also pose a profound ego threat. Ego refers to our sense of self, our innate desire as human beings to appear in a good light. Ego defensiveness can lead beleaguered decision-makers to lapse into denial. For example, they may insist problems are exaggerated, that set-backs are temporary, and engage in wishful thinking by focusing only on feedback that supports preconceived views, whilst downplaying disconfirming data. They may also try to bolster their convictions by seeking additional information that confirms their preconceived views. They may simply be deluded. For example, when TAURUS a mega IT project led by the London Stock Exchange was running over eighteen months late, a member of the project monitoring group said, ‘“I think they [the project team] ... couldn’t believe that it wouldn’t work. They believed they knew how to make it work but it would just take longer and would cost a bit more,”’. 
Another form of denial is blaming setbacks on uncontrollable external factors such as freak weather conditions and currency fluctuations. Furthermore, beleaguered decision-makers are likely to be more interested in rooting out information that justifies past decisions than addressing pressing problems. Since these psychological traps tend to operate unconsciously, decision-makers may genuinely believe that a success is close, when objective analysis clearly suggests otherwise.
Persistence may also be driven by “goal substitution” effect. That is, as economically poor projects advance, the original success goal gets forgotten as completion becomes all important. For instance, the technical team realized that TAURUS would never work as envisaged. Latterly, all they cared about was finishing the task, ‘“Let’s get the bloody thing done and behind us,”’.
Admitting failure publicly may be even more costly than admitting it privately as decision-makers strive to appear competent and avoid embarrassment. Moreover, culturally, decision-makers are expected to keep their promises, finish what they have started, to appear resolute and ‘not for turning’. Culturally, persistence in the face of adversity connotes strength whereas withdrawal signifies weakness.
Competition can also fuel escalation. Consider the so called, ‘dollar auction’ where the second highest bidder must pay the bid price though they receive nothing. In experiments, final bids consistently exceed the face value of the prize as ‘locked-in’ bidders keep raising the stakes in a mutually destructive spiral. But bidding may also be motivated by a desire to look good in front an audience and even by thirst for revenge.
Abandoning a failed line of activity usually means incurring waste. As human being we tend to abhor waste. Waste refers to the unused portion of an item. For instance, imagine that you have two identical frozen meals in the fridge with identical “use buy” dates. The only difference is that one cost £8.99. The other was bought on special offer for £5.99. Which one do you consume?
Logically, it makes no difference. But you may be tempted to devour the more expensive meal because it seems the least wasteful course of action! That explains incidentally why mobile phone companies offer trade-ins on old phones rather the equivalent cash discount. Most of us change phones more frequently than we need to. The aim of a trade in is to make the purchase seem less wasteful.
Yet some forms of waste make economic sense. For instance, it makes sense to buy a strip of tickets for admission to a theme park even though you don’t use all of them if to do so is cheaper than buying single tickets. But people have been known to stay away, thus depriving themselves of the pleasure because although they know it makes sense to buy tickets in a strip, they just can’t bring themselves to waste part of the strip. Similarly, an owner of a failing business told me she would rather throw unsold stock in a skip than sell it too cheaply! On different scale of expense, Leeds abandoned plans for a new tram system in favour of a trolley bus service even though they had already spent about £45 million on the project. It sounds like a huge waste of public money – which on one level it clearly is. Even so, it may well have been a wise move if it corrects a false start. That is, if the trolley bus really does turn out to be cheaper and better.
Waste aside ending an economically poor line of activity can be prohibitively expensive, particularly if little can be salvaged from it. One reason why divorce rates are lower amongst middle class couples than working class couples is because of the cost involved. Owners of failing small businesses may discover that they are constrained by obligations on a lease or a warehouse full of unsold stock. On larger projects there may be penalty charges to contractors; redundancy payments, costs of ripping out partially completed works, obligations on leases and so on. For example, Accenture faced possible penalty charges of about £1 bn. (subsequently reduced to about £650 million) when they decided to exit from the NHS electronic record system – a project that became another expensive failure. Similarly, planning for the London 2012 Olympics began in 2005. By the time the world financial crisis broke in 2008, decision-makers were already well and truly committed as hundreds of contracts had been let. Sometimes it can almost as expensive to quit as it is to continue.
Ultimately, economically poor projects may be driven by their own momentum. For instance, an elaborate administrative infrastructure may have been created to support them. External stakeholders may have invested heavily in anticipation of completion. If an endeavour comes to symbolize the values and purposes of the organization, it may acquire “sacred cow” status. Some projects survive because no one dares or perhaps even thinks to criticise them – known as a “non-decision”. If so, ending it may be almost unthinkable as evidenced, for example, by Hewlett-Packard’s reluctance to abandon its unprofitable PC business. Ultimately, an economically poor line of activity may survive because ending it is it is too much trouble, “All the rules, procedures, and routines of an organization as well as the sheer trouble it takes for managers to give up day to day activities in favor [sic] of serious operational disruption can cause administrative inertia.... Sometimes it’s easier not to rock the boat”. 
Escalation and the Simple Passage of Time
So far we have considered how escalation occurs thanks to intentional re-investment in a questionable line of activity that makes quitting progressively more expensive. Yet “lock-in” can occur without having to do anything – or through making so called “decision-less-decisions”. To be more precise, there is another form of escalation known as entrapment. Entrapment results mainly through the simple passage of time. The special fascination of entrapment is that decisions get made for good return yet decision-makers somehow reach a point of “no return” where persistence can only make things worse. How does it happen?
One reason for entrapment concerns the making of so called “side-bets”. Side-bets refer to extraneous investments not directly connected to the original decision but which over time can make quitting too expensive. For example, in the 1950’s Chicago operated a waiting list for jobs in middle class schools. Many new teachers took jobs meanwhile in schools in working class districts. By the time they reached the top of the waiting list they were no longer interested in transferring. Having written their materials for working class children, they were not willing to undertake the enormous task of re-writing. So although staying put was sub-optimal, changing direction had become too costly. Similarly, owners of declining small businesses may end up working for less than half of the UK minimum wage. They persist because they are reluctant to forgo the prestige and independence of ownership. Another side-bet is that owners often form emotional bonds with customers. None of this may have been why they went into business, but that is what eventually holds them in place even though persistence is severely economically sub-optimal. Incidentally, that is why the UK government passed legislation to make pensions more transferrable. Eventually, non-transferable pensions made switching to a new employer too expensive.
Side-bets are not the only reason for entrapment. Imagine waiting for a bus to take you to the railway station to catch a train to London for a job interview – or some other important appointment. What do you do when the bus fails to appear? On a different scale, why did Edinburgh wait so long for their short stretch of tramway? Why did the UK’s Chinook helicopters languish in hangars for almost eight years when the machines were badly needed on operations?
One of the worst mistakes we can make is to assume that the passage of time is bringing us closer to our goal. This point was graphically demonstrated in a clever experiment by Jerry Rubin and Joel Brockner.  The authors wanted see what would happen when costs of quitting and costs of persistence were pitted against one another. In other words, how would decision-makers react in a situation where time becomes both an investment and an expense? An investment in that it brings us closer to our goal; an expense because waiting involves cost:
As time passes, the cost associated with the continued waiting increases, but so does presumed proximity to the goal. Hence, the greater the passage of time, the greater the conflict. And the greater the conflict the greater the pressure to act decisively-either by withdrawing or committing oneself to remain in the situation.
Participants were challenged to solve a crossword puzzle in return for a “jackpot” prize of $8. 00. The value of the prize dropped after the first three minutes – with thirteen minutes to work on the puzzle. Alternatively, during the first three minutes, participants could elect to leave the experiment and take $2. 40 stake money. After three minutes, the value of the stake declined by ten percent per minute. Crucially, participants were informed that a dictionary was available to help them solve the puzzle more quickly therefore better enabling them to win the jackpot, and that they were first, second or third in the queue to receive it. Some participants also received pay-off charts depicting potential earnings at the end of each minute to measure cost salience. Participants were also divided into high and low decrement conditions.
In fact there was no dictionary. Yet 87% of participants remained in the experiment beyond the break-even point of three minutes – particularly those in the low cost salience condition. Moreover, more than half the participants waited beyond a point where they could make no more than £2. 40 even if they took what was left of the jackpot, ending up with only $1.04 when they could have had $2.40 stake money had they quit sooner. Instead, they spent over a third of the time, waiting for this non-existent resource. Overall, winnings were lower when the value of the jackpot declined slowly rather than more rapidly; where participants had no payoff chart, and, when told they were first in line for the dictionary. We may infer from those experiments that waiting begets waiting. This likely to be particularly true where:
1. the goal is important;
2. the costs of giving up are high, and,
3. the costs of persistence are hidden.
Entrapment is more invidious than escalation because it tends to happen gradually without us needing to do anything and therefore without us noticing until it is too late. It happens because we fail to recognise that the passage of time may not be without cost. In other words, because we fail to think enough about what electing to wait for a valuable resource (the dictionary) might end up costing.
What can we do to protect ourselves and our organizations against costly lock-in? How can we avoid projects that are destined to become “black holes”, forever consuming resources to no avail and depriving good projects? Some escalation is almost inevitable and should be regarded as a normal living or business expense. The important thing is recognising failure sooner rather than later and doing something about it.
What are you getting into?
Since escalation scenarios become progressively harder [more expensive] to stop the most important prescription is for decision-makers to be careful what they get into. No one intends getting trapped in the dollar auction – but all participants take that risk because of the dynamics of the situation. Yet the outcome is predictable. Nor do participants in the phantom dictionary experiment expect to be kept waiting indefinitely. In each instance, “lock-in” happens because decision-makers fail to see what they might be getting into.
The dollar auction is a good metaphor for competitive escalation. For example, suicidal price wars between some airlines and supermarkets. Likewise, litigants don’t always realise the pitfalls of going to court. Early on overly confident barristers and solicitors may say they have a strong case. Then, as they delve more deeply into it, (racking up expense meanwhile) revise their expectations downwards. Litigants may win – only for opponents to appeal. More expense! What started as an estimated £80, 000 costs becomes £800, 000 – for a strip of land worth £800! Similarly, although Accenture were probably wise to cut their losses, they might have even wiser to have steered clear of the venture in the first place. Here was a project that was almost bound to become enmeshed in controversy over medical ethics and patient confidentiality to say nothing of technical challenges of building a system involving complex rules about who has authority to input data, multi-site access and so forth. Secretary of State, George Ball warned President Lyndon Johnson of the dangers of sending troops into Indochina. Ball foresaw heavy casualties amongst troops ill-equipped to fight in hostile terrain, starting an almost irreversible process that would result in national humiliation – even after paying terrible costs. The warning was remarkably prescient. We have diplomacy precisely because there is no such thing as a limited war.
Emphatically, I am not suggesting people do not take risks, only that they pause long enough to think through how a decision might play out. As the Tao counsels, only by avoiding the beginning of things can we escape their inevitable ends.
Weigh opportunity costs
It is also wise to consider opportunity costs. This is one the big arguments over the high speed rail link project known as HS2. Yes it promises to reduce journey times between London, Birmingham, Leeds and Manchester. But critics argue that the money would be better spent upgrading existing rail infra-structure. As economists tell us, the true cost of anything is what we could have had instead.
Recall, failure arrives when it becomes clear that important expectations are unlikely to be met. But if expectations are blurred to begin with it makes failure hard to recognize and contestable because it becomes impossible to measure results against expectations.
Recall, entrapment was most pronounced in the phantom dictionary experiments where the costs of persistence were hidden or not immediately obvious. Budgets are an important tool for exercising vigilance. They may be formal financial allocations or what psychologists call ‘mental budgets’. ‘Mental budgets’ refer to informal allocations of money, time, effort and emotional energy to a project. Budgets are thought to curb escalation by inducing vigilance, throwing the spotlight on deviations from plans and making decision-makers more accountable. They also force decision-makers to confront their options which can stop things from drifting (see below).
With one caveat: budgets may induce decision-makers to abandon an economically viable project. More specifically, in experiments with sophisticated individuals, as budget limits are reached, decision-makers may decrease investment. This is erroneous because, by itself, budget depletion contains no information about the state of the project and whether it is worth completing on economic grounds. It is what accountants call a “non-informative loss”. That is, budget depletion merely states that a certain amount has been expended. Yet apparently decision-makers are sometimes unable to distinguish between informative and non-informative losses.
Make persistence an active decision
Don’t wait until problems escalate into a crisis. In a sequel to the phantom dictionary experiments, the authors found that participants who had to make an active decision to remain in a game became less entrapped than those for whom persistence was the default option. The authors concluded that active decisions force participants to consider past and future costs, and, attend to that information. Confront their options, in other words. In contrast, if persistence is the default option, decision-makers may not notice what persistence is costing.
Set limits and stick to them
Some participants in the afore-mentioned experiment were also required to publicly state limits on their involvement. Other participants set limits privately whilst a control group set no limits at all. Unsurprisingly perhaps, most money was invested by participants who set no limits. Conversely, participants required to publicly state limits invested least.
But limits are only useful if decision-makers adhere. That means disabusing ourselves of the conviction that success is just around the corner. It also means accepting that “pulling the plug” may be the wrong decision. The point is, effective decision-making is not about getting things right all the time. That is an unattainable goal. Success lies in getting decisions more right than wrong.
Think the unthinkable
Another reason participants succumb to lock-in is because they never stop to ask themselves whether the dictionary might be fiction. Yet like the dictionary, some things are destined never to arrive. If so, the sooner we recognise it the better.
If possible, obtain definitive evidence. For example, when TAURUS stalled Peter Rawlins, the then chief executive of the Stock Exchange commissioned an independent firm of consultants to examine the project. The result was a severe shock. Three years into construction, the operating system was not designed, let alone built. Similarly, when the opening of Denver International Airport became severely delayed by problems with the new fully automated baggage handling system, a firm of consultant engineers isolated a loop of track (a microcosm of the whole system) and tested it under a working load. Chaos ensued. Conveyor belts jammed and baggage cars crashed, showing conclusively that the system was not fit for purpose.
Then what? Information is only useful if we act upon it. Three questions should be posed:
1. What am I trying to achieve?
2. How can I be sure of achieving it?
3. Is the only way to achieve it?”
Many lines of activity are a means to an end. Sometimes owners of small businesses can successfully downsize – they close down parts of the business in order to remain in business. Denver airport authorities broke out of a potential escalatory spiral by recognising that their goal was not to create a state of the art baggage handling system, but to get the airport up and running. So they reverted to a semi-automated system that enabled them to achieve their over-arching goal. Similarly, Leeds recognised that the goal was not to build a tram line as such, but to provide a mass transit system. A trolley bus would suffice.
If you are manager, what are you nothearing? For instance, the TAURUS monitoring group became uneasy by what was left unsaid in monthly reports:
We said, ‘How is all this going to come together?’
And they [project managers] said, ‘Well we have an integration plan, and we have a testing schedule, and we have all these other things’. And they had a lot of them. Yet there was never a sense that the project was under control. 
What matters is not what has been done, but what remains to be done. A project may be 90% complete. But that may not mean very much if the hardest part is at the end. For example, if the end involves novel technology.
It is also important to consider what the benefits will be. Is the project still relevant? Have benefits become so diluted that persistence no longer makes sense? Looking beyond conventional information sources can pay dividends. One encouraging sign for Cross-rail has nothing to do with the achievement of project milestones or staying within budget. Rather it is the rising house prices along the route – a highly significant indicator that the long term prospects are good whatever the short-term difficulties may be.
Don’t institute a “death march”
Beware escalating indecision. That means instituting on a “death march” – resetting targets, timescales and quitting points to postpone the inevitable. Death marshes may well be a tactic to mask procrastination. To procrastinate is to postpone necessary but painful action. The attraction is temporary psychological shelter. Better to declare failure and re-direct energies to something more profitable.
But how do we recognise failure if definitive information is impossible? The answer is to weigh up, as accurately as possible the probabilities. More specifically:
1. What needs to happen for the project to succeed?
2. How likely is it that things will happen in a timely fashion?
3. What are the alternatives?
Clearly, unlike spinning a coin, any assessment of real-world probabilities is approximate. In the end it isa judgment call.The point is, recognising failure sooner rather than later usually means decision-makers have more options for dealing with it.
Real options thinking
An option confers the right but not the obligation to take action in the future. For example, an oil exploration company may acquire land and licence but postpone drilling unless and until oil reaches a certain price per barrel – known as a delayed entry option. Delayed entry options avoiding making a full commitment. Conversely, in the immediate aftermath of the world financial crisis of 2008, many construction firms mothballed partly completed blocks of flats until selling prices recovered to a predetermined level – rather than abandon projects outright. This is known as a delayed exit option. Delayed exit options are useful where outright abandonment would be extremely expensive to reverse. The genius of real options thinking is that the cost of an option is fixed whilst the benefits are potentially unlimited.
Options are not a panacea, however. Not least, is working out when an option should be exercised. For instance, it is not just a question of oil reaching a pre-determined price per barrel. How long does it have to remain at that price to justify drilling? Another objection is that a large portfolio of options may not be worth the carrying cost as they may never be “in the money”. So whilst in theory, options are a way of providing for the future instead of trying to guess, they are by no means fool-proof.
Even so, before abandoning a line of activity, decision-makers should consider what options would be destroyed. For instance, if a bespoke software project is abandoned in favour of an “off the shelf” product, the potentially lucrative option to licence the software to other firms is forfeit. So, if there is a reasonable probability of success, the project might be worth persisting with even though it is proving more difficult and more expensive than expected. Similarly, if an oil field is sold all the options attached to it, such as gas exploration, options to build on land and so forth are extinguished. In other words, exiting may be more expensive than it looks.
Abandoning a successful line of activity
Ultimately what matters is net gain. That means if an opportunity offering a higher return on investment becomes available, in theory we should switch even if it means abandoning a successful line of activity. I say “in theory” because some economists argue that in practice it is only worth switching for a really large gain. In this view it is probably not worth changing jobs for a small rise in pay and benefits.
But how big is big enough? Even big gains may not live up to expectations. For instance, Tullow’s share price rocketed when the firm abandoned North Sea operations to pursue fabulous discoveries in East Africa. But some of those new wells have proved dry. As decision-makers, our best protection lies in constantly reminding ourselves that nothing is certain – perhaps not even uncertainty itself. 
© Professor Helga Drummond, March 2015
 Drummond, H. (1996). Escalation in decision-making: The tragedy of Taurus.Oxford: Oxford University, p.141.
Drummond, H. (1996). Escalation in decision-making: The tragedy of Taurus.Oxford: Oxford University, p.140.
 Department of Health: The National Programme for IT in the NHS: An Update on the Delivery of Detailed Care Records Systems, 2011.
 Staw, B. M., & Ross, J. (1987b). Knowing when to pull the plug. Harvard Business Review,65(2), p. 71.
 Becker, H. S. (1960). Notes on the concept of commitment. American Journal of Sociology, 66, 32-40.
 Rubin J. Z., & Brockner, J. (1975). Factors affecting entrapment in waiting situations: the Rosencrantz and Guildenstern effect. Journal of Personality and Social Psychology, 31, 1054-1063.
 Rubin J. Z., & Brockner, J. (1975). Factors affecting entrapment in waiting situations: the Rosencrantz and Guildenstern effect. Journal of Personality and Social Psychology, 31, p. 1055.
 Heath, C. (1995). Escalation and de-escalation of commitment in response to sunk costs: the role of budgeting in accounting. Organization Behavior and Human Decision Processes, 62, 38-54.
 Brockner, J., Shaw, M. C., & Rubin J. Z. (1979). Factors affecting withdrawal from an escalating conflict: quitting before it’s too late. Journal of Experimental Social Psychology, 17(5), 492-503.
 Monteagre, R., & Keil, M. (2000). De-escalating information technology projects: lessons from the Denver international airport. MIS Quarterly, 24, 417-447.
Drummond, H. (1996). Escalation in decision-making: The tragedy of Taurus.Oxford: Oxford University, p. 98.
 McAfee, R. P., Mialon, H. M., & Mialon, S. H. (2010). Do sunk costs matter?Economic Inquiry,48, 323-336.
 Rubin, R. E. (2003). In an uncertain world. New York: Random House.