Strategic application of data and analysis often involve driving decisions. According to Merriam-Webster, a ‘decision’ can be defined as a determination arrived at after consideration. Whether it be political debates, litigation, business cases, or household endeavors, people arrive at an incredible variety of determinations, but often with questionable consideration. While little
consideration may be warranted for example to decide whether to buy a blue or green dish soap, significant consideration should drive other determinations, such as for investments and legal judgments. Such consideration should include logical and distinguishing analytics, known as ‘decision analytics.’
Any important decision should consider what analytic methods are available and chose that which will offer the highest level of confidence with the most reasonable level of effort. Most methods can be classified into five hierarchical tiers of confidence as shown in the figure. while higher tiered methods offer greater confidence through more precise and credible analysis, their greater sophistication correlates to more stipulations to overcome and greater levels of effort. A detailed description of each tier follows.
A random decision is spontaneous and baseless, and as a result can be applied quickly and easily under any circumstance. We use this method correctly when the decision is neither important nor obvious, for example when quickly grabbing a drinking glass from the cupboard to get a drink of water. However, because this method does not consider facts or evidence and includes no analysis, no confidence is gained to indicate a good decision was made.
Expert opinion includes methods such as Delphi and pairwise comparison to gain the opinion of an expert. All that is needed is the feedback of one or more relevant experts, so this method is often best used as a sanity test to compliment higher level methods (do the comparative return on investments appear reasonable?) but alternatively can be a last resort solution for an important decision with no other available data. A physician may rely on this method when symptoms of a patient are inconclusive. Because these decisions typically involve little or no qualitative or quantitative analysis, credibility is limited to the experts’ relevant expertise, objectivity and basis for opinion.
Anecdotal analysis attempts to equate one or a small number of example outcomes to the outcome desired by comparing constituent factors that drive outcomes. In many cases, anecdotes are the best data available, so this method is used frequently when analysis is revolutionary (e.g., cost estimates for a bleeding edge design). However, the factors that drive an example outcome may be unknown and/or different from those that would achieve the desired outcome, so anecdotal analysis offers minimal confidence. In a poor use of anecdotal analysis, an activist (e.g., lawyer or politician) may use innuendo or exploit the significant uncertainty of this method to levy unreasonable harm an individual or group’s character.
Using trends to drive analysis, including statistical regression and confidence measures, attempt to use multiple example past outcomes to measure a most likely future outcome. In some cases, future outcomes can be predicted with precision regardless of the value of one or more driving factors. Trend methods are powerful because many quantitative measures can include both assessments of outcome and confidence (i.e., credibility), so this method often is used when data is rich, such as making investment decisions to balance a portfolio. Though much more objective and thorough than anecdotal analysis, the underlying math can be deceiving if aberrations such as outliers or correlation versus causation are not considered. In addition, less available or less consistent data creates more suspect trends.
Engineered methods offer the greatest credibility because they are not subject to the biases of expert opinion, the irrelevance of anecdotes, or the mathematical properties underlying trends. Instead, the details of an engineered solution can be tallied and accounted. For example, a homebuilder can use an architect’s plans and salary data to determine with confidence what profit can be made at what price for a prospective structure. As long as the details of the engineered solution can be detailed sufficiently and risks (cost, schedule, performance) can be assessed and managed sufficiently, a determination can be made with highest credibility.
While higher tiered methods clearly enable more confident decisions, their greater sophistication can be an issue. The effort for a higher tiered method may not be worth the greater confidence, or the greater data and analytic needs to achieve greater confidence may be impractical. That is why tiers exist and why a good analyst should understand their dynamics and tradeoffs.
