The NAO report I referred to in the prior post made a number of points about managing qualitative factors. In fact, this seems to be a particular area of weakness in government decision processes:
- “Central guidance on appraisal is generally thorough but there are a few areas where a fix is necessary to support better appraisal:
- Introducing a requirement for a structured consideration of qualitative factors on a common basis with quantitative analysis which is consistently applied across options, and extending associated practical guidance.
- Developing a requirement to create an explicit ‘logic model’, setting out the contributions of all factors to the subject of the appraisal, allowing the contribution of qualitative factors to be set in context.”
- “Qualitative factors are often a key element in decision-making. From 45 final Impact Assessments 31 presented a fully monetised preferred option. The remaining Impact Assessments had either monetised only costs or benefits or not monetised any impacts. Out of the 31 Impact Assessments that had provided a net present value eight had either zero or negative net present values. Thus, the preferred option was in only 23 out of 45 cases supported by quantitative cost-benefit analysis. In the remaining cases qualitative reasoning was a decisive factor.”
The report includes guidance on how to take account of both qualitative and quantitative factors in a structured way:
- “Inevitably, there will be some government interventions which will always be too difficult to quantify in money terms and will need to be discussed in other comparable ways. The current Green Book mentions two techniques which can be used to compare unvalued costs and benefits (multi-criteria decision analysis and critical success factors analysis)”
- “Incorporating wider, non-monetised issues in the policy appraisal not only ensures that the full range of impacts are appraised, but also encourages policy-makers to consider how non-monetised issues, such as environmental limits, statutory policy targets or regulatory requirements, can be built into options as a positive feature.”
In my experience, decision-makers are often uncomfortable with qualitative or ‘intangible’ criteria, and yet as NAO found, it is often the case that these intangibles turn out to be the most significant factors influencing a decision. What is required is the ability to evaluate and manage these qualitative factors, to make them tractable. This is where multi-criteria techniques come in, providing the means to bring both quantitative and qualitative factors together in the decision process in a rational and transparent way.