Analytic hierarchy process-based group assessment of quality-in-use model characteristics

Abstract

© 2015, Springer-Verlag Berlin Heidelberg. While the characteristics related to the qualities of software in use have been standardized, people managing software development processes still have to combine and prioritize attainment levels for such characteristics. Ranking them can therefore be considered a decision problem that should be solved not only in accordance with the preferences of the stakeholders involved in the decision-making process, but also by following multi-part standards, e.g., that the relative importance of quality characteristics should depend on the high-level goals and objectives for the project. This paper presents an example guide for creating such a ranking for a group of experts coming from different domains using a decision-support inspired approach. Five important quality-in-use characteristics are evaluated by sixteen experts with the analytic hierarchy process. Obtained individual preferences were aggregated by two procedures, and achieved group results were analyzed, of which one analysis included testing of their conformity to individual results. For this case study, the group opinions indicated the top-valued quality in use was effectiveness, followed by satisfaction, freedom from risk, efficiency and context coverage in this ranked order (freedom from risk and efficiency were of nearly equal importance). Implications for the future work of applying different decision-making models such as social choice theory for studying quality attributes are also discussed.