Risk Informed Decision Making In Information System . PDF

15d ago
1.96 MB
10 Pages

Proceedings of the 50th Hawaii International Conference on System Sciences 2017Risk –Informed Decision Making in Information System ImplementationProjects: Using Qualitative Assessment and Evaluation of Stakeholders’Perceptions of RiskMonica SchurrUniversity at [email protected] De TuyaUniversity at [email protected] successful implementation of a new softwaresystem at any organization requires identificationand management of risks as well as insight into thedecision-making process throughout the informationsystem lifecycle. Risk assessment of software systemsaids in planning, implementation and adoption stagesand helps identify potential problems before theyoccur. This study utilized a qualitative case studymethod and an interview design for data collection togather, organize and make sense of key stakeholders’perceptions of risk for decision making in theimplementation of a new department-widecomputerized system. Top stakeholder risks identifiedinclude executive sponsorship support; adoption ofthe new technologies and processes; andinteroperability. The results of the analysis ofperceptions of risks allowed the organization and theteam responsible for the implementation of the newsystem to make decisions about mitigating strategiesaligned with stakeholders’ expectations; forecastpotential issues within the implementation timelinebased on activities associated with identified risks;and make implementation and process decisionsbased upon the risk assessment. This study extendsthe research on IT risk management and decisionmaking by demonstrating the utility and efficacy of aqualitative case study method for eliciting theinformation needed from stakeholders in order tomake decisions regarding system implementation,specifically in an organization that lacks theappropriate risk management maturity level toconduct an exhaustive quantitative analysis of risksassociated with the project.1. IntroductionThis paper describes how a qualitative approachfor assessing and evaluating risks in order to informURI: http://hdl.handle.net/10125/41903ISBN: 978-0-9981331-0-2CC-BY-NC-NDKathryn NollRensselaer Polytechnic [email protected] making and risk response benefited amanufacturing organization during and after theimplementation of a new department-wide softwaresystem. The purpose of the system was to move theorganization out of a paper-based manufacturingprocess and into managing their production processvia automated workflows able to control theexecution of the manufacturing steps. Theorganization implementing the system could becharacterized as highly concentrated on achievingexcellence in their core competencies. Such corecompetencies, as identified by company executives,fell within the quality assurance and manufacturingareas, with the latter being the actual owner of thesystem, making the project a department-wideimplementation. Nevertheless, the scope of thesystem required the establishment of a crossfunctional implementation team to ensure that crossdepartmental processes were considered whenconfiguring the software solution.The risk assessment for this study identified risksassociated with the new information system,hereinafter referred to as “the system”. Risksassociated with the system were based on perceptionsfrom areas of business such as Manufacturing,Information Technology (IT), Quality Assurance(QA), Supply Chain, Process Controls, andManagement. Traditionally, risk assessments forsoftware implementation projects are performedutilizing a variety of quantitative methods. In the caseof the organization being studied, there was a lack ofexpertise in performing such assessments, inparticular for software projects. Using a qualitativemethod allowed the implementation team access tokey organizational representatives of the areas beingaffected by the system.This paper is organized into six sections: sectionone includes an introduction to the paper in generaland this information system implementation projectspecifically; section two provides a brief literaturereview; section three describes the methodology;6120

section four presents results; section five analyzesand discusses the interviews and identified risks, anddetails the risk-informed decision making processthat resulted from this work; section six presentslimitations; and section seven presents contributionsand concluding remarks.The interviews for this paper were conductedbetween the end of the planning phase and thebeginning of the implementation phase.2. Literature Review2.1. Risk Management for IT Projects1.1 Description of the Information SystemImplementation ProjectThe main goal of the project was to improveproductivity and reduce cycle-time in the total time toproduce a manufacturing order. Automating theprocess of controlling manufacturing records wouldoptimize production activities within the organizationand most likely bring additional synergies wheninteracting with external manufacturers. The potentialbenefits identified during the development of thebusiness cases were, among others, increasedproductivity, savings in labor costs, enhancedmanagement capabilities, shorter reaction time tochanging market conditions and higher availability n.The proposed approach was to find a best of breedsolution that could be integrated into theorganization’s current technology landscape and longterm business and IT strategy. A transformativeinitiative like this required the establishment of agovernance body that included members of theleadership team acting as executive sponsors as wellas active members of a steering committee. A projectmanager from the IT department was in charge of theformal management of the project across the areas ofthe organization needed in the definition andexecution of the project deliverables.As part of the management of the project, acomprehensive project timeline was produced, whichlisted a 17-month implementation strategy thatincluded the definition of user and functionalrequirements, definition of interfaces with otherexisting applications, unit, system and integrationtesting phases and a final user acceptance testingphase followed by a month-long deployment into theproduction environment. The effort was divided intophases as per project management best practices: a planning phase where high-levelrequirements were gathered, vendors werescreened and selected and budgets weresubmitted for approval an implementation phase to design andconfigure the system a testing and deployment phaseRisks are classified as events that have adverseoutcomes. Risk management is a process involvingassessment, response and mitigation that can helpprevent risk from occurring, as well as minimizedamage and contain the cost of recovering from risk,if risk does occur [1]. While risk can never beentirely eliminated from a system, performing riskassessment aids in identifying current or potentialrisks associated with the implementation andoperation of a computerized system in a givenorganization [2]. Additionally, it can providestrategies to manage identified risks at a level that isacceptable for the organization [3]. Risks areassessed by examining magnitude and likelihood [1,3, 4], and risk response involves the organizationcreating and implementing both preventative andcorrective controls to ensure risk is minimized [5].Additionally, risk mitigation acts to introducecontrols that reduce potential risks within a system, toaddress risks and generate solutions to reduce andresolve threats [6].Risk management within IT systems is vital toensure that systems operate within specificperformance and computational accuracy thresholdspreviously agreed upon in the form of userrequirements and made official via Service LevelAgreements (SLAs) [2, 3]. Generally speaking,managing risks in a software implementation projectis a three phase process. Each of these phases maypresent different types of risks and, accordingly,different methods for managing them [7] .The objective of a risk analysis and identificationprocess is to provide information to facilitate thedecisionmakingprocessrelatedtotheimplementation of risk management strategieswhether it is acceptance, elimination or reduction [8].Traditionally, risk assessments for softwareimplementation projects rely heavily on a variety ofquantitative methods [9-11] that concentrate on therisk analysis and mitigation efforts to project-specificdeliverables or processes, which lead to a eless, software implementation projectsproduce business-specific (operational) risks thatshould be quantified and, if needed, managed [9, 13].To that end, research has demonstrated that involving6121

business subject matter experts (SMEs) positivelyimpacts the performance of the implementation teamand creates a sense of ownership for the SMEs whenthey perceive the system as their own creation [14].Appropriately addressing user (SMEs) perceptions ofrisk have been linked to increased levels of alignmentacross the business as well as higher levels oforganizational awareness [14].The field of risk assessment and decision makingis multifaceted and the processes multidisciplinary,which must be taken into account when considering ascientific platform and/or framework for risk [15].Many theories explaining risk and decision makingform the foundation of quantitative studies for riskanalysis and management, including decision theory,the behavioral view of risk, and the real options viewof risk [16-18]. While many consider quantitativerisk assessment (QRA) the method for estimating andquantifying risk, one must also consider that “societalrisk decision making” – which stems from identifyingsuch risks – requires consideration of stakeholders’understandings as well as contextual factors [15]. Aqualitative risk assessment targets the elicitation ofsuch important information (i.e. the answers to“what” and “how” questions) and thus providespragmatic grounds for an exploratory method, whichcould also lay the groundwork for theorydevelopment [19].2.2. Assessing and Evaluating RiskUnderstandings from the field of education withregards to assessment and evaluation can provide atheoretical framework for the development of aqualitative interview protocol, the collection of dataon specific risk indicators (assessment) as well as theuse of the information gathered from these qualitativeinterviews to inform decision making on riskmanagement, mitigation, and reduction (evaluation).One approach in education is to separate theconcept of assessment from testing and grading, andunderstand it as the extent to which one has attained alearning goal; and evaluation can be thought of asapplying that assessment information to inform andmake decisions [20-22]. For purposes of clarity andprecision when measuring attainment, broad learninggoals can be written at very specific levels.Specifying (learning) indicators at a fine-grainedlevel as opposed to a coarse-grained level [23, 24]allows for collection of useful information and thusclear and specific measurement of attainment(assessment) as well as actionable evaluation (usingthe information to inform decisions) and eliminatesthe potential for confusion that is wrought withvague, broad, and general statements/indicators [21,25-27].In applying this educational assessment andevaluation perspective to the assessment of risk in anIT project, the indicator of the presence of learning(i.e. learning goal) can instead be framed in terms ofan indicator of presence of risk (or, as the case maybe, the perceived presence of risk). Furthermore, theconcept of coarse-grained and fine-grainedinformation can be applied in terms of broadindicators of risk (e.g. issues with documentmaintenance) that can be broken down into morespecific indicators (e.g. issues with record storage,ease of access, maintaining paper records and needfor backups, among others). As in the field ofeducation, collecting this information at such a finegrained level can inform decisions (what we will callor consider a form of risk evaluation) as much as theactual actions. An example of this is users drivingorganizational change management, as discussed inthe next section.2.3. Users Driving Organizational ChangeManagementA determining success factor for theimplementation of computerized systems is the levelof readiness achieved by the organization prior todeploying the new technology [28, 29]. Such a stateof readiness is achieved by the appropriate planningand execution of an organizational changemanagement process [30], which consists of makingthe organization aware of the change, educating usersand secondary stakeholders on the consequences ofthe change and how to deal with it and creating thecorresponding mechanisms so that the new status isadopted as seamlessly as possible [28, 30].A specific approach for facilitating organizationalchange consists of involving non-supervisor membersof the organization in a semi-crowdsourcing mode ofproblem solving, also known as participativeleadership[31]. Research has positionedparticipative leadership not only as a generator oftrust, but as a driver for enhanced organizationalperformance [31] and it is also positively influencedby higher degrees of information sharing fromsupervisors [32]. This approach provides subjectmatter experts, acting as subordinates of the projectleadership team, with intrinsic motivation for findinginnovative and effective solutions for specificorganizational needs [30, 33].The inclusion of users (Subject Matter Experts orStakeholders) in the risk management process shouldprovide a better understanding of perceived riskswithin the organization [15]. Such risks and their6122

corresponding mitigation could either hinder orpromote the organizational change managementprocess required for the successful implementationand eventual maintenance of a transformativecomputerized system [11,15]. Applying anassessment and evaluation approach in this contexthelps to frame the change management process interms of specific intended outcomes for saidprocesses. Employing assessment at a fine-grainedlevel allows identification of specific risks;employing evaluation allows us to use theinformation that results from the assessment to makedecisions in terms of implementation andmaintenance. One can then gather information as towhether the intended outcomes have occurred byusing evaluation techniques at the program level(see, for example, [34] for a discussion of standardsfor program evaluation).3. A Qualitative Approach3.1 Case Study Research MethodCase studies facilitate the gathering of informationnecessary for making decisions, as well as focusingon the factors that influenced decisions within eachcase and then comparing such factors in order to testexisting theoretical constructs and relationships [35].Traditionally, for software implementation projects,risk management i

appropriate risk management maturity level to conduct an exhaustive quantitative analysis of risks associated with the project. 1. Introduction This paper describes how a qualitative approach for assessing and evaluating risks in order to inform decision making and risk response benefited a