The Effectiveness of Idea Generation through a  Decision Making Support System

The Effectiveness of Idea Generation through a  Decision Making Support System

(Version 1/24/06)

John Newman

Introduction

 Creativity can be defined as either a personality trait or an achievment (Eysenck, 1994).  As a personality trait, creativity is a dispositional variable charcteristic leading to the production of an act, items, and instances of novelty.  As an achievement, creativity results in a product from a process -- for example, the output of the decision making process.

Creative acievement may depend in part upon the trait of creativity, but it also depends on  much more.  Formal research has found variables that affect creativity as an achievement include: cognitive variables (intelligence, knwledge skills, and others), environmental variables (cultural and socioeconomic factors) and personality variables in addition to creativity as a trait (motivation, confidence, and others) (Eysenck, 1994, p. 209).  Furthermore, researchers have established that creativity can be learned and improved and is not as strongly dependant on individual traits as originally thought.  Creativity may not so much be the result of genius as being in an idea-nurturing work environment (Gatigon et al, 2002; Schmitt and Brown, 2001; Sebell et al., 2001; Leifer et al., 2000; Turban et al., 2005).  This literature suggests that tools that enhance creativity can be made available to decision makers.  Moreover, such availability may enhance the decision making process.           

According to a popular model, decision making involves a series of phases and steps (Dean and Sharfman 1996; Turban,  Aronson and Liang 2005).   Creativity is useful during most of these phases and steps.  For example, creativity can assist in problem design by helping the decision maker to identify relevant alternatives during the design phase of the process (Hilmer and Dennis, 2000; Pissarra and Jesuino, 2005; Satzinger and Garfield, 1999).  In addition, the selection of an appropriate evaluation model is a creative process.  Hence, creativity can facilitate the choice phase of decision making (Aronson and Myers, 2000; Dennis and Wixom, 2001;  Rees and and Koehler, 1999).

There has been evidence that creativity enhances the performance of persons in a variety of tasks, including decision making (Hughes, 2003;  Kurtzberg, 1998;  Massetti, 1996;  Shneiderman, 2000).   However, decision makers may be unaware, and/or lack proficiency in the use, of creativity enhancing tools.  It may be useful, then, to deliver the creativity enhancing support through an information system.  In theory, such delivery should improve the effectiveness of decision making support.

This paper tests the theory.  First, the paper presents a creativity enhanced decision making support system.  Next, there is an empirical analysis of the system concept. Then, the paper discusses the studyís implications for decision making support.

Idea Generation through a Creativity Enhancing Information System

A number of information systems exist to generate knowledge for decision making support.  These systems collectively can be called Decision Making Support Systems (Forgionne, 2002). Usually, the support is offered in a fragmented and incomplete manner with little, if any, delivery of creativity enhancing tools.  Yet, the integration of enhancements, including creativity support, within DSS, theoretically, can enhance the quality and efficiency of the decision-making support, create synergistic effects, and augment decision-making performance and value (Conteh and  Forgionne, 2005; Klein and Methlie, 1995; Potter, Byrd, Mille, and Kochut, 1992;  Power and Karparthi, 1998; Shim,  Warkentin,  Courtney,  Power, Sharda, and Carlsson ,  2002). 

Based on previous research (Forgionne, 2002;  Forgionne,  Clements,  and Newman, 1995), the resulting creativity enhancing decision making support system (CDMSS) will have the conceptual architecture shown in Figure 1.

Figure 1 shows that the CDMSS captures and stores as inputs problem specific knowledge (ideas and concepts) and creativity enhancing tools.  Ideas and concepts may come from conventional wisdom, documents detailing standard operating procedures, case studies, or other sources, while creativity enhancing tools include morphological analysis, metaphors, convergent and divergent thinking mechanisms, brainstorming, calculus, and other methodologies.

The decision maker utilizes computer technology to: (a) organize (chiefy categorize and classify) the problem knowledge, (b) structure ideas and concepts into problem elements and relationships, and (c) simulate conceptual problem solutions.   Results are reported as problem elements (status reports), the problemís conceptual structure (criteria, alternatives, events, and relationships), and/or forecasted outcomes from the conceptual analyses.              

Feedback from the user-controlled processing guides the decision maker through the design stages of the decision making process and  identifies the parties affected by the conceptual analyses (Lipp and Carver, 2000).  This identification helps the decision maker to develop an implementation plan and put the plan into action.  Created problem elements and structures are stored as additional inputs for future or additional  processing (Sheetz and Tegarden, 2000).

In theory, such support should improve decision making performance and add value to the userís decision making.  The improvement can occur through an enhanced process (for example, better problem design) or better outcomes (for example, improved user learning or organizational performance).

Empirical Analysis

The theory suggests the following research question and hypotheses:  

A traditional DSS is used as the baseline to provide a fair test for the CDMSS. 

To answer the research question, an experiment, involving a complex  semi-structured decision situation, was used to collect data and test the hypotheses. The experimental study followed a research plan developed and successfully utilized previously (Forgionne,  Clements,  and Newman, 1995).

Subjects

The subjects were upper level undergraduate students at a regional public university.  These subjects were enrolled in nine different sections of the same course in technology fluency.  They were divided into nine twenty four (24) person teams.   Each team played a management game called AIRLINE where each rival team competed for profits in a simulated market environment.

One team was designated as the control group; four were denoted as DSS groups; and four others were labeled as creatively enhanced decision making support system (CDMSS) groups.  The subjects were randomly assigned to each team within each class; that is, each subject in each of the class sections had an equal chance of being selected for membership in any of the teams.  As a result, each of the three teams contained members from each of the nine classes from which the teams were formed.  Chi-square tests of crosstabulated demographics against team and group data verified (at the alpha = .05 level) that team membership and group composition were both demographically homogeneous.  All nine sections of the class were told that 50% of their course grade was dependant upon their teamís ranking at the conclusion of the game.

Decision Task

Each of the nine teams took over the management of a small airline that transported passengers and cargo along less traveled routes.  All teams had the same starting position and the same opportunities to purchase market information, expand or contract markets served, purchase or lease additional aircraft, and so on.  During the game, each team entered a set of decisions every week for the duration of the game, 15 weeks.   The decisions involved marketing, expansion, personnel and finance.

Decision Aids

All teams were provided student manuals that described the simulation. The control group was not given any computer based system to aid them in the decision making process.  The control group was, however, allowed to complete a decision, receive feedback, and repeat this process.  Figure 2 shows the  ìdecision form.î

The DSS group was furnished a basic decision support system constructed from Microsoft ExcelÆ and Microsoft Project ManagertÆ.  This system allowed the users to perform ìwhat ifî and goal seeking sensitivity analyses outside the game prior to submitting decisions.

The CDMSS group was given the same DSS embellished with a creativity enhancement tool, Axon Idea Processor (AIP) (http://web.singnet.com.sg/~axon2000/).  AIP, which is based on the Prolog computer programming language, serves as an electronic sketchpad for visualizing, generating, and organizing ideas.  AIPís main idea processing tools are: ANALYZER (analyzes sentence structure, word frequency, etc.); CHECKLISTS (captures and stores knowledge and wisdom); CLUSTERS (organizes ideas into trees and branches) and SIMULATOR (makes concepts come alive using simulation techniques).  Figure 3 gives an example screen from the AIP software. 

Data Capture

AIRLINE output provided game data on net profit after taxes (NET PROFIT) and other operating statement statistics.  Each week a questionnaire was administered to each team eliciting Likert-scaled self assessment of proficiency in decision making (PROCESS), the number of ideas generated (IDEAS), and the time in minutes needed to reach a decision (TIME).   Figure 4 shows this decision process form, while Table 1 shows the usable data for the key variables.

Data Capture

AIRLINE output provided game data on net profit after taxes (NET PROFIT) and other operating statement statistics.  Each week a questionnaire was administered to each team eliciting Likert-scaled self assessment of proficiency in decision making (PROCESS), the number of ideas generated (IDEAS), and the time in minutes needed to reach a decision (TIME).   Figure 4 shows this decision process form, while Table 1 shows the usable data for the key variables.

Table 1 Data for Key Airline Variables

Outcome and Process Measures

Decision value can be  measured in terms of the outcome and process of decision making (Forgionne, 1999).  Outcome occurs through the process of decision making, and this process can be characterized as intelligence, followed by design and choice, and concluded with implementation.    

In this experiment, outcome is measured by the AIRLINEís net profit after taxes (NET PROFIT).  This measure is metric.

The AIP creativity enhancement tool provided the users with visual representation of data and information, and it encouraged working at a higher level of abstraction ñ dealing with ideas and concepts rather than words.  This perspective enabled the users to more efficiently and effectively use the decision making process. The users discerned the nature of the problem and the opportunities presented (intelligence), generated problem elements and structures (design), qualitatively evaluated alternatives and selected a management strategy (choice), and gained confidence in, and executed, the decision through entry on a decision form (implementation).

Process, then, can be measured by the Likert-scaled self assessment of decision making proficiency (PROCESS).    This measure is nonmetric.  Additional process measures are the number of ideas generated (IDEAS) and the time in minutes needed to reach a decision (TIME).  These additional measures are metric.

Data Analysis

For outcome, an Analysis of Variance (ANOVA) was used to test the differences in net profit after taxes between the control, DSS, and CDMSS groups.  The SPSS statistical package was used to perform the analysis, and the results are reported in Table 2. 

The results from Table 2 indicate that: (a)  there is a significant difference in net profit between the no decision aid, DSS, and creativity enhanced DSS groups, and (b) the creativity enhanced DSS group had significantly higher profits than any of the other groups.  These tests, then, support the conclusion that the CDMSS will result in an improvement in net profit when compared to a traditional decision support system (DSS).

Several process tests were conducted.  For the metric measures (IDEAS and TIME),  ANOVA was used to test for differences in these variables  between the control, DSS, and CDMSS groups.  The SPSS statistical package was used to perform the analyses, and the results are reported in Table 3. 

The results from Table 3 indicate that: (a) there is a significant difference in the number of ideas generated between the no decision aid, DSS, and creativity enhanced DSS groups, and (b) the creativity enhanced DSS group had significantly more ideas than any of the other groups.  These tests, then, support the conclusion that the CDMSS will result in an improvement in the number of ideas generated when compared to a traditional decision support system (DSS).

Table 3ís results are somewhat mixed for the TIME variable.  These results indicate that:  (a)  there is a significant difference in time needed to make a decision between the no decision aid, DSS, and creativity enhanced DSS groups, and (b) the creativity enhanced DSS group had different times needed to make decisions than the no decision aid, but not either of the other DSS, groups.  These tests, then, support the conclusion that the CDMSS will result in a the same time needed for decision making when compared to a traditional decision support system (DSS).

For the nonmetric measure (PROCESS), a Kruskal-Wallis statistic was used to to test the differences in self-assessed process proficiency between the control, DSS, and CDMSS groups.  The SPSS statistical package was used to perform the analysis, and the results are reported in Table 4.

The results from Table 4 indicate that there is no significant difference in process ratings between the no decision aid, DSS, and creativity enhanced DSS groups.  This test, then, does not support the conclusion that the CDMSS will result in better process ratings when compared to a traditional decision support system (DSS).

Although not central to the research question, an additional statistical analysis was performed on the relationship between the outcome and process measures.  Such testing helps determine the process source of the outcome improvement.  This linkage can facilitate future system design and strategic planning. 

In this case, the previous statistical analysis revealed significant differences between the groups regarding net profit, the number of ideas generated, and the time needed to make a decision.  All of these measures are metric.  To determine if there is a significant relationship between outcome (NET PROFIT) and process (IDEAS and TIME), a regression analysis was performed.  The results, from SPSS, are summarized in Table 5.

The results from Table 5 indicate that: (a) IDEAS and TIME each have significant effects on NET PROFIT, and (b) collectively, the variation in IDEAS and TIME account for about 67% of the variation in NET PROFIT.  These findings support the conclusion that the process improvements from the CDMSS result in enhanced decision outcome.

The regression analysis also identifies the specific improvements in outcome that can be expected from the enhanced decision making.  Table 5, for example, shows that each additional idea generates approximately $4660 in net profit, while each additional minute of time savings generates an extra $253 in net profit.  These values are point estimates, but it is also possible to determine interval estimates of the gains for specified levels of confidence.

In summary, the statistical tests indicate that: (a) CDMSS users generated more ideas in the same amount of time as DSS users, (b) CDMSS users had more net profit than DSS users, and (c) the variation in net profit was largely accounted for by the variation in the number of ideas and the time needed to make decisions.  These findings suggest that the null hypothesis should be rejected.  Consequently, there is statistical support for the alternative hypothesis that the CDMSS improves decision making when compared to a traditional DSS.

Conclusions and Implications

The broad conclusion from the conducted experimental study is that the CDMSS, relative to the DSS and no decision aid groups, helps improve the process of, and outcome from, decision making. Moreover, the improvement occurs because decision maker can generate more ideas through the CDMSS than with any of the other tested systems. Ideas faciltate intelligence, design, choice, and implementation in decision making. The results imply that the creativity enhanced concept is superior to traditional decision support system approaches in guiding the decision maker toward an effective policy or strategy. 

This conclusion supports the findings from previous studies.  The reported AIRLINE study involved a different decision problem and different creativity enhancing tools than the previous studies, and each of the previous studies involved disparate decision situations and disparate creativity enhancing tools.  The commonality of findings, then, suggests that the theory has validity across various decision situations.

Unlike the previous studies, the AIRLINE study established an explicit and precise relationship between the process improvement and outcome enhancement.  In particular, the AIRLINEís regression equation can be used to establish the dollar value of process improvements, thereby establishing an imputed economic value for CDMSS delivered process support.  The regression equation can also be used to evaluate the relative value of competing process improvements.  These explicit and precise relationships offer an objective basis for system design and strategic planning. 

There are some limits on these conclusions.  The number of users in the AIRLINE study is small, and the user group itself is limited in scope.  This limitation can be alleviated by doing further studies with larger sampled, more diverse users.  In addition, the AIRLINE study had a limited number of outcome and process measures.  Further studies should include additional criteria.  Moreover, the multiple criteria could conflict.  Hence, it would be desirable to consolidate the multiple criteria into an overall measure of decision value.

Despite these limitations, this study does indicate that the creativity-enhanced decision making support system concept has considerable promise and deserves further study.

REFERENCES

Aronson, JE, Myers, RM,(2000, June), ìTime Pressure Impacts on Electronic Brainstorming,î Informatica, Vol 24, No 2.

Avon Idea Processor http://web.singnet.com.sg/~axon2000/

Conteh NY, Forgionne G (2005a) The Design of Dynamic Just-in-time Agent Technology through Simulation Modeling, Frontiers in Artificial Intelligence and Applications Knowledge-Based Intelligent Engineering Systems. IOS Press B.V., Amsterdam, Netherlands.

Dean JW, Sharfman MP (1996) Does Decision Process Matter? A Study of Decision- making Effectiveness Academy of Management Journal, Vol. 39:2 368-396

Dennis, AR, Wixom, BH (2001, June) ìUnderstanding Fit and Appropriation Effects,î MIS Quarterly, Vol 25, No 2.

Forgionne G (2002) Decision-making Support Systems Architecture. In: Forgionne GA, Gupta JND, Mora M (eds.) Decision-making Systems: Achievements and Challenges for the New Decade. Idea Group Publishing, Hershey PA

Forgionne G (1999) An AHP model of DSS effectiveness. European Journal of Information Systems, Palgrave Macmillan Ltd Houndmills, Basingstoke, Hampshire, RG21 6XS, England.

Forgionne, G. A., Clements, J., and Newman, J. (1995) Qualitative thinking support systems (QTSS), Journal of Decision Systems Vol. 4, no. 2, pages 103 - 137.

Gatignon H, Tushman ML, Smith W, Anderson P (2002) A Structural Approach to Assesing Innovation: Construct Development of Innovation Locus, Type, and Charecteristics. Management Science, Vol. 48, No. 9.

Hughes, G. D. (2003). Add creativity to your decision processes, Journal for Quality and Participation, The Summer, pp. 1-12.

Hilmer, KM and Dennis AR (2000), ìStimulating Thinking in Group Decision Making,î Proceedings of the 33d Annual Hawaii International Conference on System Sciences (HICSS-33), IEEE Computer Society Press

Klein MR, Methlie LB (1995) Knowledge-based Decision Support Systems ñ with applications in Business. New York: John Wiley & Sons. 527 p. ISBN 0-471-95295-8.

Kurtzberg, T. R. (1998). Creative Thinking, Cognitive Aptitude, and Integrative Joint Gain: A Study of Negotiator Creativity, Creativity Research Journal 1998, Vol. 11, No. 4, pp. 283ó293.

Leifer R (2000) Radical Innovation: How Mature Companies Can Outsmanrt Upstarts. Boston, Harvard Business School Press.

Lipp, A and Carver, CY (2000, January) ìUsing Web Groupware and Cognitive Mapping in a CIS,î Proceedings of the 33d Annual Hawaii International Conference on System Sciences (HICSS-33), IEEE Computer Society Press

Massetti, B. (1996). An Empirical Examination Of The Value Of Creativity Support Systems On Idea Generation, MIS Quarterly,Vol. 20, No. 1, pp. 0276-7783.

Pissarra, J and Jesuino, JC (2005), ìIdea Generation through Computer-Mediated Communicationî, Journal of Managerial Psychology, Vol 20, No æ, pp 275-291.

Potter W, Byrd T, Mille J, Kochut K (1992) Extending Decision Support Systems: The Integration of Data, Knowledge, and Model Management. Annals of Operations Research, Vol. 38.501-527.

Power DJ, Karparthi S (1998) The Changing Technological Context of Decision Support Systems In: Berkeley D, Widmeyer G, Brezillion P, Rajkovic V (Eds.) Context-Sensitive Decision Support Systems. London: Chapman and Hall.

Rees, J, and Koehler, G (1999) ìBrainstorming, Negotiating, and Learning in Group Decision Support Systems,î Proceedings of the 32d Annual Hawaii International Conference on System Sciences (HICSS-32), IEEE Computer Society Press

Satzinger, JW and Garfield, MJ (1999, spring), ìThe Creative Process: The Effects of Group memory on Individual Idea Generation,î Journal of Management Information Systems, Vol 15, No 4.

Schmitt, BH, Brown, L (2001) Build Your Own Garage: Blueprints and Tools to Unleash Your Companyís Creativity., New York: Free Press.

Sebell MH, Yocum J, Prahalad CK (2001) Ban the Humorous Bazooka. Chicago: Dearborn Trade Publishing.

Shneiderman, B. (2000). Creating creativity: user interfaces for supporting innovation, ACM Transactions on Computer-Human Interaction, Volume 7, Issue 1, Pages: 114 ñ 138.

Sheetz, SD and Tegarden, LF (2000, August), ìA WWW Approach to Mapping to Support Case-Based Learning,î Proceedings of the American Conference for the Association for Information Systems, Milwaukee

Shim JP, Warkentin M, Courtney JF, Power DJ, Sharda R, Carlsson C (2002). Past, Present, and Future of decision support technology. Decision Support Systems. Elsevier Science B.V. 33(2), 111-126.

Turban E, Aronson JE, Liang T. (2005) Decision Support Systems and Intelligent Systems, Prentice Hall, Upper Saddle River, NJ.