Developing an ideation method to be used in cross-functional inter-organizational teams by means of action design research

Abstract

Using ideation methods in an optimal way has a great potential to increase the number of ideas which a team can contribute during the conceptual phase of product development in industry. Previous studies on ideation methods have been mainly statistical studies in a laboratory setting. In the present study, however, the aim was to develop tailor-made ideation methods in a specific context, among actors on a deregulated railway market, through close interaction between researchers and engineers in a real-world context. Considering previous relevant research on ideation methods, associative memory models and the findings from tests of established ideation methods performed in the same group, a number of preliminary design principles were formulated and implemented in an ideation method that combines individual phases of rotational and gallery viewing with phases of verbal group interaction. The method was tested and refined in a cross-functional inter-organizational group comprising participants from different actors in the railway sector. Besides its provision of qualitative and quantitative test results, the present study has considered the opinions of the participants in detail, which can give important insights into the factors determining whether such a method will be implemented in industry. The participants found the method to be more useful and to generate more ideas that could be used in practice compared to the established ideation methods which they had tried. The learning derived from the specific case was formalized into a number of design principles for ideation methods to be used in cross-functional inter-organizational groups.

1 Introduction

The creation of new ideas is of the utmost importance to engineers during the conceptual phase of product development. A great number of ideation methods, proposed by practitioners as well as researchers, are available to assist the engineer in the process of inventing and creating new products. However, little empirical data exist to assist the engineer in using these methods or selecting the optimal method. The most famous method, brainstorming (Osborn 1957), has been extensively studied in the social psychology literature, but although much can be learnt from these studies, it is not possible to transfer the results directly to the area of engineering design for at least three different reasons. First, these studies let the participants use words to express ideas, whereas designers use sketches to communicate. Second, the typical psychology study involves topics that do not require domain-specific knowledge to solve problems, whereas engineers engaged in product development deal with problems that do require such knowledge. Third, whereas these studies have typically involved students, engineers engaged in product development typically work in cross-functional teams involving participants with complementary skills and knowledge. These differences imply that ideation methods have to be studied and developed in the context of engineering design to extract and provide guidance as to how engineers can make the best use of these methods. Gibson (2015) found that groups using a formal ideation method outperformed groups using the informal best practice in the engineering organization at which the study took place in terms of quantity as well as variety, novelty and quality. Linsey et al. (2011) examined how two key factors influenced the outcome when mechanical engineering students worked with a design problem. These key factors were the way in which a group’s ideas are displayed and the form of communication between group members. They found that over a 40-min ideation session, 50% more ideas were generated with the most favourable combination of key factors than with the least favourable combination. Linsey and Becker (2010) made a complementary study letting nominal groups (groups of non-interactive individuals who ideate on their own and whose non-redundant ideas are pooled and compared to the outcome of an interactive group) ideate on the same problem given the same amount of time. They found that real teams in several of the ideation conditions generated a larger number of ideas than equivalent nominal groups (Linsey and Becker 2010), which is a very interesting finding, since the extensive research on brainstorming in the field of psychology unambiguously shows that verbally interactive group work decreases the amount of ideas compared to the amount obtained through nominal groups (e.g. Mullen et al. 1991; Byron 2012). Using ideation methods in an optimal way clearly has a great potential to increase the number of ideas which a team can contribute during concept generation. This is especially important since past research on brainstorming in the psychology field has found that the quantity and quality of ideas are highly correlated (Dugosh et al. 2000; Paulus et al. 2015), and some supporting evidence for this finding has been found in the area of engineering design (Yang 2009; Linsey et al. 2011).

Previous studies on ideation methods have been mainly statistical studies exploring how a method or some parameter of a method influences the number of ideas generated, and in some cases the quality or creativity of ideas. A few studies of professionals using ideation methods in a natural context exist (e.g. Sutton and Hargadon 1996; Jackson and Poole 2003). The findings of these descriptive studies are available for the engineer to interpret how best to apply them in a specific context. However, no studies have, to the best of the authors’ knowledge, tried to generate prescriptive knowledge regarding the design of ideation methods through the interaction between researchers and practitioners (in the present case engineers) in a real-world context. In the present study, the aim has been to develop and implement tailor-made ideation methods in a specific context, among actors on a deregulated railway market. To this end, action design research (ADR) (Sein et al. 2011) was applied within the field of engineering design. The learning derived from the specific case was formalized into a number of design principles for ideation methods which were generalized for the class of field problems of ideation in cross-functional inter-organizational groups. The findings add to the body of knowledge that can guide engineers as to how ideation methods can be used most effectively in different contexts. The following research questions were posed to meet the aim of the study.

• What are the goals of and the requirements for such an ideation method?

• What did the participants think about the developed method?

• How did the participants behave during the application of the different versions of the method?

• How do the individual ideation and group review steps differ during the application of the methods?

• How did the application of the method affect the ideation outcome compared to the established ideation methods that had been tested by the same group?

• What are the general design principles for such an ideation method?

2 Background

2.1 Background of the study

The study was performed in the context of the Swedish railway sector, with a focus on turnout development. A turnout (see Fig. 1) guides trains from one track to another and is a vital part of the railway infrastructure. A failure of a turnout, especially in a critical location, can cause significant delays and societal costs. In 2012, turnout-related failures were among the top ten causes of hours of disruption in Sweden (Trafikverket 2013). Turnouts cause at least 13% of the maintenance costs associated with the Swedish railway (Nissen 2009). Therefore, the OptiKrea project, run by Luleå Railway Research Center at Luleå University of Technology, was initiated with the intention of promoting the technical development of turnouts, especially from the point of view of maintenance and life cycle cost. The deregulation of the Swedish railway has resulted in different market actors managing, supplying, maintaining, utilizing and researching the railway. The idea behind the OptiKrea project was that, by integrating the different perspectives on and knowledge about the turnout that the different actors possess, better solutions would be found. A parallel goal of the project was to develop working methods, including ideation methods, which would facilitate innovation, would be tailor-made for the railway sector and could be used in the future. At the heart of the project is the so-called “creative team”, which consists of representatives from each collaborating actor. The team is cross-functional in the sense that it represents the different functions that are important when developing a turnout, i.e. research, design, manufacturing, management, maintenance, and disposal.

Fig. 1

A model of a turnout

2.2 Theoretical framework

In associative memory models (AMMs) used in cognitive psychology (e.g. Collins and Loftus 1975; Anderson 1983), information in the long-term memory is treated as a network of interconnected nodes. The inter-relatedness of two distinct nodes is determined by their associative strength, i.e. closely related pieces of information are connected to each other more strongly than less related pieces of information. A stimulus idea activates (and hence makes more accessible) other nodes that are connected to it, and primarily those with a strong associative connection.

New ideas cannot be directly retrieved from memory, but stored knowledge must be used to generate ideas, and therefore idea generation necessarily involves retrieval processes (Nijstad et al. 2002) and these can be described by AMMs. Nijstad et al. (2002) suggest that idea generation is a two-stage process in which a cue activates a set of localized strongly interconnected and semantically related features. The idea production stage follows, where these sets are used to generate (new) ideas by combining knowledge, forming new associations or applying knowledge to a new domain. These ideas are added to the search cue to activate more sets in memory, leading to additional idea generation (Nijstad et al. 2002). Since semantically related images have strong mutual ties, successively activated images will often have strong associative connections which lead to a “train of thought” exploring connected ideas (Nijstad et al. 2002).

AMMs offer a framework for understanding the benefits of ideation involving participants with different functional backgrounds and knowledge, and are used to explain and justify theoretically the method developed in the present paper. The way information is retrieved from memory implies that a group of participants with different functional and organizational backgrounds within a relevant field would have an advantage compared to a functionally homogeneous group. A functionally homogeneous group may possess similar information and connections in the part of their associative memory network that applies to the ideation topic at hand. Then it could be expected that the ideas which they would come up with as a reaction to the ideation topic would be similar, as this topic would activate similar nodes of information. To achieve a performance gain through idea exchange, an ideating subject must receive ideas belonging to categories which would have a low probability of being surveyed if the subject were to work alone (Perttula et al. 2006), and therefore exchanging ideas with each other to stimulate further activation would not be very helpful in a functionally homogeneous group, since these ideas would primarily continue to activate ideas similar to those which the individual would generate on their own. Brown and Paulus (2002) represented a brainstormer’s knowledge of a given problem as a matrix of category transition probabilities. Simulations of the so-called associative memory matrix model showed that, if a brainstormer is presented with ideas belonging to categories of low accessibility, the number of ideas belonging to that category increases and also the total number of ideas overall, i.e. this makes the individual a more productive brainstormer (Brown and Paulus 2002). In a functionally heterogeneous group, the associative memory networks of the participants could be expected to have different configurations. Hence, the ideation topic would activate different information depending on the participant and lead to the generation of ideas belonging to different categories. If a participant then was to be exposed to ideas from other group members, these ideas might activate parts of their network that would not initially be activated by exposure to the ideation topic, eventually leading to a group set of both more ideas and ideas belonging to more different categories than would be expected from a group with the same number of people with homogeneous backgrounds with respect to the ideation topic at hand. In support of this, Brown and Paulus (2002) found through simulations that interactive brainwriting (i.e. group members ideating in silence and exchanging ideas in written form) is not universally superior to individual brainstorming, but most effective when the members of the group possess different knowledge of the ideation topic.

2.3 Related work on ideation

Linsey et al. (2011) compared rotational viewing (as applied, for example, in Method 635) to gallery viewing (as applied in the gallery method) in an experiment involving mechanical engineering students working on the design of a device to shell peanuts. Whereas the participants are exposed to all the generated ideas simultaneously during gallery viewing, they are only exposed to one subset of ideas at a time during rotational viewing. Linsey et al. (2011) found that rotational viewing generated the highest number of ideas if both sketches and texts were used for describing ideas. Tests of established ideation methods in the same group as that participating in the present study (Petersson et al. 2017) showed that Method 635 generated most ideas in the shortest time compared to the gallery method and the SIL method. Gibson (2015) compared rotational viewing in a modified version of C-sketch to an unregulated technique that had some similarities with brainstorming in an experiment involving practitioners working on a snow removal problem. Gibson (2015) found that the modified C-sketch method outperformed the unregulated technique in terms of quantity, variety, novelty and quality. Paulus and Yang (2000) have shown that a failure to improve production in interactive groups compared to that in nominal groups through cognitive stimulation may be caused by participants simply not attending to stimulus ideas. Both rotational viewing and gallery viewing encourage the participants to consider the ideas from the other group members. The processing of stimulus ideas is, however, performed within the limited capacity of short-term memory, and therefore it is reasonable to assume that a person can only attend to a limited set of complex ideas simultaneously (Perttula et al. 2006). Consequently, rotational viewing may be more beneficial than gallery viewing because it helps the participant to consider only a limited set of ideas at a time. On the other hand, if the attention paid to stimulus ideas becomes excessive, the performance decreases, as has been shown experimentally by Paulus and Yang (2000) and through simulations by Brown and Paulus (2002).

Linsey et al. (2011) found that gallery viewing produced more global product solutions (corresponding to concepts in the present study) and more high-quality product solutions than rotational viewing, while the average diversity of the product solutions was greater for the rotational viewing condition. One explanation may be that participants in the gallery viewing condition focus on a limited set of ideas from the entire pool during most of the ideation session and develop similar solutions with increased quality, while during rotational viewing they are forced to consider a new set of ideas in each successive round, which might enhance the variety by activating other parts of their memory network.

Perttula et al. (2006) found that individuals generated more ideas when they discussed ideas with other individuals. Although there were some doubts about why this effect arose, there is other research (Pelled et al. 1999; Seidel and Fixson 2013) showing that teams debating ideas come up with more novel innovations, indicating a more thorough exploration of the solution space. One explanation for this might be that, while discussing, new aspects of ideas appear which participants associate with parts of their memory network that have not yet been activated.

2.4 Action design research

The action design research (ADR) method was proposed by Sein et al. (2011) in an effort to blend design research with action research. Design science is the study of artefacts in their context (Hevner et al. 2004), whereas action research, taken generally, is intervention in a social situation to both improve this situation and learn from it (Susman and Evered 1978). The purpose of ADR is to generate prescriptive design knowledge through learning from the intervention of building and evaluating an IT artefact in an organizational setting to address a problem (Sein et al. 2011). According to Rogerson and Scott (2014), it is uncommon that a design-based intervention will turn out as planned at the first attempt, especially when addressing a social situation with practitioners involved. Therefore, ADR is problem-driven and aims to build design principles based on iterative cycles in the same context (Wieringa and Morali 2012).

ADR was initially proposed within the field of information systems and has been applied in several different contexts, e.g. addressing the learning needs of the deaf community (Golding and Tennant 2013), managing projects funded by a third party (Gröger and Schumann 2014) and greenhouse gas emission reporting in the meat industry (Hilpert et al. 2013). An IT artefact is a specific bundle of hardware and software that is assembled to fulfil information needs. Therefore, the present authors considered it reasonable to assume that the development of non-IT-based tools for information handling could also profit from ADR. As the current study aimed to develop an ideation method suitable in a certain context through interaction between researchers and engineers, it was deemed that ADR would be useful in guiding the study, as its strict and explicit principles and iterative cycles allow the researcher to deliver a practical outcome for the involved organization whilst simultaneously meeting academic standards.

By developing a method in close cooperation with practitioners, we expected that the user satisfaction would be higher compared to that achieved through the established methods tried by the group. It was further expected that the method would be easy to apply in new situations with participants from different organizations and different functional backgrounds, although it was not within the scope of the present study to test this expectation, but rather to test it in a future field test. By implementing in the method being developed the findings of other researchers concerning the optimization of idea generation, we expected that the quantity of ideas generated would be greater compared to that generated using established ideation methods (Petersson et al. 2017).

3 Method

3.1 Procedure

ADR consists of four stages (Sein et al. 2011): (1) problem formulation; (2) building, intervention (i.e. test of the artefact in the target environment), and evaluation (BIE); (3) reflection and learning; and (4) formalization of learning. Each stage is based on certain principles and involves the execution of certain tasks. An overview of the method is shown in Fig. 2. A full explanation of the principles and tasks and an account of how ADR was applied in the development of the novel ideation method, as well as reflections on the feasibility of using ADR for developing methods in the area of engineering design, are to be found in Petersson and Lundberg (2016). How each stage was carried out in the present study is briefly summarized in this section.

Fig. 2

Adapted from Sein et al. (2011)

The stages of ADR and their adherent principles and tasks.

3.1.1 Problem formulation

The trigger for the present study was the insight that, as the different functions involved during the life cycle of railway products were spread out over different actors as a result of deregulation, special efforts were needed to achieve a holistic picture of a product when developing it. Previous research has shown examples of both the customer’s (Hannola et al. 2009) and the subcontractor’s (Liker et al. 1998) importance for the introduction of improvements. By involving representatives from several actors when ideating, more viewpoints on the product to be developed would be shared and thus a better product would eventually be designed. The initial research question formulated was therefore how ideation should be performed to capture the benefits of collaboration during product and process development in the given context, and specifically, how an ideation method should be designed. The participants in the project were the infrastructure manager of the Swedish railway (the Swedish Transport Administration (STA)), a turnout manufacturer, a maintenance contractor, and researchers from Luleå University of Technology. Representatives from each participating organization formed the so-called “creative team”, in which issues concerning turnouts were proposed and addressed by means of the strategies proposed by the researchers managing the project. In Sweden, the infrastructure manager typically runs its development and research projects in cooperation with or through research institutes and universities, and therefore academic researchers were relevant participants in the group. After each test, the participants in the creative team gave their views on the method. The researchers analysed the data from the test and proposed changes to the method, and the participants of the creative team commented on the changes before the next test.

To the best of the authors’ knowledge, no ideation method has been developed specifically for groups with members possessing different functional knowledge and representing different organizations. According to Straus et al. (2011), there is no knowledge of what happens in inter-organizational groups to which different actors bring different cultures and agendas. Therefore, it was concluded that design principles for ideation methods for cross-functional inter-organizational groups would be interesting for other fields as well, as deregulation and outsourcing have become increasingly common. As a result, the research task was framed as addressing the following class of field problems: ideation in cross-functional inter-organizational groups. A literature review was performed on the subject of ideation methods and an exploratory case study was performed of a cross-functional inter-organizational group (the same group as that participating in the current study) testing three different established ideation methods (Petersson et al. 2017), to both validate previous research results and find empirical evidence that could be used to make an initial design of the ideation method to be developed.

3.1.2 Building, intervention and evaluation

The BIE cycles in the development of the method were divided into an alpha phase comprising iterations in the creative team (described in the present paper) and a beta phase involving future field tests. The first step of the alpha phase involved identifying the basic requirements for the method to be developed. These basic requirements, together with the results and conclusions obtained from testing established ideation methods (Petersson et al. 2017) and from the previous research literature, served as a basis for working out a proposal for a novel method. The suggested method was discussed in the team and a few changes were introduced before the test of the first version of the method, referred to as OptiKrea method version 1 (OKMv1) throughout this paper. The method was tested, evaluated and changed in two iteration steps. After two iterations, the participants were satisfied with what became the second version of the method (OKMv2) and did not want to make further modifications.

3.1.3 Reflection and learning

The ideation method developed in the present study was adapted as the study progressed, according to the evaluations and analysis that took place, to reflect the increasing understanding of how the method performed in reality. The increased understanding of the method characteristics clarified what broader class of problems the method may be applied to.

3.1.4 Formalization of learning

Sein et al. (2011) suggest viewing the outcome as a solution that addresses a problem, and performing a generalization by making a conceptual move on three levels: (1) generalization of the problem instance, (2) generalization of the solution instance, and (3) derivation of design principles, i.e. recommendations on how solutions addressing the same class of problems should be designed, on the basis of the design research outcome. The third level requires a reconceptualization of the learning from the specific solution instance into design principles for a class of solutions as identified in level 2. In the present study, the learning from the specific instance was translated into design principles for ideation methods to be used in cross-functional inter-organizational teams.

3.2 The creative team

The creative team was a group of six participants comprising four railway professionals and two academics. The professionals each had 20–30 years of experience from the railway sector. Two worked with turnout-related issues at STA, one at an international company manufacturing turnouts and one at a contractor performing maintenance. The academics had experience from railway-related projects performed in close cooperation with STA and industry. The group had not worked together before the OptiKrea project, some of the group members, however, knew each other from before. Details of the participants are provided in Table 1. Before the OptiKrea project, only participant E had experience of ideation methods.

Table 1 Details of participants

3.3 Procedure

The goals of and the requirements for the method to be developed were emerged continuously during the OptiKrea project. Comments relevant to the goals and requirements were noted down at meetings and workshops. These data were continuously considered by the researchers in relation to the findings that were made, and the goals and requirements regarding the method were shaped during this process. The final goals and requirements that emerged as a result were presented to and approved by the participants of the OptiKrea Group. The goals and requirements were then used to direct the development of the method.

The two versions of the developed method were tested by the group with an interval of 4 weeks between the tests. The established methods in the exploratory case study were tested over a 3-month period that commenced about 1 year before the tests of the developed method were initiated. Between the two tests of the developed method, the participants worked with their normal tasks and did not use any ideation methods.

A different ideation topic was chosen for each ideation method to avoid learning effects. Our aim was to find equivalent problems that were based on actual needs, were domain-specific and open-ended, and had a large amount of possible solutions. First, the participants were asked to come up with topics which they wanted to work on within the OptiKrea project and which based on their experience, they thought were equivalent. Thereafter, the authors of the paper made a review of the suggestions to pick out the topics that best fulfilled the equivalence criteria. The participants were informed of this selection and agreed upon the ideation topics about 1 week before each ideation session. The chosen ideation topics are presented in Table 2. The issues on which the ideation was performed required at least a basic technical understanding in general and railway-specific knowledge in particular.

Table 2 Ideation topics used for each ideation method test

At the start of each session one of the participants presented the ideation topic, followed by a common group analysis of the topic to identify causes of and known solutions to the problem involved (30–60 min long), before the test of one of the ideation methods started. Before a test started, the participants were instructed to try to create many ideas and not to criticize each other negatively during the application of the ideation method. They were instructed that thorough scrutiny (both negative and positive) would be possible in a subsequent step of the project. The execution of the established methods which was tested in the exploratory case study (Petersson et al. 2017) and to which the developed methods will be compared in terms of outcome is presented in Table 3. A detailed description of the tests of the established ideation methods is to be found in Petersson et al. (2017). The first author presented the instructions for the method concerned to the group by means of a projector, ran a timer to keep track of the time, announced when the group should move on to the next stage of the method and answered questions about the instructions during the session. In this capacity, the first author is referred to as the “supervisor” throughout the paper.

Table 3 The execution of the established ideation methods that was tested (Petersson et al. 2017)

3.4 The views of the participants

Both after the test of version 1 of the developed method and after the test of version 2, the participants answered a questionnaire and a short group interview regarding the participants’ experience of the applied method took place to capture the participants’ immediate reactions to the ideation method that they had tried. The questionnaire presented different statements about each method, and the participants assessed their level of agreement with these statements, answering on a continuous scale from “Do not agree at all” to “Agree completely”, by making a mark on a line which was about 100 millimetres in length and where 0 represented “Do not agree at all”. The position of the mark was measured in millimetres from 0 with a ruler and divided by the total length of the line. The average value for each statement was calculated, and despite the small number of participants, the average values for the subgroups “academics” and “railway professionals” to see if there were any differences in their views. The group interviews were audio-recorded and the transcribed recordings, along with the questionnaires, served as the basis for analysing the participants’ views on the methods. A linear regression was performed between each pair of statements in the questionnaires to find out if any correlation existed. All the five methods tested by the creative team during the project were included, resulting in 30 data points.

3.5 Number of concepts and ideas

Each method was divided into individual ideation phases (IPs) and group review phases (GPs), and an analysis was made of how the number of non-redundant concepts and ideas varied over the different phases and between the methods. Further, the number of ideas reused in a later phase of the versions of the developed method was counted.

Each set of sheets of paper containing the suggestions from each ideation session and each recording from the group phases were examined carefully. As a starting point, concepts were defined as sketches and text that clearly belonged together and were marked off from other sketches and text on the sheets of paper. Ideas were defined as the units which each concept could be systematically decomposed into, and which could be expressed as a key phrase consisting of a verb phrase containing a maximum of one verb, or a noun phrase. All the ideas from an ideation session were organized systematically and the key phrases that implied physically doing the same thing were defined as being the same idea, although different words were used to describe that idea. When all the ideas had been organized and compared, the total number of ideas from each phase could be counted, as well as the number of ideas contained in each concept. Some concepts contained variants that were not compatible. If half or more of the ideas were incompatible, the concept was split and counted as two different concepts that shared some ideas. Similarly, if more than half of the ideas of each of two concepts were shared by those concepts, the two concepts were counted as one concept containing variants. A summary of the rules for counting concepts is to be found in Table 4, while the corresponding summary for ideas can be found in Table 5.

Table 4 Summary of the rules for counting concepts

Table 5 Summary of the rules for counting ideas.

During the test of the established ideation methods (Petersson et al. 2017), the quality in terms of viability and ability was assessed by letting the group members act as expert judges and individually rate the concepts. However, the rating by each participant of a particular concept varied to such an extent that it was difficult to make any conclusions regarding the quality of the ideas. We therefore decided to focus on quantity when assessing the ideas during the present study. We believe this is acceptable since earlier research has shown that quantity and quality are correlated (Dugosh et al. 2000; Yang 2009; Linsey et al. 2011; Paulus et al. 2015).

3.6 Behaviour of the participants

Observations made during the ideation sessions, together with the transcribed audio recordings of the sessions, served as a basis for analysing the behaviour of the participants. A variant of the coding system developed by Jackson and Poole (2003) was used to analyse the content of the ideation workshops. The number of words used in the different activities was counted. The main activities specified by Jackson and Poole (2003) were used: idea statement, elaboration, criticism, direction, and going off at a tangent. A description of each activity is to be found in Table 6.

Table 6 Activities used to code the ideation sessions, drawing upon Jackson and Poole (2003)

The distribution of the words spoken by the participants during the ideation session was analysed. The distribution of the speaking frequency, i.e. how many times each participant spoke during a session, was also calculated. During this analysis, agreeing expressions consisting of a maximum of three words that were not an answer to a direct question were neglected (e.g. yes, h’m). This was partly because such expressions did not add anything to the discussion and partly because they were difficult to hear on the recordings (especially when uttered by those participants sitting furthest away from the microphone) and counting would therefore not give a fair picture of the speaking distribution. However, when a speaker was interrupted by such a comment and then had to “speak up” again to continue, this was counted as speaking twice, since speaking up again after being interrupted takes an effort and is important in understanding the speaking distribution. The same rule applied when a speaker said something that made the others laugh and then had to start speaking again. It was not possible to interpret all the verbal communication of the recordings of the workshops, and the uninterpretable part was not included in the analysis. However, this part is so small (estimated to be roughly 1% of the total number of words) that it would only have had minor effects on the results of the analysis and would not have changed the overall conclusions.

If the speaking distribution had been equal between the participants, each participant (in the group of six participants) would have spoken an ideal fraction corresponding to one-sixth of the number of words remaining after subtracting the number of words uttered by the supervisor from the total number of words spoken. The ideal fraction was compared to the actual fraction and the average of the absolute values for how much the participants deviated from the ideal fraction was calculated to be able to compare the “equality of speech” between the methods. A corresponding calculation was performed for the speaking frequency.

An overview of the areas of focus during the evaluation, the sources of evidence and the methods applied to analyse the data are presented in Table 7.

Table 7 Parameters that were analysed during the evaluation of the tests of the ideation methods

4 Results and discussion

Section 4.1 presents how the initial design of the method was reached and describes what changes were made to the method after the first test in the creative team. The changes were introduced to meet needs that were identified during the evaluation of the initial design. The remainder of Sect. 4 is dedicated to a detailed presentation and discussion of the results from the evaluation of the tests of the different versions of the developed method and a comparison of these results with those from the test of the established methods which the developed method is based upon.

4.1 Design of the method

4.1.1 Requirements and demands set for the method

Two overall goals emerged for the method to be developed: (a) the method should produce a great number of ideas with breadth and potential, which was broken down into the requirements “must generate a great number of ideas” and “must generate ideas with breadth and potential” and (b) the method should be implemented by STA after the project finishes. The prerequisites for an implementation were found to be that the intended users must feel that the method is easy to initiate (a low usage barrier) and that the inclusion of different participants (from different organizations) on different ideation occasions must be easy. It emerged that, to meet these prerequisites, the method must fulfil the requirements “must be easy to use and understand” and “must be attractive to the users”. Despite previous findings that facilitators (e.g. Kramer et al. 2001) and training (Parnes and Meadow 1959) can enhance the productivity of ideation, both of these alternatives were ruled out, because they would raise the usage barrier too much. Partly for the same reason, special equipment or software was ruled out. Another reason for excluding software was that organizations typically (e.g. in the present case) do not allow external software to be downloaded onto their employees’ computers. In addition, computational tools for the conceptual stage do not yet provide the same usability, the same ease of annotation of design sketches with rough dimensions, and the same possibility of making notes as do pen and paper (Yang 2009).

The results from the tests of established ideation methods showed that during the SIL method the participants were quite confused about what they were supposed to do and asked a large number of questions about the procedure during the entire SIL workshop (Petersson et al. 2017). It was therefore concluded that the SIL method would not be feasible to explore further since it would be impossible to achieve the goal of easy implementation with that method; therefore, the SIL method was excluded from the scope of this paper. Neither Method 635 nor the gallery method encountered similar problems and were therefore judged to be a better starting point for the method to be developed.

4.1.2 Individual and group work

Drawing upon the implications of AMMs, it was determined that a feasible design would be to start with a first step of individual ideation where the participants would document the ideas which they would first come up with as a reaction to the ideation topic. In this way, they would not be affected by the other participants’ ideas initially. After the first individual step, the participants should be exposed to other ideas than their own to activate not-yet-accessed parts of their memory network. Participants can be exposed to ideas generated by others in the group or external ideas such as pictures of objects from nature. Previous research suggests that the effectiveness of stimuli depends on the appropriateness of the stimuli with respect to the given topic, participants and context (Perttula and Sipilä 2007). It was decided that the group would use internally generated ideas during the ideation method because (a) it was expected that these ideas would have a higher probability of appropriateness than selected external stimuli, (b) someone would have had to make the effort of selecting the external stimuli before the ideation session, which would have been a barrier to using the method, and (c) the different functional backgrounds of the participants would provide ideas with enough variety.

4.1.3 Sharing of ideas

There are several ways of exchanging ideas that can activate the memory network in different ways. Since the ideation method to be developed did not have any given time restrictions, it was decided that a mix of idea-exchanging strategies would be a good starting point. Linsey et al. (2011) suggested that “an improved process for idea generation consists of first using a gallery communication method to generate a large number of high-quality product solutions and then moving to a rotational viewing method using words and sketches to develop the details of the product solutions and a large number of functional ideas”. However, in the light of the reasoning in Sect. 2.2, one should first aim to achieve a large pool of ideas with great variety that can activate different parts of the memory network. Therefore, the method developed in the present study starts with individual ideation with rotational viewing followed by an IP with gallery viewing. After each viewing phase, the participants exchange ideas through presentation and discussion in verbal interaction steps. The participants of the present study thought that presenting and discussing ideas were very important for reaching good solutions, and this feature of the method made it particularly attractive, since the participants really enjoyed interacting verbally (Petersson et al. 2017). Further, the presentation steps let each participant gain an overview of the entire set of ideas generated by the group. One difference between interactive and nominal groups is the fact that, after the ideation session, the members of an interacting group will have knowledge about the entire set of generated ideas, whereas a member of a nominal group will only know about the subset of ideas that he or she has generated. In consideration of this, it is not very surprising that the participants in an interactive group typically feel more productive (Stroebe et al. 1992; Paulus et al. 1993) and enjoy the ideation session more (Furnham 2000). It is reasonable to assume that the sharing of ideas is an important factor in motivating individuals to participate and engage in the ideation session. The procedure of the first version of the developed method is shown in Table 8. It is acknowledged that GP2, the presentation and discussion of ideas from IP2, might not add very many new ideas, but rather is used to conclude the session for the group. However, knowing that others will view the ideas which one has come up with might enhance the idea generation in the earlier steps through social comparison (Perttula et al. 2006). Further, it might be possible that this last step may activate new thoughts among the participants, who may come up with ideas directly after the session, after a period of incubation, or later when interacting with some other stimuli.

Table 8 The execution of the two different versions of the developed method

4.1.4 Form of representation

According to Linsey et al. (2011), embodied cognition theories suggest that external representations such as sketches are helpful for the performance of design tasks since they reduce the cognitive load, as the amount of information which an individual has to represent internally is reduced. On the other hand, whereas information such as geometry and configuration tend to be easier to express in drawings, other information such as abstract concepts might be easier to convey in words (Linsey et al. 2011). Additional information in words might also be helpful when other individuals interpret a sketch. Particularly worth noting is that individuals have different preferences and some individuals simply do not like to sketch. Therefore, the participants were instructed to use both sketches and text, to the extent desired by them, when documenting their ideas. An advantage of the developed method is that all the ideas are documented in sketches and text by the participants continuously during the workshop, so that no special person needs to be appointed to record ideas. Table 9 sums up the design principles used to set up the initial version of the new method.

Table 9 Design principles for ideation methods to be used in cross-functional inter-organizational groups, their justification and implementation

4.1.5 Length of each round of rotational viewing: adjustment made in OKMv2

Two changes were made to the initial design of the method after the first test. The time allocated for each round of IP1 was prolonged from 5 to 10 min, as the participants all agreed that 5 min was too short a time to be able to document all their own ideas during the first round, and to read through and understand the ideas on the sheet of paper which they had received and then come up with new ideas during the remaining rounds. This is in line with Nijstad et al.’s (2002) reasoning that stimulus ideas can interfere cognitively with a person’s train of thought if they belong to a different semantic area of memory. The train of thought is then cut off, leading to a loss of potential ideas. Therefore, the time allocated for each round must be long enough to allow the participants to reach the end of their train of thoughts, as activated by exposure to the ideation topic and stimulus ideas. With each successive round, each sheet of paper included more and more information, and it took a longer time for the participants to acquaint themselves with this increased amount of information during the remaining rounds.

4.1.6 Distribution of spoken words: adjustment made in OKMv2

The second change concerned the unequal distribution of spoken words among the participants, which was found after the first version of the developed method had been tested and is dealt with in greater detail in Sect. 4.3. The distribution of spoken words is important, since previous research claims that it is essential that all the perspectives and knowledge are contributed in the group (Milliken et al. 2003). New information can activate new areas in the participants’ memory network, and to access and use the information which each participant possesses, all the participants must speak. Some persons are inclined to speak more and others less (e.g. Burke 1974), and therefore it was important to find a simple procedure that could even out the distribution of spoken words. Instead of letting the discussion be completely free, the change was introduced that, after each participant had presented his suggestions, the other participants took turns to comment on the presented ideas. Whenever questions and urgent comments arose, the participants were still allowed to speak out of turn. The steps of the second version, OKMv2, are shown in Table 8.

4.2 The participants’ views

Sections 4.2.1 and 4.2.2 present the results regarding the participants’ views on the versions of the developed ideation method and rely on data from the group interviews. Quotes (translated from Swedish into English) are included to exemplify qualitative data that contributed to the results. The letter in brackets after each quote indicates who is quoted, according to Table 1.

4.2.1 The participants’ views after testing OKMv1

All the participants thought that OKMv1 was the best of all the methods which they had tried thus far, i.e. Method 635, the gallery method, the SIL method and OKMv1. They liked combining elements from Method 635 and the gallery method, since they thought that this gave them the opportunity to exhaust the ideation topic more thoroughly than they had been able to do with the methods tested previously. This is exemplified by the following quote:

“Directly after this [Method] 635 we got to do this [gallery method]. […] On the same ideation topic, and then we could exhaust the topic more.” (F).

All the participants felt that they had progressed further regarding the complexity and details of the concepts and ideas, and that the concepts and ideas had evolved in more directions and in greater detail than when they had tried the established ideation methods.

All the participants agreed that 5 min in each round of IP1 was too short a time. They had not had enough time to document their own initial ideas in the first round. Four participants mentioned specifically that, when the next round had started, they had forgotten their own remaining ideas and had concentrated on making associations with the ideas on the paper which they had received. However, reading through and understanding these ideas had consumed most of the minutes that had been available, and therefore there had not been much time left to build on these ideas. This is exemplified by the following quotes:

“Once you get the sheet from your neighbour you get a bit stressed, you don’t have the time to read through and understand all […] the innovations or ideas before you have to [add your own ideas], there is so little time left to try to come up with something new yourself.” (F).

“When I got [my neighbour’s sheet with ideas…], I sat wondering what I should do with them, but I had my own ideas, of course, and [… his ideas] were completely different from what I was thinking about. […] But then I lost [my own ideas before I got to write them down]. […] Now I was supposed to understand what he had [suggested…].” (D).

Two participants mentioned specifically that they were surprised and impressed by the breadth of the ideas that had emerged during the session.

When asked if they would have preferred working in groups of two, for example, during IP2, all the participants agreed that they preferred working alone during the individual steps. All the participants thought that they could create ideas with more breadth by working individually that working on their own saved time because working in pairs or teams required more time for explaining, and that there was enough group work during the presentation steps. Also mentioned was the fact that it was good to have time to develop one’s own ideas on one’s own. This is exemplified by the following quotes:

“We have better possibilities of [covering a wide range of ideas] if we work individually in this third step, in my view. The generation of ideas and then the presentation of them, that is when you have the common discussion. Then the group is working. (D).

“I agree […] that it was a suitable mix of working individually and working together.” (E).

Two of the participants mentioned specifically that they had not been in a good creative mood on the day when OKMv1 had been tested, in one case because of a lack of sleep.

4.2.2 The participants’ views after testing OKMv2

All of the participants agreed that OKMv2 represented another step forward and was the best method that they had tried thus far. This is exemplified by the following quotes:

“I wrote that it is a step forward, that I think that this very fact that [we have more time] means that we can process, think about and grasp what people mean instead of just spurting out ideas. So I feel, as it were, that this is a method that, as I see it, could be put into practice.” (D).

“I agree, this was the best we accomplished, I believe. It felt completely right, all the way. [Sometimes] there were even some spare minutes, that was first-rate, I thought.” (C).

“Oh, best so far, definitely. […] I agree, I was hardly even close to thinking about drainage [before] and yet lots and lots of [ideas] emerge and then the way in which the quality is improved from [the Method 635 step] to the [gallery step], yes, it’s clear as a bell. Terrific.” (B).

The participants all thought that prolonging the time for the rounds of rotational viewing was excellent, but had somewhat different opinions about how to distribute the time. One participant thought that it would be better if the first round was longer than the others. The other participants, however, thought that 10 min was appropriate for every round. All the participants thought that there was sufficient time to understand, consider and make associations with other people’s ideas, as well as to document one’s own ideas. This is exemplified by the following quote:

“No, ten [minutes] was excellent, it’s quite sufficient, because somehow I felt there was peace and quiet and we had the time to read through [the ideas] before starting to comment; [during OKMv1] we started to comment directly, as it were, one didn’t have the time.” (D).

Four of the participants mentioned that, during the last rounds of rotational viewing, they had actually had a few minutes left which they had thought could perhaps have been used for IP2, the second individual step, instead. None of the participants wanted to remove IP2, despite the fact that IP1 had been prolonged. They wanted to use IP2 to summarize or to develop concepts, or to be able to work on new ideas which they had created during GP1. Half of the participants thought that the time allocated for IP2 had been sufficient, whereas the other half felt that they could have used a few minutes more.

Three participants mentioned specifically that they were surprised by the large amount of ideas which they had conceived regardless of whether they had been very familiar or less familiar with the ideation topic. This is exemplified by the following quotes:

“I hadn’t expected to discover any ideas, but it felt as if my brain got going anyway, and I think that was good.” (F).

“Just to see visually what we [created], what we got down on paper, my goodness, we were never close to that any time before. And on a topic which I think, [although there is good knowledge of it within the group,] as for me, I have only scant knowledge of it, yes, well, incredibly good.” (C).

Letting all the participants sitting around the table speak in turn was appreciated, because this meant that everyone was given a hearing.

One participant suggested that the developed method could be improved by having each group member think of three ideas on the topic in question before the ideation session and bring it to the session as a starting point for IP1. The other participants, however, were reluctant to accept this suggestion, because they thought that it would be difficult to arrange a presentation and clarification of the ideation topic in advance of the session, that “crazy” ideas would disappear and that it would not be possible to achieve the quick responses and communication which the developed method made possible.

4.2.3 Questionnaire

The participants’ average levels of agreement with the statements in the questionnaire are shown in Fig. 3. The responses to the statements presented in Fig. 3 were also analysed separately for the academics and railway professionals. It turned out that the variation between all the individual participants was greater than that between the two subgroups, and therefore no conclusions about possible differences between the two subgroups could be established. The only statement for which the variation between the individuals was less than that between the subgroups was the statement “Many ideas were new to me”. The academics exhibited a higher level of agreement with this statement during the test of OKMv2, which is perhaps not surprising, as the other participants had a longer experience from the railway sector. Moreover, there was a trend that the professionals from STA agreed to a lower extent than the others with the statement “Many interesting ideas were presented during the workshop”.

Fig. 3

The participants’ average level of agreement with different statements about each method

Figure 3 shows that the engagement in all the ideation issues was quite high, although the engagement in the issue which was ideated on during OKMv1 was somewhat lower. This may be one explanation for the lower engagement felt during the OKMv1 ideation session compared to the engagement felt during the gallery method or OKMv2. However, when examining the individual ratings, it was found that it was the very low rating given by one of the two participants who had experienced a “bad day” which had pulled down the average level of engagement felt by the participants during the OKMv1 workshop. The same explanation applies to how satisfied the participants were with their own contribution.

When testing OKMv2, it was observed during IP1 that the participants had been less active during the last minutes of the last rounds. The participants reported that they had experienced difficulty coming up with new ideas at the end of these rounds and this is also reflected in the questionnaire. OKMv2 scored much lower on “If I had had more time, I would have come up with more ideas” compared to the other methods. Another way of judging whether a topic is more exhausted is to analyse the types of ideas appearing at the end of the ideation session. It was observed that the ideas that had emerged at the end of IP1 of OKMv2 had obviously been more crazy and unrealistic (e.g. killing plant life using nuclear radiation or introducing animals that like water to address insufficient drainage at turnouts) than previous ideas. It has been reported in the literature that, when “the rate of suggested ideas has diminished almost to a standstill”, more intriguing and unusual ideas start to appear, but only a few people contribute then and most people would prefer to do something else (Byron 2012). It could therefore be argued that with OKMv2 this point in time was reached, which did not happen during the other tests, and therefore the topic dealt with during OKMv2 was more exhausted. The point in time when the participants run out of ideas is probably dependent on the ideation issue, but it can be concluded that it takes quite a long time to exhaust the types of questions which have been dealt with in the present study; for example, in the case of OKMv2, the participants used 63 min during IP1. Once this point is reached, there is a choice of two alternatives: ending the ideation (or in the present case, advancing to the next step of the method) or trying to squeeze out unusual ideas at a lower rate while the members possibly start feeling that they are wasting time. Nijstad et al. (1999) found that an individual working alone or in a group stops brainstorming when they feel that it is no longer worth the effort, and this feeling is based on their subjective estimate of the probability of them being able to generate more ideas on the ideation topic. The continuation of ideation requires some effort to be made to overcome this stage, and perhaps additional measures must be taken. Although many of the ideas produced in this phase are obviously unrealistic, a few of them could lead to a breakthrough through association.

With regard to how the participants viewed the ideas generated during the ideation, OKMv1 scored highest on the number of interesting ideas presented and OKMv2 on the newness of the ideas to the participants. The variations concerning these parameters between the methods may, however, be a result of how familiar the participants were with the ideation topic. When rating the extent to which the presented ideas could be used practically, the participants awarded higher scores to the two versions of the novel method than they awarded to Method 635 and the gallery method. When analysing the data from the interviews, we could observe that this could be due to the discussions during the group phases and the possibility of refining ideas in several steps. OKMv2 scored highest on how useful the method was thought to be, which is reflected in the comments by the participants that this was the best of the methods that they had tried thus far. As expected, the developed method outperformed the other methods regarding user satisfaction.

Considering the correlations between different statements in the questionnaire, a few points are interesting to note. However, it should be kept in mind that due to the small number of data points the findings should be regarded as tentative. The only correlation with any significance at all, see Fig. 4, was between how engaged a participant felt during the workshop and how satisfied that participant was with his own contribution (R² = 0.68). This tentatively indicates that there might be a causal relationship between the engagement felt by the participants and the effort which they make during the workshop. However, another possibility is that, if a participant feels more engaged, they will perceive their own contribution as better although it might not objectively have been so. Similarly, their engagement might bias their view of how useful the method is. Between the engagement felt by the participants and how useful they found the method, an R² = 0.44 was found. Previous research concerning brainstorming found that group members enjoyed their work more and were more satisfied with their own performance than individual brainstormers, although they were actually less productive (e.g. Stroebe et al. 1992), and that one reason for this might be a tendency to appropriate ideas of others in the group (Paulus et al. 1993). Gibson (2015) found that groups using a verbally interactive informal best practice approach reported a higher level of comfort than groups using a modified C-sketch method, although the latter outperformed the first on all metrics. This underlines the importance of taking both objective and subjective results into account when determining what method is most promising, since the views of the participants might be biased by their engagement. Since most participants had similar levels of agreement with most of the statements in the questionnaire, the pairwise plotting of the statements against each other in most cases resulted in a clustering of most of the data points on which each linear regression was performed, as is exemplified in Fig. 4.

Fig. 4

Linear fit (dashed line) between the participants’ engagement during the workshop and how satisfied they were with their own contribution (R² = 0.68). “Do not agree at all” corresponds to 0 and “Completely agree” to 1

The absence of correlations for some pairs of statements is interesting. How useful the participants found the method and how satisfied they were with their own contribution during the workshop did not correlate (R² = 0.18). A possible explanation for this is that the participants tried to judge how useful the method would be in general, while they rated their own contribution in the specific instance. How engaged the participants were in the ideation topic did not correlate to how engaged they felt during the workshop (R² = 0.10) or how useful they found the method (R² = 0.01). Other researchers (e.g. Isaksen 1998; Bolin and Neuman 2006) have suggested that ownership of the task or the strength of the incentive for high performance is important for the outcome of a brainstorm. One would intuitively think that a high engagement in the topic should lead to a high engagement during the workshop and a better outcome compared to a lower motivation to address the topic on the part of the group. The absence of a correlation between the engagement in the topic and the engagement felt during the workshop indicates that other factors are more important. However, other factors that could be expected to influence the engagement felt during a workshop, for example, the extent to which the method resulted in ideas that could be used in practice (R² = 0.06), how many ideas were new to the participants (R² = 0.01) or how interesting the generated ideas were (R² = 0.06), were also uncorrelated. Neither were these factors correlated to how useful the participants found the method (R² = 0.18, 0.00 and 0.12, respectively). This may be because the right factors were not investigated in the questionnaire or because the relations between these parameters are more complex and cannot be explained by linear correlations between different pairs of statements. In any case, more instances of participants being less engaged in the topic would be required to obtain a reliable correlation.

4.3 Distribution of spoken words

Table 10 shows the average deviation from an equal distribution of the spoken words and speaking frequency among the participants during the group phases. If the number of spoken words had been completely equally distributed among the participants, the deviation would have been zero. Table 10 shows that the average deviation from an equal distribution of the spoken words and speaking frequency among the participants decreased when the participants took turns to comment on the ideas (in OKMv2), suggesting that this might be a feasible procedure, although the same people as before still spoke the most (see Table 11). The distribution of spoken words is not only dependent on the design of the method, but also, for example, on the personality of each group member and how they are affected by the behaviour and personalities of the other group members. Therefore, it is not likely that the observed speaking patterns would disappear completely by taking turns. Some persons are more outgoing and talkative (more extrovert), others tend to be more reserved (more introvert), and therefore some group members are more likely to dominate the interactions in a group. During individual interviews performed in connection to the tests of the established ideation methods (Petersson, 2017), group members mentioned their different personalities as the reason for the unequal distribution of words. Other possible reasons for the uneven speaking distribution could be the asymmetric dependence of different actors on each other and that an informal hierarchy exists among the participants. The intention was that the presentation and discussion of concepts and ideas taking place during GP1 would act as input to IP2. If some group members speak more than others during GP1, they might affect the outcome during IP2 more than the others. However, speaking the most does not necessarily mean contributing additional ideas or perspectives, and could possibly hinder other group members’ perspectives from being shared.

Table 10 Average deviation from an equal distribution of the spoken words and speaking frequency (absolute %)

Table 11 Distribution of spoken words between the participants (%)

All the participants thought that letting all the group members take turns to speak was a good improvement. From observations it appeared that taking turns to comment on the ideas had helped to keep the discussion focused and had made it clear to the group what was going to happen next all the time, which had streamlined the time for interaction. The participants themselves jointly initiated a procedure whereby the sheet of paper containing suggestions presented by one participant was circulated among the other participants for them to comment on the ideas in turn. This made it clearer when the next person should start giving feedback and easier for the participants to avoid forgetting any comments which they might have thought of.

In Table 11 it can be observed that the supervisor spoke much less during GP1 and GP2 of OKMv1 and OKMv2 than during GP1 of the gallery method, indicating that the group worked more independently when applying the developed method. However, since each member of the studied group has had the time to understand and learn the methods, as well as acquaint themselves better with the other group members, one cannot conclude that a group with “beginners” would be able to work through the method without a supervisor.

4.4 Ideation outcomes

The group generated concepts that included physical solutions to the problem of the ideation topic, organizational solutions and maintenance solutions. The concepts were described by words, a sketch, or by a combination of a sketch and words. Examples of concepts from each ideation session are shown in Figs. 5 and 6, respectively. In all the examples of concepts and ideas presented throughout this paper, the text has been translated from Swedish into English. Examples from the tests of Method 635 and the gallery method are to be found in Petersson et al. (2017).

Fig. 5

Examples of concepts from the OKMv1 ideation session on how to adjust the position of a modular turnout

Fig. 6

Examples of concepts from the OKMv2 ideation session on drainage in turnouts

From Table 12 it is obvious that there is a great difference in the idea generation rate between the individual and group phases. During IP1 and IP2, where the participants work individually, significantly more ideas were generated than during the group phases, as can be seen in the far right column in Table 12. It might be tempting to draw the immediate conclusion that, to maximize the total idea generation rate, only individual phases should be included. However, as was observed during IP1 of OKMv2, the participants eventually ran out of ideas. Although some ideas were expressed verbally, it can be seen from Table 13 that most of the spoken words were dedicated to the elaboration of ideas during all the group phases. The longer time the participants had to ideate individually, the longer was the time it took to present, explain and discuss the ideas in the subsequent step. After the group phases, the intention was that the participants would be able to create associations with what had been presented and come up with new ideas. Although it is difficult to quantify the extent of inspiration, it was found that the ideas that had turned up in IP2 in many cases had been based on some issue that had been brought up during GP1. The average concept generation rate was lower in IP2 than in IP1, but the average number of ideas per concept was higher in IP2, reflecting the fact that several of the participants aimed to combine ideas from previous phases with new ideas to achieve a complete solution to the problem of the ideation topic (as opposed to producing several unrelated ideas). An example is shown in Fig. 7. The fraction of ideas from earlier phases that each participant reused in concepts generated in IP2 varied from 0 to 100%, but was on average 57% for OKMv1 and 54% for OKMv2, i.e. more than half. It is especially interesting that, although the participants had difficulties creating any more ideas during the last rounds of IP1 of OKMv2, despite exchanging ideas through rotational viewing, after the verbal interaction in GP1 they could squeeze out 32 new ideas in IP2. From these results it is concluded that the verbally interactive steps are relevant in the method. Kohn et al. (2011) found that interactive brainstorming groups generated combinations of previously generated ideas of higher utility than nominal groups. Nijstad et al. (1999) found that when groups brainstorm verbally without a time constraint, the productivity loss in relation to the productivity of nominal groups decreases, because the groups continue to ideate for a longer time. These findings indicate that verbal group interaction may be beneficial in group ideation if it is applied in the right instance, and may perhaps be especially beneficial in the longer ideation sessions that may be required in the field of engineering design. This may be especially fruitful when the absolute number of ideas is more important than the idea rate. One would expect that industrial companies would rather use some extra hours during ideation to assure that as many ideas as possible would be collected before starting the selection process. Although the idea generation rate is an interesting parameter for evaluation purposes, the total number of ideas in a day or even a week is likely to be more relevant to companies and other organizations. It might therefore be more interesting to focus on how to make a group create more ideas on a topic which they think they have exhausted than mainly to consider the idea generation rate.

Table 12 Number of concepts and ideas, length of time and idea generation rate for each step of each method

Table 13 Distribution of spoken words between activities (%)

Fig. 7

Example of a concept generated during IP2 that combined several ideas and concepts from IP1 and GP1

It was expected that the developed method would outperform the other methods with regard to the quantity of ideas generated and this expectation was fulfilled if one considers the total number of ideas. However, one can question whether the novel method outperformed the other methods concerning the overall concept and idea generation rate. The generation rate for new ideas during IP1 tended to decrease as the length of this step increased, which is consistent with previous research on verbal brainstorming showing that the idea flow declines with time (e.g. Paulus and Dzindolet 1993; Byron 2012). However, although the idea generation rate decreased somewhat, the second version of the novel method had a surprisingly good idea generation rate during this step, considering the fact that the time was doubled compared to that allocated for the first version. All of the participants thought that it was better to spend some extra time, despite a lower idea generation rate, to achieve a greater absolute number of ideas.

Method 635 and the similarly executed IP1 of OKMv1 were of equal length, but the former generated more ideas. Several parameters can explain this result, but the fact that two of the six participants experienced a “bad day” when testing OKMv1 and were unhappy with their own effort probably contributed. Nevertheless, the group was very satisfied with the results of the workshop, especially with the breadth of the ideas generated during the workshop. Therefore, another explanation may be that the scope of the topics dealt with in Method 635 and OKMv1 differed somewhat in width, despite efforts to find relevant topics of a similar scope, and that in relation to the topic the performance of IP1 of OKMv1 was good.

When testing OKMv1, the participants felt that they had not had enough time to follow through on their own ideas created during the first round, but had been interrupted by the ideas on the sheet of paper given to them by their neighbour after 5 min. Linsey et al. (2011) hypothesized that “rotational viewing encourages the participants to spend more time understanding other [teammates’] ideas”, which is in line with the experience from the present study. Attending to the ideas of others is a critical factor for the observation of stimulation effects in groups (Paulus et al. 2015). The first ideation phase of the developed method actually consists of (a) attending to the content of the sheet of paper and (b) generating new ideas. To allow the participants both to comprehend the existing ideas on the sheet of paper received and to generate more ideas, the length of time for each round was successfully increased to 10 min. However, it is important not to prolong the duration of each round unnecessarily, since previous research has shown that individuals under time pressure work at a faster rate and that the less time there is available, the greater is the task focus shown by groups; however, the quality of the performance may suffer if too high a pressure is applied, since information is processed less thoroughly then and a narrower range of alternatives are considered (Karau and Kelly 1992). Too long a time might also allow the participants to start thinking about disadvantages of their ideas and decide not to share them with the group. It was concluded that 10 min was the right length of time for the topics considered in the present study. Linsey et al. (2011) used 10 min for individual idea generation prior to four subsequent 7.5 min periods where each group of five participants viewed either all the ideas (gallery viewing) or a new subset of ideas during each period (rotational viewing). Gibson (2015) used a cycle time of 6 min when testing a modified C-sketch method in groups of five participants. The total times of 40 min in the study performed by Linsey et al. (2011) and of 30 min in the study performed by Gibson (2015) are shorter than the time used for IP1 in the present study, but longer than the total ideation time reported for the typical study in social psychology literature, which does not deal with design problems. An intuitive assumption is that the appropriate total and cycle times will depend on the ideation topic and the size and experience of the group concerned, and developing guidelines for setting these times as a function of these parameters is an interesting area for future research.

4.5 Formalization of learning and limitations

The present study is one of several efforts to understand better how ideation methods can be utilized within the area of engineering design, and the study has focused especially on a real-world context with ideation topics decided by the participants themselves. Besides the quantitative outcomes from the ideation sessions, the present study has considered the opinions of the participants in detail, which is unusual in research on ideation methods and can give important insights into the factors determining whether such methods will be implemented in industry. This study is limited to a single group, and therefore, statistical generalizations are not applicable. The findings and conclusions of the paper are valid for this particular group. Using another group might have resulted in different findings due to different group characteristics exerting different influences. Possible influencing factors in this connection are the personalities of the group members, their experience, backgrounds, gender, age and other personal characteristics. Further, the group acted in the railway sector, and studying a cross-functional inter-organizational group from another sector could result in other findings. Through the guidance of ADR, the findings have been generalized to design principles that can be used as a starting point for developing ideation methods in a similar situation that other researchers might face within other organizations. The last stage of ADR requires the researcher to make a conceptual move from the specific and unique to the generic and abstract on three levels (Sein et al. 2011). The present study has developed an instance-specific solution that has been reconceptualized into a class of solutions. The developed ideation method has been viewed as belonging to the class of ideation methods for cross-functional inter-organizational groups. Preliminary design principles for this class of solutions were formulated when designing the initial version of the developed method. These were confirmed during the tests and two additional design principles were identified during the test of the initial version of the method. Further, it was realized that the method had inherently encouraged the participants to attend to the other participants’ ideas and that this was an important design principle. The design principles are to be found in Table 9.

Although AMMs explain how information in the long-term memory is accessed, other factors exist that can further stimulate or interfere with activation, such as motivation. Stimulation requires that the subject concerned should attend to the stimulus idea (e.g. Dugosh et al. 2000), which is a process that requires the person to be motivated to do so. The motivation for a participant to create ideas or share the ideas created might be low for different reasons. One factor which could lower the motivation of a participant is a fear of other participants stealing good ideas from them due to the inclusion of competing actors in the group (Petersson et al. 2017). Almefelt and Claesson (2015) applied systematic design methods in a collaborative project including an automotive manufacturer and 35 automotive suppliers and found that in some teams where the parties represented competing suppliers, the work progress was hindered by conflicts of interest. Therefore, no competitors (actors competing in the same functional segment) were involved in the creative team of the present study, but there is still a risk that some participants might have kept some ideas for themselves owing to a fear of having them spread outside the group.

Because only a single group participated in this study, we were forced to change the ideation topic between the ideation sessions. Although the different topics were chosen with care so that they would be equivalent, and although they were found to lead to similar engagement among the participants (see Fig. 3), it cannot be excluded that they influenced the ideation outcomes differently. Some of the comments made by the participants during the interviews indicated that they had expected that it would be more difficult to generate ideas concerning drainage of the superstructure compared to the other topics compared to the other topics, although they eventually seemed to believe that they had not been constrained by this during the ideation (see Sect. 4.2.2). How well the participants knew the topic from before and how easy it was to associate from it to other domains are examples of factors that might have influenced the outcome. In retrospective, it would have been beneficial to assess how difficult the participants found it to be to generate ideas on each topic and how familiar they were with it from before. The number of non-redundant ideas and concepts was carefully counted according to a specified procedure to make the count as fair as possible between the methods. However, even if great care is taken to count ideas in a neutral way, subjectivity will to some extent influence the counting, and in the present case the different nature of the concepts made the identification of single ideas challenging. Because of these uncertainties originating from the change of ideation topic and the counting of ideas, our aim has not been to make conclusions about which method is the “best” with respect to the idea generation rate, but rather to focus on interesting trends in the data that can be useful in helping us to understand how to apply ideation methods in an optimal way within the area of engineering design.

The tests of the methods were performed on different occasions. This was the only possible way to organize the tests, because (a) the participants were busy professionals located in different parts of Sweden and (b) the evaluation of the first test had to be completed before the second version could be tested. Therefore, the conditions of each test may have varied with respect to the mood and fitness of the participants, and this was especially evident during the test of OKMv1, where two participants explicitly mentioned that they had experienced a bad day with respect to creativity.

This study has focused on developing and testing a method suitable for application in a functionally heterogeneous group on the basis of implications of AMMs. However, we have not tested the method in a homogeneous group and compared the results to be able to verify these implications. One of the starting points for developing the method was that participants with different functional backgrounds would be able to provide different types of ideas as a reaction to the ideation topic in the individual ideation phase at the start of the method, and that these ideas would be used as stimulus ideas for the other participants throughout the method. If the participants of a group have a similar background, competence and perspective with respect to the ideation topic, it is expected that the ideas that the group can produce at the start of the method will have less variety and hence not serve the purpose of functioning as stimulus ideas for the group. Consequently, the outcome is expected to be worse compared to that for a group that is heterogeneous with respect to the ideation topic. In the case of a very homogeneous group, external stimulus ideas might be a better approach. It has, however, not been within the scope of the present study to investigate this, but it is a very interesting area for future research.

Several researchers have found that studying team performance in work settings is difficult because it is hard to obtain access to such settings (Shah 1998; Paulus et al. 2015). However, studies of ideation methods in organizations and industry are important for understanding how the use of ideation methods in such contexts can increase the effectiveness of teams working with innovations (Shah et al. 2000; Paulus et al. 2015). From our experience, the company or organization must be convinced that they will profit from this type of study to provide the researchers with access. This implies that the researcher has to adapt to the situation and the on-going work at the company or organization, and that the main aim of the activity for which the researcher is given access is that it should be fruitful for the organization. Further, setting up a completely controlled experiment necessarily means procedures that would never take place in real life, and therefore the study necessarily becomes less realistic, although it still takes place in a work setting. Therefore, we found that it is critical to find a balance between letting the participants work in a realistic setting and exerting enough control of the experiment to be able to accomplish a meaningful evaluation. Performing studies of ideation methods in work settings will necessarily imply more uncertainties and a smaller sample than performing such studies in controlled experiments with students, and therefore statistical significance is a challenge. In our view, such studies are still worthwhile, since they can shed light on findings from statistical studies and indicate if these findings are relevant in real work settings, as well as identify relevant topics for future research.

5 Future work

The present paper has presented the results of a study which has developed an ideation method within one cross-functional inter-organizational group. The next step is to make field tests where the method will be tested in real-life projects on real problems by participants who are new to the method.

6 Conclusions

The goals for the method to be developed were found to be that the method should:

• produce a great number of ideas with breadth and potential,

• be implemented by STA after the project finishes.

The requirements for the method were found to be that it must:

• generate a great number of ideas,

• generate ideas with breadth and potential,

• be easy to use and understand,

• be attractive to the users.

The participants thought that the developed method was the best method that they had tried. The method was found by the participants in the group to be more useful and to generate more ideas that could be used in practice compared to Method 635, the gallery method and the SIL method.

The participants did not report having and were not observed to have difficulties understanding and applying the method. Most of the spoken words were dedicated to the elaboration of ideas during all the group review phases. The distribution of spoken words between the participants became more equal when the procedure of taking turns to speak was put into practice in the second version of the method.

Considerably, more ideas were generated during the individual phases than during the group review phases. The group review phases were mainly used to present and elaborate on concepts and ideas. The average fraction of ideas from earlier phases that each participant reused in concepts generated in the second individual phase was more than half.

The developed method outperformed the other methods with regard to the total number of generated concepts and ideas, but not with regard to the overall concept and idea generation rates.

The learning derived from the development of an ideation method in a single group was formalized into the following design principles for ideation methods to be used in cross-functional inter-organizational teams:

• development of the method based on methods proven to be easy to understand,

• the initial generation of a pool of ideas with great variety,

• exposure to stimulus ideas generated internally within the group,

• mixing idea-exchanging strategies,

• the use of external representation in the form of sketches and text,

• including time for discussion and debate,

• optimization of the cycle time,

• development of a strategy for distributing the verbal interaction,

• paying attention to stimulus ideas.