Internal Open Innovation—Lessons Learned from Internal Crowdsourcing at SAP

Abstract

Crowdsourcing provides companies with access to widespread knowledge pools and constitutes a well-established inbound open innovation practice. More recently, some companies have introduced the approach of open innovation within their company boundaries. Using internal crowdsourcing (IC), companies can apply open innovation principles to overcome information silos. Multinational corporations often have thousands of employees around the globe, embedded in divisions and virtually separated from each other. Although a large proportion of companies nowadays use social IT to mitigate problems of distance, only a few companies can access their employees’ wisdom effectively—let alone efficiently. With almost 100,000 employees worldwide, SAP possesses significant resources, which IC can help to unlock and develop. In this business case study, we report the findings of our investigation of five IC implementations at SAP. Based on interviews and secondary data, we analyze the process and related governance tasks of the different IC approaches. The applications for IC range from the search for new and sustainable business models to an approach that uses crowdsourcing for the competence development of SAP’s employees. Our paper contributes to our understanding of open innovation and crowdsourcing by conceptualizing IC as a form of internal open innovation. Further, from our observations, we derive six lessons learned to support managers in implementing and executing IC initiatives successfully. Our findings will subsequently help managers to increase the innovation capabilities of their companies, create more sustainable business models, further the entrepreneurial mindset of their employees and thus provide a competitive advantage.

1. Introduction

Over the past decades, companies have increasingly opened up their external boundaries with the intention of allowing knowledge to flow in and out of the organization to create a competitive advantage. This phenomenon—coined open innovation—was first conceptualized by Chesbrough [1] and today constitutes a new imperative for how innovation is organized [2]. More precisely, open innovation can be defined as “a distributed innovation process based on purposively managed knowledge flows across organizational boundaries, using pecuniary and non-pecuniary mechanisms in line with each organization’s business model. These flows of knowledge may involve knowledge inflows to the focal organization (leveraging external knowledge sources through internal processes), knowledge outflows from a focal organization (leveraging internal knowledge through external commercialization processes) or both (coupling external knowledge sources and commercialization activities)” [3].

Managing knowledge inflows—the inbound mode of open innovation—constitutes a well-established and well-researched practice of how firms can tap into external knowledge pools to complement their knowledge base and subsequently increase their innovation performance [4,5,6,7,8]. Especially crowdsourcing, defined as “inviting an undefined group of contributors to self-select to work on tasks” [9], has seen a tremendous uptake in interest over the recent years [10,11,12]. The principal raison d‘être of crowdsourcing is based on it providing access to the wisdom of the crowd, i.e., the idea that crowds of people under certain circumstances produce superior results, even when compared to experts’ efforts [13]. Crowdsourcing, therefore, allows firms to access diverse knowledge pools, turning distant into local knowledge, and increasing their innovation performance [10,12,14]. While most of the literature on crowdsourcing deals with innovation contests, crowdsourcing can be used for a large number of tasks. Geiger and Schader [15] classified four archetypes of crowdsourcing: (1) crowd rating, (2) crowd processing, (3) crowd creation and (4) crowd solving. The versatility of crowdsourcing is immense, ranging from big data analysis [16], to promoting disaster resilience [17], to sustainable last mile delivery [18].

While external crowdsourcing constitutes a rather well known and widely adopted phenomenon, companies have only recently embraced IC to leverage their internal crowds more effectively and efficiently [19]. The most comprehensive definition of IC can be found in [20]. Based on a structured search of the literature resulting in 74 relevant articles, they define IC as an “IT-enabled group activity based on an open call for participation in an enterprise” [20]. While remaining a niche phenomenon in the body of literature on crowdsourcing, IC has seen an upsurge in interest as of late [19,20,21,22], revealing how firms can leverage the knowledge and ideas of their employee crowd for innovation purposes. In recent years, successful applications of IC have already been reported, e.g., at Siemens, McKinsey & Company, Eli Lilly [23], Allianz [21], International Business Machines Corporation (IBM) [24] and NASA [25].

In this paper, we postulate that the open innovation paradigm has an intra-organizational perspective that has not been fully acknowledged yet. Most of the existing literature on open innovation is dealing with inbound or outbound knowledge flows across the outer organizational boundaries, e.g., when collaborating with suppliers, competitors or research organizations [8,26]. We argue that firms apply open innovation principles on the intra-organizational level when they purposively manage knowledge flows across intra-organizational boundaries (e.g., business units, foreign headquarter and subsidiaries)—a phenomenon coined “internal open innovation” [27]. As IC facilitates this search by providing efficient and effective access to a large crowd of individuals spread over the entire organization, we classify it as an internal open innovation strategy. In order to enhance internal knowledge flows, appropriate financial and managerial resources must be allocated [27]. As the management of IC is not a straightforward [21,28], our understanding of the associated governance tasks must be improved [20].

As early as 2001, SAP recognized that not only are external sources of information and the employees of the research and development department relevant for its innovation projects, but also that the entire pool of its 100,000 employees constitutes a valuable and rich source of new ideas [29]. The multinational software corporation has many years of experience in dealing with IC. It employs the technology for various tasks in its innovation process—from the development of innovative and sustainable business ideas, to the solution of smaller development tasks, to the strategic competence development of its employees.

The present paper provides insights into the following questions: (1) how and to what purpose does SAP use IC and (2) how is IC managed at SAP? Based on interviews and data from secondary sources, we describe five different IC initiatives at SAP and derive six lessons learned to support managers in implementing and executing IC initiatives successfully. Our results show that well-designed incentive systems, a collaborative, transparent and open corporate culture, the provision of sufficient resources, an appropriate submission evaluation procedure and the successful integration of IC are of central importance for its success. Further, the results of the present paper contribute to our understanding of the open innovation framework [26]. First, by conceptualizing IC as an internal open innovation practice. Second, by improving our understanding of the management tasks related to IC. Third, we introduce an IC initiative that is specifically interesting within the context of sustainability, emphasizing the importance of open innovation to reach sustainability goals and develop more sustainable products [30,31,32]. More specifically, we add to this perspective by showing how more sustainable ideas and business models can be created by applying internal open innovation principles via IC. Our case study shows how SAP was able to collect ideas and create internal businesses that apply their technologies to address sustainability issues like disaster relief and food waste. The results will enable managers to unleash the full innovation potential of their employees and to create a competitive advantage of their company.

2. Theoretical Background

2.1. Internal Crowdsourcing

IC highlights the potential of internal knowledge markets for innovation activities of firms [23]. It allows companies to open up the innovation funnel and to tap into the comprehensive knowledge of all of its employees [33,34], specifically about customers, products and services [35]. Especially multinational corporations with many geographically dispersed employees can use this technology to overcome information silos and to utilize the full potential of the company crowd more effectively and efficiently [21]. Sharing resources available throughout the organization potentially also leads to cost reductions [21]. Supported by an intranet- or internet-based platform, a shift towards a more collaborative, instead of an adversarial culture, is possible [36,37]. Additionally, a company may hold internal competitions for ideas to promote unity among employees and encourage creativity and entrepreneurial skills [38]. IC enables employees to make their ideas and innovative solutions available to a wider corporate audience, thereby communicating to employees that their ideas are valued and taken seriously by the company [28]. It conveys the message to the employees that anyone can submit an idea and realize it [39]. Additionally, the satisfaction and productivity of employees can be increased when they engage in more meaningful, significant and creative tasks [40]. These characteristics of IC can ultimately lead to more committed employees [28,36,39].

Compared to external crowdsourcing, IC accommodates for delicate tasks that involve sensitive information [20,41] and is better suited for tasks that require specific, in-depth expert knowledge [42], such as developing new business areas, incremental innovations [38] or enterprise strategy [40]. In contrast to external crowdsourcing, strategic decisions concerning intellectual property rights usually only play a minor role [41].

2.2. Internal Open Innovation with Crowdsourcing

When Chesbrough [1] introduced the concept of open innovation, he wrote: “Open Innovation is a paradigm that assumes that firms can and should use external ideas as well as internal ideas, and internal and external paths to market, as the firms look to advance their technology.” While most of the existing literature is dealing with knowledge flows across the outer organizational boundary of the focal firm, e.g., when collaborating with suppliers, competitors or research organizations [4,7,26], we postulate that the open innovation paradigm has an intra-organizational perspective that has not been fully acknowledged. From our understanding, Chesbrough’s definition also includes opening up the innovation process to internal knowledge sources that might not commonly be accessed to generate new ideas and products (e.g., business units, foreign headquarter and subsidiaries). This behavior is in contrast to organizations creating innovations by relying mainly or solely on their internal research and developing departments. Opening up the innovation process to all employees allows the organization to tap into previously unused knowledge pools. Actively searching for knowledge from various sources dispersed across the internal organization and, therefore, across intra-organizational boundaries, can thus be referred to as internal openness [27]. Subsequently, we refer to the act of purposively managing knowledge flows across intra-organizational boundaries, i.e., the application of the concept of open innovation to the intra-organizational context, as “internal open innovation”.

IC provides the organization with a “mechanism for participatory and co-creative innovation in large organisations” [35]. It facilitates internal search for solutions and innovations by providing efficient and effective access to a large crowd of employees spread over the entire organization. IC, therefore, bridges intra-organizational boundaries as it constitutes a “distributed organizational model used by the firm to extend problem solving to a large and diverse pool of self-selected contributors beyond the formal internal boundaries of a multi-business firm” [34]. Subsequently, we postulate that IC constitutes a method with which open innovation principles are applied to the intra-organizational level [41]—classifying it as an internal open innovation practice.

2.3. Managing the Crowd

To enhance internal knowledge flows, appropriate financial and managerial resources must be allocated [27]. Specifically, the management of IC is not a straightforward task [21,28]. Analogous to external crowdsourcing [43,44], firms have to develop capabilities to identify, absorb and commercialize widespread knowledge in order to leverage the full potential of IC. If IC is not managed appropriately, it may not produce the expected results and even be detrimental to the corporate culture. Hence, to fully exploit its potential, careful management and coordination of the IC process is paramount. However, despite their importance, the governance tasks associated with IC are not yet well understood [20].

To derive lessons from this case study, we use the conceptual framework of Zuchowski et al. [20] to guide our analysis. Based on a structured literature review, they develop a conceptual framework that allows a meaningful description of the governance tasks related to IC. These include “all actions and policies used to govern, manage, and steer the crowd” [20]. Based on previous work by [45,46], Zuchowski et al. [20] distinguish corporate culture and change management, incentive design, task definition/decomposition, quality assurance, community management and regulations management. In the following, we will briefly introduce the main concepts related to these tasks. A brief overview of the governance tasks can be found in Figure 1.

2.3.1. Corporate Culture and Change Management

This governance task involves creating an open and collaborative corporate culture in which ideas, people and information can move freely across the organization [36,41,47,48]. Companies need to actively break down hierarchical structures [49], appreciate employees’ ideas and efforts [35,38,50], take on a proactive and supportive leadership style [51], provide sufficient resources [22] and take measures to sustain activity on the platform [21,37]. Careful change management is paramount to manage IC [52,53].

2.3.2. Incentive Design

A strategic examination of the design of the incentive structures is of immense importance to ensure a high participation of the crowd. While some authors assume that a specific incentive structure for the employees is not necessary because it is already sufficiently covered by salaries and bonus payments [54,55], others advocate specific incentive structures to ensure the long-term commitment of employees [56,57]. Should the organization use incentives, material and immaterial incentives are possible. Material incentives include monetary compensation or prizes [23,28,58]. The most notable immaterial incentives are recognition by colleagues [42,59], learning new skills [38,60] and the realization of ideas [28,61]. While immaterial incentives have been reported to be more important [62], combining them with material incentives is optimal to promote employee motivation [22,23].

2.3.3. Task Definition and Decomposition

This governance task is about defining crowdsourcing tasks in a way that individual participants can solve them and is of great importance to ensure that the solutions can later be reintegrated into complex structures [20,42]. Most studies advocate to use IC to address direct business needs. An excellent summary of this critical step in creating a fruitful basis for any IC initiative is that “the best approach is to define the scope, provide context, identify constraints and clear goals, and remove as many assumptions as possible” [21].

2.3.4. Quality Assurance

Activities performed to ensure a high quality of submissions and results are summarized under the term quality assurance [20]. Ensuring quality helps in establishing the credibility of IC campaigns [61]. Often crowdsourcing campaigns generate large numbers of submission that need to be analyzed, evaluated, selected and developed. Being able to do that effectively and efficiently is crucial for the success of crowdsourcing [45,63]. Evaluation of ideas is either based on a form of crowdvoting [61], an evaluation against specific performance criteria [21], or a combination of the two. Despite the importance of the voting and comments from the crowd, their evaluation is sometimes distorted by social aspects like popularity, status and centrality of ideas [61,64,65]. In contrast, evaluations of experts and managers are more aligned with company goals [64].

2.3.5. Crowd Selection

While Zuchowski et al. [20] introduce community management as the fifth component of their framework, they mainly deal with the question of who should be allowed to take part in such a campaign. As we understand community management as a more broad term [66], we will refer to it as crowd selection. It is usually assumed that greater openness will increase diversity and, therefore, the probability to detect innovations [33,38] might result in creative rebound effects [37]. Furthermore, the openness of the platform signals that contributions of all employees—independent of hierarchies—are welcome [60]. Contrary to common belief, employees “below the radar or working in remote offices” often submit the majority of successful ideas [36].

2.3.6. Regulations Management

For IC to run smoothly, standards, strategies and rules have to be set [20]. This governance task represents the least investigated so far [67]. Issues that need to be addressed are intellectual property rights [41,59], a code of conduct for participants [11], anonymity of participants [28], barrier-free access [68], and the inclusion, consensus and cooperation of the works council. Above all, transparency regarding the whole IC campaign, processed personal information and the overall objective of the initiative is paramount.

3. Methodology and Case Description

The present business case study represents a descriptive, practice-oriented research approach [69]. The basic idea is to identify, describe and compare different implementations of IC and to derive findings for the successful implementation and execution of IC that are of relevance—especially for the practitioner. Although we only observe one company, we still derive our lessons learned from five different applications of and approaches to IC within SAP, which makes this a comparative business case study. Answering questions on how and why, i.e., questions that have a more explanatory character, case studies are considered to be a particularly suitable research strategy [70]. For example, case studies have been used successfully to analyze sustainability issues like transitions of family firms towards a circular economy model [71], or transitions of established energy and utility firms towards energy informatics business models [72]. The German software manufacturer SAP was selected as the object of investigation for this case study. The selection was made based on the following four reasons: First, SAP has many years of experience with IC and was one of the first companies in Germany to implement IC in its innovation process [29]. The insights and implementation strategies observed here are, therefore, of particular interest and value to current and future implementors. Second, we were able to observe five different implementations of IC at SAP. Considerable variations could be found concerning the individual governance structures of the different approaches. Third, SAP is the global market leader in business software and, as such, also offers its software implementation for IC to its customers. The insights gained from developing this software makes SAP a pioneer in the field of crowdsourcing software. Fourth, the willingness of the interviewed employees to share their knowledge enabled us to examine the different implementations from the perspectives of a wide range of stakeholders, i.e., we interviewed contest participants as well as managers responsible for the implementation and execution of the programs.

A total of 10 in-depth interviews were conducted over 14 months in 2018 and 2019. The interviews lasted between 28 and 92 min, with a total length of 555 min. The interviews were transcribed and subsequently analyzed using the IC governance framework introduced above. The interview data was enriched with information from secondary sources, such as press releases, blog entries, publications in newspapers and on websites, and annual reports. The collected secondary data enriched the background information on the selected firm and was integrated with the interview data in a triangulation process to gain a deeper understanding of the cases and assure the validity of the research [70,73]. The guideline for the interviews consisted of open questions on the employee’s function and background, their understanding and knowledge of IC in general and at SAP specifically, experienced successes and barriers with respect to the managerial tasks described above, and their evaluation of the impact of IC at SAP. Details about the individual IC campaigns are reported in

Table 1. Summary comparison of crowdsourcing initiatives at SAP based on [59].

Process Stage	Criterion	InnOvaTor Challenge	1 Billion Lives	Intrapreneurship	HR Crowdsourcing	SAP Blue
preparation phase	Task	generating ideas and solutions for given or self-identified problems	identifying business ideas for internal start-ups	identifying business ideas for internal start-ups; advising and supporting employees during implementation	sourcing of employees for projects of the consulting unit	organizing product development across teams and locations
	crowd	global since 2017, previously only Germany	all SAP APJ employees to date; from 2018 onwards: all SAP employees	all SAP employees	consultants at SAP	all employees of SAP India; worldwide introduction planned
	primary objective	competence development through long-term and in-depth engagement with forward-looking technologies	using SAP technologies to promote sustainable ideas with a high social impact	achieve agility and speed of start-ups in product development; gain competitive advantage	fast and efficient matching of demand and supply concerning specific employee profiles	fit between idea generators and developers; promote entrepreneurial thinking
	secondary objectives	ideas can be transferred to the intrapreneurship program if necessary; values, norms and innovative corporate culture can be conveyed	establishing an entrepreneurial way of thinking; external impact through social commitment	convey an entrepreneurial mindset; identify capable employees and teams	recording qualifications of employees in the consulting department	generate competitive advantages; faster development; efficient use of resources
	incentive design	partly material bonuses; enable learning of new skills and deepening of existing ones	implement sustainable ideas with significance; gather experience and knowledge around the establishment of a start-up; career prospects through start-up; start-up capital of 200,000 EUR; 20% of working time for implementation	the joy of implementing your idea; gathering experience and knowledge about starting a start-up; agile working environment; recognition by colleagues; career prospects through founding a business	be able to select projects yourself; pursue your interests	pursue individual interests; self-realization; use, share and expand abilities
	tool	generic software (SAP Jam and e-mail); introduction of specific solution planned (SAP Innovation Management)	specific solution (SAP Innovation Management) and SAP Jam for team collaboration	specific solution (SAP Innovation Management)	generic software (e-mail)	no information
	intellectual property rights	owned by SAP	owned by SAP	owned by SAP	irrelevant	no information
	commitment	bottom-up approach; small team which carries out project in part-time work; high credibility among employees	high annual budget and openly communicated to the outside world; top-down approach	bottom-up approach; nowadays direct support of the executive board; very effective in public relations; comparatively many employees	top-down approach	no information
	resources	200–300 EUR per team for hardware purchases; coaches and mentors; expert network	1 million EUR per year; coaches and mentors, expert network, workshops; accelerator	independent business unit with many employees; seed investments; mentoring program; workshops; accelerator	HR department	no hierarchies, self-sufficient as far as possible; team to select and evaluate the most valuable ideas
execution phase	frequency and duration	specific campaign; once a year for six months	specific campaign, once a year	permanent open call; specific campaigns in the form of business plan competitions and hackathons	permanent use with specific calls	permanent use with specific calls
	submission mode	competitive between teams, collaborative inside teams	Competitive between teams, collaborative inside teams	competitive between teams, collaborative inside teams	competitive	competitive
	submission format	closed submission; no collaboration possible on submission platform	open and transparent platform; commenting, evaluation and collaboration possible	open and transparent platform; commenting, evaluation and collaboration possible	closed submission	open and transparent platform; commenting, evaluation and collaboration possible
evaluation stage	evaluation criteria / evaluation mode	jury voting after final presentation of the results at a closing event	crowdvoting results; expert votings according to feasibility, social significance; combination of the results of the expert voting, crowdvoting and final presentation	team composition; submission of a video; complex catalogue of criteria; small circle of experts conducts evaluation; final presentation after accelerator phase	fit with required qualifications; fit with the inquiring customer; qualifications of the applicant; job interview with the customer	idea providers evaluate submitters themselves based on the submitted solution proposal
implementation stage	accomplishments	2016: 35 participants; 2017: 400 participants; 2018: similar to 2017	2016: almost 100 ideas and three start-ups; 2017: >90 ideas and five start-ups	more than 1500 business ideas have already been evaluated; six start-ups received funding	no information	128 projects with around 600 participants in the first two years; product and process innovations have already been incorporated into the SAP portfolio

.

Company Description

SAP is the world’s leading provider of business software. In terms of market capitalization, SAP is the world’s third-largest software producer and the largest software producer outside the United States. SAP was founded in 1972 by five computer analysts from IBM. Their goal was to enable the central processing and coordination of the entire value chain. The result was the development of an information system for companies called SAP ERP (Enterprise Resource Planning). SAP ERP enables companies to bring together all steps of the operational value chain in a single system. The current version of the system—R/3—uses artificial intelligence and machine learning to further increase efficiency, transparency and agility. Today, SAP offers a wide range of digital solutions for businesses. These include offerings for human resources management, artificial intelligence, machine learning, cloud platforms, databases, and procurement. All of these offerings can be combined and connected to make the most of the information generated in day-to-day business [74].

In 2018, SAP achieved an annual revenue of 24.708 billion EUR. The company sells its software solutions to over 425,000 customers in 180 countries around the world. Over 90% of Forbes Global 2000 companies are SAP customers. With approximately 96,498 employees and more than 100 innovation and development centers around the globe, SAP possesses exceptional resources for the application of IC [74]. It is precisely the geographical distribution and the large number of employees that make SAP a particularly interesting object of investigation regarding IC. Additionally, the high level of research and development expenditures, the high share of employees with tertiary education and the comparatively flat hierarchies that create and support an innovative environment are additional factors making SAP such a promising candidate for the application of IC. Given the case study design of this study, our results cannot be easily generalized [70]. However, given the careful case selection, in-depth interviews and collected secondary data, this research will be a basis for future theoretical and empirical work [73].

4. SAP’s Internal Crowdsourcing Initiatives

We observed a total of five different IC initiatives (for a summary, see Table 1). As most interviewees were not aware of any other existing crowdsourcing initiatives than the ones they were involved in, it can be assumed that the implementations described here do not render a complete picture of the crowdsourcing landscape at SAP. Nevertheless, the observed spectrum of applications ranges from internal recruitment, to crowdsourcing of sustainable business models, to competence development. For a brief overview, see Figure 2.

4.1. Initiative 1: The InnOvaTor Challenge

The InnOvaTor Challenge aims to develop essential skills of SAP employees concerning recent technological developments like AI and blockchain. The basic idea of the program is to create a competitive challenge as an alternative learning environment for innovative technologies, complementing existing training courses or virtual courses. The opportunities for competence development within the InnOvaTor Challenges go far beyond established models of continuing professional education. Instead of merely imparting specialist knowledge, IC can convey values, norms and corporate culture. Thus, crowdsourcing can be understood as a tool to change from skill-centered training to a more holistic approach to competence development [75]. Unlike most other forms of IC at SAP, the InnOvaTor Challenge is not primarily concerned with solving existing problems or creating new products or services but instead focuses on long-term and in-depth engagement with future-oriented technologies.

The program was initially launched in 2016 as a regional pilot program in Germany. However, the presentation of the final results was so successful that a global rollout took place as soon as 2017. The initiative was initially managed, planned and implemented by a single employee—in addition to their day-to-day workload. They had originally introduced the idea of a challenge-based competence development program and was entrusted with its implementation on an ad hoc basis. While initially the challenge was launched as a pure Internet of Things (IoT) challenge, this narrow focus was abandoned in favor of a broader thematic orientation that includes all business areas of SAP (e.g., blockchain, machine learning, etc.).

The InnOvaTor Challenge takes place once a year over a period of six months and can be divided into three phases. First, ideas and possible questions are collected. Subsequently, the campaign is started and advertised on the intranet. Participation is open to all SAP employees worldwide. Employees either participate in working on existing problems or enter the challenge to implement their original ideas. To incentivize employee participation, both intrinsic and extrinsic incentives are used. Purely financial incentives in the form of bonus payments do not exist. Non-cash bonuses are sometimes awarded to the winners but are not communicated a priori. The primary extrinsic motivation is recognition within the company. Winning such a challenge generates attention and strengthens the employee’s internal profile. However, the primary motivation of the participants is to learn new skills and to solve the problem itself.

In the second phase teams are formed, which are then given six months to work on implementing the idea. The participants are neither assigned exact time quotas nor explicit monetary resources. There is, however, access to cloud solutions and all other SAP products. Furthermore, the teams are connected to coaches, experts and mentors who support them with their know-how and networks. Within the program, so-called “enablement sessions” are organized in which experts give employees an insight into existing SAP solutions (e.g., SAP Hanna). In the third phase, the projects are completed and evaluated based on a final pitch during a closing event. The events were held at six SAP locations all over the world. Following the presentations, a jury decides on the final ranking. In the past, crowd voting approaches were also tested. However, it turned out that teams tended to vote strategically. This behavior might reflect the competitive nature of this initiative. Particularly promising ideas can be transferred to other SAP internal programs for further development.

4.2. Initiative 2: One Billion Lives (1BL)

“One Billion Lives” is an IC campaign initiated by SAP in 2016 under the leadership of Adaire Fox-Martin in the Asia Pacific Japan (APJ) region. The initiative aims to use the ideas of SAP employees to improve the lives of one billion people on this planet by 2020. SAP technologies are used to implement sustainable ideas with social significance that are economically feasible. To this end, SAP APJ invests 1 million EUR annually to finance internal social startups. The start-ups are mainly focused on the business areas of health and disaster management. More than 190 ideas were submitted in the first two years. The eight selected submissions range from improving the effectiveness of cancer therapies in India with the help of machine learning, to making geriatric care more efficient with the help of the Internet of Things. The program is open to all employees at SAP. What is unique about the 1BL campaign is the degree of complexity of the task. It is not only a matter of evaluating a proposal or asking for a specific customer or market knowledge but a complex creative task that ultimately encompasses the entire conception and implementation of an internal start-up.

The campaign takes place once a year. Ideas are collected, and teams formed through design thinking workshops or directly via an online platform. As with all forms of IC at SAP, the intrinsic motivation of employees is the most critical driver of participation in the 1BL campaign. The opportunity to implement sustainable ideas with social significance as part of the program is of great importance to participants. Additionally, some incentives arise from the start-up experience. Setting up an internal start-up, a separate business unit, or an independent company is a complex and rewarding experience. Often there exists a strong sense of ownership of the idea and the resulting start-up. The employees are also given a lot of freedom and support for the implementation of their idea. In addition to the positive effects on the employee’s network, the learning aspect is essential. In the workshops held, the employees learn many skills related to the founding of a company that would typically not be accessible to them. Extrinsic factors are also relevant drivers in the decision to participate in such a campaign. Winning such a prestigious challenge generates a high level of attention and strengthens the internal profile of the employee. Another incentive is undoubtedly the options that SAP provides in the event of a successful start-up. If the developed product is close enough to SAP’s business interests, it is possible that it could be included in the company’s own product portfolio. To this end, a separate business unit will be set up and the idea owners will be given the opportunity to lead. Another option could be to establish a company outside of SAP.

Ideas are selected in a three-step process. First, the best 20 ideas are chosen by experts to reduce the number of ideas to a reasonable amount to be subsequently voted on by all employees during one-week crowd voting session. Additionally, a vote among experts is carried out, who evaluate the ideas based on various aspects such as feasibility and social significance. The best 10 ideas are then selected from the combination of the two votes.

The selected teams take part in a one-week preparatory workshop to develop their ideas. Subsequently, further workshops organized by SAP.io, SAP’s internal accelerator, take place to provide teams with the necessary knowledge and support to create a minimally viable product. The workshops are focused mainly on the theory of starting a business and require the participants to reflect and further develop their ideas. Finally, the teams present their ideas to an investment committee of experts and board members, who determine the winners of the challenge.

The ideas ultimately selected receive 200,000 EUR in seed capital and become part of SAP’s internal accelerator program. The start-up capital can be used for SAP software licenses and travel expenses. If the ideas are selected by the investment committee, the submitting team can dedicate themselves to the start-up full-time. If the committee is not willing to make this high investment expenditure but is still convinced of the idea, it can still allow employees to continue working on the idea. For this purpose, employees are granted a time quota of 20% of their working time for the development of the idea after the campaign.

Spotlight—Relief.iO

One hundred forty million people are affected by unforeseeable natural disasters such as earthquakes, tsunamis and typhoons every year. If such a disaster occurs, a large number of organizations are involved on the ground to help those affected. These include, for example, non-governmental organizations (NGOs), intergovernmental organizations (IGOs), but also national governmental organizations in the affected countries.

Despite the commitment of the United Nations Office of the Coordination of Humanitarian Affairs (UN OCHA) to coordinate the work of the aid organizations involved, distributional inefficiencies regularly arise due to overlapping fields of activity and competencies.

Real-time optimization of supply chains to deliver the necessary relief supplies at the right time to the people who need them most is the biggest challenge in disaster management. This is exactly where Relief.io comes in. The internal start-up, which emerged from the 1BL campaign in 2017, provides a collaboration platform for disaster relief that can be used to make supply chains more efficient. Discussions with disaster relief workers on site revealed that mobile devices and messenger services in particular are used in the daily work environment. Therefore, an app was developed in which employees can enter field activities on site. Aid organizations are thus informed about new activities and required help in real-time and can process and coordinate them more quickly. This enables improved collaboration between field and office staff. The rapid availability of information means that on-site assistance can be provided more efficiently and, therefore, more cost-effectively.

The tool is also scalable, i.e., the tool can be used by various aid organizations at the same time. This enables collaboration between different aid organizations, strengthening their cooperation and further reducing duplication of effort using advanced machine learning and artificial intelligence tools. Relief.iO has already been successfully used by UN OCHA to support the coordination of relief activities during a dam burst in Laos in July 2018 and Typhoon Mangkhut in September 2018.

4.3. Initiative 3: Intrapreneurship with SAP.iO

Within SAP, SAP.iO represents an ecosystem that provides financial and consulting support to external and internal entrepreneurs and their business ideas. Launched in 2014 in a bottom-up manner, the program focuses on ideas that use existing SAP applications to generate innovative products. The basic idea behind the introduction of SAP.iO was to reduce the inefficiencies of a large company in product development by taking advantage of the agility and speed of start-up companies while relying on the support network and resources of a large company. SAP.iO’s Venture Studio is responsible for identifying, promoting, and financing start-up ideas by SAP employees, i.e., it represents SAP’s intrapreneurship program. As in 1BL campaigns, SAP.iO sets high demands on the creativity and motivation of employees as it requires the entire conception and implementation of an internal start-up.

There is no explicit incentive structure for the intrapreneurship program and no monetary extrinsic incentives. The employees’ willingness to participate is mainly based on the intrinsic motivation to implement their ideas and to learn the necessary skills for a successful start-up. In addition, there is the opportunity to change their fixed everyday work structures for an agile international working environment. The experience of setting up a start-up and receiving the support of a global corporation in the form of mentoring and financing constitutes the greatest incentive for employees. The entire program receives a great deal of attention at SAP, rendering recognition among peers another extrinsic motivation.

Calls for campaigns and hackathons are made public via an internal platform and participation is entirely open to all employees. The SAP.iO team evaluates all ideas and then provides feedback to the submitter. In some cases, these campaigns result in several hundred submissions. This may lead to vast amounts of work for the evaluation team since it is essential to select the right groups and to sort valuable ideas early on. Evaluation is based on the potential market, the problem to be solved, market size and the qualifications of the team. The process is iterative, designed to prevent the erroneous exclusion of valuable ideas and to make the evaluation process as transparent as possible.

For selected teams, an accelerator program, start-up financing and a mentoring program will be provided. More precisely, the successful teams are initially given some time to further develop the ideas presented, while SAP.iO offers coaching and mentoring. Subsequently, the teams participate in a six-week accelerator program. During this phase, the teams work full-time on the submitted idea. Finally, there is an investor meeting where the teams pitch their ideas. The consulting services provided by SAP.iO cover the entire start-up life cycle, from idea generation to establishment on the market. Thus far, more than 1500 ideas have been submitted. Six internal start-ups have received an initial investment of one to two million each.

Spotlight: Atlas & Ruum

Two companies that emerged from SAP’s intrapreneurship program are Ruum and Atlas. Both start-ups use SAP solutions and their unique integration to offer innovative products.

Ruum is a project management application that allows one to translate business processes into project plans. It provides a platform for task management, planning horizons, file storage and team collaboration. Artificial intelligence supports processes such as follow-ups with colleagues and creates daily status reports on milestones and essential developments in projects. Particularly noteworthy is the integration with other SAP solutions, such as SAP Sales Cloud and SAP Marketing Cloud.

The start-up Atlas provides geo-referenced data on demographic composition and customer frequency. These can be queried with a single click on a map and are intended to help companies in the hospitality and retail industries to select suitable locations for new branches. Key figures on customer frequency, household income and willingness to buy are to be calculated in real-time based on mobile phone, B2B and credit card data. Artificial intelligence and the processing of large amounts of data make it possible to compare locations and predict developments.

4.4. Initiative 4: Internal Crowdsourcing of Human Resources (HR)

IC is also used in the consulting unit of SAP to find skilled employees. There is no central platform for this, but the job posts are made available via a mailing list. Employees active in consulting can subscribe to various mailing lists and receive regular emails with offers for jobs. Members of the sourcing team share information about available positions based on the provided interests specified by employees. Hence, suitable employees for specific customer inquiries can be identified in a short amount of time at low costs. The positions advertised range from project managers to technical specialists. The e-mails are highly standardized to ensure a high degree of clarity and comprehensibility. They include required seniority, function, role description, language requirements, industry focus, the scope of the task, number of consultants needed, location and duration of the engagement.

Each employee’s profile is stored in a database. The profile encompasses competencies, a curriculum vitae and previous project experience. The information is made available to the customers before the actual job interview. The profile allows a first appraisal of the candidate based on which the HR crowdsourcer makes the pre-selection of suitable candidates.

The advantage of this type of tailored job suggestion service compared to a platform is the lower requirement for proactivity on the part of the employees. Consultancy would otherwise have to invest time to search on a platform to identify interesting and suitable offers. With the standardized mails, part of the work is already done in advance by the HR crowdsourcer and the employee can decide with minimal effort whether a job matches his skills and requirements. If an employee is interested in a specific position, he or she can simply reply to the respective mail, after which contact is established with the corresponding HR crowdsourcer.

4.5. Initiative 5: SAP Blue

SAP Blue is an IC platform for SAP India that allows organizing product development across teams and locations [76]. The platform connects owners of an idea (sponsor) with developers (crowd). In principle, any employee can submit an idea to the platform. Subsequently, developers apply and are selected by the sponsor. The motto of the platform is “build, learn, unite and enjoy” [77]. The platform was launched in August 2015. Within the first two years, 128 projects were initialized on the platform by about 100 sponsors and attracted more than 600 participants [78]. The completed projects resulted in both product and process innovations that have already been integrated into the SAP portfolio. Subsequently, SAP introduced SAP Blue in Brazil, France, the USA, and China [79]. In the future, the platform will be launched worldwide and constitute a central component of the existing idea management system at SAP [78].

The basic idea of the platform is to bring together supply and demand in one place, which enables and facilitates creativity and experimentation. Through the crowdsourcing principle, a sponsor can implement his idea despite having few resources of his own. Employees are provided with the opportunity to pursue individual interests, hone skills and share experiences. Furthermore, the projects enable them to gain experience with new technologies and realize their ideas. Thus, no additional incentive structures had to be installed to achieve a high level of participation. Many projects received their first applications from the crowd within five minutes, and usually more people applied than available positions were advertised. The platform itself is mostly self-sufficient. It is not hierarchically structured and therefore allows a free exchange of ideas. This creates a culture of curiosity, which should increase innovativeness in the long run [78].

In order to focus on potentially relevant innovations, the ideas submitted must be in line with SAP’s goals and mission [80]. Additionally, there is a team that selects the most valuable and promising ideas by evaluating whether the idea can be integrated into existing products or whether new business areas can be developed [78].

5. Lessons Learned

Based on our observations at SAP, we derive six lessons learned that can help managers to fully exploit the potential of IC as an internal open innovation strategy.

5.1. Lesson 1: Provide Dedicated Time Continuously

For employees, participation in IC represents an additional task that is often not covered by their job description and can, therefore, be perceived as an additional burden. In addition to the day-to-day business, there is often too little time to deal with sophisticated campaigns in an adequate manner. An interviewee reported that dealing with issues was only possible in times of relative low workloads when participation did not interfere with plans of direct superiors: “So we can invest 20% of our working time in the idea. That’s the theory. If there is a lot of work to do, the boss expects you to do 120%, which means the 20% is done on the side.” Due to this circumstance, work on the prototype had to be delayed during busy times. Although managers might be afraid of losing the capacity of their direct employees to IC initiatives, we argue that sufficient time resources must be allocated to IC activities on a continuous basis, and not only when capacities allow it. The same is true when additional human resources are necessary; as one interviewee put it: “It is particularly difficult to recruit people internally to do something like development. These tasks require a lot of work. [...] We are dependent on the willingness of volunteers to give us their time.” Engaging in IC represents a risk for employees because it can be associated with high opportunity costs. This happens when focusing on their everyday job is perceived as more valuable by other employees or superiors. Interviewees mentioned that engaging in IC might come off as if the employee is not working to capacity, a phenomenon that has been reported in previous studies [81]. As IC encourages and educates entrepreneurial thinking and therefore creates a more innovative working environment, time spend on campaigns should not be regarded as time wasted [37,38]. To summarize, we recommend that clear rules on time quotas with respect to IC should be set and communicated by top-management. Furthermore, the organization should clearly communicate that participation in IC is desired and might also recognize IC participation in target agreements.

5.2. Lesson 2: Selection of Solutions Should Be Made by Experts

Usually, companies either rely on a selection process based on the crowd itself or an evaluation by experts, usually resting on previously defined criteria [21,61]. On the contrary, quality assurance at SAP relies mostly on experts. Selection by the crowd is of rather small or no importance. For example, to evaluate the submitted business ideas within the intrapreneurship program, a great deal of experience and very specific knowledge is required, which can only be gained by long-term experience with the subject area. For this reason, the number of decision-makers has been steadily reduced over the years. As one interviewee stated, “Crowdsourcing of evaluations is a very bad idea in our context. It is very difficult to learn this kind of evaluation. The learning curve is relatively long. They [the evaluators they crowdsourced the evaluation to] just weren’t close enough to the process and I don’t think they have enough ownership to be able to apply hard criteria. That was very problematic. The circle of people making the selection has become smaller and smaller over time. Now it’s just our team.”

In the 1BL campaign, participants can vote on ideas, but the crowd’s vote makes up only a small percentage in the overall evaluation. Additionally, a small group of experts conducts a pre-screening of the submitted ideas, “so that the employees do not have to choose between hundreds of ideas and people do not make purely emotional choices because they do not like something or they know the person that submitted the idea. Or because the idea appeals to people’s emotions but is not feasible.”

In the InnOvaTor Challenge, crowdvoting was tested but ultimately dismissed because participants used their votes strategically for their benefit. Our interviewee stated that “it turned out that the teams coordinated their voting strategically and less objectively. This behavior reflects the fact that the format creates a certain competitive atmosphere.” Crowd voting often favors popular ideas and ideas from well-connected employees and rarely reflects the actual value creation potential of the ideas submitted [61,82]. On the other hand, expert ratings carry the risk of reducing the diversity of ideas, as they are likely to reject ideas that are unrealistic or not in line with the strategic goals of the company. Nevertheless, if companies keep these shortcomings in mind, we find relying on experts for the evaluation of submissions to be more promising, especially at the later stages of the IC process.

5.3. Lesson 3: Promote Bottom-Up Development, Demand Top-Down Support

Our case study shows that crowdsourcing initiatives seem to enjoy particularly high recognition and acceptance when initiated bottom-up. Employees may not perceive the initiatives as additional work assigned to them from management, but rather as an opportunity to explore and solve problems in their daily work. The InnOvaTor challenge, as well as the internal accelerator, were bottom-up initiatives, i.e., they were perceived and pushed by individuals within SAP. Both initiatives received significant support early on and rapidly grew to become substantial elements of SAP’s internal innovation landscape, i.e., early support is essential. This point can be illustrated by one employee that attributed the success of the initiative to the fact that they “had management support very early on. I had the freedom to do what I thought was right. I did not have to compromise on the important issues. I find it hard to believe that the same program could have been set up within a traditional business unit.” Nevertheless, such approaches might initially receive less attention than top-down initiatives like 1BL. At the same time, however, bottom-up initiatives seem to emanate a certain degree of credibility among the staff, that top-down approaches do not always have. Additionally, almost all interviewees mentioned sufficient resources as the most important prerequisite for successful IC. In particular, this concerns time, personnel and financial resources; for example, one interviewee stated, “we used part of the seed funding for the initial coding. However, it soon became clear that the money would be gone too quickly and would not be enough. […] Ultimately, of course, we are making a much slower progress than if we could concentrate fully on this. What we were hoping for was that maybe we would get a person who could work on it full time. But in the end no team got that. We were told that we simply had salaries that were far too high for us to work full-time on the ideas and keep getting our salaries.” Here, support from middle and senior management is essential [21,38,83]. Both must provide capacities and resources, encourage employees to participate and inform about possibilities of participation.

5.4. Lesson 4: Better Integration of IC into the Overall Company-Wide Innovation Process

The integration of IC into the day-to-day work of employees is of vital importance [21,68]. We found that many of the interviewed employees only know few or none of the other existing crowdsourcing initiatives taking place at SAP. We find that higher awareness, integration and collaboration of the initiatives could create opportunities for interaction and cross-pollination. Greater awareness bears the potential for learning from previous experiences and procedures that are now frequently neglected. This is especially true for large and geographically dispersed companies like SAP. For example, after our interview with a challenge winner of the 1BL campaign, they realized the potential of crowdsourcing to compensate problems of resource scarcity, which they experienced in developing their prototype: “The reason for the idea [crowdsourcing the development] is the scarcity of resources, which were not correctly anticipated in the campaign plan. The idea is now to crowdsource the individual development blocks. The roadmap is divided into small, self-contained work packages and tendered via crowdsourcing. In this way we want to crowdsource the development itself.” The existing roadmap was divided into small, self-contained tasks and crowdsourced via the InnOvaTor program. This frees up capacities within the teams, as the team’s “developers can now rather be used to integrate the results and ensure quality”. To summarize, a higher awareness of different initiatives could lead to positive externalities between the initiatives and produce synergies for employees engaging with such new forms of work.

Next to awareness, process-related barriers must be overcome in the implementation of IC. Interviewees repeatedly stated that agile working must be supported and that departments must accept that employees will be unavailable for a certain amount of time. One interviewee summarized it in the following way: “There is a whole series of hurdles on the process level, compliance level and legal level that need to be clarified. Frameworks are needed to work with such campaigns. […] Certain standard processes that do not support true agile working.” Furthermore, accounting must be flexible to permit employees to enter more elaborate programs—like an internal accelerator—that requires transitory relocation as in the case of the intrapreneurship program.

The integration of ideas and solutions generated by IC into the commercialization structure is vital [38]. Successful submitters must be informed about the possibilities of further development of their idea in the future. It must be clear what resources will be available and what strategy is in place to integrate the idea into the company’s portfolio. One interviewee stressed how important the options of developing his ideas into a full-grown start-up was for his motivation to stay on the program: “After I found out that the whole thing can also develop in the direction of a company, where I would potentially get a very good position, this of course motivated me a lot more. Of course, you also think about your own career.”

5.5. Lesson 5: Encourage Ownership of Ideas

The idea of giving employees responsibility for their tasks or projects is not new, of course, but we find it to be particularly important in the context of IC. In this way, the employee can benefit if his idea generates great interest and becomes a success. Being allowed to implement one’s idea increases employee self-satisfaction and stronger identification with the company’s goals. As one employee stated, “First [his main motivation] it was the purpose. Not that I don’t like my job or anything. But there is a difference between working for something as positive as saving people’s lives and helping some customers to maximize their profits. […] The exceptional aspect of the 1BL campaign is that it goes far beyond the idea, because in the end an independent company is to be founded. Therefore the commitment is very high and intrapreneurship is promoted. It goes far beyond having only a specific idea for improving a process or a specific issue.” However, we observed that there sometimes exists a lack of accountability and commitment. Of all employees who take part in a challenge, ultimately often only a few are fully committed, meaning that ideas are sometimes abandoned or fall short of their potential. As one participant noted, “There are 85 people who register for a challenge. Maybe 60 of them will actually take part. Only 40 of them do it seriously and 5 of them actually go through with it. But no one really cares. The worst case is, you don’t win. But there are no other consequences, no ownership.”

In most cases, participation has only positive effects for the employee. There are no obligations that would result in any responsibility. At the same time, this disadvantage is one of the great advantages of crowdsourcing, as employees can submit their ideas without fear of negative consequences. We find that responsibility also increases the commitment of employees, because they are linked to the result, either positively or negatively. On interviewee put it like this: “You have to give an incentive that offers an interesting upside but also a downside when things don’t work—a certain ownership.” Letting people implement their ideas like at SAP creates a sense of ownership that efficiently solves the problem at hand. While commitment is not mandatory, employees that are attached and invested in their ideas, as a result, want to be accountable. This effect can even be leveraged when teams are created around an idea. Ownership binds teams to the success but also the potential failure of an idea and therefore might increase the probability of success.

5.6. Lesson 6: Openness and Transparency of the IC Process Is Key

Regardless of the stage of the crowdsourcing process, the highest possible level of transparency must be guaranteed. Therefore, certain aspects of the campaigns must be defined a priori. This includes the benefits for participants, how submissions will be evaluated, what happens to the ideas and solutions submitted—especially if they are not selected—and expectations concerning the outcome [28,38,41]. Not playing by the book will always be conceived negatively by employees and might deter them from future participation. Companies should also commit to resources that will be invested in integrating successful ideas and communicate them openly from the start.

In line with earlier studies, e.g., [84], we find that providing timely feedback for submission constitutes an excellent approach to increasing the openness and transparency in the evaluation process and shows appreciation for individual participants. In the intrapreneurship program, for example, each participating team receives a short feedback to show possibilities for further development. “We communicate the results to the teams [...] where we give feedback to the teams and explain our decisions, but also give the teams the opportunity to comment. [...] The feedback is usually very well received and highly appreciated by the teams. But of course it is a very big effort.” With the help of this feedback, participants better understand the decisions and might be more willing to take them into account in further developing their idea. However, as the interviewee emphasized, the workload associated with such thorough feedback loops must be considered when planning resources for crowdsourcing initiatives.

6. Conclusions and Limitations

In recent years, IC has played an essential role in providing companies access to the innovative ideas and solutions of their employees. Intranet- or Internet-based IC platforms efficiently connect employees, building a powerful tool to create and gather ideas, new products, services and business models, and new markets—regardless of organizational or geographical boundaries. IC promotes an open, transparent and innovative corporate culture, increases the commitment and entrepreneurial skills of employees and can provide companies with a decisive competitive advantage. However, to fully exploit its potential, careful management of the IC process is paramount. Within our business case study, we identified, documented and analyzed IC initiatives and their governance at SAP. We provide a detailed description of the different approaches, use cases and put a spotlight on especially interesting success stories. Subsequently, we provided six lessons learned that will help managers to implement and execute successful IC initiatives in the future. Additionally, we theoretically developed IC as an internal open innovation practice, furthering our understanding of the open innovation paradigm [26].

The expectations regarding the success of crowdsourcing initiatives of many employees and especially management are often excessively high [61,81] due to the invested funds and the use of enormous time resources. However, the success of crowdsourcing initiatives is often difficult to quantify and many of the ideas submitted cannot be directly implemented or only to a limited extent. Companies must take into account the positive effects of IC that are not directly measurable. In addition to quantifiable factors, such as cost savings through process innovation and increases in sales through new product innovation, IC promotes a more innovative, open and collaborative corporate culture. The exchange of knowledge between employees and across intra-organizational boundaries is supported and the identification of employees with the company strengthened. Campaigns such as 1 Billion Lives can also take social and environmental aspects into account and thus can be part of a corporate social responsibility strategy. If successes, ideas and campaigns are also communicated outside the company, IC can furthermore improve the company’s image and enhance its reputation [21]. Our case study shows how SAP is able to collect ideas and create internal businesses that apply their technologies to address sustainability issues like disaster relief and food waste. Hence, we emphasize the importance of open innovation to reach sustainability goals and develop more sustainable products [30,31,32]. More specifically, we add to this perspective by showing how more sustainable ideas and business models can be achieved by applying internal open innovation principles.

In conclusion, we find that SAP has very successfully implemented a wide range of crowdsourcing solutions, ranging from crowdsourcing software development tasks, to ideas, innovations and new and sustainable business models. Across all programs, the number of participants is continuously increasing, and the concept of IC seems well accepted by employees. In addition to the positive effects of such a program on employee satisfaction, competence development and company culture, we also observed economic successes. The intrapreneurship program and the 1BL campaign have already produced the first start-ups. Further, the first products and services that were developed with the help of SAP Blue can already be found in SAP’s product portfolio.

7. Limitations

The present work has several limitations, which must be mentioned. First, the analysis is limited to a single company, which means that the results are naturally only a description of a specific case. External validity is, therefore, not given and is not intended in this case. The intention was instead to describe applied IC in a company with many years of experience to derive valuable insights. Nevertheless, we describe five IC approaches that have very specific differences in size, scope, aim and resources.

Second, it follows from the case study design that contextual factors, such as country-specific social and market structures, must be taken into account when interpreting and, in particular, deriving implications [85]. Especially in the case of SAP, industry-specific characteristics should be highlighted here. Within ICT industries, a high level of innovativeness and a high diffusion rate of social information technologies can be observed. On the one hand, this may signal that conclusions from this case study cannot necessarily be transferred to other industries without adaptation. At the same time, it may also implicate that the software industry is one step ahead of other sectors and that the derived lessons learned are, therefore, of particular value.