The Role of Digital Maturity Assessment in Technology Interventions with Industrial Internet Playground

Abstract

The use of digital technologies such as Internet of Things and advanced data analytics are central in digitally transforming manufacturing companies towards Industry 4.0. Success cases are frequently reported, and there is clear evidence of technology interventions conducted by industry. However, measuring the impact and effect of such interventions on digital maturity and on the organizational adoption can be challenging. Therefore, the research aim of this paper is to explore how the combination of the different methods of Industrial Internet Playground (IIP) pilots, Shadow Infrastructure (SI) and digital maturity assessment can assist in conducting and documenting the technical, as well as organisational, impact of digital interventions. Through an elaborate literature review of existing digital maturity assessment tools and key dimensions in digital transformation, we have developed a digital maturity assessment tool (DMAT), which is presented and applied in the paper to identify digital development areas and to evaluate and document the effects of digital interventions. Thus, the paper contributes with new knowledge of how the IIP pilot and SI combined with digital maturity assessment can support effective, transparent and documented digital transformation throughout an organisation, as explored through theory and a practice case.

1. Introduction

Numerous initiatives in industry have been put into motion to fundamentally transform manufacturing activities towards Industry 4.0. Whilst providing greater connectivity across an enterprise and due to the ability to act on production intelligence, Industry 4.0 offers endless opportunities to improve operations, create new value and respond to challenges such as climate change [1,2,3].

Today, manufacturing leaders are seeing results and gaining competitive advantages from the Industrial Internet of Things (IIoT). However, most have much work ahead of them. As stated by a report by MPI Group, only 11% of manufacturers have implemented a strategy to apply IoT technologies to production processes, and around half state that they are struggling with the basics of defining and implementing an IoT strategy [4].

The barriers to implement such strategies are recently gaining academic attention. Boyes et al. point to challenges such as ambiguous and contradictory terms, implications of working in existing operational architectures and lack of assessments of the risks in interventions [5]. The challenge of brownfield systems and retrofitting equipment is also prevalent in practice, but lacks sufficient study [6,7].

Furthermore, Pessot et al. describe the need for unique approaches for digital transformation beyond the dimension of technology [8]. Frysak et al. provide claims that companies require more support to use such frameworks. Where experts are involved, their focus on a dominant technical viewpoint results in neglecting much of the framework. With many standards integrated categorically, there is no prescriptive method or advice to work across layers, leaving it to the practitioner to devise their own methods [9].

Ferreira et al. outline challenges for organisational and technical design of Industry 4.0 architecture. The first is the ‘necessity of defining required business entities’, in relation to how they participate in the value creation process. The second is integration of knowledge from existing legacy systems not compliant with standards and reference frameworks [10].

According to Mu et al., to assimilate technologies requires broad adaptation, both of technology itself and organisation [11]. A significant proportion of companies cannot accomplish this adaptation mentioned above; therefore, many of the benefits are not achieved through the adoption of different corporate systems, as discussed by Carr [12]. Thus, effective technology adoption is a key challenge to many companies. However, Yi et al. argue that particularly human and social factors could play a role in the adoption of technology [13]. The literature on the adoption of new technology has investigated various aspects of technology acceptance and adopted multiple levels of analysis. These include among others: Theory of Diffusion of Innovations (DIT) [14], Theory of Reasonable Action (TRA) [15], Theory of Planned Behaviour (TPB) [16,17], Decomposed Theory of Planned Behaviour [18], Technology Acceptance Model (TAM) [19,20], Technology Acceptance Model 2 (TAM2) [21] and Technology Acceptance Model 3 (TAM3) [22].

Thus, technology adoption has been a core topic for years, and particularly the human and organisational side of technical interventions and digital transformation are viewed as critical for the users’ effective adoption of technology across an organisation and its employees [23]. Therefore, in this paper, we experiment with the use of two concepts, the Industrial Internet Playground (IIP) and the Shadow Infrastructure (SI), as methods (see Section 2.1) to provide an easily accessible and transparent digital intervention approach for companies undergoing digital transformation towards Industry 4.0. To evaluate and document the organisational effects of the interventions using these methods, we developed a digital maturity assessment tool that is applied to measure the digital maturity on different organisational and strategic dimensions of the company before and after the technology interventions. In the study, we detail the application of IoT proof-of-concepts (POCs) into production environments to understand and measure the impact of IoT POC interventions across different dimensions of digital maturity.

With this interdisciplinary study, we explore technology adoption (specifically IIoT) using two different methodologies from two different research disciplines (engineering and business social science) in one case. Thus, the research aim of the study is to explore how the application of IIP and digital maturity assessment in combination can ensure more effective digital interventions with faster progression and documented effects on the digital transformation of organisations.

The theoretical contributions of the study show how to apply participatory design in combination with Industry 4.0 reference models, such as the Industrial Internet Reference Architecture (IIRA) [24] and Reference Architectural Model Industry 4.0 (RAMI4.0) [25]. The theory indicates an effective way to initiate IIoT technology adoption and digital transformation. Specifically, by framing requirements elicitation with the constraints of a pre-defined data taxonomy in a deployed IIoT infrastructure, it supports prototyping POCs closer to the design requests of stakeholders, in turn resulting in organisational understanding of the technologies introduced and creating strategic imperative.

In addition, this study contributes to our existing understanding of digital transformation by providing new knowledge of how to evaluate and document digital transformation from a technical and organisational side. This is done by (1) providing a new digital maturity assessment tool (DMAT) to be applied as a framework in supporting the organisational adoption of technologies during an intervention processes, and (2) showing, via a company case, how such a tool can be applied in leveraging technology adoption and increasing technological and organisational impact.

The empirical contributions reveal, through developing three IIoT POCs with the IIP method, how SME manufacturers can initiate digital transformation, practically, through interventions in brownfield production environments, which contribute to technology adoption at an organisational level.

Finally, the study contributes with empirical knowledge of how to apply digital maturity assessment tools in practice in organisations to identify digital development areas before interventions and in evaluating and documenting digital maturity after interventions. With a case company, the process and application of the methods are illustrated with managerial implications of how to work actively with digital maturity assessment and IIP for more effective technology adoption.

2. Materials and Methods

The methods used are based on the DMAT and IIP Intervention methods, as depicted in Figure 1 below.

2.1. The Industrial Internet Playground (IIP)

The Industrial Internet Playground (IIP) is a facilitated innovation program for Industry 4.0 pilots in manufacturing companies. IIP was created by Aarhus University and PulseLabs and first applied in the case study of this paper. The pilots follow a service-designed program, typically hosted over eight-weeks and optionally repeated with a new topic, with regard to a problem or technology focus, or group of participants which may include end users, managers, customers or another group of stakeholders.

The IIP pilot is framed around a methodology combining several distinct methods for applied innovation. The methods include: (1) IoT focused context mapping, ideation and co-creation [26]; (2) agile development and deployment for design and prototyping [27]; (3) and evaluation episodes [28], supporting the transition from lab to field-tests with artificial and naturalistic activities.

Each IIP program iteration follows a four-phase project-lifecycle, as presented in Figure 1 and further explained below. The activities carried out in each phase come from their respective focus discipline, which include (1) co-creation, (2) agile, (3) dev ops and (4) design evaluation. The specific methods used are dependent on the needs of the intervention and specific iteration, which are selected on a project basis. Iterations is treated as an independent project which begins with an IoT focused co-creation workshops to explore problems and design a solution, followed by agile development of designs with the result of working POCs that can then be deployed to the operational SI and evaluated against initial design criteria during a typical two-week trial with the different stakeholders.

The pilots may take different forms in terms of time spent on each iteration or technology resources made available for the pilot. They do, however, always begin with the deployment of SI solutions, one or more program iterations and a conclusion step to determine of POCs should remain operational.

The development of IIP responds to and builds upon the results from participatory-design methods found in (however not limited to) the domain of Smart City, such as Living Labs [29], Open Innovation [30], and co-creation. Specifically, the lack of attention to apply these in an industrial context was a particular driver for IIP. For example, the FormIT model by Stählbröst [31] is a well-referenced methodology, but lacks technical competencies for industry uses. Corallo et al. attempt to extend FormIT as a Living Lab for the industrial sector [32]. However, the concept is only in ‘planning phase’ with no empirical contributions or results from manufacturers. Lastly, Siemens offered the ‘LivingLab for process industries’; available references, however, indicated no methodology or results [33].

In contrast, IIP has prioritised a focus on the needs of industry sectors, integrating topics from reference frameworks such as IIRA, RAMI4.0 and the IIoT Analysis Framework [5]. No work existed at the time of starting the IIP initiative to apply participatory design with industrial reference frameworks.

2.2. Shadow Infrastructure (SI)

During the program, a Shadow IT [34] infrastructure is deployed, whereby an IoT middleware and a network is locally installed at the pilot company. It allows for low-risk interventions in the production domain, using the retrofit sensors, devices and monitors, with remote update capabilities, to completely avoid machine-integrations and physical access during operations once deployed.

The SI is crucial to allow for cyber-physical system (CPS) interactions without disruption to the company’s operations. This gives common access allowing integration of a variety of developed POCs, production assets’ data and functionality, cloud services and operators in a specific industrial scenario.

Several template applications are offered in the program with libraries for microcontroller input/output, hybrid web app for tablets, mobiles and wearables, as well as an extendable web app for reporting live and historic data visuals. Solutions such as uptime reporting, abnormal vibration detection and bearing analysis are used as demonstrators in the program with the initial devices installed. Beyond these components, additional system features are integrated in each iterative program.

An IIP SI architecture model (Figure 2) provides complete function for use cases requiring data acquisition at varying sampling rates, from battery or mains powered sensors, from basic input/output sensors to video/audio, as well as user interfaces for operator interaction. The implementation SI architecture model has resulted in a complete cloud based IoT platform with a modern and scalable ecosystem for apps, analytics and data management. The SI deployed in this study is the first instantiation of the SI architecture in an IIP pilot.

Before deploying SI, a value-stream map of manufacturers’ operations is used to identify key assets for initial installation of retrofit machine monitors, environmental sensors and wearables to collect data on machine and operator performance. Furthermore, data loggers are available for standard serial interfaces (such as RS485), for internet-protocol enabled systems with appropriate application programming interfaces (API) and Cloud-to-Cloud Open API adapters for integration with third-party solutions a company may use, such as Microsoft PowerBI. The SI software implementation uses container microservices to host the core app and API, supporting synchronous and asynchronous functionality communication and event-driven programming scripts.

A general purpose IIP data model (Figure 3) represents domain entities for environment, actor, assets and product, stream and event data formats for sensing and actuation, and, lastly, classes to represent the components of SI required for management and services. The IIP data model is the result of many iterations to support a schema that can easily model the interactive stakeholder context map activity whilst mapping devices and data, which can be transferred to the technology implementation of a co-created concept.

The data model is used for resource identifiers across the SI implementation giving a common structure amongst different protocols for persistence, views, read, writes and service actions.

Table 1. Example of domain entity taxonomy used in co-creation workshops mapping to technical infrastructure protocols and URL structure, source: Own research.

UI view	https://{{shadow-infra-1-host}}/app/<domain>/<environment>/<agent>/<construct>
REST endpoint	https://{{shadow-infra-1-host}}/api/<domain>/<environment>/<agent>/<construct>
Websocket URL	wss://{{shadow-infra-1-host}}/<domain>/<environment>/<agent>/<construct>
MQTT Subscription Topic	subscribe <domain>/<environment>/<agent>/<construct>
Database collection	mongodb://<domain>/<environment>/<agent>/<construct>

provides an example on how the data model is readily visible across the different protocols of the SI, which has been attributed to support faster development efforts between platforms.

The combined use of techniques and technologies brings several benefits to IIP facilitated pilots: (1) relative cost-effectiveness compared to complete Industry 4.0 pilots, (2) individual focus on manufacturers’ value stream processes through co-creation, (3) de-risks intervention efforts on production operations through the dedicated SI, and, finally, (4) ability to rapidly develop and integrate (digital) technologies on stakeholder request.

The results of an IIP program consist of a co-created design specification, functional prototype and results from a production trial where operators can evaluate and decide whether the solution has valid applications in day-to-day work. IIP programs are concluded with a company decision to commission POC solutions, continued iterations or stop the intervention and remove all traces of SI.

2.3. The Digital Maturity Assessment Tool

In establishing a proper methodology to identify digital development areas and to evaluate and assess the organisational effects of the IIP project and all other types of digital interventions, the Digital Maturity Assessment Tool (DMAT) was developed (Supplementary Materials) and applied in the context of the selected case company to show how digital maturity assessment can assist in support technology adoption and implementation.

Generally, maturity models are applied in providing a normative description of practices in each area and dimension, building a ranked order of practices (i.e., from low to high maturity) [35,36,37]. While maturity can be captured qualitatively or quantitatively in a discrete or continuous manner [38], most maturity models are based on a scoring method for maturity assessment, which is subsequently defined to identify the criticalities in implementing the digital transformation and to subsequently drive the improvement of the entire system [35].

Today, maturity models have become a widely established management instrument to conceptualise and measure the maturity of an organisation, a functional entity or a process regarding some specific target state [39]. Multiple consultancy companies have developed a corresponding framework to measure digital maturity, including KPMG, McKinsey, Boston Consulting Group and Capgemini Consulting [40,41,42]. The application of maturity models is not limited to any particular domain [36], and they can be used both as an assessment tool and as an improvement tool [43]. The challenge with existing consultancy models for digital maturity assessment is that they are not theoretically derived and the methodology of how they have been developed is not described or validated.

Existing studies provide evidence that firms with higher digital maturity earn superior corporate performance [44]. However, in leveraging superior performance through digital interventions, there are still many challenges for companies to overcome, which can be classified into:

• Leadership (difficulty in creating urgency, vision and direction for the digital transformation)
• Institutional (resistance to change in the form of attitudes of old employees, legacy technology, innovation fatigue and politics) [45].

The concept of digital maturity can be divided into:

• Digital capabilities (e.g., strategy, technological expertise, business models, customer experience)
• Leadership capabilities (e.g., governance, change management, culture) [46].

In developing a new tool for assessing digital maturity in digital transformation and digitalisation processes, we conducted an extensive literature review of recent literature and established consultancy frameworks on digitalisation using a systemic literature review following the methodology approach by Fink [47]. Thus, the DMAT and the six dimensions were theoretically derived, statistically validated and launched in June 2020. The tool has been further validated through the data captured from over 500 companies worldwide and across sectors that have applied the DMAT tool to access their digital maturity.

The key dimensions of digital maturity critical to an organization’s digital maturity were identified through the study and constitutes the following six dimensions: Strategy, Culture, Organisation, Processes, Technology and, finally, Customers and Partners. Each of these dimensions were statistically validated to play a critical role in a company’s digital maturity and ability to digitally transform, and thus the company’s ability to adopt technology.

The summarized results of the literature review of the six critical dimensions of digital transformation can be viewed below.

• Strategy

This dimension is explained as the company’s digital business strategy, which constitutes a pattern of deliberate competitive actions undertaken by a firm as it competes to offer digitally enabled businesses, processes, products, and services [48]. The study by Kane et al. consistently found that strategy is the strongest differentiator of digitally maturing companies [49]. In the 2017 study by MIT Sloan Management Review and Deloitte of more than 3500 business executives, managers revealed digital strategy as a key denominator in digital transformation, together with the willingness to commit resources to achieve a digital vision [50].

2.

Culture

This dimension constitutes the digital culture of the company and can be seen as an emerging set of values, practices and expectations regarding the way people act and interact digitally and within the contemporary network society in business and as individuals [51]. Kane et al. continue to place the role of humans, organisational culture and the need for formal strategic planning at the heart of successful digital transformation initiatives [52]. Thus, actively facilitating a digital culture that is conductive to broad and constant learning, radical change and fundamental innovation is critical to digital transformation [53].

3.

Organisation

This dimension incorporates the digital organisation and refers to how an organisation organises and applies their competences to adopt to the digital transformation and how to integrate digital business development more effectively throughout the organisation [54]. Thus, digital transformation requires workforce transformation [55]. With the increasing importance of big data in digital transformation, organisational and business structures must develop based on the potential to develop new value streams based on new data processing solutions [56].

4.

Process

This dimension addresses the digital processes, which include the existing and new routines and processes developed by the company to gather, analyse and apply data throughout the business and its processes more effectively [57]. Businesses embarking on digital transformations need to acquire and build big data analytics capabilities and fundamentally transform their decision-making processes. To adopt and assimilate big data analytics requires transformations regarding structure, capability, culture and procedures across the entire organisation [58].

5.

Technology

This dimension refers to the various combinations of digital technologies (e.g., IoT, machine learning, AI, VR/AR) that companies include in their business, processes, products, services and digital business development [59]. Digital transformation is based on direct and indirect effects of the application of digital technologies and techniques [60]. The combination of new technologies with innovative methods of data processing and analysis not only improves and disrupts existing business processes, but also enables completely new business models and markets [61].

6.

Customers and Partners

This dimension includes the ways and activities planned and carried out to involve and engage customers and other partners and stakeholders in the digital business development across the company’s value chain and ecosystem [62]. In a recent study, 75% of executives indicated that their competitive advantage is not determined internally, but by the strength of partners and ecosystems they choose to work with [63]. Customers play a key role in digital transformation as digital technologies allow consumers to co-create value, e.g., by designing and customising products [64,65,66].

3. Results

The acoustic panel manufacturer case involved an SME manufacturer of a popular acoustic panel. A company with their core business focused on a high-volume low-mix product with additional customisation of paints and profiles. With increased customer demand and continuous growth, the firm had recently invested in a second, modernised, production line, theoretically doubling manufacturing capacity. Although large investments were made in assets (automation equipment, articulated robots, etc.), challenges in operational processes and technical aptitude remained. The company searched for solutions using digital technologies and data-driven processes to improve performance as well as encouraged an organisational shift to approach and solve problems using data. It was agreed that three IIP program iterations would be carried out exploring the potentials of IoT technologies in three different contexts in the production environment.

Prior to the IIP intervention, the company was assessed with the DMAT to measure digital maturity on the six dimensions. The first assessment was based on the answers from CEO, one of the two Production Managers and an internal Lean Specialist. The result of the assessment served as a guide for IIP facilitators to focus efforts based on the baseline report provided by the DMAT. Each of the six dimensions were measured on a scale from 1 to 5 and was based on an average score of 3–5 Likert scale questions. A score between 1 and 3 equals a low level of digital maturity, with a mean of 3. Thus, a score between 3 and 5 equals an above average to a prominent level of digital maturity. The results of the CEO’s answers before assessment are displayed in

Table 2. DMAT results before the IIP intervention, source: Own research.

1.  Strategy	2.  Culture	3.  Organisation
3.25	3.20	3.40
4.  Processes	5.  Technology	6.  Customers and Partners
3.25	2.75	3.25

below.

The radar chart below in Figure 4 illustrates the digital maturity on each of the six dimensions assessed through the DMAT by the CEO. The findings revealed that the case company’s digital maturity on all six dimensions was assessed as average (close to 3) with Technology on the lowest score (2.75) and Organization on the highest score (3.40).

The IIP was presented to the stakeholders at a kick-off meeting in May 2019, and the program iterations ran during the period from the end of August 2019 to August 2020. The finalisation of the last iteration was prolonged by the COVID-19 situation, as the production crew was burdened by additional compliance, and there was limited access to the production site.

For each program iteration, the relevant resources (template applications, preconfigured IoT devices, developers) were organised and made ready based on company interviews, value stream mapping, initial DMAT results and previous iterations if carried out. The aim was to customise workshops, reduce the IIoT topics and ensure that the program was building upon relevant problems identified by the stakeholders.

The iterations focused on different topics with a range of technologies applied in the production environment.

Table 3. Summary of IIP developed proof-of-concepts, source: Own research.

Iteration	1	2	3
Topic	Proactive maintenance	Quality control	Operations
Problem	Maintenance call out is slow, reactive and not trackable. Past operational data is not easy to access and understand by maintenance	QC station is manual, and no data is recorded about scale or frequency of defects encountered	Acquiring and correlating defect data with mixing process data from ERP system is too time consuming to react in a timely way
Solution	Linewatcher—A wearable device operators can use as an ‘andon’ notifier of production start, stop and support requests	Defect tracker buttons—Programmable buttons to log defect type and timestamp	Line integration PLC microservice—Integrate PLC data from moisture, line speed and stoppage reasons to indicate defect causes
Unique technologies	Android wearable app Live dashboard	Particle Argon microcontroller Live Dashboard Historic Dashboard	PLC telemetry translator Live Dashboard Job and Shift Scorecard
SI components	Hybrid web-app REST API Smart TV	Microcontroller library REST API Tablet Scorecard analysis	Microservice MQTT API Tablet Scorecard analysis
Ability to reach users’ needs	No—wearable was too unnatural to wear and did not provide enough detailed input options	Yes—accepted and used beyond the trial period to understand current issue	Yes—accepted and used in complete correlation dashboard monitor

provides a problem-solution mapping of each iteration and the instrumental components used from the deployed IIP SI.

The result of the three iterations led to a functional defect tracker and programmable logic controller (PLC) integration, resulting in a performance correlation system for surface and structural issues in the acoustic panels. Collectively, they offered functionality to support the operations team with weekly report calculation (metrics for performance via uptime monitoring and quality via the defect buttons). The overall components of the SI and developed POC interactions between each can be seen in Figure 5.

A view of the key elements and increase in sophistication for each iteration is presented in Figure 6. Using agile development, the means of prototyping in each iteration was not predetermined, but evolved with the project needs. Some solutions remained actively used, even taking on new functionalities, whilst others were rejected for low impact once evaluated on the factory floor. Figure 6 also outlines the IoT devices applied, data sources and data views made available to the operators and managers at the manufacturing site, as well as the user interface method.

4. Analysis and Discussion

Through this case with the SME acoustic panel manufacturer, the DMAT and IIP were used together in an intervention with applied, theoretical and value creating implications. The IIoT technologies applied allowed the manufacturer to view and merge data across machines, environment and manual operator actions, enabling consistent analysis and correlation across operations. This provided unseen insight into overall performance of the production. It further contributed to significant outcomes in terms of enabling data-driven decision-making, as well as a large potential for enhanced quality management, leading to a reduction in previously untraceable defects, quantification of waste and evidence for informed product recipe improvements.

Table 4. Perceived effects and value created based on interviews with key staff sections, source: Own research.

Key Effects and Value Created in Staff Sections
Production Operator Teams	Production Manager Team	Technical Team	C-Level
A more comprehensive understanding of the overall impact of the specific quality processes for which each operator is in charge. Easier monitoring of product up-time and defects, due to the automated button registration system of manual visual defect/error detection. Team leaders’ understanding of the production line settings and effects is supported by the live visualisation of the production line in the hybrid web app.	Ability to analyse abstract machine data for trend identification in operational processes and procedures, e.g., for forecasting. Overview of production efficiency with a benchmark graph, which continuously detects product up-time and defects, compared against the current recipe. An eye-opener that showed more production line errors than previously assumed. Ability to compare the operators’ button-registered defects in the production line with the overall number of rejects in the final quality control of the produced batch. Cause and effect insights through the automated visualisation of the correlation between environmental factors (telemetry data from sensors) and manual action factors (e.g., error detection buttons). Team meetings and planning are now supported by the live visualisation of the production line in the hybrid web app.	Knowledge of basic ways to collect data, connect machines and users with IoT smart devices and the types of data that are collectable. Objective insights into process changes and machine configurations that enable maintenance based on data. Data to begin analysis of the raw materials affecting the processing and mixing parameters on the end quality. Enable operators to adjust configurations in an informed way. Closer collaboration with machine suppliers based on a joint interest in making equipment data an asset.	Data insights that are valuable when approaching new international partners and customers. Data-driven strategic decision-making becomes a reality, as no subjective/approximate estimates affect the calculated figures.

below shows the perceived effects and value created during the period of the IIP intervention. The table summarises the experienced effects and the value created in different staff sections based on the interviews with the CEO, technical manager, production manager and external technical specialist during the interviews in October 2020.

The organisational impact of the technology interventions was documented and evaluated through a DMAT assessment after the final IIP program iteration. The results of the CEO’s answers before and after assessments are displayed in

Table 5. DMAT results before and after the IIP Intervention, source: Own research.

1.  Strategy	2.  Culture	3.  Organisation
Before 3.25 After 4.00	Before 3.20 After 3.60	Before 3.40 After 3.80
4.  Processes	5.  Technology	6.  Customers and Partners
Before 3.25 After 3.50	Before 2.75 After 4.00	Before 3.25 After 3.50

below. Findings from this comparison revealed an increased level of digital maturity on all six dimensions. However, the improvement was particularly articulated in the Technology dimension, which had increased from 2.75 to 4.0. Additionally, the Strategy dimension had improved from 3.25 to 4.0, which was surprising, as no workshops or interventions had been carried out in relation to strategy development as part of the IIP program. Thus, and as expected, the IIP intervention did not just have an impact on the technology of the company, it also had an impact on strategy, the organisation, culture, processes as well as customers and partners. This indicates that digital interventions using IIP and SI may allow for more effective adoption of the technology due to a more accessible and transparent method. Additionally, the application of DMAT as a method in the process made it easier to document and communicate the human and organisational impact of the technology intervention. In addition, the assessment had increased the employees’ understanding of the company’s digital intervention and digitalisation by inviting key stakeholders across different departments to carry out the DMAT assessment. With the enhanced digital capabilities brought by the IIP intervention, data-driven decision making and insights for the further digital transformation journey were developed. Finally, through the mini report derived by the DMAT, key digital development areas were identified and explained, and a sector-benchmark was provided, enabling the company to conclude on digital competitive advantages that may serve as future avenues of development for the company.

Figure 7 below illustrates the radar diagram with the CEO’s DMAT assessment before and after the intervention of the IIP program. The time between the DMAT before and after assessment was 11.5 months.

IIP as a facilitated method was used to develop three distinct POCs using a repeatable program format. The resulting POCs were interoperable via the common SI architecture and data model, which were combined and commissioned as a complete reporting solution for the company. Furthermore, post-intervention interviews with the case company stakeholders and IIP facilitators were analysed to understand how the six digital maturity dimensions changed during the intervention.

Table 6. Technology-related results of the IIP on the six dimensions, source: Own research.

Dimension/Change	Company’s Influence on POC	Direct Technology Change	Impact of Intervention
Strategy +0.75	Low digital capabilities resulted in weak influence over the POC directions. No aligned vision across stakeholders	No immediate change measured	Raised awareness of complexity in systems (production, support, IT) and the scale to which change must be considered
Culture +0.4	Digital culture was minimal with the result that user experience became a priority to reduce “data overload” for users	No immediate change measured	Increasing operator willingness to contribute across intervention and POC acceptance indicates change, but sustainability is unclear
Organisation +0.4	Stakeholders understood weak points and problem areas, which resulted in a common requirement for each POC	Potential new way for the production team to carry out weekly meetings and root-cause analysis	Multiple departments worked together to carry out intervention; however, final artefact benefited only one team
Process +0.25	Critical importance due to realisations of bottleneck issues and clear wasteful manual methods documented via co-creation	Introduction of data logger buttons, user interface screens, IoT Platform and new integrations with PLC	Complete digital solution with data-driven reporting was devised to replace subjective configurations to processes
Technology +1.25	Good level of digitisation knowledge (automation, sensors), low digitalisation (IT/IS). Designs were limited to support spreadsheet and web browser formats for accessibility	New addition of sensing, user interfaces and data-driven dashboards to operations	New technologies introduced encouraged digitalisation of existing process and offered examples and showed potential for new methods (e.g., proactive maintenance)
Customers and Partners +0.25	Multi-stage production relied on a range of international suppliers, yet very little was fully digitalised and ready to be integrated with SI	New integration between PLCs and IoT suppliers required clearer roles and responsibilities than what the typical way of working offered	Allowed one supplier to develop a new feature in their product. Case company found that this sort of innovation benefited the supplier more than them

synthesises these findings based on initial company influence, IIP impact and technological implications.

The findings outlined in Table 6 indicate a positive change across each of the dimensions. The degree of operational efficiency gains is expected to show benefits; however, it is too soon to quantify in detail the amount of waste items reduced, and it would require further study.

For IIP, this case provides a unique opportunity for analysis of the theories and literature on which it is based. The use of participatory design methods [28,32] combined with relevant technological expertise of Industry 4.0 best practices [5,24,25] has proven successful for an SME manufacturer to improve operations whilst simultaneously contributing to strategic efforts to improve digital maturity.

The overall outcomes validate the benefits of using a SI reference architecture model, IIP data model and SI implementation in digital interventions. The combination of these three artefacts provides an effective bridge between industrial knowhow of the selected reference frameworks and the required guidance from participatory design approach. This makes it not only useful for industrial sectors, due to incorporating appropriate reference frameworks, but makes digital interventions more likely to succeed by grounding the company’s (and stakeholders’) knowledge of their own legacy systems, processes and business entities at the forefront of the participatory approach. This appears as a clear solution to the challenges discussed by Frysak et al. and Ferreira et al. [9,10].

Furthermore, we observe specific gains of using a general data model of ‘domain entities’ between the explorative context-mapping activities in co-creation and hands-on implementation via SI, which was instrumental in the success of the intervention. This is by far the greatest perceived benefit of using SI for digital interventions involving CPS.

The literature on Shadow IT provided a foundation to explore and to extend the concept in order to consider not only IT systems, but also CPS in an industrial context, leading to the Shadow IoT Infrastructure. This goes beyond the Shadow IT to include interactions with existing physical machinery, operators, operational processes and inclusion of novel IIoT technologies that reduce the barriers to acquire data and interact. We emphasise that ready to use, retrofit and non-evasive devices are a core characteristic of a SI. Wherever it is possible, it is suggested that direct integrations, which could directly alter the production behaviour of a machine (such as software or firmware) should be avoided during interventions. The data collected from any retrofit sensor or system should always come with insight on the quality, accuracy and relevance before operators are prompted to act on production systems that could potentially cause damaging effects.

5. Conclusions

This paper presents a new tool for digital maturity assessment while using IIP as a key enabler for IIoT interventions. The theoretical contributions of the paper are the exploration and further development of existing methods for technology adoption and existing theories and frameworks for digital transformation and digital maturity assessment. The empirical contributions lie in revealing how a DMAT and IIP intervention can assist companies in unleashing the potentials of their digital transformation across six different dimensions. The managerial implications underline the potentials and approaches of using DMAT and IIP in enhancing technology adoption and performance.

However, the paper also has limitations, which provide venues for further research. For one, although the DMAT has been tested in over 500 companies, maturity assessment is still a self-assessment process influenced heavily by the companies’ own digital self-perception. Second, the IIP methodology is yet untested in a wider technological context, but promising results of this applied case provide a good basis for further research. Finally, the study is not longitudinal, so elaboration on the performance and long-term impact of applying the frameworks is not possible, yet is very interesting and a venue for further studies.

Both IIP and DMAT were developed independently and function individually. The use of DMAT in other technological interventions also appears promising whereas IIP may be used in other contexts related to digital transformation in general.

Combining methodologies from engineering, computer science and social science into an interdisciplinary approach is very challenging. At present, we are successfully leveraging participatory, agile and engineering elements in developing IIP interventions. The addition of DMAT as an assessment methodology shows promising results in refining this approach; however, further work is needed.