Farming smarter with big data: Insights from the case of Australia's national dairy herd milk recording scheme
Introduction
The application of big data in smart farming has been a focus of recent scholarly interest. Big data, the collection, aggregation and analysis of large volumes of data and information sets (Bronson and Knezevic, 2016), is suggested to revolutionise timely decision making on farm and realise large gains in productivity and profitability. On-farm application of big data could generate up to up to AUD $19.1bn of additional value across the Australian agricultural sector (Perrett et al., 2017) through capture, discovery, and analysis (Kamilaris et al., 2017) in a way that allows farmers and related organisations to extract economic value from data. However, to maximise the benefits of big data necessitates large investments in data storage and processing infrastructure. Big data analysis includes practices that advance traditional analytics through the amalgamation of real-time, multi-sourced information – derived on and off-farm – providing farmers better opportunities to diagnose risks, identify alternatives, and project future consequences (Lioutas et al., 2019). Smart farming involves the enhancement of existing management tasks and decision making in a specified context, situation and/or location (Kamilaris et al., 2017). In smart farming, digital technologies assist in the continuous monitoring and measurement of the physical environment. This results in large quantities of data – often from heterogeneous sources - requiring large-scale collection, storage, pre-processing, modelling and analysis. Recently the term ‘digital agriculture’ has been used to encompass the concepts of smart farming, precision agriculture, decision agriculture and Agriculture 4.0 (Klerkx et al., 2019).
There are a range of impediments for farmers to realise the benefits of digital agriculture. Constraints surrounding data sovereignty and the socio-ethical dimensions of use of data include; a lack of structure and governance related to agricultural big data (Kamilaris et al., 2017), data collection and use arrangements which favour multinational commercial interests in agriculture over user capability and autonomy (Boyd and Crawford, 2012; Bronson and Knezevic, 2016; Wolfert et al., 2017; Carolan, 2018) and inequality amongst beneficiaries from big data (Carbonell, 2016; Eastwood et al., 2017b; Fleming et al., 2018; Zhang et al., 2017). Big data can also disrupt long-standing relationships between actors such as farmers and agribusinesses or supply chain organisations (Bronson and Knezevic, 2016). A lack of trust between farmers (as data contributors) and those third parties who collect, aggregate and share their data has been noted(Wiseman et al., 2019; Klerkx et al., 2019). Public-private partnerships (PPP) are suggested as a mechanism to balance public and private interests in progressing agricultural innovation more broadly (Hermans et al., 2019). However, the sheer scale of private interests pushing proprietary big data platforms and solutions to farmers is a challenge for the organisation of beneficial digitalization in agriculture (Wolfert et al., 2017; Carolan, 2018).
Secondly, numerous ongoing technical challenges have been reported (Wolfert et al., 2017 provide a comprehensive review) which also impact farm interest and adoption of big data applications. Technical challenges include: data interpretation at scale; data integration; reliable infrastructures to collect and analyse big data; human resources and expertise; and a need for new business models that are attractive to solution providers while enabling a fair share between different stakeholders (Kamilaris et al., 2017 pg 29; Wolfert et al., 2017). The cost of smart farming technologies, ease of use, lack of compatibility between devices and the availability of impartial advisory services have been identified as key farm-level constraints (Tey and Brindal, 2012; Pierpaoli et al., 2013; Eastwood et al., 2017a; DEDJTR, 2018). Better support for learning is important for progressing from data to practical farming decisions (Evans et al., 2017) with digital literacy and the skills and capacity of some groups of farmers to manage digital tools reported as impediments (Pierpaoli et al., 2013; Carolan, 2018; Fleming et al., 2018). The need for incentives to increase farm exposure and transition in digital agriculture, particularly first-time users, has been noted as an important support to adoption (Tey and Brindal, 2012; Yigezu et al., 2018). Evans et al. (2017 p79) suggest information and services will need to be provided flexibly, and at varying frequencies for ‘different user segments’. The lack of proven results in transforming data to better decisions, and the focus of many digital technologies on measurement not decisions is a key concern amongst farmers in the European Union and Australia (Jochinke et al., 2007; Zhang et al., 2017; Fleming et al., 2018; Knierim et al., 2018)
With few proven applications of big data, this range of impediments represents large uncertainties and risks for farmers so it is conceivable to question why farmers would engage with big data. Regan (2019) argues that ‘risk’ in big data applications is ambiguous and perceived differently amongst different actors. They suggest risk management needs to be discourse-based and negotiated collaboratively over time. This requires collaborative governing. Because farmers are citizens (Carolan, 2018) and/or co-creators (Lioutas et al., 2019), we need to consider ways to involve them in governing, Ayre et al., 2019 argue that digital agriculture requires harnessing and mobilizing diverse skills, knowledge/s, materials and representations for translating digital data, digital infrastructure and digital capacities into better decisions for farm management. The role of farm advisors in big data applications has received increasing attention. The importance of back-office advisory roles and how these will need to adapt with the demands of big data has been highlighted by Eastwood et al. (2019). They suggest new roles will emerge beyond information gathering, to include remote data interpretation and sensemaking, as well as roles for technology suppliers as specialist advisors at the technology and farm management interface. These intermediary roles are suggested to continually adapt to enable greater value from data-driven smart farming to be captured by farmers.
Addressing the gap between opportunity and reality of big data application for farm decision making has been suggested as a key area for scholarly concern. Carolan (2018) calls for multiple understandings of agri-digital applications with attention to the interdependencies between individual and collective arrangements and the implications for farmer autonomy and capability. While Wolfert et al. (2017) called for social research related to practical and commercial implications of big data developments, including organisation and governance issues. Bronson and Knezevic (2016) suggest systematically tracing the digital revolution in agriculture as an important goal for big data scholarship. To generate value from big data applications at farm level, Lioutas et al. (2019) emphasise the importance of understanding how farmers interact with data and their motivations in doing so. Primarily it is argued farmers' engagement with big data is about improving farm decision making, hence a need to follow farmers' decisions though case studies.
By extension, current farmer involvement in big data applications and use of data to make informed decisions and the institutional arrangements that have supported or hindered this engagement can feasibly provide insight to these gaps. We use the case of farmer participation in dairy herd recording in Australia to: 1) examine the use of big data to add value to farm decision making; and 2) explore factors and processes, including institutional arrangements, which influence farmer engagement with and use of big data.
With a history spanning more than a century, dairy herd recording (herd testing or milk recording) is one of the oldest examples of digitalization and big data in agriculture, thus it lends itself well as a case study. A herd test captures information about milk volume and quality attributes of individual cows through the collection (and analysis) of individual milk samples (Andrews, 2016). Milk sample analysis and data compilation is undertaken by herd test organisations (HTO); intermediary organisations with multi-faceted roles in the data pipeline (Fig. 1). Characteristic of big data analysis, HTO amalgamate herd test records with data captured from other sources on-farm (Fig. 1). Reports and summary statistics generated can be used to assist on-farm decision making in areas such as cow health, milk quality and feeding (Zottl et al., 2015). With one herd test milk sample capable of generating 899 spectral data points and ~20 milk parameter estimates, dairy cows accumulate hundreds of pieces of information within a year, illustrated in Fig. 2. Nationally, the Australian dairy industry's central data repository has close to 200 million herd test records and is growing by 2.9 million herd test records per year (DataGene Ltd, 2018). HTO also play a role in the aggregation of data from multiple farms for submission into this database (see Section 4.1.1) where it is used collectively for regional reporting, research and national herd genetic improvement. Such data sharing arrangements between HTO and organisations undertaking genetic evaluation services are seen world-wide. These relationships enables additional value from data, without creating extra work on-farm (Zottl et al., 2015).
The paper is organised as follows. First the conceptual framework for the study is presented, followed by a description of the case study methodology and details of the studies conducted. This is followed by results and discussion.
Section snippets
Conceptual framework
To address the research questions, we required an analytical framework that theorized both farmers' engagement with and perception of value in big data, including the role of intermediaries such as farm advisors, and the institutional arrangements that support farmer engagement in big data applications. We drew on theories of farm decision making, intermediary practice and collaborative governance.
Method
To trace the development of herd recording as an example of farmer engagement in big data applications, a case study methodology was chosen. Having the national dairy herd recording system as the focus of the case study allows an in-depth appreciation of farm decision making and collaborative governance relating to big data in a real context. Whilst there are different types and uses of case studies in scientific inquiry, in this study we chose herd testing as an ‘instrumental’ case (Stake, 2005
Big data emergence & a history of collaborative investment and adaptation
Within the evolution of herd recording over 100 years, three key processes are identified as influential in the continuation of herd recording: continual innovation in data collection, analysis and reporting practices; strategic cycles of government investment; and collaborative effort from major stakeholders to develop a sectoral strategy and implement governance arrangements summarised in Table 2. Fig. 5 outlines changes in the number of cows under herd test in Victoria alongside key events
Discussion
In this paper, we drew on the case of herd recording in the Australian dairy sector to investigate factors and processes influencing farmer engagement with big data applications and how farmers use big data in decision making. The importance of the user production context in adoption, and the usability and usefulness of data to decision support has been further reinforced in our study (Evans et al., 2017 p79; Rose et al., 2018). Lioutas et al. (2019) emphasise that an understanding of farmer
Conclusion
In this paper, we have traced the development of herd recording in Australia as a case for understanding: 1) the use of big data to add value to farm decision making; and 2) factors and processes which influence farmer engagement with and use of big data, including institutional arrangements. Following an evolution over 100 years through cycles of digitization and digitalization, the case provides some insights to the broader context of farmer’s acceptance of big data applications.
In this study
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This research was completed as part of the ImProving Herds Project. ImProving Herds is funded by the Gardiner Dairy Foundation (Melbourne, Australia) and Dairy Australia (Melbourne, Australia) and led by Agriculture Victoria (Victoria, Australia) with collaborative support from DataGene Ltd (Melbourne, Australia), Holstein Australia (Melbourne, Australia) and the National Herd Improvement Association of Australia (Melbourne, Australia). We also acknowledge the important contributions of
References (62)
- et al.
Supporting and practising digital innovation with advisers in smart farming
NJAS-Wageningen J. Life Sci.
(2019) - et al.
Invited review: Mid-infrared spectroscopy as phenotyping tool for milk traits1
Journal of Dairy Science
(2014) - et al.
Dynamics and distribution of public and private research and extension roles for technological innovation and diffusion: Case studies of the implementation and adaptation of precision farming technologies
Journal of Rural Studies
(2017) - et al.
Making sense in the cloud: Farm advisory services in a smart farming future
NJAS Wageningen J. Life Sci.
(2019) - et al.
Public-private partnerships as systemic agricultural innovation policy instruments – Assessing their contribution to innovation system function dynamics
NJAS - Wageningen Journal of Life Sciences
(2019) - et al.
Ordering adoption: materiality, knowledge and farmer engagement with precision agriculture technologies
Journal of Rural Studies
(2017) - et al.
Recommendations arising from an analysis of changes to the Australian agricultural research, development and extension system
Food Policy
(2014) - et al.
The adoption of precision agriculture in an Australian broadacre cropping system—Challenges and opportunities
Field Crop Res.
(2007) - et al.
A review on the practice of big data analysis in agriculture
Computers and Electronics in Agriculture
(2017) - et al.
A review of social science on digital agriculture, smart farming and agriculture 4.0: New contributions and a future research agenda
NJAS - Wageningen Journal of Life Sciences
(2019)
Key questions on the use of big data in farming: An activity theory approach
NJAS - Wageningen Journal of Life Sciences
How private-sector farm advisors change their practices: An Australian case study
Journal of Rural Studies
Drivers of precision agriculture technologies adoption A literature review
Procedia Technol.
Symposium review: Building a better cow—The Australian experience and future perspectives
Journal of Dairy Science
‘Smart farming’ in Ireland: A risk perception study with key governance actors
NJAS - Wageningen Journal of Life Sciences
Mid-infrared prediction of bovine milk fatty acids across multiple breeds, production systems, and countries
Journal of Dairy Science
Triggering change: Towards a conceptualisation of major change processes in farm decision-making
Journal of Environmental Management
Governance of big data collaborations: How to balance regulatory compliance and disruptive innovation
Technological forecasting and social change.
Ethics of smart farming: Current questions and directions for responsible innovation towards the future
NJAS - Wageningen Journal of Life Sciences
Invited review: A perspective on the future of genomic selection in dairy cattle
Journal of Dairy Science
Big Data in Smart Farming - A review
Agricultural Systems
Enhancing adoption of agricultural technologies requiring high initial investment among smallholders
Technological forecasting and social change
Comparison of records from in-line milk meters and conventional herd testing for management and genetic evaluation of dairy cows
The Hico story. Dairy Herd Improvement in Gippsland and Colac
Dairy co-op Murray Goulburn cuts milk prices, MD Gary Helou departs
Australian dairy herd improvement scheme
Proc. Assoc. Advmt. Anim. Breed. Genet.
Critical questions for big data - Provocations for a cultural, technological, and scholarly phenomenon
Information, Communication and Society.
Big Data in food and agriculture
Big Data Soc.
The design and implementation of cross-sector collaborations: Propositions form the literature
Public Administration Review
The ethics of big data in big agriculture
Internet Policy Rev.
Smart farming technologies as political ontology: access, sovereignty and the performance of neo liberal and not so neo liberal worlds
Sociol. Rural.
Cited by (51)
“How can we?” the need to direct research in digital agriculture towards capacities
2023, Journal of Rural StudiesBirth of dairy 4.0: Opportunities and challenges in adoption of fourth industrial revolution technologies in the production of milk and its derivatives
2023, Current Research in Food ScienceThe enabling and constraining connections between trust and digitalisation in incumbent value chains
2023, Technological Forecasting and Social ChangeCitation Excerpt :These path dependencies are difficult to overcome when there is a limited basis of trust, or even distrust, that supports a collaborative approach towards digitalisation and its possibilities. In a sense organisations or even entire value chains become ‘locked-out’ or excluded from new (digitalisation) opportunities, due to the level of (dis)trust (Newton et al., 2020). A key implication for those concerned with managing innovation in value chains is that there needs to be sufficient space for exploration to see what the possibilities and challenges are in both the short and longer term.
Dimensions of digital transformation in the context of modern agriculture
2022, Sustainable Production and ConsumptionCitation Excerpt :Still, on the subject of data security and data sharing, Van Es and Woodard (2017) points out that coordinating data sharing and making access to this information more transparent are vital to improving data security. To help farmers have more choices in their interactions with their own data, Newton et al. (2020) recommend establishing data-sharing pipelines and policies. As one of the more effective components of DT for agroecological and small farmers, Rotz et al. (2019b) cite data sharing and open-source technologies.
Reducing greenhouse gas emissions through genetic selection in the Australian dairy industry
2022, Journal of Dairy Science