Participatory design of digital agriculture technologies: bridging gaps between science and practice

Cara Stitzlein1, Simon Fielke2, Aysha Fleming2, Emma Jakku2 & Martijn Mooij1

1 CSIRO, Data61, Australia,

2CSIRO, Land and Water, Australia

Commonwealth Scientific Industrial Research Organisation (CSIRO) College Road Sandy Bay, Tasmania Australia [email-masked]

Abstract. The relationship between agricultural productivity and sustainable land management is highlighted in a growing number of initiatives stressing the need for clean water and healthy soil. Consequently, on-farm practice changes are being recommended to landholders through these programmes. We report on research within Australia where participatory design methods were used to promote on-farm practice change through digital tools. Two projects are presented: one focused on promoting participation in a carbon farming programme and the second focused on nitrogen management in sugarcane production. We report on how engagement between researchers, farmers and advisors can work to incentivise on-farm practice change by aligning technology design and information presentation with different values held by targeted end users. Our experience demonstrates how to apply participatory design methods during the product development process, thus shaping the digital experience in ways that promote the practice changes consistent with scientifically derived insights and initiatives promoting sustainable outcomes.

Keywords: digital agriculture, carbon farming, sugarcane, participatory design, product development.

Introduction

For digital technology to be adopted, it must be considered useful by the people using the tools (Kuehne et al. 2017). To be useful, these technologies should be designed with their targeted end users in mind and ideally with their close and continuous engagement (Higgins et al. 2017). However, those usually responsible for technology development can be quite distant from the people who use the tools (Rose et al. 2018). This is often the case when these tools are developed from scientific research, resulting in a need to translate insights into a language and value proposition appropriate for general audiences (Weigold 2001).

Digital agricultural technologies can be tools, analytical processes or user interfaces that have some form of digital or computer processing applicable to work within the agricultural sector, either on or off farm (Wolfert et al. 2017; Shepherd et al. 2018; Klerkx et al. 2019). We report on two projects completed within the Digiscape Future Science Platform, a research and development effort aimed at developing digital agricultural technologies that help to 'harness the digital revolution for the Australian land sector' (CSIRO 2017, 2019). The goal of this paper is to report on how different participatory design methods were used across the two projects. In this way, we provide multiple instances of how engagement can be achieved, and the technology provided to farmers and advisors (referred to as end users) made more useful (Yin 2014).

The digital tools that were developed in the two projects provided information and incentivised on-farm practice changes related to sustainable agriculture (namely land-based carbon sequestration and nitrogen fertiliser management). Each project team was interdisciplinary, with skills across the biological, social and computer sciences (henceforth called the research teams). In both projects, the methods emphasised engagement between the research teams and targeted end users, helping to tailor the digital solution such that the information was communicated with an appreciation of what was understandable and relevant. The methods elicited feedback about personal values associated with land use changes and personal experiences working with digital technology. They considered aspects of modern farming which have broad economic implications, driven by financial and other values. Through this work, we advocate that participatory design methods can be helpful for ensuring that digital technologies are fit for purpose. Going beyond the delivery of an easy-to-use interaction, these methods provide insights into how to incentivise an on-farm practice change. That is, the methods helped the research teams articulate a solution suitable for the context of why an on-farm practice change (like registering a carbon farming project or reducing nitrogen application) may be desirable and worthwhile for an individual to consider.

Background

Project context and motivation

Existing scholarship points to failures of technology transfer approaches in complex environments, particularly where new practices and investments are required for users to obtain benefits (McCown et al. 2009; McCown et al. 2012). More recently, the process of developing technologies with human-centred design considerations has been touted as a solution to the failures of the science into practice pipeline ( Meynard et al. 2017; Rose et al. 2018; Ayre et al. 2019; Glover et al. 2019; Hardy et al. 2019). In this paper we demonstrate how embedding social science and human centred design methods in research, development and product development activities can support this process (Fleming et al. 2019; Thorburn et al. 2019). This contributes to existing work aimed at bridging disciplinary boundaries in real-world research and development efforts (Allen et al. 2014; Ayre & Nettle 2015; Musvoto et al. 2015; Polk 2015; Scholz & Steiner 2015; Duncan et al. 2018). Two projects are described, where digital technologies are intended to promote a change to existing farm management actions (on-farm practice change) towards a more sustainable outcome.

Carbon farming project: Card sort method to inform information design

The first project is set in the broad context of a carbon farming initiative and seeks to use a digital technology (e.g. mobile phone application) to promote on-farm practice changes that either increase carbon storage (carbon abatement) or reduce emissions associated with intensive farming practices (emission reduction) (Stokes & Howden 2010). Carbon farming refers to farm practices associated with the storage of carbon in soils or vegetation (Evans et al. 2015). Such practices include activities like retaining stubble after harvesting crops or planting trees. Other carbon farming practices are associated with a reduction in harmful greenhouse gases and include activities such as reduced application of nitrogen-based fertilisers, limiting livestock grazing and more efficient energy usage. These are not new agricultural practices but are now being rewarded in policy initiatives for meeting national sustainability targets in Australia, targets for achieving greenhouse gas reduction, carbon neutrality and improving overall farm sustainability (IPCC 2019).

Co-benefits refer to non-financial improvements of a land use change, such as increased biodiversity. Recent work proposes that making co-benefits a more deliberate and significant part of the decision to adopt carbon farming may increase motivation for uptake by landholders (Fleming et al. 2019). There is growing awareness that the values embedded in co-benefits, such as those around mitigating effects of climate change or improving the value and legacy of the farm, are present within farmer groups (e.g. Farmers for Climate Action n.d.; AbCF n.d.) but so far these values are rarely leveraged to full effect to drive behaviour change.

During the product development process, results of a card sort study and questionnaire (common human-centred design methods) informed how information about the incentives associated with carbon farming (e.g. co-benefits like biodiversity) was organised and presented to farmers (IDEO 2015). Card sorts are a recognised method for learning how people understand and interact with information content, based on their ranking or sorting of a group of items in a personally relevant or important way. When combined with perceptions and reports of personal experience, this method provides a view into what end users find intuitive, familiar and what they value.

Sugarcane farming project: Conceptual framework to track motivations towards using a digital technology

The second project is specific to sugarcane farming and seeks to use digital technology to show farmers the relationship between nitrogen fertiliser use and water quality in adjacent waterways (which drain into the Great Barrier Reef). The desired practice change was to reduce nitrogen inputs on-farm by providing farmers with relevant information (e.g. nitrogen concentrations or river heights) about how their nitrogen fertiliser application affected both crop yield and water quality of the water that flowed into the lagoon of the Great Barrier Reef.

Leveraging an external partnership between the Digiscape research team and a community of sugarcane farmers, a suite of mobile applications was developed that used near real-time water monitoring systems and a scientific modelling capability to display the nitrogen and water flow, rainfall and other parameters (Thorburn et al. 2019). Participatory design methods including interviews and design workshops were used by the research team to better understand whether a) the information provided by the application supported an understanding of water quality and run off, and b) if through that understanding a practice change was likely.

Methodology

We recognised that for the two digital technologies to be considered useful, the scientific insights about possible and desirable practice changes must be presented in a way that is understood and relevant (valued) to the targeted Australian farmers and farm advisors (Jakku & Thorburn 2010). Our product development approach looks to integrate feedback from farmers throughout the development process (Lindblom et al. 2017; Rose et al. 2018). The process also includes input from social scientists (rural sociologists and human centred designers) who can inform scientists in the research team on the broader social context, potential barriers and change agents. Advisors and farmers contributed to the development of digital technologies to promote an on-farm practice change (Table 1).

Table 1. Number of agricultural advisors and farmers participating in the research

Carbon farming project

Sugarcane farming project

Advisors

16

8

Farmers

17

12

Total

33

20

Ethics approval was obtained before participants were recruited from different populations. Participants in the carbon farming project were recruited by email invitation and the sample included various advisory and professional networks across Australia. Participants in the sugarcane farming project were recruited from an organisational partnership and located in North Queensland, Australia.

In the carbon farming project, the goal was to form a set of categories that would resonate with the end users as advantages and disadvantages of implementing a carbon farming project on their land. An online card sort study was used to collect information from farmers and advisors and findings from the study were used to inform the design of how co-benefits and dis-benefits were presented in the digital tool.

In the sugarcane farming project, the goal was to better understand farmer and advisor barriers to digital technology use, while at the same time eliciting feedback that might improve the likelihood of using a tool that provided information about the effects of the inputs on their farm. Formal face-to-face and telephone interviews were conducted with a strong orientation towards understanding ‘digital’ at different points in time in the product development process. Design workshops with the research team and end users were used to analyse content from the interviews and capture aspects of the maturing digital experience of the participating farmers and advisors.

Carbon farming project

In the carbon farming project, a card sort study was conducted to identify how farmers and advisors conceptualised co-benefits (Wood & Wood 2008). The material for the card sort study was developed by the Digiscape research team who conducted a review of the scientific literature and identified a set of potential benefits (both positive and negative) associated with carbon farming practices. From this review, co-benefits were categorised by the research team into five groups: farm productivity, soil health, biodiversity and conservation, water quality and quantity, and socio-economic benefits. Instead of implementing the categories derived by the research team, farmers were involved in the product development process so that their perceptions could be integrated into the tool. An online testing tool called Optimal Workshop was used by participants who grouped the co-benefits into a structure that they identified with (Figure 1).

In the study (conducted in 2019), participants provided demographic information, described their familiarity with carbon farming practices and described their motivation to implement these practices. Then they completed the card sort activity, naming each group that they created. Finally, participants commented on whether the cards seemed complete, accurate, or whether anything was missing. Participants’ comments about the relative importance of items, examples of how they are realised in farming practice and perceptions of current financial opportunities (i.e. carbon farming) were captured in a set of open response questions.

Figure 1. Screenshot of the card sort activity interface*

Source: Optimal Workshop, n.d.

*Participants moved individual items (left hand-side list) into groupings and named them (right hand-side).

Sugarcane farming project

In the sugarcane farming project, twenty in-depth interviews were conducted with sugarcane farmers, advisors and researchers in late 2018. Two workshops were held in late 2018 and early 2019 (approximately 10 researchers at each), for the purpose of classifying farmers, advisors and researchers’ level of interest in the digital tools over time. The approach was based on a recently developed concept of “digi-grasping” in the literature that attempted to capture aspects of understanding, use and living with digital technologies (e.g. “modes”) (Dufva & Dufva 2019).

First, the research team coded interview responses to a set of behaviours based on the digi-grasping concept. This involved an iterative three-stage coding process whereby responses were divided into relevant question response categories, then thematically grouped. A workshopping process and a series of participant engagements was conducted to refine the coding approach and explore how an individual’s digital journey could be captured (Table 2). For example, refinement occurred when the research team introduced the modes to farmers and advisors in later engagements, they revised digi-MART to digi-MAST due to farmer aversion to the term “roused”. In the final stage of coding, the research team classified farmers and advisors to a mode, this time with greater sensitivity to how participants were responding to the proposed digital tool over time.

Table 2. The digi-MAST framework captured potential digital journeys, using modes from uncertainty towards acceptance (rows) and across time (columns)

The digi-MAST framework was developed by the social scientists and human centred designers embedded within the project team (with input from the broader Digiscape research programme) and used methodologies from both disciplines, mainly through a series of co-design activities involving the user community and the scientific research team. In other words, the digi-MAST framework provided a way for the research team to reflexively consider the barriers and opportunities captured in the participatory design activities (Beers & van Mierlo 2017).

The implications of using these methods in the two projects is presented in the next section. We then discuss insights for developing digital technologies that are fit for purpose.

Results

Carbon farming project

The card sort study elicited qualitative data including conceptual groupings, descriptions and perceptions. This data was analysed for themes and the entire set of co-benefits assessed for completeness and relevance to how farmers and advisors considered the on-farm practice change. The social scientists on the research team analysed data using a content analysis and a similarity matrix (matrix produced by the Optimal Workshop application). Participants rated the three top or most important co-benefits as: biodiversity (10 votes), increased yield/bottom line (10 votes), and climate adaptability/resilience (9 votes).

Participants described reasons for their selection, for example:

Climate adaptability is important, as it will improve my business's ability to remain viable in the face of climate change. Biodiversity is important, as it will improve the health of my property, my business and my personal wellbeing. Income streams [are] important as diversification will be key to financial success (Participant ID 10 from Western Australia, 20 July 2019).

In the analysis, the research team looked for how the popular co-benefits were reflected in the groupings formed by the study’s participants. Qualitative data captured in the open response questions provided additional insights into participants’ values and beliefs related to carbon farming. For example, based on several comments that the dis-benefits did not belong with co-benefits, a unique category was created for these items. A subset of results from a card sort method are presented in a similarity matrix (Figure 2).

Figure 2. Similarity matrix*

Source: Optimal Workshop, n.d.

*Numbers indicate how often cards were grouped in the same category

The similarity matrix shows the extent of agreement between participants’ responses (based on pairs), with higher numbers indicating that more participants grouped items together. When items are grouped together, they are typically related or considered belonging to the same category.

Results from the card sort method were evaluated against the research team’s categorisation and a revised set of categories was implemented in the digital tool. Table 3 shows the categories proposed by the research team (based on the scientific review) and those extracted from the card sort study (based on farmers and advisors’ groupings and comments).

Table 3: Categories developed by the research team (left) versus participating farmers and advisors (right)

Research scientist categories

Farmer and advisor categories

Farm Productivity

Farm Profitability

Soil Health

Farm Resilience

Biodiversity and Conservation

Broad Environmental and Social Benefits

Water Quality and Quantity

Dis benefits

Socio-economic

The categories developed by the research team focused on physical and financial aspects of the farm/farm business and separated physical components such as soil and water. The categories created by farmers and advisors focused on benefits found on the farm versus the ones that are distributed in the broader community and landscape. The categories developed by research scientists, farmers and farm advisors are neither right nor wrong. The key difference between the two groups of people is that the categories developed by study participants reflect an arguably more integrated and holistic conceptualisation of co-benefits. When designing digital tools for farmers, technology developers should consider that collecting feedback from farmers can be used to present information in a useful and relevant way (Lindblom et al. 2017; Rose et al. 2018).

Sugarcane farming project

The digital modes within the digi-MAST framework were used to communicate that perceptions of a digital technology or tool needed to progress from the user feeling Mystified, to Aware, to Sparked (or roused or engaged), to having their lives or world views Transformed in some way. Table 4 provides examples of the type of phrases that the research team used to classify the twenty interview participants into different modes, each expressing a different sort of perception related to digital technology.

Table 4. Statements that helped classify the digi-MAST mode of interviewees at different points in time

Mode

Example verbal indicators

Mystery

I have no idea, I don’t care, I don’t see X as important, I don’t have the resources to consider X

Aware

I have heard of X, tell me more, help me understand the value of X so that I can consider it further

Spark

I want to play with X! This is fun/a hobby, I’ll have a go, I’m understanding how X can fit in with my values and physical world

Transform

I am going to invest in X because it could change how I do things, I am gaining confidence, I am able to share with others

Figure 3 shows results from a workshop where the difference between advisors and farmers was captured in the different modes of behaviour. The analysis suggested that advisors were moving beyond the” aware” state with greater interest in using the tool, as indicated by their perceptions captured by the “spark” mode. Conversely, farmers’ perceptions were best captured by “mystery” and “aware” modes, without indicating increased desire or interest in engaging with this specific digital tool.

These data suggest differences exist between farmers and advisors consistent with differences detected between end users and next users. For example, next users or intermediaries, are farm advisors who indicated they were further along the digi-grasping journey (Munthali et al. 2018), and they could help champion the use of digital technology and on-farm practice change. With these results, the research team was able to prioritise remaining project work and devote resources to improve the impact of their investment (e.g. financial planning activities, developing strategies to extend research outputs through the final phases, and additional workshopping). The application of the digi-MAST framework within this project allowed the research team to capture how end users and next users were operating at different levels of technical skill and capability. The combined use of in-depth interview data with the framework reinforced that a gap existed between the realities of farmers’ worldviews, the expectations of the interdisciplinary research team and those who might ultimately use the digital tool. The insights derived from these research activities highlighted important differences that otherwise would have not been addressed within the project and limited its ability to have practical impact.

Figure 3. Research team classification of the digi-MAST mode for a set of participating farmers and advisors in relation to the digital tool (n=8)

Discussion

To promote an on-farm practice change, digital technologies must affect a range of agricultural stakeholders in a way that both informs and incentivises their actions. In both projects, the research teams considered the context in which farmers operate, which is often overlooked, for example in terms of privacy concerns (Wiseman et al. 2019). In both projects, feedback was obtained early in the product development process as one potential means to help promote trust between the research team developing the tool and the intended end users (Jakku et al. 2019). The approaches reported here suggest how to recognise and incorporate farmers’ beliefs systems into digital tools in order to increase the relevance of scientific research and thus the ability to have both scientific and societal impact (Bornmann 2012).

In the carbon farming project, engagement with farmers and advisors during the product development process influenced how digital information was presented, making it more relevant to how farmers conceptualised the carbon farming opportunity. The study revealed that the research scientists and farmers think about land conditions differently. The results from the card sort study were used to design the information content based on user perceptions instead of researchers’ preconceived notions. By integrating farmer perceptions into scientific research outputs, we believe this work helps to create a bridge between science and practice. In the final product, the information content was strongly related to farmers’ holistic and interconnected values, resulting in a broad conceptualisation that could further incentivise on-farm practice change. This has the potential to make the tool more fit for purpose and we believe that providing such relevance may incentivise action better than information presented in a scientific context.

In the sugarcane farming project, engagement with farmers and advisors was evaluated with a conceptual framework of how their experience with ‘digital’ may evolve over time. The digi-MAST framework and design workshop approach helped the interdisciplinary research team respond to perceptions of the farmers and advisors, increasing the reflexive capacity of the researchers, helping them to build technical capability and an awareness of digital technological requirements beyond basic function and usability. Using the framework contributed to positive project outcomes because it captured important project relationships (e.g. between farmer and advisor, and with technology) and provided a way to classify the interactions between the end users, next users and the product being developed. Early evidence of using this framework has indicated where the strongest potential for project impact is and how remaining investment may be maximised – by becoming more strategic in focus and helping the research team prioritise their finite resources. Importantly, recognising that while the on-farm practice change may be implemented by farmers, the incentives for the change are recognised by advisors interacting with the technology. The contributions of both farmers and advisors in the development of this project were essential. In addition to creating this bridge between the science and practice of tool use, the research highlighted elements of the broader policy and farming space that affects proposed digital tools in this context. This approach may help overcome limitations of traditional research, development and institutional constraints in achieving buy-in from community stakeholders to achieve impact (Cullen et al. 2014; Glover et al. 2019).

Since this research was completed, both digital technologies have been released. The carbon farming project tool, https://looc-c.farm/ launched in December 2019, and there is evidence that it is being used in the promotion of state-sponsored initiatives, where landholders are encouraged to use the tool during a project application process (Land Restoration Fund n.d.). The tool developed in the sugarcane farming project, https://1622.farm launched in January 2020, with further details on the design process recently published (Vilas et al. In press).

Practical implications

When considering the benefits of using participatory design methods, two lessons are clear. Firstly, in the sugarcane farming project, the methods involving targeted end users helped manage parallel streams of research, product development and interdisciplinarity. This finding supports vast existing literature (McKee et al. 2015; Rauschmayer et al. 2015; Rosendahl et al. 2015; Thompson et al. 2017; Turner et al. 2017). Secondly, both projects demonstrate how incorporating participatory design methods can help build in adaptability and agility in science innovation research. We found that human-centred design methods and social scientific conceptual frameworks helped to manage expectations of project team members: they can better experience and endorse the project achievements. These interventions provided evidence to better inform project decisions, as discussed by Glover et al. 2019.

The findings reported here have limitations that are consistent with exploratory qualitative research, such as using specific samples which may not be replicated in other contexts. As a result, this study makes no explicit claims to broad generalisability. Rather, the projects demonstrate the potential usefulness of participatory design methods for interacting with end users and using those results to design technology which is better able to affect behaviour change.

Conclusion

Our experience using participatory design methods led to a better understanding of how to frame incentives of an on-farm practice change in the design of digital agriculture technologies. Human-centred design processes and social science methodologies, like those described here, were utilised early and throughout the product development process. Applying these methods helped to manage expectations and maximise the understanding by all stakeholders about what the digital tools were intended to do. We believe in the absence of this approach, the outputs of the two projects would have been less appropriate for the intended users and context of use.

If this approach becomes widespread, there is an opportunity to use such digital tools for the promotion of sustainable policy initiatives and on-farm practice changes (e.g. through carbon markets or nutrient management programmes). Early engagement results in better design outcomes and an opportunity to reflect on the desired decision making and engagement more broadly. It provides a space to develop responsiveness to shifting environmental policy and discuss options to maintain adaptive. In other words, the ways of linking product design with gathering participant feedback as described here can enable researchers and end users to connect and build an understanding of each other, helping to align values to enhance the resulting impacts of research investment. A capability such as this is critical if digital technologies are to take hold and mature in the agricultural sector and deliver on the hype of agriculture 4.0.

Acknowledgments

This project was supported by the CSIRO Digiscape Future Science Platform and study approval was granted by the CSIRO Human Research Ethics Committee 086/18 (Carbon farming project) and 106/17 (Sugarcane farming project). We acknowledge members of the research teams who were directly involved in activities reported here: Stephen Roxburgh, Jeff Baldock, Peter Fitch, Peter Thorburn, Aaron Davis, and Anthony Webster. We are grateful to all the participants who took the time to provide feedback both formally and informally. We appreciate the constructive comments received by the reviewers of this journal.

References

1622TM n.d., Water Quality, https://wq.1622.farm/ [10 August 2020].

AbCF n.d., Aboriginal Carbon Foundation, https://www.abcfoundation.org.au/ [5 August 2020].

Allen W, Ogilvie S, Blackie H, Smith D, Sam S, Doherty J, McKenzie D, Ataria J, Shapiro L, MacKay J, Murphy E, Jacobson C & Eason C 2014, 'Bridging Disciplines, Knowledge Systems and Cultures in Pest Management', Environmental Management, vol. 53, no. 2, pp. 429-440, https://dx.doi.org/10.1007/s00267-013-0180-z.

Ayre M, Mc Collum V, Waters W, Samson P, Curro A, Nettle R, Paschen J-A, King B & Reichelt N 2019, 'Supporting and Practising Digital Innovation with Advisers in Smart Farming', NJAS - Wageningen Journal of Life Sciences, vol. 90-91, p. 100302, https://doi.org/10.1016/j.njas.2019.05.001.

Ayre M & Nettle R 2015, 'Doing Integration in Catchment Management Research: Insights into a Dynamic Learning Process', Environmental Science & Policy, vol. 47, no. 1, pp. 18-31, https://dx.doi.org/ 10.1016/j.envsci.2014.10.011.

Beers PJ & van Mierlo B 2017, 'Reflexivity and Learning in System Innovation Processes', Sociologia Ruralis, vol. 57, no. 3, pp. 415-436.

Bornmann L 2012, 'The Societal Impact of Research', EMBO reports, vol. 13, no. 8, pp. 673-676.

CSIRO 2017, The Digiscape Future Science Platform, https://research.csiro.au/digiscape/ [10 August 2020].

CSIRO 2019, Helping Sugarcane Farmers Protect the Great Barrier Reef, https://research.csiro.au/digiscape/digiscapes-projects/great-barrier-reef-and-sugarcane-production/ [10 August 2020].

Cullen B, Tucker J, Snyder K, Lema Z & Duncan A 2014, 'An Analysis of Power Dynamics Within Innovation Platforms for Natural Resource Management', Innovation and Development, vol. 4, no. 2, pp. 259-275, https://dx.doi.org/10.1080/2157930x.2014.921274.

Dufva T & Dufva M 2019, 'Grasping the Future of the Digital Society', Futures, vol. 107, pp. 17-28, https://doi.org/10.1016/j.futures.2018.11.001.

Duncan R, Robson-Williams M, Nicholas G, Turner JA, Smith R & Diprose D 2018, 'Transformation is ‘Experienced, Not Delivered’: Insights From Grounding the Discourse in Practice to Inform Policy and Theory', Sustainability, vol. 10, no. 9, p. 3177.

Evans M, Carwardine J, Fensham R, Butler D, Wilson K, Possingham H & Martin T 2015, 'Carbon Farming via Assisted Natural Regeneration as a Cost-effective Mechanism for Restoring Biodiversity in Agricultural Landscapes', Environmental Science & Policy, vol. 50, pp. 114-129.

Farmers for Climate Action n.d., https://farmersforclimateaction.org.au/ [5 August 2020].

Fielke S, Garrard R, Jakku E, Fleming A, Wiseman L & Taylor B 2019, 'Conceptualising the DAIS: Implications of the ‘Digitalisation of Agricultural Innovation Systems’ on Technology and Policy at Multiple Levels', NJAS - Wageningen Journal of Life Sciences, p. 100296, https://dx.doi.org/10.1016/j.njas.2019.04.002.

Fleming A, Stitzlein C, Jakku E & Fielke S 2019, 'Missed Opportunity? Framing Actions Around Co-benefits for Carbon Mitigation in Australian Agriculture', Land Use Policy, vol. 85, pp. 230-238, https://doi.org/10.1016/j.landusepol.2019.03.050.

Glover D, Sumberg J, Ton G, Andersson J & Badstue L 2019, 'Rethinking Technological Change in Smallholder Agriculture', Outlook on Agriculture, vol. 48, no. 3, pp. 169-180, https://dx.doi.org/10.1177/0030727019864978.

Hardy J, Phelan C, Vigil-Hayes M, Su NM, Wyche S & Sengers P 2019, 'Designing From the Rural', Interactions, vol. July-August, http://interactions.acm.org/archive/view/july-august-2019/designing-from-the-rural.

Higgins V, Bryant M, Howell A & Battersby J 2017, 'Ordering Adoption: Materiality, Knowledge and Farmer Engagement with Precision Agriculture Technologies', Journal of Rural Studies, vol. 55, pp. 193-202, https://doi.org/10.1016/j.jrurstud.2017.08.011.

IDEO 2015, The Field Guide to Human-centered Design, http://www.designkit.org//resources/1 [5 August 2020].

IPCC 2019, Summary for Policymakers of the Special Report on Climate Change and Land, https://ipcc.ch/report/srccl [5 August 2020].

Jakku E, Taylor B, Fleming A, Mason C, Fielke S, Sounness C & Thorburn P 2019, '“If They Don’t Tell Us What They Do With It, Why Would We Trust Them?” Trust, Transparency and Benefit-sharing in Smart Farming', NJAS - Wageningen Journal of Life Sciences, vol. 90-91, p. 100285, https://doi.org/10.1016/j.njas.2018.11.002.

Jakku E & Thorburn PJ 2010, 'A Conceptual Framework for Guiding the Participatory Development of Agricultural Decision Support Systems', Agricultural Systems, vol. 103, no. 9, pp. 675-682, http://dx.doi.org/10.1016/j.agsy.2010.08.007.

Klerkx L, Jakku E & Labarthe P 2019, 'A Review of Social Science on Digital Agriculture, Smart Farming and Agriculture 4.0: New Contributions and a Future Research Agenda', NJAS - Wageningen Journal of Life Sciences, vol. 90-91, p. 100315, https://doi.org/10.1016/j.njas.2019.100315.

Knierim A, Labarthe P, Laurent C, Prager K, Kania J, Madureira L & Ndah TH 2017, 'Pluralism of Agricultural Advisory Service Providers – Facts and Insights From Europe', Journal of Rural Studies, vol. 55, pp. 45-58, https://doi.org/10.1016/j.jrurstud.2017.07.018.

Kuehne G, Llewellyn R, Pannell DJ, Wilkinson R, Dolling P, Ouzman J & Ewing M 2017, 'Predicting Farmer Uptake of New Agricultural Practices: A Tool for Research, Extension and Policy', Agricultural Systems, vol. 156, pp. 115-125, https://doi.org/10.1016/j.agsy.2017.06.007.

Land Restoration Fund, Queensland n.d., https://www.qld.gov.au/environment/climate/climate-change/land-restoration-fund/about [5 August 2020].

Landscape Opportunities and Options Carbon- Calculator, LOOC-C 2019, https://looc-c.farm/ [5 August 2020].

Lindblom J, Lundström C, Ljung M & Jonsson A 2017, 'Promoting Sustainable Intensification in Precision Agriculture: Review of Decision Support Systems Development and Strategies', Precision Agriculture, vol. 18, no. 3, pp. 309-331.

McCown RL, Carberry PS, Hochman Z, Dalgliesh NP & Foale MA 2009, 'Re-inventing Model-based Decision Support with Australian Dryland Farmers. 1. Changing Intervention Concepts During 17 Years of Action Research', Crop and Pasture Science, vol. 60, no. 11, pp. 1017-1030, https://doi.org/10.1071/CP08455.

McCown RL, Carberry PS, Dalgliesh NP, Foale MA & Hochman Z 2012, 'Farmers Use Intuition to Reinvent Analytic Decision Support for Managing Seasonal Climatic Variability', Agricultural Systems, vol. 106, no. 1, pp. 33-45, https://doi.org/10.1016/j.agsy.2011.10.005.

McKee A, Guimarães MH & Pinto-Correia T 2015, 'Social Capital Accumulation and the Role of the Researcher: An Example of a Transdisciplinary Visioning Process for the Future of Agriculture in Europe', Environmental Science & Policy, vol. 50, pp. 88-99, http://dx.doi.org/10.1016/j.envsci.2015.02.006.

Meynard JM, Jueffroy MH, Le Bail M, Lefevre A, Magrini MB & Michon C 2017, 'Designing Coupled Innovations for the Sustainability Transition of Agrifood Systems', Agricultural Systems, vol. 157, pp. 330-339, <https://dx.doi.org/10.1016/j.agsy.2016.08.002>.

Munthali N, Leeuwis C, van Paassen A, Lie R, Asare R, Van Lammeren R & Schut M 2018, ‘Innovation Intermediation in a Digital Age: Comparing Public and Private New-ICT Platforms for Agricultural Extension in Ghana’, NJAS - Wageningen Journal of Life Sciences, vol. 86-87, pp. 64-76, <https://doi.org/10.1016/j.njas.2018.05.001>.

Musvoto C, Mason N, Jovanovic N, Froebrich J, Tshovhote J, Nemakhavhani M & Khabe T 2015, 'Applying a Transdisciplinary Process to Define a Research Agenda in a Smallholder Irrigated Farming System in South Africa', Agricultural Systems, vol. 137, pp. 39-50, http://dx.doi.org/10.1016/j.agsy.2015.03.008.

Optimal Workshop n.d., https://www.optimalworkshop.com/ [10 August 2020].

Polk M 2015, 'Transdisciplinary Co-production: Designing and Testing a Transdisciplinary Research Framework for Societal Problem Solving', Futures, vol. 65, pp. 110-122, http://dx.doi.org/10.1016/j.futures.2014.11.001.

Rauschmayer F, Bauler T & Schäpke N 2015, 'Towards a Thick Understanding of Sustainability Transitions - Linking Transition Management, Capabilities and Social Practices', Ecological Economics, vol. 109, pp. 211-221, https://dx.doi.org/10.1016/j.ecolecon.2014.11.018.

Rose DC & Chilvers J 2018, 'Agriculture 4.0: Broadening Responsible Innovation in an Era of Smart Farming', Frontiers in Sustainable Food Systems, vol. 2, art. 87, https://dx.doi.org/10.3389/fsufs.2018.00087.

Rose DC, Morris C, Lobley M, Winter M, Sutherland WJ & Dicks LV 2018, 'Exploring the Spatialities of Technological and User Re-scripting: The Case of Decision Support Tools in UK Agriculture', Geoforum, vol. 89, pp. 11-18, https://doi.org/10.1016/j.geoforum.2017.12.006.

Rose DC, Sutherland WJ, Barnes AP, Borthwick F, Ffoulkes C, Hall C, Moorby JM, Nicholas-Davies P, Twining S & Dicks LV 2019, 'Integrated farm Management for Sustainable Agriculture: Lessons for Knowledge Exchange and Policy', Land Use Policy, vol. 81, pp. 834-842, https://doi.org/10.1016/j.landusepol.2018.11.001.

Rosendahl J, Zanella MA, Rist S & Weigelt J 2015, 'Scientists’ Situated Knowledge: Strong Objectivity in Transdisciplinarity', Futures, vol. 65, pp. 17-27, http://dx.doi.org/10.1016/j.futures.2014.10.011.

Scholz R & Steiner G 2015, 'The Real Type and Ideal Type of Transdisciplinary Processes: Part II—What Constraints and Obstacles Do We Meet in Practice?', Sustainability Science, vol. 10, no. 4, pp. 653-671, http://dx.doi.org/10.1007/s11625-015-0327-3.

Shepherd M, Turner JA, Small B & Wheeler D 2018, 'Priorities for Science to Overcome Hurdles Thwarting the Full Promise of the ‘Digital Agriculture’ Revolution', Journal of the Science of Food and Agriculture, https://dx.doi.org/10.1002/jsfa.9346.

Stokes C & Howden M 2010, Adapting Agriculture to Climate Change: Preparing Australian Agriculture, Forestry and Fisheries for the Future, CSIRO publishing, Canberra. https://www.nccarf.edu.au/sites/default/files/attached_files_publications/AGRICULTURE_A4Printable.pdf

Thompson MA, Owen S, Lindsay JM, Leonard GS & Cronin SJ 2017, 'Scientist and Stakeholder Perspectives of Transdisciplinary Research: Early Attitudes, Expectations, and Tensions', Environmental Science & Policy, vol. 74, pp. 30-39, https://doi.org/10.1016/j.envsci.2017.04.006.

Thorburn P, Fitch P, Zhang Y, Shendryk Y, Webster T, Biggs J, Mooij M, Ticehurst C, Vilas M & Fielke S 2019, '‘Digital Agriculture’ Helping Farmers Reduce Impacts of Cropping on the Great Barrier Reef', in 2019 Agronomy Australia Conference, Proceedings of the 2019 Agronomy Australia Conference, Wagga Wagga, www.agronomyaustralia.org/conference-proceedings.

Turner JA, Klerkx L, White T, Nelson T, Everett-Hincks J, Mackay A & Botha N 2017, 'Unpacking Systemic Innovation Capacity as Strategic Ambidexterity: How Projects Dynamically Configure Capabilities for agricultural innovation', Land Use Policy, vol. 68, pp. 503-523, https://dx.doi.org/10.1016/j.landusepol.2017.07.054.

Vilas MP, Thorburn PJ, Fielke S, Webster T, Mooij M, Biggs JS, Zhang Y-F, Adham A, Davis A, Dungan B, Butler R & Fitch P In Press, '1622WQ: A Web-based Application to Increase Farmer Awareness of the Impact of Agriculture on Water Quality', Environmental Modelling & Software, 104816, https://www.sciencedirect.com/science/article/pii/S1364815220301596.

Weigold M 2001, 'Communicating Science: A Review of the Literature', Science Communication, vol. 23, no. 2, pp. 164-193.

Wiseman L, Sanderson J, Zhang A & Jakku E 2019, 'Farmers and Their Data: An Examination of Farmers’ Reluctance to Share Their Data Through the Lens of the Laws Impacting Smart Farming', NJAS - Wageningen Journal of Life Sciences, vol. 90-91, 100301, https://doi.org/10.1016/j.njas.2019.04.007.

Wolfert S, Ge L, Verdouw C & Bogaardt M-J 2017, 'Big Data in Smart Farming - A Review', Agricultural Systems, vol. 153, pp. 69-80, https://doi.org/10.1016/j.agsy.2017.01.023.

Wood JR & Wood LE 2008, 'Card Sorting: Current Practices and Beyond', Journal of Usability Studies, vol. 4, no. 1, pp. 1-6.

Yin RK 2014, Case Study Research: Design and methods, Sage Publications, New York.