Is= it the model or is it the process of using it? Extension officers evaluate ADOPT a= s a tool to assist planning in the pastoral sector
Oscar Montes de Oca Munguia1,2, David J. Panne= ll1 & Rick Llewellyn1,3
1School of Agriculture and E= nvironment, The University of Western Australia, Crawley WA 6009
2Scion Research, 49 Sala St= reet, Rotorua 3010, New Zealand
3CSIRO, Glen Osmond SA 5064=
Email: oscar.montes@scionresearch.com
Abstract. Agricultural extension professionals a= re aware of the complexity surrounding farmers’ decisions to adopt a new technology or practice. These extension officers often need to design strat= egies to improve adoption though planning processes, which are commonly run collaboratively by expert groups and through deliberation rather than individually. Models have been used to assist these deliberations, but it is not clear which aspects of the model or the deliberative process are more useful for extension planning. In this study, we research how ADOPT, a model that predicts adoption, may assist decision making in planning for agricultural extension. In 2018, we used AD= OPT in three workshops with extension officers from the pastoral sector in New Zealand to analyse the adoption of four well-known practices in the industr= y. We identified important features of the model and the process used in the workshops and asked participants to rank their usefulness. The components w= ere: a conceptual model of adoption, a comparison between the predicted diffusion curve and actual uptake, a sensitivity analysis of the results, and a structured discussion around these components. We found that using ADOPT changed participants’ perceptions on the feasibility of forecasting adoptio= n. We also found that participants believe the process of discussing and using AD= OPT was just as important, or more important, than the model’s results.
Keywords: Technology diffusion, agricultural extension, adoption, modelling, pastoral farming, ADOPT.
Planning for extension in agriculture requires consideration of the complex nature of the process via which farmers adopt = new practices. In this process, many elements dynamically interact with each ot= her over time and future conditions are uncertain. Interacting elements include= the population of potential adopters, the technology or practice in question and the external context in which adoption takes place (e.g. biophysical, economic and regulatory conditions). These factors mak= e each adoption case unique.
Agricultural extension agents are aware that a farme= r’s decision to adopt technologies and practices in agriculture is not as simpl= e as choosing the option offering more economic benefits (V= anclay 2011). According to Nicholson et al. (2015, p. 1), farm decision-making consists of '…choosing a path that provides a farming business with accepta= ble reward for acceptable effort at an acceptable amount of risk'. For example, some farmers are willing to accept lower profits in ord= er to maintain a lifestyle and production system that suits their goals and value= s, while others might be willing to operate at a high level of risk in order to maximise returns while knowing they are foregoing other opportunities. Moreover, agriculture depends on biology and climate, so its performance ca= n be affected by factors beyond operator control. Nicholson et al. (2015) point = out that agriculture operates in one of the most challenging business environme= nts due to the combination of volatile production and prices. Kaine et al. (201= 1) support this idea and propose that landowners normally configure resources = with technologies and practices to realise family and business objectives while managing exogenous constraints. The interdependence of all those elements imposes restrictions on how farmers can respond to opportunities and constraints and therefore can influence their decisions on whether to adopt= new practices.
On the other hand, one of the aims of extension is to increase and/or accelerate the adoption of beneficial technologies and practices amongst a target population of potential adopters. The design of = the extension strategy used is often developed in a planning process, which commonly is done collaboratively rather than individually. Collaborative planning is thus an iterative process where decisions emerge from discussio= ns amongst a designated group rather than from individuals working alone. Plan= ning theorists argue that groups are better than individuals at assessing and choosing amongst options for the future (Innes & B= ooher 1999). They propose that the main advantage of group discussions is having = the opportunity to expose the group members to the unique knowledge of each participant about aspects of a problem that they understand better than any= body else does.
However, due to the complexity of adoption decisions= , it is difficult to achieve a common understanding of what drives adoption, especially when individual knowledge and beliefs differ substantially from = one another (Pannell & Claassen 2020). Wilkinson (2011, p. 47) has suggested that '…the word "adoption" is so entrenched in the language that everyone who uses the word thinks they know what is meant by it, but that i= ts interpretation varies. This lack of common understanding is well documented= in the adoption literature (Wauters & Mathijs 2014; Liu et al. 2018) and it can potentially= make it difficult for experts to undertake effective planning sessions.
Models of various types have been used successfully = to understand complexity and assist decision making. It has been argued that, = when making decisions, the more complex the problem, the greater the potential benefits of a model (Walker 2000; Vicsek 2002; Epstein 2008; Veldkamp 2009). In order to assist with planning for agricultural extension, a model of adoption needs to logically reflect the realities of landowners making a ch= oice about adoption (Nicholson et al. 2015) by considering the interplay between rewards, effort and risk, in terms of both the decision makers’ preferences= and the characteristics of the practice itself (Montes de Oca Munguia & Llewellyn 2020). The model also needs to clearly identify drivers related to potential adopters and the practice itself, distinguishing between drivers = that are at the disposal of participants to design extension interventions and drivers that are outside their control. Finally, the model needs to show the effects that intervention strategies can have in the system under certain contexts.
There are many examples in the agriculture and natur= al resource management literature showing the use of different modelling approaches in different types of deliberative processes. Models include: economic models (e.g. Rosegrant et al. 2002), information flow models (e.g. Fountas et al. 2006), integrated models (e.g.= Antle et al. 2014; Kuehne et al. 2017), and agent-bas= ed models (e.g. Laciana & Oteiza-Aguirre 2014; Schreinemachers & Berger 2011). Collaborative processes include ‘robust decision-making’ (Lempert et al. 2006; Haasnoot et al. 2013; Kalra et al.= 2014; Maier et al. 2016), ‘theory of change’ approaches (Prinsen & Nijhof 2015; Allen et al. 2017; Douthwaite & Hoffecker 2017; Thornton et al. 2017) and more recent efforts to develop tools to facilitate planning for the scaling out of innovations in complex developing country scenarios (e.g. Sartas et al. 2020).
The literature is not clear in defining what a ‘good deliberative process’ is for extension planning and the contributing role t= hat models may play. Using a model in group settings requires a degree of compromise between the individuals’ perception of how the system works and = the simplified representation of the system used in the deliberation. It is therefore inevitable that this would cause a degree of scepticism that will affect the individuals’ perception of the model’s results. Nevertheless, participants who are open to the use of models in their deliberations often= do it pragmatically, or as the common aphorism attributed to the statistician George Box states, they may embrace the view that: 'all models are wrong, b= ut some models are useful' (Box & Lucefio 1998= ).
The objective of this paper is to improve our understanding of the usefulness of models in planning for extension, and whether participants perceive the model or the process of using it to be mo= re useful. There is already evidence in other fields that the successful use of models depends on a good deliberative process (Jakku & Thorburn 2010), and that '...A good process can survive a bad model, = but a bad process isn’t helped by a good model' (Lempert 2015). Furthermore, Phillips & Linstone (20= 16) suggest that the real objective of using models in planning, especially forecasting models, is not necessarily to be 'right', but to '… help us be better prepared to understand the range of possibilities and react with flexibility and resilience to future events', and that '…the most precise forecast is not necessarily the most useful forecast' (p. 163).
In general terms, we considered that the usefulness = of a model and a deliberative process were based on their ability to generate focused technical discussions, as proposed by Forester (1999). Usefulness t= hus depends on facilitating detailed, focused technical arguments amongst participants about the range of options at their disposal to design extensi= on interventions, to assess the potential performance of these interventions u= nder a specific context, and to methodically analyse the uncertainty surrounding drivers and their effects on adoption. Thus, we consider an adoption model = to be useful for extension if it can improve participants’ understanding of the complex environment in which they operate by: a) illustrating how the system works and identifying key driving forces, b) quantitatively predicting the outcomes from the system in a particular context, and c) methodically analy= sing the uncertainty surrounding drivers and their effects on outcomes.
It is also not clear in the literature if there are specific aspects of a model that can be identified as being more useful than others. For this research, we identified three model components that were evaluated independently and are aligned to one of the three points above: a) the model’s specifications of drivers and their causal relationships (e.g. functional form), b) the model’s output (e.g. diffusion curve), and c) the outputs’ sensitivity analysis (e.g. scenario evaluation) (Kalra et al. 2014= ).
For this research, we use the <= /span>ADOPT model (Adoption and Diffusion Outcome Prediction Tool), developed by Kuehne= et al. (2017), in a case study with professional extension specialists working in the pastoral sector in New Zealand.
ADOPT was selected because we consider it offers four features that can be evaluated separately in terms of their contribution to effective deliberation: ADOPT includes a conceptual framework that can be u= sed to illustrate how the system works identifying key driving forces; ADOPT produces a predicted diffusion curve; ADOPT includes a sensitivity analysis= to methodically analyse the uncertainty surrounding drivers and their effects;= and it includes a structured process that can be used by groups for deliberative design.
The process of using ADOPT
consists of three steps. These steps include: first, presenting and discuss=
ing the
conceptual model; second, methodically answering a sequence of questions to
define the model’s inputs – this step requires the group to discuss each
question, understand its relevance and reach consensus for each answer; and
third, discussing the effects of different variables on the model’s output
using sensitivity analysis.
Most importantly, ADOPT is one of the few tools avai= lable that explicitly provides ex-ante adoption analysis in agriculture. We consi= der these features can generate the level of technical enquiry suggested by For= ester (1999) for planners to 'assess practically, comparatively and prescriptivel= y a range of viable options at their disposal'. Given its features, ADOPT may be able to facilitate discussions about the potential effects of drivers on the adoption of a technology or practice, where experts in different fields are encouraged to make contributions throughout the process, as suggested by In= nes & Booher (1999).
Three separate workshops were conducted in 2018 with= 34 professionals representing a range of organisations involved in research and extension in the pastoral sector in New Zealand after h= uman research ethics approval was granted by the University of Western Australia= . These participants have extensive knowledge of the population of pastoral farmers in New Zealand, and some were also experts on the specific technolo= gies used in this research. Pastoral farming dominates the rural agribusiness sector in New Zealand, more specifically 'sheep and beef' farming (producing meat and fib= re) and dairy cattle farming. Technology has always played an important role in= the sector. Organisations represented in the workshops were: Dairy NZ (an industry organisation), the Red Meat Profit Partnership (a research program= me), AgResearch (an agricultural research company), = the Ministry for Primary Industries, the Alliance Group (a meat processing company), Beef & Lamb New Zealand (an industry organisation), and Linco= ln University.
The number of participants for each workshop were 18= , 10 and 6, with no repetition. The first two workshops were attended by dairy industry specialists and the third workshop was attended by sheep and beef farming specialists.
Extension officers were asked to complete a question= naire before their workshop and one at the end, to detect shifts in their percept= ions about using models before and after their participation (Montes de Oca Mung= uia 2020). Reponses were summarised as a group and used to define statistical models. The pre-workshop questionnaire was sent alongside the invitation to participate two weeks prior to the workshop. This questionnaire included questions regarding the participant’s area or work, their perception of the usefulness and the feasibility of predicting adoption, and their opinion on= the importance of several drivers of adoption to be considered while thinking a= bout adoption.
The post-workshop questionnaire included questions a= bout their perceptions on the usefulness of the group discussion generated in the workshop and different components of ADOPT. Participants were also asked whether their participation in the workshop had changed their opinion about= the ability of a model to forecast adoption and whether their opinion on the importance of different adoption factors had changed from their original perceptions. Finally, they were asked what actions they were likely to take= in relation to predicting adoption after the workshop. A total of 31 participa= nts completed the pre-workshop questionnaire. Of those, 24 completed the post-workshop questionnaire.
Participants were made aware that the objective of the workshop was to use their expert opinion to evaluate the use of ADOPT for extension planning using an example of a well-known technology available to pastoral farmers in New Zealand. We selected familiar practices as a refere= nce point rather than a new practice so that participants would be more confide= nt in assessing the model’s output. We clarified that the intention of the workshop was not to validate the model results against measured adoption. Current adoption of these practices was estimated using the results of a su= rvey of pastoral farmers in New Zealand (Montes de Oca Munguia 2020). Each group= was asked to select an example from four available options:
· Use = of body condition scoring. The assessment of Body Condition Scoring (BCS) is us= ed to estimate body fat reserves in both cows (visual) and ewes (feeling backb= one with fingers and thumb). BCS is used as a management tool to determine feed requirements and improve reproductive performance.
· Use = of pasture management software. Use of computers, tablets and smartphone a= pps or programmes to calculate feed demand, feed availability and feed quality = for sheep and cattle at any time of the year and for different levels of production. Information can be used for both tactical and strategic decisio= ns.
· Use = of Plantain and/or Lucerne for summer grazing. Plantain and Lucerne are us= ed to increase the amount and quality of summer feed in grazing systems. Plant= ain is more often used as a pasture mix, but it can also be used as a special purpose crop, lasting 2-3 years. Lucerne is used on soils with low soil moisture holding capacity to increase production.
· Use = of a formal, audited nutrient management plan. Used to actively manage nutri= ents (N and P) on the farm in a formal, audited way. They can be developed in conjunction with a fertiliser consultant or as part of an environmental plan developed by industry or local government. Nutrient management may include managing the type, placement and timing of ferti= liser applications; crop rotations; precision application; and stock exclusion fr= om waterways.
All practices have been available in the industry for at least=
20
years, but they all showed different levels of current adoption by the time=
the
workshops took place. The survey found body condition scoring was the most
widespread practice amongst surveyed farmers, reaching 74% adoption. This w=
as
followed by using Plantain and Lucerne for summer grazing (56%), the use of
pastoral management software (45%) and the use of nutrient management plans
(38%).
Participants at each workshop were asked to select t=
he
practice they were more familiar with or more interested in. The =
use of
formal, audited nutrient management plans amongst dairy farmers was used as=
the
example for the first workshop. The use of pasture management software amon=
gst
dairy farmers was used as the example for the second workshop, and the use =
of a
formal, audited nutrient management plans amongst sheep and beef farmers wa=
s used
as the example for the third workshop.
The workshops consisted of three facilitated steps:
· A presentation and discussion about ADOPT’s conceptual model, covering t= he model’s specifications of drivers and their causal relationships.
· Running ADOPT on-line to produce a diffusion curve and compare it with the survey’s measured adoption for the selected practice.
· An exercise to use ADOPT’s sensitivity analysis to adjust the initial forecast= to be closer to the survey’s measured adoption curve, discussing the assumptio= ns behind observed discrepancies.
The ADOPT conceptual model shown in Figure 1 was pre= sented by the facilitator and discussed by the group. The presentation also covered the mechanics of using ADOPT in the workshop: participants were required to discuss and agree on the answers to 22 questions about the numbered variabl= es in the four quadrants represented in Figure 1. In ADOPT, responses are used= in mathematical functions to predict time to peak adoption and peak adoption l= evel using cause-effect relationships for each relationship shown in the concept= ual model.
The first high-level driver of adoption in the ADOPT= model is relative advantage. Theories related to relative advantage for modelling= are well supported in literature and include subjective expected utility theory (Tversky & Kahneman 1981), prospect theory (Laibso= n & Zeckhauser 1998), and multi-attribute uti= lity theory (Huang et al. 2011). The application of these theories in models of decision-making have been reviewed extensively (e.g. B= ehzadian et al. 2010). The use of different population orientations (i.e. preference= s) to weight characteristics of the practice when calculating overall relative advantage in the conceptual model illustrates the interplay between rewards, effort and risk in decision making in agriculture, as proposed by Nicholson= et al. (2015).
Figure 1. The ADOPT concep=
tual
model
Source: Kuehne et al. (2017)
The second high-level driver of adoption in the ADOPT model is learning, which is a key focus for extension professionals. The AD= OPT conceptual model identifies cause-effect relationships affecting the likeli= hood of adoption and the time lag from the availability of the innovation to the decision to adopt it in three stages as suggested by Lindner et al. (1982). They are: first, the discovery stage – the time it takes for the producer t= o be aware of the existence of the innovation; second, the evaluation stage – the time from awareness to first use, on a trial basis; and third, the trial st= age – the time from the initiation of trial use to the acceptance of the innovation. Thus it covers the stages of awarene= ss, trialling and adoption as outlined by Pannell et al. (2006).
The process of using ADOPT in all workshops consiste= d of running the on-line version of ADOPT and having a 1 to 3-minute group discussion about each of the 22 questions until agreement was reached on an answer. ADOPT results were then presented, compared with the survey results, and discussed.
Figure 2 shows an example of the ADOPT questions participants were asked to answer. The question and its explanation are displayed on the top-right. After group discussion, participants agreed on = an answer (bottom-right) and moved on to the next question. The list of variab= les and their correspondence of each variable to a quadrant in the conceptual m= odel is displayed in the list on the left-hand side of the screen. A printout of= the conceptual model (Figure 1) remained visible to participants throughout the process, so the facilitator could point out the links between each numbered question and their relationships with other variables.
Table 1 shows, for each workshop, the estimated curr= ent uptake level and average time to adopt from the survey and the ADOPT outputs resulting from the group’s discussion. The table shows differences between = each group’s predictions and the survey estimates of current adoption levels. Pa= rticipants were reminded that the intention of the comparison was not for validation purposes, but rather to reflect on the characteristics of the practice and = the population that might be behind the differences.
Workshop 1 participants produced a peak level predic= tion relatively close to the estimated current uptake level, but an unusually lo= ng time to peak. In contrast, workshop 2 participants produced a prediction of= the time to reach peak adoption close to estimates of current adoption times but predicted peak adoption levels were much higher than current uptake levels. Predictions generated by workshop 3 participants were much higher than curr= ent adoption levels and involved a much longer time to reach the predicted peak than the average time to adoption experienced by adopters so far.
Figure 2. Example of on-line ADOPT used in Workshop 2
Table 1. Simulated results in relation to current lev= els of uptake from a recent survey
|
Workshop |
Practice |
Measured current adoption level from s=
urvey |
Average time to adoption estimated from
survey |
Predicted peak adoption level from
participants’ ADOPT run |
Predicted time to near-peak adoption f=
rom participants’
ADOPT run |
|
<=
span
style=3D'mso-bidi-font-size:8.0pt'>1 |
Audited nutrient management plans |
38% |
6 years |
59% |
21 years |
|
<=
span
style=3D'mso-bidi-font-size:8.0pt'>2 |
Pasture management software |
45% |
12 years |
94% |
13 years |
|
<=
span
style=3D'mso-bidi-font-size:8.0pt'>3 |
Audited nutrient management plans |
38% |
6 years |
93% |
13 years |
The last step in the workshop was an exercise to all= ow adjustment of participants’ inputs, informed by ADOPT’s sensitivity analysis and the predictions generated by the initial inputs. The purpose of this exercise was to allow participants to identify the variables that, if adjus= ted, would have the most impact on the outputs. Participants were encouraged to explore different combinations of variables that would adjust their initial prediction and discuss the assumptions behind their answers. In this case, = the sensitivity analysis identifies the variables that h= ave the most effect on the model’s predictions of peak level of adoption and ti= me to peak adoption. Figure 3 shows ADOPT’s sensitivity analysis for Workshop 2.
The horizontal axis shows the q=
uestion
numbers, which correspond to each numbered variable in the conceptual model=
. In
each graph, the vertical axis shows the step changes that occur in each out=
put
by changing the answer to the question one level up or down. The change in =
peak
adoption level occurs in terms of adoption percentage and the time to peak
adoption is measured in number of years.
Figure 3. ADOPT’s sensitivity analysis diagram used to analyse the use of
pasture management software in Workshop 2
Sensitivity analysis allowed pa= rticipants to make separate adjustments to both their prediction of peak adoption leve= ls and time to reach peak adoption by exploring different sets of variables (as illustrated in the conceptual model – Figure 1). For example, participants = in workshop 2 aimed at reducing their predicted peak adoption level while retaining their predicted time to reach peak adoption. Figure 3 shows that = the most sensitive variable for changing the predicted peak adoption level on t= he use of pasture management software was the level of profit benefits in years that the practice is used (question 16), followed by profit benefits in the future (question 17) and environmental costs and benefits (question 19). Fi= gure 3 shows that, for example, adjusting the answer to question 16 one level do= wn would reduce the prediction by 21%. In their example, this would adjust the= ir initial peak adoption prediction from 94% to 73%. Participants agreed to re= duce this variable by one level. After making this adjustment, the process was repeated to decide whether to make more adjustments to the same variable or adjust other variables, making constant references to the conceptual model<= /span>. Table 2 shows, for each workshop, the variables th= at were adjusted from the initial run and the resulting adjusted predictions.<= o:p>
The wrap-up discussion after the
sensitivity analysis centred on the feasibility of predicting adoption, the
general use of prediction models in extension and the ‘strengths and
weaknesses’ of using ADOPT for extension planning. There was discussion abo=
ut
factors that affect the diffusion of innovations that are not only outside =
the
individual farmer’s control but also outside the scope of any model. For
example, participants highlighted the role of impending legislation in the
future uptake of audited environmental farm management plans. Experts obser=
ved
that the ‘pressure to comply’ amongst the farming community has increased
recently, and therefore peak adoption level in a voluntary basis has not be=
en
reached for this practice yet. Participants did not feel this pressure coul=
d be
captured explicitly in the relative advantage equation.
Table 2. Comparison between initial and adjusted ADOPT predictions for each workshop
| <= /td> |
Workshop 1 |
Workshop 2 |
Workshop 3 |
|||
|
Predictions |
Initial |
Adjusted |
Initial |
Adjusted |
Initial |
<=
span
lang=3DEN-US style=3D'mso-ansi-language:EN-US;mso-fareast-language:EN-AU;
mso-bidi-language:EN-US'>Adjusted |
|
Peak adoption |
59% |
48% |
94% |
66% |
93% |
<=
span
lang=3DEN-US style=3D'mso-ansi-language:EN-US;mso-fareast-language:EN-AU;
mso-bidi-language:EN-US'>35% |
|
Time to near peak adoption |
21 years |
10 years |
13 years |
13 years |
13 years |
<=
span
lang=3DEN-US style=3D'mso-ansi-language:EN-US;mso-fareast-language:EN-AU;
mso-bidi-language:EN-US'>12 years |
|
ADOPT answers |
|
|
|
|
|
<=
span
lang=3DEN-US style=3D'mso-ansi-language:EN-US;mso-fareast-language:EN-AU;
mso-bidi-language:EN-US'> |
|
1. Profit orientation |
a
minority |
|
almost all |
|
a
minority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
2. Environmental orientation |
about
half |
|
a minority |
|
a
majority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
3. Risk orientation |
a
majority |
|
a minority |
|
a
majority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
4. Enterprise scale |
almost
all |
|
almost all |
|
almost
all |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
5. Management horizon |
a
majority |
|
almost none |
|
a
majority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
6. Short-term constraints |
almost
all |
|
almost all |
|
almost
none |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
7. Trialing ease |
moderately
easy |
easily |
easily |
|
moderately
difficult |
d=
ifficult |
|
8. Innovation complexity |
very
difficult |
slightly
difficult |
difficult |
|
difficult |
m=
oderately
difficult |
|
9. Observability |
moderately |
easily |
difficult |
|
difficult |
m=
oderately
difficult |
|
10. Advisory support |
a
minority |
about
half |
about half |
|
a
majority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
11. Group involvement |
a
minority |
|
a minority |
|
a
minority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
12. Relevant existing skills & knowledge |
almost
all |
a
majority |
about half |
|
a
majority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
13. Innovation awareness |
a
minority |
about
half |
almost all |
|
a
majority |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
14. Relative upfront cost of innovation |
moderate |
|
minor |
|
large |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
15. Reversibility of innovation |
very
easily |
|
very easily |
|
easily |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
16. Profit benefit in years that it is used |
small
profit advantage |
|
moderate |
small |
moderate |
s=
mall |
|
17. Profit benefit in future |
moderate
profit ad |
|
moderate |
small |
small |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
18. Time for future profit benefits to be realized |
3 to 5
years |
6 to 10
years |
1 -2 years |
|
3 to 5
years |
6=
to 10
years |
|
19. Environmental impact |
small |
|
no net |
|
large |
s=
mall |
|
20. Time for environmental impacts to be realized |
3 to 5
years |
|
not applicable |
|
1 to 2 |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
21. Risk |
small
reduction |
|
small |
|
moderate |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
|
22. Ease and convenience |
no change |
|
small increase |
|
small
decrease |
<=
span
style=3D'mso-bidi-font-family:Calibri'> |
Some participants suggested that
ADOPT could be complemented with other models analysing ‘macro’ variables, =
such
as bioeconomic models, but many participants felt that extension planning
should not be ‘run by models’. However, there was also discussion about the
strengths of ADOPT. Participants expressed that the model included the fact=
ors
than need to be in place to assess the ‘adoptability’ of an innovation. Aft=
er
this discussion participants were asked to fill in the post-workshop
questionnaire and the workshop concluded.
The frequency of responses = from both pre<= /span> and post-workshop questionnaires were summarised. Due to the ordinal nature of the data and the small sample size= , we grouped and reported responses for each question (collected on a scale from= 1 to 10) into three intervals: low scores (e.g. rarely, disagree, not useful), medium scores (e.g. sometimes, moderately agree, moderately useful) and high scores (e.g. often, agree, useful).
We also co= nducted non-parametric testing to explore the statistical relationships between responses. Recognising the limitations arising from our small sample size, = we consider these tests could be used more confidently in subsequent studies w= ith larger samples. Firstly, we conducted a paired-samples Wilcoxon test (i.e. Wilcoxon signed-rank test) to detect a shift in the perception of participa= nts using the responses to four questions that were asked both before and after= the workshop. Those questions were:
·&nb= sp; In your view, is it possible to predict adop= tion adequately?
·&nb= sp; In your view, how important are the following factors in driving adoption:
o <= /span>Characteristics of the farmer.
o <= /span>Characteristics of the technology or practice.
o <= /span>External factors, out of farmers’ control.
Secondly, = we defined five variables based on the responses of a selection of post-worksh= op questions (Table 3). The variables were used to build an ordered logit regression model to identify the level of contribution of each ADOPT featur= e in the participant’s perception of usefulness of the discussion.
Table 3. Variables used to test the usefulness of the workshop components
|
|
Question |
|
Dependent variabl= e |
How useful did you find the workshop to focus technical discussion of adoption? |
|
Independent varia= bles |
How do you rate the contribution of the fo= llowing model features explored today to the discussion: Diffusion s-curve Sensitivity analysis Conceptual model Pr= ocess of group discussion |
Participants were asked to identify tasks in their w= ork that required them to think about future adoption. Many participants were involved in the evaluation of research projects involving farmers (35%) or = in decisions to invest in new technologies or practices (38%). In addition, mo= st participants were directly involved in designing (88%) and implementing (74= %) extension strategies to increase adoption of beneficial practices.
Participants were asked how often they considered the likely adoption of a technology or practice in their work. Results showed t= hat 15% of participants seldomly considered adoption, while about a quarter considered adoption occasionally (24%). Almost two-thirds of participants indicated they needed to consider adoption regularly (62%).
Prior to the workshop, most participants considered = that predicting the adoption of a new technology or practice was useful. Results showed that 41% of participants considered it very useful, 44% considered it moderately useful and 15% only slightly useful.
However, while most respondents to the pre-workshop = survey considered predictions useful, 38% did not believe it was possible to predi= ct adoption adequately (Table 4).
When asked about the importance of the characteristi= cs of the farmer in driving adoption, over two-thirds of participants considered = it important (71%), 24% or participants considered it neutral, and 5% consider= ed it not important. Participants rated the importance of the characteristics = of the technology in driving adoption as important (81%), or neutral (19%). Finally, opinions before the workshop were more divided amongst participant= s about the importance of external factors (factors out of farmers’ control) in dri= ving adoption. Half the group considered them important (52%), 33% considered th= em neutral and 14% considered them not important.
After the workshop, participants were asked whether = their participation in the workshop changed the way they understood adoption. More than half of respondents agreed with this statement (55%), 27% were neutral= and 18% of participants disagreed. Table 4 shows the responses to questions included in both the pre and post-workshop questionnaires. The largest changes to participants’ perceptions following = the workshop were an increase in the perception that it is possible to predict adoption adequately, and an increase in the importance ascribed to external factors out of farmers’ control as influences on adoption.
Table 4. Participants’ changes in opinion before and = after the workshop
|
In your view,=
is
it possible to predict adoption adequately? |
Disagree |
Neutral |
Agree |
|
|
Pre-workshop |
38% |
29% |
33% |
|
|
Post-workshop |
14% |
24% |
62% |
|
|
<=
b>In your
view, how important are the following factors in driving adoption |
Not import=
ant |
Neutral |
Important<= o:p> |
|
|
C= haracteristics of the farmer |
Pre-workshop |
5% |
24% |
71% |
|
Post-workshop |
5% |
21% |
74% |
|
|
C= haracteristics of the technology or practice |
Pre-workshop |
0% |
19% |
81% |
|
Post-workshop |
0% |
5% |
95% |
|
|
E= xternal factors, out of farmer’s control |
Pre-workshop |
14% |
33% |
52% |
|
Post-workshop |
5% |
16% |
79% |
We also asked participants to rate the usefulness of= the workshop and the different features of ADOPT to generate focused technical discussio= n of adoption. All participants considered that the workshop was useful. Table 5 shows how participants rated the usefulness of different components we evaluated.
Table 5. Participants’ rating of the usefulness of wo= rkshop components
|
How do you r=
ate
the usefulness of the following information and model features explored t=
oday |
Not useful |
Neutral |
Useful |
|
Conceptual m=
odel |
0% |
4% |
<=
span
lang=3DEN-NZ style=3D'mso-ansi-language:EN-NZ;mso-fareast-language:EN-NZ'=
>96% |
|
Process of g=
roup
discussion |
0% |
0% |
<=
span
lang=3DEN-NZ style=3D'mso-ansi-language:EN-NZ;mso-fareast-language:EN-NZ'=
>100% |
|
Diffusion S-=
curve |
8% |
25% |
<=
span
lang=3DEN-NZ style=3D'mso-ansi-language:EN-NZ;mso-fareast-language:EN-NZ'=
>67% |
|
Sensitivity
analysis |
4% |
13% |
<=
span
lang=3DEN-NZ style=3D'mso-ansi-language:EN-NZ;mso-fareast-language:EN-NZ'=
>83% |
All components of the workshop, including the ADOPT conceptual model, ADOPT results and the process of using it were considered useful by most of the participants. The presentation of the ADOPT conceptual model was considered useful by most participants and the process of group discussion was considered useful by all participants. Regarding ADOPT outpu= ts, most participants considered them useful, with some participants being neut= ral about them and one or two participants considered them not useful.
Finally, we asked participants what actions they were likely to take in relation to predicting adoption after the workshop. A minority (4%) said they would take no action, 25% said they would reassess their practice, 33% said they would change their approach or advice, and the majority (58%) said they would seek extra information or training.
We had limited success regarding the two analyses us= ed to explore the statistical relationships between responses, possibly due to the low number of observations (n=3D24). However, the paired samples Wilcoxon t= est detected a significant shift in two of the perceptions of participants listed in Tab= le 4. The test indicated a significant difference in the perceived ability of models to predict adoption after the workshop (p =3D 0.046), and the importance of external factors (p =3D 0.044). According to the test, participants’ opinion did not change significantly regarding the importance= of characteristics of farmers and the practice after the workshop. On the other hand, the regression analysis aimed at identifying which factors from Table= 5 had the most influence on the rating of the overall usefulness of the works= hop did not identify any statistically significant factors (p<0.05).<= /p>
Was it the model or the process of using it? Our res= ults show that all participants rated the process of group discussion to be usef= ul, although a minority of participants considered the model to be not useful. = Our results therefore support previous research finding that the process of usi= ng models is considered more valuable than the model itself (Jakku & Thorburn 2010; Lempert 2015).
However, previous studies gener=
ally
only focus on contrasting model results vs process. We extended the analysi=
s by
investigating three specific aspects of a model that may influence its
usefulness for extension planning: understanding the system, forecasting
In line with previous studies (=
e.g.
Phillips & Linstone 2016), our results showed that the model’s predicted diffusion
curve (i.e. model results) was ranked by participants only as the third most
useful component. Even though ADOPT was able to generate adequate predictio=
ns
of adoption, more participants found the other two components even more
valuable than the predictions per se.
The most useful model component=
was
the conceptual model. Participants used it to discuss how the system works =
and
to identify the key driving forces and relationships behind the adoptabilit=
y of
an innovation, identifying how drivers affect either the likelihood of adop=
tion,
the speed of adoption or both.
This was followed by the
sensitivity analysis. Participants were able to evaluate alternative outcom=
es,
identifying drivers that where under their control (e.g. advisory support,
improve awareness) and those they cannot control (e.g. the priority given by
the farmers to issues such as profit, risk and t=
he
environment). They used sensitivity analysis to discuss how different adopt=
ion
strategies could work in practice.
We thus consider that this study can move the discus=
sion
of model usefulness further by offering a more nuanced analysis of a model’s
ability to generate focused technical discussions, as proposed by Forester
(1999) and reduce uncertainty in decision making (Kalra et al. 2014).
There are two points not covere=
d in
this study that could limit its findings. We did not investigate how planni=
ng
occurs in the absence of a model and we did not investigate the role of mod=
els
in participants’ learning. Regarding the second point, we believe there is =
an
opportunity to further understand models as learning tools. Participants in=
our
workshops reported a change in their understanding of adoption,
suggesting that their participation was useful in dee=
pening
their knowledge of the adoption process. They also indicated an intention to
change their current practice because of the workshop, either by reassessing
their current practice, changing their approach or advice, or seeking extra
information or training.
Conclusion
The objective of this paper was to improve our understanding of the ‘usefulness’ of models in planning for extension, and whether extension officers perceived the model or the process of using it t= o be more useful. We conducted a series of workshops using ADOPT as the model to analyse the adoption of well-known practices and asked participants in our workshops to evaluate the model and the process of using it in assisting extension planning. Our results support previous research finding that the process of using models is considered more valuable than the model outputs,= but we also offer a more nuanced understanding of model usefulness that goes be= yond using models as ‘black boxes’ to produce adoption projections.
In our workshops, we found that
participants have pragmatic attitudes towards the use of models in planning,
even when some felt that extension planning should not be ‘run by models’. =
At
each workshop, there were multiple discussions about the general use of
prediction models and the ‘strengths and weaknesses’ of using them for
extension planning. It was concluded that ADOPT was useful to think about t=
he
‘adoptability’ or ‘adoption potential’ of an innovation, but that external
factors that are outside of the scope of any model should also be kept in m=
ind
when using the results.
Lastly, this study highlights the importance for mod= el developers considering a full range of model features to better assist extension planning to im= prove the uptake of beneficial agricultural innovations.
We acknowledge the support of A= gResearch and Landcare Research in the development of the research underpinning this study. We also acknowledge Denise Bewsell from = the Red Meat Profit Partnership and Callum Eastwood and Ina Pinxterhuis from Da= iry NZ for their valuable contribution in the selection of technologies and practices to be included in the survey and workshop organisation.
Allen W, Cruz J & Warburton B 2017, ‘How decision support systems can benefit from a theory of change approach’, Environmental Management, vol. 59,= no. 6, pp. 956–965, https://dx.doi.org/10.1007/s00267-017-0839-y.
Antle JM, Stoorvogel JJ & Valdivia RO 2014, ‘New parsimonio= us simulation methods and tools to assess future food and environmental securi= ty of farm populations’, Philosophical Transactions of the Royal Society of London, vol. 369, no. 1639, pp. 20120280, https://dx.doi.org/10.1098/rstb.2012.0280.
Behzadian M,
Box GEP & Lucefio A 1998, ‘Quality quandaries models, assumptions, and robustness’, Quality Engineering, vol. 10, no. = 3, pp. 595-598, https://dx.doi.org/10.080/08982119808919174.
Douthwaite B & Hoff= ecker E 2017, ‘Towards a complexity-aware theory of change for participatory rese= arch programs working within agricultural innovation systems’, Agricultural Systems, vol. 155, pp. 88–102, https://dx.doi.org/= 10.1016/j.agsy.2017.04.002.
Epstein JM 2008, ‘Why Model?’, Journal of Artificial Societies and So= cial Simulation, vol. 11, no. 412. Available from: <http://jasss.soc.surrey.ac.uk/11/4/12.html> [18 June 2020].
Forester J 1999, The deliberative practit= ioner: encouraging participatory planning processes, MIT Press, Cambridge, MA, USA.
Fountas S, Wulfsohn D, Blackmore BS, Jacobsen HL & Pedersen SM 2006, ‘A model of decision-making and information flows for information-intensive agriculture’, Agricultural Systems, vol. 87, no. 2, p= p. 192–210, https://dx.doi.org/10.1016/j.agsy.2004.12.003.
Haasnoot M, Kwakkel JH, Walker WE = & ter Maat J 2013, ‘Dynamic adaptive policy pathways: A method for crafting robust decisions for a deeply uncertain world’, Global Environmental Change, vol. 23, no. 2, pp. 485–498, https://dx.doi.org/10.1016/j.gloenvcha.2012.12.006.
Huang IB, Keisler J
& Linkov I 2011, ‘Multi-criteria decision
analysis in environmental sciences: Ten years of applications and trends’, =
Science of the Total Environment, =
vol. 409,
no. 19, pp. 3578–3594, https://dx.doi.org/10.1016/j.scitotenv.2011.06.022.<=
/p>
Innes JE & Booher DE
1999, ‘Consensus building and complex adaptive systems’, APA Journal, vol. 65, no. 4, pp. 412–423. Jakku E & Th=
orburn
PJ 2010, ‘A conceptual framework for guiding the participatory development =
of
agricultural decision support systems’, Agricultural
Systems, vol. 103, no. 9, pp. 675–682, https://dx.doi.org/10.1016/j.ags=
y.2010.08.007. Kaine G, Wright V, Cooksey R & Bewsell D 2011, ‘Identif=
ying
potential adopters of an agricultural innovation’, in Changing land management adoption of new practices by rural landhol=
ders,
eds. DJ Pannell DJ & F =
Vanclay, CSIRO Publishing, Collingwood, Vic, pp. 69–8=
6. Kalra N, Hallegatte S, Lempert RJ, Brown =
C, Fozzard
A, Gill S & Shah A 2014, ‘Agreeing on robust decisions new proce=
sses
for decision making under deep uncertainty’, World Bank Policy Research Working Paper, no. 6906,
https://dx.doi.org/doi:10.1596/1813-9450-6906. Kuehne G, Llewellyn R, Pannell DJ, Wilkinson R, Dolling P, Ouzman J
& Ewing M 2017, ‘Predicting farmer uptake of new agricultural
practices: A tool for research, extension and policy’, Agricultural Systems, vol. 156, pp. 115–125, https://dx.doi.org=
/https://doi.org/10.1016/j.agsy.2017.06.007. Laciana CE &=
Oteiza-Aguirre N 2014, ‘An agent based multi-optional=
model
for the diffusion of innovations’, Physica A: Stati=
stical
Mechanics and its Applications<=
/i>,
vol. 394, pp. 254–265, https://dx.doi.org/10.1016/j.physa.2013.09.046. Laibson D & =
Zeckhauser R 1998, ‘Amos Tversky and the ascent of
Lempert RJ, Groves DJ, Popper SW &= amp; Bankes SC 2006, ‘A general, analytic method for generating robust strategies and narrative scenarios’, Management Science, vol. 52, no. 4, pp. 514–528, https://dx.doi.org/10.1287/mnsc.l= .
Lempert RJ 2015,= ‘Key note presentation - A new decision science for complex systems: A decade of enabling tools’, in IIASA Systems Analysis Conference. Luxembourg. Available from: <https://www.youtub= e.com/watch?v=3Da0I4buEvjl4> [15 January 2020].
Lindner RK, Pardey PG & Jarret FG 1982, ‘Distance to information source and the time lag to e= arly adoption of trace element fertilisers’, Australian Journal of Agricultural Economics, vol. 26, no. 2, pp. 98–133, https://dx.doi.org/10.1111/j.1467-8489.1982.tb00618.x.
Liu T, Bruins RJF & Heberling MT 2018, ‘Factors influencing farmers’ adoption of best management practice= s: A review and synthesis’, Sustainabili= ty, vol. 10, no. 2, pp. 1–27, https://dx.doi.org/10.3390/su10020432.
Maier HR, Guillaume JHA, van Delden H, Riddell GA, Haasnoot M & Kwakkel JH 2016, ‘An uncertain future, deep uncertainty, scenarios, robustness and adaptation: How do they fit together?’, Environmental Modelling & Software, vol. 81, pp. 154–164, https://dx.doi.org/10.1016/j.envsoft.2016.03.014.
Montes de Oca Munguia O 2020, Improving the forecasting of adoption in agriculture, PhD thesis, The University of Weste= rn Australia.
Montes de Oca Munguia O & Llewellyn R 20= 20, ‘The Adopters versus the Technology: Which Matters More when Predicting or Explaining Adoption?’, Applied Econ= omic Perspectives and Policy, vol. 42, no. 1, pp. 80–91, https://dx.doi.org/= 10.1002/aepp.13007.
Nicholson C, Long J, England D, Long B, Creelman Z, Mudge B & Corn= ish D 2015, Farm decision making= : The interaction of personality, farm business and risk to make more informed decisions, Grains Research and Development Corporation, Kingston ACT.
Pannell D & Claassen R 2020, ‘The Roles = of Adoption and Behavior Change in Agricultural Po= licy’, Applied Economic Perspectives and P= olicy, vol. 42, no. 1, pp. 31–41, https://dx.doi.org/10.1002/aepp.13009.
Phillips F & Linsto= ne H 2016, ‘Key ideas from a 25-year collaboration at technological forecasting & social change’, Technological Forecasting and Social Change, vol. 105, pp. 158–166, https://dx.doi.or= g/10.1016/j.techfore.2016.01.007.
Prinsen G & Nijhof<= /span> S 2015, ‘Between logframes and theory of change: Reviewing debates and a practical experience’, Development in Practice, vol. 25, no. 2, pp. 234–246, https://dx.doi.org/10.1080/09614524.2015.1003532.
Rosegrant MW,
Sartas M, Schut M, Proietti C, Thie= le G & Leeuwis C 2020, ‘Scaling Readiness : science and practice of an approach to enhance impact of research for development’, Agricultural Systems<= /i>, vol. 183, pp. 1–52, https://dx.doi.org/10.1016/j.agsy.2020.102874.
Schreinemachers = P & Berger T 2011, ‘An agent-based simulation model of human-environment interactions in agricultural systems’, Environmental Modelling and Software, vol. 26, no. 7, pp. 845–859, https://dx.doi.org= /10.1016/j.envsoft.2011.02.004.
Thornton PK, Schuertz T, Forch W, Cramer L, Abreu D, Vermeulen S & Campbell BM 2017, ‘Responding to global change: A theory of change approach to making agricultural research for development outcome-based’, Agricultural Systems, vol. 152, pp. 145–153, https://dx.doi.org/10.1016/j.agsy.2017.01.005.
Tversky A & Kahneman D 1981, ‘The Framin= g of Decisions and the Psychology of Choice’, Science, vol. 211, no. 4481, pp. 453–458, https://dx.doi.org/10.1126/science.7455683= .
Vanclay F 2011, = ‘Social principles for agricultural extension in facilitating the adoption of new practices’, in Changing land manage= ment adoption of new practices by rural landholders, eds DJ Pannell & F = Vanclay, CSIRO Publishing, Collingwood, Vic, pp.51–68= .
Veldkamp A 2009, ‘Investigating land dynamics: future research perspectives’, Journal of Land Use Science, vol. = 4, no. 1, pp. 5–14, https://dx.doi.org/10.1080/17474230802645592.
Vicsek T 2002, ‘= The bigger picture’, Nature, vol. 4= 18, p. 131, https://dx.doi.org/10.1038/418131a.
Walker WE 2000, ‘Policy analysis: a systemat= ic approach to supporting policymaking in the public sector’, Journal of Multi-Criteria Decision Analysis, vol. 9, no. 1, pp. 11–27, https://dx.doi.org/10.1002/1099-1360(200001/05)9:1/3<11::AID-MCDA264>3.0.CO;2-3.
Wauters E & = Mathijs E 2014, ‘The adoption of farm level soil conservation practices in developed countries: a meta-analytic review’, International Journal of Agricultural Resources, Governance and Ecology, vol. 10, no. 1, pp. 78–102, https://dx.doi.org/10.1504/IJARGE.2014.061058.
Wilkinson R 2011, ‘The many meanings of adop= tion’, Changing land management adoption o= f new practices by rural landholders, eds. DJ Pannell & F Vanclay, CSIRO Publishing, Collingwood, Vic, pp.39–50.
Rural Extension & Innovation Systems Journal, 2020= 16(1) – Research © Copyright APEN
Rural Extension & Innovation Systems Journal, 2020= 16(1) – Research © Copyright APEN
Rural Extension & Innovation Systems Journal, 2020= 16(1) – Practice © Copyright APEN
Rural Extension & Innovation Systems Journal, 2020= 16(1) – Practice © Copyright APEN