Page 1 of 25
Advances in Social Sciences Research Journal – Vol. 11, No. 1
Publication Date: January 25, 2024
DOI:10.14738/assrj.111.16172.
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with
Instructive Feedback in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
Services for Science and Education – United Kingdom
Comparison of Two Maintenance Training Arrangements with
Instructive Feedback in Children with Autism
Daniel Carvalho de Matos
ORCID: 0000-0002-6793-0101
Universidade Federal do Maranhão, Instituto Evoluir
and Universidade Ceuma, São Luís – Maranhão, Brazil
Danielle Juliana Ribeiro Loureiro
ORCID: 0000-0001-8502-1510
Universidade Ceuma, São Luís, Maranhão, Brazil
Katiane Reis da Silva
ORCID: 0000-0001-8428-5215
Universidade Ceuma, São Luís, Maranhão, Brazil
Pollianna Galvão
ORCID: 0000-0001-7579-8852
Universidade Federal do Maranhão, Instituto Evoluir
and Universidade Ceuma, São Luís, Maranhão, Brazil
ABSTRACT
Instructive feedback (IF) can improve the efficiency of teaching in children with
autism spectrum disorder (ASD) by establishing emergent skills. The purpose of
this study was to compare the effects of two types of listeners responding
maintenance teaching with IF on acquisition of four untaught repertoires in three
children with ASD. In one of the teaching cases, attending and tact responses of
visual stimuli were required. The emergence was partial for all learners, with no
great difference in the efficiency of the types of teaching. Two of the new repertoires
were not demonstrated. Errorless performance of some relations was shown by all,
but for only one child immediate emergence was verified. The results were
discussed regarding the possibility of new investigations on alternative procedure
to IF that could produce the emergence of new repertoires.
Keywords: autism spectrum disorder, primary target, untaught repertoires, instructive
feedback.
INTRODUCTION
Children with Autism Spectrum Disorder (ASD) frequently show impairments in several non- verbal and verbal behavioral repertoires, demanding a comprehensive intervention curriculum
to ameliorate the deficits. According to Skinner [1], non-verbal and verbal operant behaviors
are shaped by consequences. These are called reinforcers when the likelihood of future
emission of the behaviors increases. Procedures in Applied Behavior Analysis (ABA) commonly
Page 2 of 25
2
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
aims to establish repertoires in children with ASD through the arrangement of reinforcement
contingencies [2, 3]. A nonverbal operant is the product of interactions with the physical
environment (e.g., a child reaches out to grab a toy from a shelf). A verbal operant behavior is
shaped by a mediated reinforcing consequence. During a verbal episode, the speaker emits the
verbal behavior and the listener, specially trained by the verbal community, delivers a
reinforcer to the speaker (e.g., a child asks his mother for a cookie, and she allows him access).
Research on ABA focused on developing procedures to expand skill acquisition in children with
ASD. One case, called instructive feedback (IF), involves the definition of secondary target
during the teaching of primary target. As an example, a given child is taught to say “airplane”
upon the presentation of a picture of airplane (primary target). When he/she contacts
differential reinforcement (e.g., praise and a tangible item), the interventionist provides an IF
information (e.g., by saying “airplane is a transportation”). Thereafter, a probe shows that the
child can tact (label) the picture according to class (e.g., by saying “transportation”) and upon
the provision of a supplementary question (e.g., “what is an airplane?”). Tact is a type of verbal
behavior that involves the emission of a vocal verbal response under the control of a non-verbal
discriminative stimulus, and the response is maintained by a generalized conditioned
reinforcer (an established form of attention) [1]. In the case of the example presented, the tact
according to class as a new skill (secondary target) is established in the participant’s repertoire
without direct teaching, and it is considered that the variable IF plays a role in this. The
literature points out that IF increases instructional gains [4].
Recent investigations assessed the effects of providing IF information during the maintenance
teaching of primary targets previously established in learners’ repertoires. It is considered that
this kind of instructional arrangement can make the process of establishing secondary targets
without direct teaching less effortful [5-8]. Since primary target maintenance training with IF
was one of the concerns of the current research, it is important to describe the previous
literature in detail regarding the methodological characteristics and main results.
Tullis et al. [5], for two children with ASD as participants, compared different locations in which
IF information was administered during the reinforcement of primary targets. Primary and
secondary targets consisted of tact pictures and listener responding by function, feature, and
class (LRFFC), respectively. It was previously said that the tact is a type of verbal behavior [1].
LRFFC, in turn, represents a listener repertoire with the emission of non-verbal stimuli
selection responses based on instructions that specify function, feature or class to which the
stimuli belong [3, 9]. As an example, when an array of three different pictures is shown to a
learner (e.g., pictures representing soap, sharpener, and bottle), he/she must select the picture
corresponding to a given instruction (e.g., selecting the picture of soap under the verbal
instruction “show me toiletry”). No differential reinforcement was provided in LRFFC trials in
the study. Two sets of three pictures were defined for both primary and secondary targets.
The tact of all stimuli was previously established in the participants’ repertoire. In baseline,
both children did not demonstrate LRFFC responses (secondary targets), or performance
remained below 20% correct responses. During intervention condition, two treatments
(involving tact response maintenance training) were compared through an adapted alternating
treatments design. For one of the treatments, contingent to the emission of tact correct
Page 3 of 25
3
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
responses, IF information was provided and it was followed by praise (e.g., after saying “soap”
under the picture of soap, an experimenter said, “soap is a toiletry, good job!”).
The other treatment was administered similarly to the first, but, contingent to correct
responses, praise was delivered before the IF information (e.g., after saying “bag” under the
picture of bag, the experimenter said “well done! The bag has a handle”). For both treatments,
whenever a learning criterion was reached in three consecutive sessions, a probe to verify
emergence of secondary targets was administered 30 min later (other activities were
conducted with the children before the probes). As a result, the secondary targets for both
participants were acquired faster when, after the emission of primary targets in training, the IF
information was presented before praise. Since this one was the best treatment, it was
implemented for both sets of stimuli. The effects of training tact pictures with IF were lasting,
since, for one participant, maintenance of secondary targets was demonstrated for 16 weeks.
For the other participant, maintenance occurred for 18 weeks. The participants’ parents
considered the procedures highly effective and socially acceptable.
The authors discussed data in the sense that the effectiveness of the procedures with IF was, in
part, due to the occurrence of indiscriminable contingencies (learning by observation). They
also emphasized that the participants showed strong tact and echoic repertoires. The echoic
(vocal imitation) is a type of verbal behavior that involves the emission of a vocal verbal
response under the control of a vocal verbal discriminative stimulus with which it maintains
point-to-point correspondence, and the response is maintained by a generalized conditioned
reinforcer (e.g., an interventionist presents the instruction “say dog”. A learner then responds
“dog” and receives verbal praise) [1]. In the study, although the participants did not overtly
echo the IF information during primary target maintenance training, it is possible that they did
so covertly, which may have influenced acquisition of secondary targets. Finally, Tullis et al. [5]
also considered that the characteristics of instructional context possibly influenced acquisition
of secondary targets, since all sessions were conducted in an environment in which ABA
therapy sessions were typically provided to the participants. Plus, the experimenters were
professionals with whom they were familiar.
Frampton and Shillingsburg [6] assessed the effects of listener responding maintenance
training, with IF information in the consequent portion of the contingency (e.g., pointing to the
picture of the State of Tennessee under the instruction “show me Tennessee”). After a correct
listener response, it was provided an IF information (e.g., “Nashville is the name of the capital
of that State”) and the effects were assessed on acquisition of new multiple targets without
direct reinforcement in two children with ASD as participants. These new targets were the
following: 1) listener responding under control of a characteristic (e.g., selecting the picture of
Tennessee under the question “what is the capital of Nashville?”); 2) tact of a characteristic (e.g.,
saying “Nashville” in the presence of the picture of Tennessee and the question “what is the
capital of this State?”); 3) intraverbal (e.g., saying “Nashville” under the question “what is the
capital of Tennessee?”); 4) reverse intraverbal (e.g., saying “Tennessee under the question
“what State has Nashville as capital?”). The training of each of three sets of stimuli in the study
consisted of three primary target maintenance sessions with IF. After each series of three
sessions, a probe to verify emergence of the new multiple repertoires was administered. As a
result, until the third probe, nearly all new targets emerged for the two participants. The
acquisition of the new skills was not delayed, indicating the efficiency of the training with IF.
Page 4 of 25
4
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
For one child, the emergence of tact of characteristic was not demonstrated. The children did
not echo the IF information during maintenance training, but the authors pointed out the
possibility of emission of covert echoic, influencing the establishment of secondary targets for
both children. The new multiple targets were maintained for two weeks.
In Frampton and Shillingsburg [6], the difference between the two listener responding
repertoires organized was the type of the verbal discriminative stimulus used. Both skills
involved the emission of selection responses under the control of verbal instructions. The
listener responding repertoire whose maintenance was taught represented a simpler case, that
is, each participant had to select a picture from an array under the control of the verbalization
of its name by an experimenter. The more complex case (as a new probed repertoire) consisted
of each participant selecting a picture corresponding to an instruction about a characteristic
(or feature) without including the picture’s name. Tacts had been previously defined in this
manuscript. As to the intraverbal, it involves the emission of a verbal response under the
control of a verbal discriminative stimulus with which it lacks point to point correspondence.
The response is maintained by a generalized conditioned reinforcer. This was true for the two
types of intraverbals defined in the research [1, 3, 9].
Laddaga Gavidia et al. [7] conducted a systematic replication of the research by Frampton and
Shillingsburg [6] in two children with ASD as participants. However, they made some additions
in the sense that: 1) along training, they demanded attending response to visual stimuli during
the delivery of IF; 2) measured the emission of echoic of IF information provided during
training; 3) included fill in the blank intraverbal probes. Similar to the previous research, the
dependent variables consisted of the following: 1) listener responding under control of a
characteristic (e.g., pointing to the picture of a scientist under the question “who helps making
discoveries?”); 2) tact of a characteristic (e.g., saying “make discoveries” in the presence of the
picture of a scientist and the phrase “she helps to...”); 3) fill in the blank intraverbal (e.g., saying
“make discoveries” in the presence of the phrase “the scientist helps to...”); 4) intraverbal under
a question (e.g., saying “making discoveries” under the verbal question “how does the scientist
help?”); 5) reverse intraverbal under a question (e.g., saying “the scientist” under the question
“who helps making discoveries?”).
Three sets of stimuli were programmed for each participant. During intervention, listener
response maintenance teaching was administered (e.g., pointing to the picture of scientist
under the instruction “show me scientist”). The emission of each correct and prompted
response was followed by praise and 20s access to a preferred tangible reinforcer. After this,
an experimenter positioned the picture close to the participant’s eyes and pointed to it. If the
participant did not look at the picture in 5s, the experimenter said “look”. If 5s more elapsed
without the participant observing the picture, it was positioned close to a preferred item until
the participant looked at it. Once the picture was clearly being observed, the IF information was
provided (e.g., “she makes discoveries”).
Along the study by Laddaga Gavidia et al. [7], after each one of several series of three
maintenance sessions to teach primary targets with IF, a probe to verify the emergence of all
the new multiple targets previously mentioned was conducted. As a result, emergence of the
new repertoires was demonstrated by one participant regarding all three sets of stimuli. The
acquisition was delayed, that is, it demanded several probes along additional training. As to the
Page 5 of 25
5
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
case of the other child, emergence of new skills occurred only for the first set. Both children
echoed the IF information in most trials along training of primary targets, which may have
influenced acquisition of new repertoires (in a less extent for the participant who showed
acquisition of new targets related only to one set of stimuli). Plus, the authors suggested that
demanding attending responses to the picture while providing IF information possibly
established listener and speaker behaviors, also facilitating the establishment of the new
targets of the study. Another important aspect facilitating the acquisition was the fact that both
participants had a robust repertoire of tacts and a history of listener behavior emerging after
the teaching of speaker behavior and vice-versa, referred as bidirectional naming [10].
One important limitation of Laddaga Gavidia et al. [7] was that for one participant procedural
modifications were applied. For two of three sets of stimuli, the new targets did not emerge
after training the primary target with IF over 30 sessions. So, the probes were no longer
conducted under extinction, that is, differential reinforcement for correct responses was
administered. However, this modification also seemed ineffective. So, a new modification
consisted of increasing, in each probe trial, the interval between the instruction and the
response from 5s to 10s. Considering the results for the two participants, the authors suggested
that the extent of the emergence of new repertoires through IF may be idiosyncratic. They also
suggested that the use of tangible reinforcers during the training of primary targets with IF may
have prevented the emergence of new repertoires. They said that tangible reinforcers are not
consumed fast, and they may compete with IF information, making attending responses to them
unlikely.
Another possible reason for the lack of emergence for one participant was the absence of
prerequisite skills for the emission of intraverbals, which require conditional discrimination.
Intraverbal questions contained two elements, which jointly should control the responses (e.g.,
“where does the eagle live?”). In the mentioned example, the word “live” alters the evocative
function of the word “eagle”, and vice-versa, and they are necessary together to avoid errors.
The participant who lacked emergence of new targets for two sets of stimuli possibly did not
have prerequisites for this kind of complex intraverbal [11, 12].
Cariveau et al. [8] also assessed the effects of primary target maintenance teaching on the
establishment of secondary targets without direct teaching in a child with ASD as participant.
In this study, the delivery of IF information during the training of primary targets occurred
intermittently, that its, after three teaching trials on average. The investigation was conducted
in two different moments. During the first assessment, the participant was 4 years old. The
second assessment happened when the participant was 6 years old. On both occasions, data
collection on echoic of IF information were systematically taken. Primary targets consisted of
listener responses (following instructions for the emission of simple motor actions) or simple
motor imitation responses.
Descriptions on how to teach the mentioned skills may be obtained in manuals which describe
training strategies to establish them. Through discrete trial teaching procedure, a successful
trial concerning each skill involves providing an instruction as a discriminative stimulus and up
to 5s for the emission of a target response by a learner. After a correct response, a reinforcer is
delivered (e.g., verbal praise; access to a tangible item). As an example of following instruction,
a learner claps his/her hands when an interventionist provides the instruction “clap your
Page 6 of 25
6
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
hands” as discriminative stimulus. By emitting the target response, the learner is praised and
get access to a preferred item. In the case of simple motor imitation responses, the learner
would emit the same type of response and access reinforcers, but the discriminative stimulus
in this case would consist of providing the non-verbal model of the clapping response [2, 3, 13].
After the emission of primary target correct responses in Cariveau et al. [8], the IF information
(e.g., “you raise your hand”) was presented along with a reinforcer and the participant had up
to 3s to emit an echoic of the information given. Fill in the blank intraverbals were defined as
secondary targets (e.g., saying “hand” in the presence of the verbal stimulus “you raise your...”).
Four and three sets of stimuli were programmed during first and second assessment,
respectively.
In Cariveau et al. [8], each set of stimuli involved three secondary targets, whose IF information
were presented three times during each primary target maintenance training session. Each
session included approximately 27 demands. Maintenance sessions were conducted once or
twice a day (there were one to three meetings with the participant per week). On each day, a
secondary target probe (intraverbal) was conducted before the training of primary targets.
Along several probes, if performance remained low on a given stimuli set, intraverbal training
would be implemented using a constant prompt delay procedure. During the second
assessment with the participant, an intervention with a more intrusive correction procedure
was needed (the antecedent stimulus and echoic prompt were presented three times). At the
end of the study (considering the two assessments together), the participant demonstrated
acquisition of two sets of secondary targets.
For three other sets of stimuli, the direct teaching procedure with constant prompt delay was
needed to establish secondary targets. Other two sets, however, demanded the more intrusive
teaching procedure. Data were discussed in the sense that learners not always show acquisition
of secondary targets through IF, that is, without direct teaching. As in the study by Laddaga
Gavidia et al. [7], Cariveau et al. [8] maintains that the delayed acquisition of new repertoires
through IF should not be expected. A more intrusive approach should be warranted instead.
Regarding the emission of echoic of IF information, which the literature suggests it is important
for acquisition of new targets, there was an increase of overt responses during the first
assessment. However, few echoic responses were demonstrated during the second assessment.
As in the case of other studies described in this manuscript, the authors hypothesized that the
echoic responses were emitted covertly, although the lack of secondary target emergence in
some sets of stimuli seems to not support such a hypothesis. Lastly, the authors pointed out the
definition of a single participant as a methodological limitation, limiting the conclusions
regarding the generality of the findings.
Other recent studies also investigated the effects of training primary targets with IF on the
establishment of new multiple targets without direct teaching, although in the beginning of
these studies the primary targets were not established in the learners’ repertoires (no
maintenance training sessions were programmed in the beginning)[14, 15]. Since these studies
assessed the influence of IF on the acquisition of multiple targets, which was one of the concerns
of the current research, they should be described in detail.
Page 7 of 25
7
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
In the research by Cordeiro [14], conducted with four children with ASD as participants,
primary targets consisted of simple tact responses (according to name). During training,
correct responses were followed by praise and IF information (e.g., saying “São Paulo” in the
presence of the picture of the State of São Paulo. Thereafter, the experimenter delivered praise,
the information “São Paulo is a State in the southeastern region” and showed a picture
representing the southeast region). For each participant, a set with three stimuli (pictures) was
programmed. The learning criterion during training consisted of two sessions without errors.
When the criterion was reached, a probe was conducted to check emergence of each of four
new repertoires. They were the following: listener responding according to class (e.g., selecting
the figure of the State of São Paulo under the instruction “show me a State in the southeast
region”); tact according to class (e.g., saying “southeast region” in the presence of the picture of
the State of São Paulo and the question “what region is the State of São Paulo from?”); arbitrary
visual pairing (e.g., relate the picture of the State of São Paulo to another in the southeast
region); intraverbal of listing names according to class (e.g., saying “São Paulo, Rio de Janeiro
and Minas Gerais” under the instruction “name States from the southeast region”). Along the
study, if a single probe session after training did not result in robust emergence of the
secondary targets, additional primary target training sessions with IF would be implemented,
as well as new probes to verify emergence of the new repertoires.
The teaching of skills regarding tact, listener responding and intraverbal was previously
explained as the relevant literature on the theme of this research was presented. As to the
repertoire concerning arbitrary visual pairing, a successful teaching trial, although this skill was
not directly taught through differential reinforcement in the study by Cordeiro [14], involves
providing a visual discriminative stimulus (e.g., an interventionist shows the picture of a dog
breed as a model or sample); the emission of a pairing response by a learner (e.g., the learner
selects a corresponding picture of another breed of dog from an array of different pictures); the
delivery of reinforcers by the interventionist [2, 3, 13].
As a result, in Cordeiro [14], all participants acquired the primary targets (simple tacts)through
differential reinforcement. Regarding the other targets according to class, they fully emerged
(without direct teaching) for one participant after only two primary target training sessions in
which criterion was reached. For a second participant, two of the new repertoires emerged
without errors (arbitrary visual pairing and intraverbal) and the other two emerged partially
(but many correct responses were emitted). One limitation was that additional training and
probe sessions were not implemented for this participant because his participation in the study
could no longer continue. In the case of a third participant, five probes indicated a very partial
emergence effect (performance, in fact, got worse along probes). For the last participant, no
emergence effect was demonstrated. Due to the result for this participant, the direct teaching
through differential reinforcement was established for one of the repertoires according to class
(tact according to class). Along this training, four additional probes for the remaining
repertoires according to class were conducted and they all emerged without errors.
Data for the last participant of the research by Cordeiro [14] is in accordance with the argument
by Cariveau et al. [8] that the delayed acquisition of new repertoires through IF should not be
expected. However, the new approach by Cordeiro was not very intrusive because only one of
the four secondary targets according to class was directly taught and emergence of the
remaining was demonstrated thereafter. Cordeiro [14] discussed these data in the sense that,
Page 8 of 25
8
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
if the IF procedure is not successful for a given learner, an alternative approach to produce
emergence of new repertoires should be considered. In fact, that was what happened to the last
participant. Plus, the emergence of one or more repertoires according to class because of
training others is consistent with the results of some learners from the previous literature
involving children with ASD as participants [16, 17]. Direct teaching of all repertoires
representing secondary targets should be planned and implemented as a last resort.
Tullis et al. [15], as Cordeiro [14], trained simple tacts (e.g., saying “calculator” in the presence
of the picture of calculator and the question “what is this?”). The IF information was delivered
in the consequent portion of the contingency (e.g., the experimenter said to the participant “a
calculator has batteries”). The secondary targets were the following: LRFFC (e.g., selecting the
picture of calculator under the instruction “touch the item that has batteries”); tact under
control of function, feature or class (e.g., saying “calculator” under the picture of calculator and
the instruction “say the name of the item that has batteries”); intraverbal (e.g., saying
“batteries” under control of the verbal antecedent “a calculator has...”); reverse intraverbal
(e.g., saying “calculator” under the verbal antecedent “something that has batteries is...”). A
probe for each of the secondary targets was conducted after each series of three primary target
training sessions with IF. The three participants (children diagnosed with ASD) successfully
acquired both the primary and secondary targets (considering three sets of stimuli). The
acquisition of the secondary targets was delayed since the emergence occurred in subsequent
tests along training with IF. Maintenance of the repertoires was demonstrated for two months.
In short, the recent literature on IF and its effects in children with ASD as participants showed
that new repertoires may be established without direct teaching. This was demonstrated both
through maintenance training sessions, in which primary targets were already established in
the learners’ repertoire [5-8], and by teaching unknown primary targets [14-15].
Some of these recent studies were concerned with measuring the potential of IF in producing
the acquisition of several new targets in children with ASD, increasing the efficiency of the
procedure [6, 7, 14, 15]. Nevertheless, in these cases, not all children demonstrated a fast and
robust emergence effect of new skills after IF (or any emergence at all). The research by
Frampton and Shillingsburg [6] was an exception, since the two participants showed a fast
emergence of all, or nearly all, new repertoires. The participants from Tullis et al. [15] also
showed emergence of several untaught skills, but it was delayed. In other words, it happened
after several probe sessions along more primary target training sessions with IF.
In Frampton and Shillingsburg [6], the primary target maintenance training with IF was very
efficient in the sense that, across three sets of stimuli, three series of three training sessions
sufficiently produced acquisition of several new skills without direct teaching. Regarding the
other studies focused on emergence of multiple unknown targets, the only cases with
participant who matched the efficiency of those from Frampton and Shillingsburg, were one
participant from Cordeiro [14] and another one from Laddaga Gavidia et al. [7] (third set of
stimuli). As to other participants from these studies and those from the research by Tullis et al.
[15], emergence of new targets, when occurred, was delayed. For some learners, the emergence
was not demonstrated.
Page 9 of 25
9
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
Since the recent investigations on IF in children with ASD as participants showed mixed results
(depending on the learner, the emergence of new repertoires was immediate, delayed or not
observed), new scientific research on variables, which may possibly influence the efficiency of
IF, is warranted. The literature showed that primary target maintenance training with IF, which
effectively produced emergence of new targets in some participants, requires less effort to
respond than the case in which primary targets are unknown. In this sense, the current study
sought to extend the investigation on primary target maintenance training (simple listener
responding) with IF and its effects in producing emergence of multiple new repertoires in
children with ASD as participants. An alternating treatments design was used to compare the
efficiency between two interventions with IF. In one case, attending responses to visual stimuli
in training were demanded, as in the study by Laddaga Gavidia et al. [7], as well as tact of these
stimuli. In the other case, these demands were not made. Comparisons between the two
teaching conditions were conducted to assess the possible role of additional variables
(attending responses and tact of visual stimuli) in increasing the efficiency of the intervention
with IF. It was assumed that the intervention, which produced a more robust (errorless) and
immediate emergence effect of multiple repertoires according to category as secondary targets
(and directly related to primary targets and IF information), would be the most efficient one.
METHOD
Participants
The participants were three children with ASD (P1, P2 and P3) aged between 4 and 7 years.
They all received specialized interventions on Applied Behavior Analysis (ABA) in a university- based laboratory where data collection for the current study occurred. The participants had the
following repertoires according to the VB-MAPP assessment protocol [9]: 1) tact responses of
over 200 nonverbal stimuli, such as objects, pictures and actions (they could also select these
stimuli as listeners); 2) generalized identity matching to sample (relate identical pictures and
objects); 3) generalized arbitrary matching to sample performance (relate similar pictures and
objects and relate objects to corresponding pictures); 4) over 70 selection responses of pictures
and objects under verbal instructions specifying function, feature and class to which they were
related to; over 70 intraverbal responses (fill in blank type and answering “what”, “who” and
“where” questions).
All participants were able to vocally communicate using full sentences (consisting in four or
more words) regarding different verbal functions (mand and tact). Intraverbals with sentences
in general were emitted under prompt control. Considering other complex intraverbal cases,
the children demonstrated multiple responses under divergent control (e.g., saying “dog, cat
and horse” under the verbal instruction “name some animals”). They also showed some
intraverbal responses under convergent control (e.g., saying “soup” in the presence of the
instruction “name a hot meal”), but had difficulty in presenting responses under both divergent
and convergent control (e.g., saying “soup, barbecue and roast chicken” under the instruction
“name some hot meals”). Since some of the repertoires defined in the research as dependent
variables (VD) required multiple control [11, 12, 18], being a learner of these types of skills
outside the context of research represented a selection criterion.
Materials, Environment, Interobserver Agreement (IOA) and Treatment Integrity (TI)
Stimuli used in assessment and intervention conditions were organized in picture cards
measuring 7 X 10 cm. Each of them contained an image related to a category/class. Overall, the
Page 10 of 25
10
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
stimuli consisted of nine pictures of animals regarding three subcategories (land animals,
marine animals, and flying animals) and nine pictures of transportation regarding three
subcategories (ground transportation, marine transportation, and air transportation). Data
collection occurred in a university-based research laboratory where behavioral interventions
based on ABA were provided to the participants once a week. The room where the study
procedures were undertaken contained a table and three chairs. Each participant and an
experimenter sat facing each other. The experimenter conducted assessment and intervention
tasks and systematically collected data on the participants’ performance using a data sheet. A
second observer, not familiar with the research objectives, in approximately 30% of the
sessions also took data.
IOA was determined for these sessions (two probe sessions and two intervention sessions)
regarding all participants. IOA consisted of agreements and disagreements using a trial-by-trial
basis. For each assessment or intervention trial, an agreement was verified if both
experimenter and second observer recorded the participant’s response in the same way. IOA
was calculated in the following manner: number of trials with agreement divided by the total
number of trials in each session. To determine a percentage, the result was multiplied by 100.
IOA for all participants was 100%.
In approximately 30% of sessions, the second observer took data on the experimenter’s TI. This
was also determined for two probe sessions and two intervention sessions. It was used a
checklist on the conclusion of the following components: 1) use of relevant stimuli during tasks;
2) appropriate administration of instructions; 3) waiting 5s for the emission of a response by
the participant in each trial; 4) appropriate use of reinforcers and IF information; 5)
appropriate use of correction procedures, if needed. For each session in which TI was
determined, the total number of components concluded correctly was divided by the total
number of components. To establish a percentage, the result was multiplied by 100. Mean TI,
considering sessions with the three children, ranged from 80% to 100%. Procedural errors,
when committed, referred to not waiting up to 5s for the emission of a response by the
participant.
Dependent Variables and Independent Variables
In this investigation, the dependent variables (DV) consisted of the following untaught
repertoires according to category or class (secondary targets): listener responding according
to category (e.g., touching the picture of shark under the instruction “show me a marine
animal”); tact according to category (e.g., saying “marine animal” under the picture of shark and
question “what is a shark?”); arbitrary visual pairing (e.g., relating the picture of shark to the
picture of whale); intraverbals of saying item names according to category (e.g., saying “shark,
whale and dolphin” in the presence of the verbal instruction “name some marine animals”). One
of the independent variables (IV) consisted of teaching simple listener responses (primary
targets) (e.g., touching the picture of shark under the instruction “touch shark”) and presenting
an IF information (e.g., “the shark is a marine animal”). Another IV consisted of teaching simple
listener responding, also demanding attending responses to the pictures used and the simple
tact of the pictures (e.g., touching the picture of airplane under the instruction “touch the
airplane”. Thereafter, after looking at the picture and upon the question “what is this?”, the child
says “airplane”) and these responses were followed by an IF information (e.g., “the airplane is
a transportation”).
Page 14 of 25
14
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
Figure 1: Percentage of Correct Responses According to Category for P1
Note. The upper graph represents the percentage of correct responses according to category
before (pre-intervention probe) and after (post-intervention probe) the implementation of IV
with animals: listener responding according to category (black bar), tact according to category
(grey bar), arbitrary visual pairing (white bar) and intraverbal of saying item names according
to category (crosshatched bar). The lower graph represents the same kind of data before and
after the implementation of IV with transportation. The second segmented vertical line
represents the insertion of the IV, that is, primary target maintenance training (data not
shown).
According to Figure 1, regarding the animal category, P1 already had the repertoire of listener
responding according to category before intervention. About arbitrary visual pairing
repertoire, 44% correct responses were demonstrated in the last pre-intervention probe
session. During the first post-intervention probe session, performance improved to 100%
correct responses. Although performance fell to 44% in the following session, no errors
occurred in the last two post-intervention probe sessions. Tacts and intraverbals according to
category were not demonstrated in pre-intervention probe sessions and very discrete
improvements (22% and 11%, respectively) occurred until the last post-intervention probe
session. As to the transportation category, there was an improvement in listener responding
according to category from 56% correct responses in the last pre-intervention probe session to
100% in the first post-intervention probe session. Although performance fell to 44% in the
following session, it improved again to 100% in the last two post-intervention probe sessions.
Regarding arbitrary visual pairing, there was an improvement from 44% in the last pre- intervention probe session to 100% in both penultimate and last post-intervention probe
session. Tact according to category was unaffected by intervention and no correct responses
were emitted. In the case of intraverbal according to category, there was only a slight increase
in performance to 11% in the penultimate post-intervention probe session. Figure 2 shows the
Page 15 of 25
15
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
percentage of correct untaught repertoires according to category (secondary targets) for P2
along pre-intervention probe sessions, and after implementation of primary target
maintenance training with delivery of IF information (post-intervention probe sessions).
Figure 2: Percentage of Correct Responses According to Category for P2
Note. The upper graph represents the percentage of correct responses according to category
before (pre-intervention probe) and after (post-intervention probe) the implementation of IV
with animals: listener responding according to category (black bar), tact according to category
(grey bar), arbitrary visual pairing (white bar) and intraverbal of saying item names according
to category (crosshatched bar). The lower graph represents the same kind of data before and
after the implementation of IV with transportation. The second segmented vertical line
represents the insertion of the IV, that is, primary target maintenance training (data not
shown).
Figure 2 for P2, regarding the animal category, shows that listener responding according to
category improved from 44% correct responses in the last pre-intervention probe session to
100% correct responses in the last post-intervention probe session. In the case of arbitrary
visual pairing, a high percentage of correct responses had been demonstrated until the last pre- intervention probe session (89%). Performance improved to 100% in the penultimate post- intervention probe session, but it fell to 78% during the last session. Regarding tact according
to category, there was a slight improvement from 0% in the last pre-intervention probe session
to 22% in the penultimate post-intervention probe session. However, no correct responses
were emitted during the last post-intervention probe session. As to intraverbal according to
category, there was an important improvement from 0% in the last pre-intervention probe
session to 67% in the penultimate post-intervention probe session. Nevertheless, performance
fell to 11% in the last post-intervention probe session. Considering the case of transportation
category, performance in listener responding according to category improved from 44% in the
last pre-intervention probe session to 78% in the first post-intervention probe session. In the
following session, performance fell to 44%.
Page 16 of 25
16
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
It was more robust in the penultimate session with 89% correct responses, and, in the last
session, 78% correct responses were emitted. Arbitrary visual pairing improved from 22% in
the last pre-intervention probe session to 100% during the second post-intervention probe
session. Although performance fell to 67% in the following session, it improved again to 100%
in the last session. Tact according to category improved slightly from 0% in the last pre- intervention probe session to 22% correct responses in the penultimate post-intervention
probe session, but performance dropped again to 0% in the last session. Intraverbal according
to category also only improved very slightly from 0% in the last pre-intervention probe session
to 11% correct responses in both penultimate and last post-intervention probe sessions. Figure
3 shows the percentage of correct untaught repertoires according to category (secondary
targets) for P3 along pre-intervention probe sessions, and after implementation of primary
target maintenance training with delivery of IF information (post-intervention probe sessions).
Figure 3: Percentage of Correct Responses According to Category for P3
Note. The upper graph represents the percentage of correct responses according to category
before (pre-intervention probe) and after (post-intervention probe) the implementation of IV
with animals: listener responding according to category (black bar), tact according to category
(grey bar), arbitrary visual pairing (white bar) and intraverbal of saying item names according
to category (crosshatched bar). The lower graph represents the same kind of data before and
after the implementation of IV with transportation. The second segmented vertical line
represents the insertion of the IV, that is, primary target maintenance training (data not
shown).
According to Figure 3, for the animal category, P3 showed an improvement in listener
responding according to category from 22% in the last pre-intervention probe session to 100%
correct responses in the last post-intervention probe session. In the case of arbitrary visual
pairing, in the last pre-intervention probe session correct responses were not emitted. After
intervention, the first post-intervention probe session showed 44% correct responses, but
performance fell to 0% in the following sessions. As to tact and intraverbal according to
category, performance was unaffected by intervention, that is, correct responses were never
emitted. In the case of the transportation category, there was an improvement from 44%
Page 17 of 25
17
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
correct responses in the last pre-intervention probe session to 100% in the last post- intervention probe session of listener responding according to category. As to arbitrary visual
pairing, after 56% correct responses in the last pre-intervention probe session, performance
was unaltered in the first post-intervention probe session. However, no correct responses were
demonstrated in the following probe sessions. Tact and intraverbal according to category were
not emitted at any time during data collection.
Table 2 shows the number of correct trials per secondary target demonstrated by all
participants along probe sessions (both pre-intervention and post-intervention probe
sessions) for each of the two defined categories (animals and transportation). It is important to
remember that, for each untaught repertoire according to category (secondary target), nine
trials were administered per session. Therefore, the maximum number of correct responses
possible by each participant for each repertoire corresponded to nine out of nine trials (which
represented 100% correct responses).
Table 2: Number of Correct Responses (Secondary Targets) in Probe Sessions
Pre-intervention probe sessions (animal
category)
Post-intervention probe sessions (animal
category)
Listener
responding
Arbitrary
visual
pairing
Tact Intravebal Listener
responding
Arbitrary
visual
pairing
Tact Intravebal
P1 9; 9; 9 2; 4; 4 0; 0;
0
0; 0; 0 9; 9; 7; 9 9; 4; 9; 9 0; 0;
1; 2
0; 0; 1; 1
P2 4; 4; 4 6; 8; 8 0; 0;
0
1; 0; 0 4; 8; 6; 9 6; 6; 9; 7 0; 1;
2; 0
0; 0; 6; 1
P3 3; 0; 2 4; 0; 0 0; 0;
0
0; 0; 0 3; 1; 2; 9 4; 0; 0; 0 0; 0;
0; 0
0; 0; 0; 0
Pre-intervention probe sessions (transportation
category)
Post-intervention probe sessions
(transportation category)
Listener
responding
Arbitrary
visual
pairing
Tact Intravebal Listener
responding
Arbitrary
visual
pairing
Tact Intravebal
P1 7; 5; 5 4; 4; 4 0; 0;
0
0; 0; 0 9; 4; 9; 9 2; 0; 9; 9 0; 0;
0; 0
0; 0; 1; 0
P2 1; 4; 4 6; 1; 1 0; 0;
0
0; 0; 0 7; 4; 8; 7 2; 9; 6; 9 0; 0;
2; 0
0; 0; 1; 1
P3 3; 0; 4 4; 0; 5 0; 0;
0
0; 0; 0 4; 2; 2; 9 5; 0; 0; 0 0; 0;
0; 0
0; 0; 0; 0
Note. The left portion of the table shows the number of correct responses, regarding four
secondary targets, by each participant in three pre-intervention probe sessions for the animal
(top) and transportation (bottom) categories. The right portion of the table shows the number
of correct responses, regarding four secondary targets, by each participant in four post- intervention probe sessions for the animal (top) and transportation (bottom) categories.
The data in Table 2, similarly to the cases in Figure 1, Figure 2, and Figure 3, shows that for the
animal category, immediate emergence of arbitrary visual pairing was demonstrated by P1
Page 18 of 25
18
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
(nine correct responses or 100% in the first post-intervention probe session). Both P2 and P3
showed a delayed emergence in listener responding according to category (nine correct
responses or 100% in the last post-intervention probe session). Regarding the transportation
category, P1 showed immediate emergence of listener responding according to category (nine
correct responses or 100% in the first post-intervention probe session) and delayed emergence
of arbitrary visual pairing (nine correct responses or 100% in the third post-intervention probe
session). P2 demonstrated delayed emergence of arbitrary visual pairing (nine correct
responses or 100% in the second post-intervention probe session). P3 demonstrated delayed
emergence of listener responding according to category (nine correct responses or 100% in the
last post-intervention probe session).
DISCUSSION
In this study, primary target maintenance teaching with delivery of IF information in the
consequent portion of the contingency was established for three participants (children with
ASD). Simple listener responding regarding stimuli from two categories, animals, and
transportation, were the primary targets. No incorrect responses were emitted along training
sessions by any participant. In the process, post-intervention probe sessions to verify the
emergence of new repertoires according to category (secondary targets) were administered
each time a learning criterion was achieved for simple listener responding (that is, after every
two training sessions without errors). Gains regarding untaught repertoires according to
category (listener responding, arbitrary visual pairing, tact and intraverbal according to
category) were partial for all participants. When robust(errorless)improvements were noticed
among participants, they were related to one or two skills only (listener responding according
to category and/or arbitrary visual pairing). A partial emergence effect of new repertoires, after
primary target training with IF, had also been demonstrated by other participants with ASD
from previous studies [6, 7, 14].
In the current investigation, the emergence of new skills was not always immediate. Besides,
two of the skills probed (tact according to category and intraverbal of saying item names
according to category) never emerged, or their emission was too discrete. The efficiency of
primary target (simple listener responses) maintenance training with IF was a function of the
extent of emergence of related untaught repertoires according to category, and whether
emergence was immediate or not. Maximum efficiency of training with IF would be verified for
each untaught skill assessed, if the first post-intervention probe session showed immediate
errorless performance. For only one participant (P1), some secondary targets emerged
immediately (arbitrary visual pairing in the case of animal category and listener responding
according to category in the case of transportation category). In previous literature, immediate
acquisition of new targets, after training with IF, had also been shown by some learners [6, 7,
14].
Still about P1 in the current study more specifically, for the animal category, it was said that
arbitrary visual pairing was the only secondary target that emerged immediately without
errors (during the first post-intervention probe session). Regarding this emergent repertoire,
training with IF seemed to be more efficient than the case with transportation category in which
attending, and tact responses to visual stimuli used were also required. In this case, the
errorless emergence of arbitrary visual pairing was delayed (third post-intervention probe
session). Also, for transportation category, listener responding according to category emerged
Page 19 of 25
19
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
fully and immediately. A comparison with the case of animal category was not possible because
the participant demonstrated listener responding according to category without errors in the
last pre-intervention probe session. This represented a limitation and future studies should
only select participants who do not fully show any of the repertoires defined as DV in before
the onset of intervention.
About P2 in this research, for the animal category, errorless emergence of arbitrary visual
pairing and listener responding was delayed (only in third and fourth post-intervention probe
session, respectively). As to transportation category, the full emergence of visual pairing was
delayed (second post-intervention probe session). In the case of listener responding according
to category, although errorless emergence did not happen, few errors occurred during the third
post-intervention probe session. For this participant, there seems to be no teaching procedure
with IF more efficient than the other. Finally, in the case of P3, only listener responding
according to category emerged without errors and it was delayed for both animal and
transportation categories (fourth post-intervention probe session). No training with IF was
more efficient than the other as well.
According to Cariveau et al. [8], if the acquisition of new repertoires through IF is delayed, a
more intrusive measure should be used. That is, the direct teaching of the skills through
differential reinforcement should be established. This logic seems applicable to the participants
of the current research, since delayed acquisition of some relations was verified for all of them.
Plus, important relations (tact according to category and intraverbals of saying item names
according to category) did not emerge for any participant at all. However, the literature
discusses the possibility of additional measures to produce emergent responding. In the study
by Cordeiro [14], as told previously, one of the participants did not show emergence of different
untaught repertoires according to category (secondary targets) after simple tact (primary
target) training with IF.
Thus, an additional IV was applied, that is, one of the secondary targets (tact according to
category) was directly taught. Along training sessions, all the remaining skills (listener
responding according to category, arbitrary visual pairing and intraverbals of saying item
names according to category) emerged after several post-intervention probe sessions.
Although the establishment of these remaining skills was not immediate, the new training may
be considered efficient in the sense that it resulted in acquisition of three new skills without
differential reinforcement. Since in ABA investigations on procedures which may produce
emergence of new repertoires is important for participants with ASD, their direct teaching
should be defined as last resort. Besides, previous studies have shown that teaching tact
according to category (and other repertoires), similarly to the case of one of the participants
from Cordeiro [14], possibly influenced the emergence of another skill as well. In a study by
Grannan and Rehfeldt [16] conducted with two participants (children with ASD), the teaching
of tact according to category was part of a more comprehensive approach.
The participants were also taught simple tacts (that is, tact responses according to noun) and
arbitrary visual pairing. The direct teaching of these skills produced emergence of intraverbals
of saying item names according to category for both participants. In Grannan and Rehfeldt [16],
the effects of the taught skills were not assessed separately, so determining the extent of
emergence (of the untaught intraverbal repertoire) possibly produced by each intervention
Page 20 of 25
20
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
was not possible. However, it was discussed that teaching tact according to category played an
important role on intraverbal emergence (and that an expressive repertoire of tacts is
important for the establishment of intraverbals).
Recently, Matos and Lima [17] conducted a similar investigation involving four participants
(children with learning disabilities). The participants were taught the following repertoires in
the following order: simple tact, tact according to category and listener responding according
to category (this one instead of arbitrary visual pairing). The effects of the interventions were
also assessed on the emergence of intraverbals according to category. In this study, the effects
of the interventions were measured separately, and it was noticed that, after teaching simple
tact and tact according to category, intraverbals largely emerged for a participant. In the case
of this learner, it is worth mentioning that, among all participants, he was the one with a very
expressive and generalized repertoire of tacts at the beginning of the research. The same
applied to the participant from Cordeiro [14] to whom teaching tact according to category
produced the emergence of other repertoires according to category, including intraverbals.
In this study, simple listener responding maintenance training with IF did not produce
emergence of tact according to category and intraverbal of saying item names according to
category for any of the three participants with ASD. Considering that all of them had a robust
tact repertoire (and they were all able to tact the stimuli used in the research according to noun
as well), tact teaching according to category could have been investigated as an additional IV in
possibly producing the emergence of a related repertoire (intraverbal), as in the case of some
participants from previous studies [14, 16, 17]. Intraverbal training could also be established
to measure the possible emergence of tact according to category as well. In this sense, two
teaching sequences (tact training – probing intraverbal; intraverbal teaching – probing tact)
could be compared to determine which would be the most efficient (in the sense of producing
a better emergence effect of the related untaught repertoire).
Another aspect of the investigation demands discussion. Some of the repertoires defined as DV
required multiple control [11, 12, 18]. Listener responding according to category was
demonstrated in post-intervention probe sessions. In each session trial, a participant had to
select a picture from an array under the control of two elements of an instruction (e.g., touching
the picture of airplane under the instruction “show me air transportation”). In the example, the
words “air” and “transportation” jointly need to control the selection response and only the
picture of airplane matched the subcategory mentioned. The DV regarding tact according to
category and intraverbal of saying item names according to category also required multiple
control.
Regarding tact according to category, two stimuli needed to exert joint control as well (e.g.,
saying “air transportation” in the presence of the picture of airplane and instruction “what is
airplane?”). In this example, it is expected that the question changes the evocative function of
the picture, providing the context for tacting according to category. During post-intervention
probe sessions, in most trials, the participants just repeated the name provided in the
instruction (e.g, “airplane”). This could also be related to a history of reinforcement for the
emission of simple tacts under the control of pictures (e.g., saying “car”, “dog”, whale” under
their corresponding pictures). This result suggests that direct teaching through differential
reinforcement should be defined (Cariveau et al. [8]), or related intraverbals according to
Page 21 of 25
21
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
category could be taught and new post-intervention probe sessions administered to verify
possible emergence of tacts according to category.
In the case of the intraverbal, the responses needed to be emitted under both divergent and
convergent control[11, 12](e.g., saying “whale, shark and dolphin” under the instruction “name
marine animals”). In the example, the words “marine” and “animals” jointly control the
mentioned multiple target responses. Regarding the current study, it was previously mentioned
that, before the onset of the investigation, the participants demonstrated intraverbals under
divergent control and under convergent control separately. However, they lacked the skill of
demonstrating intraverbal responses under both divergent and convergent control (as in the
example pointed out). This represented a limitation of the research, and it is important that
future studies select participants who already demonstrate at least some intraverbal repertoire
under both divergent and convergent control. Anyway, as in the case of previous studies [14,
16, 17], it would be important to verify if teaching tact according to category as an additional
IV produces the emergence of intraverbals.
The recent literature on IF has discussed the possible influence of echoing IF information on
the establishment of new repertoires without direct teaching. In Frampton and Shillingsburg
[6], the participants did not echo the IF information during primary target maintenance
teaching. However, the authors hypothesized the emission of covert echoic by the participants,
possibly influencing acquisition of new targets. In Laddaga Gavidia et al. [7], the participants
echoed the IF information in most primary target maintenance teaching trials and suggested
that overt echoic also influenced acquisition of new targets to a greater degree for one of the
participants.
In Cariveau et al. [8], for a single participant, there was an increase of overt echoic of the IF
information during a first assessment, but few overt echoics were demonstrated in a second
assessment. The authors also considered the possibility of emission of covert echoics and that
they may have influenced the emergence of targets from some sets. In the current study, the
participants did not overtly echo the IF information during primary target maintenance
teaching, but it is also hypothesized that they emitted covert echoics, possibly influencing
emergence of one or two of the four secondary targets (depending on the participant). Finally,
in the current study for a set of stimuli, the demand to attend to visual stimuli, as in Laddaga
Gavidia et al. [7], does not seen to have significantly increased the efficiency of teaching with IF
(as well as the demand to tact the pictures used).
This study had major limitations that need to be discussed. Although it was meant to be an
experimentally based investigation using a single case research design (adapted alternating
treatments design – AATD), the two treatments, whose effects were measured and compared
for each participant with ASD, aimed at teaching skills (simple listener responses) already
established in the participants’ repertoires. Considering the case of the design used (AATD), the
mentioned skills defined as primary targets could not be part of the participants’ repertoires
before the onset of the interventions, since functional relations between IV and DV would not
be feasible. Through an AATD design with two treatments conducted in an alternated fashion,
the demonstration of experimental control occurs if there is a differentiation in target
acquisition [20], which was not possible in this study, representing a threat to internal validity.
Page 22 of 25
22
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
Plus, important recommendations to increase the chances of demonstrating experimental
control were not followed.
Cariveau and Fetzner [21] conducted a review of studies on AATD with the purpose of analyzing
adopted measures to ensure or facilitate the establishment of experimental control. They
reviewed and coded 114 articles from 27 journals (five in Behavior Analysis and the others in
Special Education) published between 1985 and 2017. It was noticed that in more than half of
the studies, one or more important methods for demonstrating experimental control were
employed. A no-treatment control condition was used in less than one third of the studies.
However, in more than half of the studies with a control condition, correct performance was
demonstrated at a level above the initial baseline, representing a potential threat to internal
validity.
In 28.1% of the studies, the AATD design to address non-reversible behaviors was embedded
into another single case design (e.g., multiple baseline/MBL design or multiple probe/MP
design). The MBL and MP designs may be defined across behaviors, settings, or individuals at
staggered points in time. Of the articles that combined AATD with MBL or MP, 65.6% showed
that correct performance did not occur at a level beyond initial baseline along staggered panels.
In this sense, Cariveau and Fetzner [21] suggested that combining designs may be a better
method than using a control condition to prevent threats to internal validity. They also
suggested that both methods (no-control condition; additional designs like MBL or MP) should
be included in research on AATD to ensure the establishment of experimental control.
Cariveau et al. [22] conducted review research to report methods to equate target sets
regarding the difficulty in studies employing the AATD design. According to Cariveau et al.,
equating target sets is also a necessary measure to ensure the validity of the AATD. If the target
sets are not appropriately equated, changes in the DV may be a function of a difference in target
difficulty and not a function of the IV. The review involved a search in five journals in Behavior
Analysis, and the search was conducted in 2018. 65 articles using the AATD design were found.
In 36 of these articles, one or more methods of logical analysis were used to equate target sets.
They were equated based on (1) the number of syllables (e.g., targets across conditions should
have similar number of syllables); (2) overlapping sounds (assigning targets with similar first,
middle and end sound to different conditions; assigning targets that rhyme to separate
conditions); (3) novelty of the responses (by assessing whether other relations regarding
individual targets are acquired); (4) visual properties (when visual targets share common
aspects, the targets should be assigned to different conditions); (5) number of motor responses
(responses that share a similar number of steps); (6) number of letters in the targets. Among
all these methods, controlling for the number of syllables was the most used one. And the most
common combined methods in the studies were the number of syllables and overlapping
sounds. Cariveau et al. [22] concluded that, even though the ideal methods are still unknown, a
combination of those from the literature reviewed based on logical analyses should be used in
future research on AATD design.
In the current study, besides the fact that the primary targets were already part of the
participants’ repertoires, the recommended methods by the literature regarding the definition
of a no-treatment control condition, combining AATD with MBL or MP design and equating the
target sets [21, 22] were not addressed. This poses a threat to internal validity indeed.
Page 23 of 25
23
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
Nevertheless, four new repertoires according to category (secondary targets) were probed for
emergence along the process of conducting maintenance training of primary targets (simple
listener responses) with IF. Even though two of the new skills defined as secondary targets did
not emerge (tact and intraverbal according to category), errorless performance of other
relations was demonstrated to some extent by all participants (regarding arbitrary visual
pairing and/or listener responding according to category).
This suggests that maintenance training of primary targets with IF possibly played a role in the
emergence of new repertoires. It is important that the types of primary targets of this research
(simple listener responses with IF) be addressed in future studies as unknown skills by
participants with ASD. In one condition, attending and tact responses to visual stimuli in
training should be demanded. The target sets should be equated (regarding the difficulty) to
better assess the possible effects of attending and tact responses in increasing the efficiency of
the IF procedure. Plus, a no-treatment control condition should be defined by adding a target
set to which differential reinforcement, correction procedures and IF are not delivered. Finally,
a multiple probe design across participants should be combined with AATD. This way, the
demonstration of experimental control will possibly be addressed with the establishment of a
differentiation in primary target acquisition compared to baseline levels of responding, and the
possible emergence of untaught repertoires representing secondary targets.
DECLARATIONS
Ethical Approval
The study was approved by an ethics committee in research with humans (Research Ethics
Committee - Comitê de Ética em Pesquisa/CEP) (authorization No. 4.284.271) from Federal
University of Maranhão, São Luís-MA (Brazil). Those responsible for the participating children
(as well as the children themselves) signed an informed consent form for participation. All
personal information was kept confidential, and the procedures could be interrupted at any
time without prejudice to those involved. No form of financial compensation was foreseen for
participation, or for travel costs. All participants would be compensated for any expenses with
their participation in the research, and for any damages they could suffer for the same reason,
and resources would be guaranteed for these expenses. Those responsible for the children were
informed that the research results would always be presented to them and that these results
would also be disclosed in events and other productions of a scientific nature only, without any
identification of the participants. All mandatory terms of presentation have been delivered to
the ethics committee (CEP) and are in accordance with resolution 466/12 of the National Health
Council (Conselho Nacional de Saúde/CNS).
Competing Interests
The authors declare no competing interests.
Authors’ Contributions
D.C.M. and P.G. wrote the main manuscript text and reviewed the manuscript. D.J.R.L. and K.R.S
prepared figures and reviewed the manuscript.
Funding
Own funding (by the authors). The study involved no external funding.
Page 24 of 25
24
Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024
Services for Science and Education – United Kingdom
Availability of Data and Materials
Raw data may be obtained by contacting the corresponding author.
References
1. Skinner BF. Verbal behavior. Prentice-Hall, Inc; 1992.
2. Greer RD, Ross DE. Verbal behavior analysis: Inducing and expanding complex communication in children
with severe language delays. Allyn & Bacon; 2008.
3. Sundberg ML, Partington JW. Teaching language to children with autism or other developmental disabilities.
Behavior Analysts, Inc; 1998.
4. Reichow B, Wolery M. Comparison of Progressive Prompt Delay with and without Instructive Feedback. J
Appl Behav Analysis. 2011; 44:327-340. http://dx.doi.org/10.1901/jaba.2011.44-327.
5. Tullis CA, Gibbs AR, Butzer M, Hansen SG. A comparison of secondary target location in instructive feedback
procedures. Advances in Neurodevelopmental Disorders. 2019; 3:45-53. https://doi.org/10.1007/s41252-
018-0090-4.
6. Frampton SE, Shillingsburg MA. Promoting the development of verbal responses using instructive feedback. J
Appl Behav Analysis. 2020; 53:1029-1041. https://doi.org/10.1002/jaba.659.
7. Laddaga Gavidia V, Bergmann S, Rader K.A. The use of instructive feedback to promote emergent tact and
intraverbal control: a replication. The Analysis of Verbal Behav. 2022; 38:95-120.
https://doi.org/10.1007/s40616-022-00171-y.
8. Cariveau T, Brown A, Platt D, Ellington P, Hurtado R. An evaluation of instructed feedback during mastered
demands. The Analysis of Verbal Behav. 2022; 38:179-189. https://doi.org/10.1007/s40616-022-00174-9.
9. Sundberg ML. The verbal behavior milestones assessment and placement program: The VB-MAPP. AVB
Press; 2008.
10. Miguel CF. Common and intraverbal bidirectional naming. The Analysis of Verbal Behav. 2016; 32:125-138.
https://doi.org/10.1007/s40616-016-0066-2.
11. DeSouza AA, Fisher WW, Rodriguez NM. Facilitating the emergence of convergent intraverbals in children
with autism. J Appl Behav Analysis. 2019; 52:28-49. https://doi.org/10.1002/jaba.520.
12. Sundberg ML. Verbal stimulus control and the intraverbal relation. The Analysis of Verbal Behav. 2016;
32:107-124. https://doi.org/10.1007/s40616-016-0065-3.
13. Matos DC. Análise do comportamento aplicada ao desenvolvimento atípico com ênfase em autismo [Applied
behavior analysis to atypical development with an emphasis on autism]. AICSA; 2016.
14. Cordeiro NCP. Ensino de repertórios por meio de feedback instrucional em crianças com transtorno do
espectro autista (tea) [Teaching repertoires through instructional feedback in children with autism
spectrum disorder (asd)] [Master’s thesis, Universidade Federal do Maranhão]. Repositório institucional da
Universidade Federal do Maranhão; 2020.
https://sigaa.ufma.br/sigaa/public/programa/noticias_desc_stricto.jsf?lc=es_ES&idPrograma=1157¬icia
=219561216.
15. Tullis CA, Gibbs AR, Priester J, Tillem A. Emergence of verbal responses using instructive feedback: A
replication and extension. Behavioral Interventions. 2022; 37:1-19. https://doi.org/10.1002/bin.1836.
Page 25 of 25
25
Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback
in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.
URL: http://dx.doi.org/10.14738/assrj.111.16172
16. Grannan L, Rehfeldt RA. Emergent intraverbal responses via tact and match-to-sample instruction. J Appl
Behav Analysis. 2012; 45:601-605. https://doi.org/10.1901/jaba.2012.45-601.
17. Matos DC, Lima ABR. Ensino de tatos e respostas de ouvinte e os efeitos sobre emergência de intraverbais.
[Teaching tact and listener responses and the effects on intraverbal emergence]. In: Vilas Boas DLO, Cassas F,
Gusso HL, Mayer PCM, editors. Comportamento em foco. Associação Brasileira de Psicologia e Medicina
Comportamental; 2018. pp. 140-153.
18. Michael J, Palmer DC, Sundberg ML. The multiple control of verbal behavior. The Analysis of Verbal Behav.
2011; 27:3-22. https://doi.org/10.1007/ BF033 93089.
19. Barlow DH, Hayes SC. Alternating treatments design: One strategy for comparing the effects of two
treatments in a single subject. J Appl Behav Analysis. 1979; 12:199-210.
https://doi.org/10.1901/jaba.1979.12-199.
20. Sindelar PT, Rosenberg MS, Wilson RJ. An adapted alternating treatments design for instructional research.
Education & Treatment of Child. 1985; 8:67-76.
21. Cariveau T, Fetzner D. Experimental control in the adapted alternating treatments design: a review of
procedures and outcomes. Behavioral Interventions. 2022; 37:805-818. https://doi.org/10.1002/bin.1865.
22. Cariveau T, Batchelder S, Ball S, Montilla ALC. Review of methods to equate target sets in the adapted
alternating treatments design. Behav Modification. 2021; 45:695-714.
https://doi.org/10.1177/0145445520903049.