Page 1 of 25

Advances in Social Sciences Research Journal – Vol. 11, No. 1

Publication Date: January 25, 2024

DOI:10.14738/assrj.111.16172.

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with

Instructive Feedback in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

Services for Science and Education – United Kingdom

Comparison of Two Maintenance Training Arrangements with

Instructive Feedback in Children with Autism

Daniel Carvalho de Matos

ORCID: 0000-0002-6793-0101

Universidade Federal do Maranhão, Instituto Evoluir

and Universidade Ceuma, São Luís – Maranhão, Brazil

Danielle Juliana Ribeiro Loureiro

ORCID: 0000-0001-8502-1510

Universidade Ceuma, São Luís, Maranhão, Brazil

Katiane Reis da Silva

ORCID: 0000-0001-8428-5215

Universidade Ceuma, São Luís, Maranhão, Brazil

Pollianna Galvão

ORCID: 0000-0001-7579-8852

Universidade Federal do Maranhão, Instituto Evoluir

and Universidade Ceuma, São Luís, Maranhão, Brazil

ABSTRACT

Instructive feedback (IF) can improve the efficiency of teaching in children with

autism spectrum disorder (ASD) by establishing emergent skills. The purpose of

this study was to compare the effects of two types of listeners responding

maintenance teaching with IF on acquisition of four untaught repertoires in three

children with ASD. In one of the teaching cases, attending and tact responses of

visual stimuli were required. The emergence was partial for all learners, with no

great difference in the efficiency of the types of teaching. Two of the new repertoires

were not demonstrated. Errorless performance of some relations was shown by all,

but for only one child immediate emergence was verified. The results were

discussed regarding the possibility of new investigations on alternative procedure

to IF that could produce the emergence of new repertoires.

Keywords: autism spectrum disorder, primary target, untaught repertoires, instructive

feedback.

INTRODUCTION

Children with Autism Spectrum Disorder (ASD) frequently show impairments in several non- verbal and verbal behavioral repertoires, demanding a comprehensive intervention curriculum

to ameliorate the deficits. According to Skinner [1], non-verbal and verbal operant behaviors

are shaped by consequences. These are called reinforcers when the likelihood of future

emission of the behaviors increases. Procedures in Applied Behavior Analysis (ABA) commonly

Page 2 of 25

2

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

aims to establish repertoires in children with ASD through the arrangement of reinforcement

contingencies [2, 3]. A nonverbal operant is the product of interactions with the physical

environment (e.g., a child reaches out to grab a toy from a shelf). A verbal operant behavior is

shaped by a mediated reinforcing consequence. During a verbal episode, the speaker emits the

verbal behavior and the listener, specially trained by the verbal community, delivers a

reinforcer to the speaker (e.g., a child asks his mother for a cookie, and she allows him access).

Research on ABA focused on developing procedures to expand skill acquisition in children with

ASD. One case, called instructive feedback (IF), involves the definition of secondary target

during the teaching of primary target. As an example, a given child is taught to say “airplane”

upon the presentation of a picture of airplane (primary target). When he/she contacts

differential reinforcement (e.g., praise and a tangible item), the interventionist provides an IF

information (e.g., by saying “airplane is a transportation”). Thereafter, a probe shows that the

child can tact (label) the picture according to class (e.g., by saying “transportation”) and upon

the provision of a supplementary question (e.g., “what is an airplane?”). Tact is a type of verbal

behavior that involves the emission of a vocal verbal response under the control of a non-verbal

discriminative stimulus, and the response is maintained by a generalized conditioned

reinforcer (an established form of attention) [1]. In the case of the example presented, the tact

according to class as a new skill (secondary target) is established in the participant’s repertoire

without direct teaching, and it is considered that the variable IF plays a role in this. The

literature points out that IF increases instructional gains [4].

Recent investigations assessed the effects of providing IF information during the maintenance

teaching of primary targets previously established in learners’ repertoires. It is considered that

this kind of instructional arrangement can make the process of establishing secondary targets

without direct teaching less effortful [5-8]. Since primary target maintenance training with IF

was one of the concerns of the current research, it is important to describe the previous

literature in detail regarding the methodological characteristics and main results.

Tullis et al. [5], for two children with ASD as participants, compared different locations in which

IF information was administered during the reinforcement of primary targets. Primary and

secondary targets consisted of tact pictures and listener responding by function, feature, and

class (LRFFC), respectively. It was previously said that the tact is a type of verbal behavior [1].

LRFFC, in turn, represents a listener repertoire with the emission of non-verbal stimuli

selection responses based on instructions that specify function, feature or class to which the

stimuli belong [3, 9]. As an example, when an array of three different pictures is shown to a

learner (e.g., pictures representing soap, sharpener, and bottle), he/she must select the picture

corresponding to a given instruction (e.g., selecting the picture of soap under the verbal

instruction “show me toiletry”). No differential reinforcement was provided in LRFFC trials in

the study. Two sets of three pictures were defined for both primary and secondary targets.

The tact of all stimuli was previously established in the participants’ repertoire. In baseline,

both children did not demonstrate LRFFC responses (secondary targets), or performance

remained below 20% correct responses. During intervention condition, two treatments

(involving tact response maintenance training) were compared through an adapted alternating

treatments design. For one of the treatments, contingent to the emission of tact correct

Page 3 of 25

3

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

responses, IF information was provided and it was followed by praise (e.g., after saying “soap”

under the picture of soap, an experimenter said, “soap is a toiletry, good job!”).

The other treatment was administered similarly to the first, but, contingent to correct

responses, praise was delivered before the IF information (e.g., after saying “bag” under the

picture of bag, the experimenter said “well done! The bag has a handle”). For both treatments,

whenever a learning criterion was reached in three consecutive sessions, a probe to verify

emergence of secondary targets was administered 30 min later (other activities were

conducted with the children before the probes). As a result, the secondary targets for both

participants were acquired faster when, after the emission of primary targets in training, the IF

information was presented before praise. Since this one was the best treatment, it was

implemented for both sets of stimuli. The effects of training tact pictures with IF were lasting,

since, for one participant, maintenance of secondary targets was demonstrated for 16 weeks.

For the other participant, maintenance occurred for 18 weeks. The participants’ parents

considered the procedures highly effective and socially acceptable.

The authors discussed data in the sense that the effectiveness of the procedures with IF was, in

part, due to the occurrence of indiscriminable contingencies (learning by observation). They

also emphasized that the participants showed strong tact and echoic repertoires. The echoic

(vocal imitation) is a type of verbal behavior that involves the emission of a vocal verbal

response under the control of a vocal verbal discriminative stimulus with which it maintains

point-to-point correspondence, and the response is maintained by a generalized conditioned

reinforcer (e.g., an interventionist presents the instruction “say dog”. A learner then responds

“dog” and receives verbal praise) [1]. In the study, although the participants did not overtly

echo the IF information during primary target maintenance training, it is possible that they did

so covertly, which may have influenced acquisition of secondary targets. Finally, Tullis et al. [5]

also considered that the characteristics of instructional context possibly influenced acquisition

of secondary targets, since all sessions were conducted in an environment in which ABA

therapy sessions were typically provided to the participants. Plus, the experimenters were

professionals with whom they were familiar.

Frampton and Shillingsburg [6] assessed the effects of listener responding maintenance

training, with IF information in the consequent portion of the contingency (e.g., pointing to the

picture of the State of Tennessee under the instruction “show me Tennessee”). After a correct

listener response, it was provided an IF information (e.g., “Nashville is the name of the capital

of that State”) and the effects were assessed on acquisition of new multiple targets without

direct reinforcement in two children with ASD as participants. These new targets were the

following: 1) listener responding under control of a characteristic (e.g., selecting the picture of

Tennessee under the question “what is the capital of Nashville?”); 2) tact of a characteristic (e.g.,

saying “Nashville” in the presence of the picture of Tennessee and the question “what is the

capital of this State?”); 3) intraverbal (e.g., saying “Nashville” under the question “what is the

capital of Tennessee?”); 4) reverse intraverbal (e.g., saying “Tennessee under the question

“what State has Nashville as capital?”). The training of each of three sets of stimuli in the study

consisted of three primary target maintenance sessions with IF. After each series of three

sessions, a probe to verify emergence of the new multiple repertoires was administered. As a

result, until the third probe, nearly all new targets emerged for the two participants. The

acquisition of the new skills was not delayed, indicating the efficiency of the training with IF.

Page 4 of 25

4

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

For one child, the emergence of tact of characteristic was not demonstrated. The children did

not echo the IF information during maintenance training, but the authors pointed out the

possibility of emission of covert echoic, influencing the establishment of secondary targets for

both children. The new multiple targets were maintained for two weeks.

In Frampton and Shillingsburg [6], the difference between the two listener responding

repertoires organized was the type of the verbal discriminative stimulus used. Both skills

involved the emission of selection responses under the control of verbal instructions. The

listener responding repertoire whose maintenance was taught represented a simpler case, that

is, each participant had to select a picture from an array under the control of the verbalization

of its name by an experimenter. The more complex case (as a new probed repertoire) consisted

of each participant selecting a picture corresponding to an instruction about a characteristic

(or feature) without including the picture’s name. Tacts had been previously defined in this

manuscript. As to the intraverbal, it involves the emission of a verbal response under the

control of a verbal discriminative stimulus with which it lacks point to point correspondence.

The response is maintained by a generalized conditioned reinforcer. This was true for the two

types of intraverbals defined in the research [1, 3, 9].

Laddaga Gavidia et al. [7] conducted a systematic replication of the research by Frampton and

Shillingsburg [6] in two children with ASD as participants. However, they made some additions

in the sense that: 1) along training, they demanded attending response to visual stimuli during

the delivery of IF; 2) measured the emission of echoic of IF information provided during

training; 3) included fill in the blank intraverbal probes. Similar to the previous research, the

dependent variables consisted of the following: 1) listener responding under control of a

characteristic (e.g., pointing to the picture of a scientist under the question “who helps making

discoveries?”); 2) tact of a characteristic (e.g., saying “make discoveries” in the presence of the

picture of a scientist and the phrase “she helps to...”); 3) fill in the blank intraverbal (e.g., saying

“make discoveries” in the presence of the phrase “the scientist helps to...”); 4) intraverbal under

a question (e.g., saying “making discoveries” under the verbal question “how does the scientist

help?”); 5) reverse intraverbal under a question (e.g., saying “the scientist” under the question

“who helps making discoveries?”).

Three sets of stimuli were programmed for each participant. During intervention, listener

response maintenance teaching was administered (e.g., pointing to the picture of scientist

under the instruction “show me scientist”). The emission of each correct and prompted

response was followed by praise and 20s access to a preferred tangible reinforcer. After this,

an experimenter positioned the picture close to the participant’s eyes and pointed to it. If the

participant did not look at the picture in 5s, the experimenter said “look”. If 5s more elapsed

without the participant observing the picture, it was positioned close to a preferred item until

the participant looked at it. Once the picture was clearly being observed, the IF information was

provided (e.g., “she makes discoveries”).

Along the study by Laddaga Gavidia et al. [7], after each one of several series of three

maintenance sessions to teach primary targets with IF, a probe to verify the emergence of all

the new multiple targets previously mentioned was conducted. As a result, emergence of the

new repertoires was demonstrated by one participant regarding all three sets of stimuli. The

acquisition was delayed, that is, it demanded several probes along additional training. As to the

Page 5 of 25

5

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

case of the other child, emergence of new skills occurred only for the first set. Both children

echoed the IF information in most trials along training of primary targets, which may have

influenced acquisition of new repertoires (in a less extent for the participant who showed

acquisition of new targets related only to one set of stimuli). Plus, the authors suggested that

demanding attending responses to the picture while providing IF information possibly

established listener and speaker behaviors, also facilitating the establishment of the new

targets of the study. Another important aspect facilitating the acquisition was the fact that both

participants had a robust repertoire of tacts and a history of listener behavior emerging after

the teaching of speaker behavior and vice-versa, referred as bidirectional naming [10].

One important limitation of Laddaga Gavidia et al. [7] was that for one participant procedural

modifications were applied. For two of three sets of stimuli, the new targets did not emerge

after training the primary target with IF over 30 sessions. So, the probes were no longer

conducted under extinction, that is, differential reinforcement for correct responses was

administered. However, this modification also seemed ineffective. So, a new modification

consisted of increasing, in each probe trial, the interval between the instruction and the

response from 5s to 10s. Considering the results for the two participants, the authors suggested

that the extent of the emergence of new repertoires through IF may be idiosyncratic. They also

suggested that the use of tangible reinforcers during the training of primary targets with IF may

have prevented the emergence of new repertoires. They said that tangible reinforcers are not

consumed fast, and they may compete with IF information, making attending responses to them

unlikely.

Another possible reason for the lack of emergence for one participant was the absence of

prerequisite skills for the emission of intraverbals, which require conditional discrimination.

Intraverbal questions contained two elements, which jointly should control the responses (e.g.,

“where does the eagle live?”). In the mentioned example, the word “live” alters the evocative

function of the word “eagle”, and vice-versa, and they are necessary together to avoid errors.

The participant who lacked emergence of new targets for two sets of stimuli possibly did not

have prerequisites for this kind of complex intraverbal [11, 12].

Cariveau et al. [8] also assessed the effects of primary target maintenance teaching on the

establishment of secondary targets without direct teaching in a child with ASD as participant.

In this study, the delivery of IF information during the training of primary targets occurred

intermittently, that its, after three teaching trials on average. The investigation was conducted

in two different moments. During the first assessment, the participant was 4 years old. The

second assessment happened when the participant was 6 years old. On both occasions, data

collection on echoic of IF information were systematically taken. Primary targets consisted of

listener responses (following instructions for the emission of simple motor actions) or simple

motor imitation responses.

Descriptions on how to teach the mentioned skills may be obtained in manuals which describe

training strategies to establish them. Through discrete trial teaching procedure, a successful

trial concerning each skill involves providing an instruction as a discriminative stimulus and up

to 5s for the emission of a target response by a learner. After a correct response, a reinforcer is

delivered (e.g., verbal praise; access to a tangible item). As an example of following instruction,

a learner claps his/her hands when an interventionist provides the instruction “clap your

Page 6 of 25

6

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

hands” as discriminative stimulus. By emitting the target response, the learner is praised and

get access to a preferred item. In the case of simple motor imitation responses, the learner

would emit the same type of response and access reinforcers, but the discriminative stimulus

in this case would consist of providing the non-verbal model of the clapping response [2, 3, 13].

After the emission of primary target correct responses in Cariveau et al. [8], the IF information

(e.g., “you raise your hand”) was presented along with a reinforcer and the participant had up

to 3s to emit an echoic of the information given. Fill in the blank intraverbals were defined as

secondary targets (e.g., saying “hand” in the presence of the verbal stimulus “you raise your...”).

Four and three sets of stimuli were programmed during first and second assessment,

respectively.

In Cariveau et al. [8], each set of stimuli involved three secondary targets, whose IF information

were presented three times during each primary target maintenance training session. Each

session included approximately 27 demands. Maintenance sessions were conducted once or

twice a day (there were one to three meetings with the participant per week). On each day, a

secondary target probe (intraverbal) was conducted before the training of primary targets.

Along several probes, if performance remained low on a given stimuli set, intraverbal training

would be implemented using a constant prompt delay procedure. During the second

assessment with the participant, an intervention with a more intrusive correction procedure

was needed (the antecedent stimulus and echoic prompt were presented three times). At the

end of the study (considering the two assessments together), the participant demonstrated

acquisition of two sets of secondary targets.

For three other sets of stimuli, the direct teaching procedure with constant prompt delay was

needed to establish secondary targets. Other two sets, however, demanded the more intrusive

teaching procedure. Data were discussed in the sense that learners not always show acquisition

of secondary targets through IF, that is, without direct teaching. As in the study by Laddaga

Gavidia et al. [7], Cariveau et al. [8] maintains that the delayed acquisition of new repertoires

through IF should not be expected. A more intrusive approach should be warranted instead.

Regarding the emission of echoic of IF information, which the literature suggests it is important

for acquisition of new targets, there was an increase of overt responses during the first

assessment. However, few echoic responses were demonstrated during the second assessment.

As in the case of other studies described in this manuscript, the authors hypothesized that the

echoic responses were emitted covertly, although the lack of secondary target emergence in

some sets of stimuli seems to not support such a hypothesis. Lastly, the authors pointed out the

definition of a single participant as a methodological limitation, limiting the conclusions

regarding the generality of the findings.

Other recent studies also investigated the effects of training primary targets with IF on the

establishment of new multiple targets without direct teaching, although in the beginning of

these studies the primary targets were not established in the learners’ repertoires (no

maintenance training sessions were programmed in the beginning)[14, 15]. Since these studies

assessed the influence of IF on the acquisition of multiple targets, which was one of the concerns

of the current research, they should be described in detail.

Page 7 of 25

7

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

In the research by Cordeiro [14], conducted with four children with ASD as participants,

primary targets consisted of simple tact responses (according to name). During training,

correct responses were followed by praise and IF information (e.g., saying “São Paulo” in the

presence of the picture of the State of São Paulo. Thereafter, the experimenter delivered praise,

the information “São Paulo is a State in the southeastern region” and showed a picture

representing the southeast region). For each participant, a set with three stimuli (pictures) was

programmed. The learning criterion during training consisted of two sessions without errors.

When the criterion was reached, a probe was conducted to check emergence of each of four

new repertoires. They were the following: listener responding according to class (e.g., selecting

the figure of the State of São Paulo under the instruction “show me a State in the southeast

region”); tact according to class (e.g., saying “southeast region” in the presence of the picture of

the State of São Paulo and the question “what region is the State of São Paulo from?”); arbitrary

visual pairing (e.g., relate the picture of the State of São Paulo to another in the southeast

region); intraverbal of listing names according to class (e.g., saying “São Paulo, Rio de Janeiro

and Minas Gerais” under the instruction “name States from the southeast region”). Along the

study, if a single probe session after training did not result in robust emergence of the

secondary targets, additional primary target training sessions with IF would be implemented,

as well as new probes to verify emergence of the new repertoires.

The teaching of skills regarding tact, listener responding and intraverbal was previously

explained as the relevant literature on the theme of this research was presented. As to the

repertoire concerning arbitrary visual pairing, a successful teaching trial, although this skill was

not directly taught through differential reinforcement in the study by Cordeiro [14], involves

providing a visual discriminative stimulus (e.g., an interventionist shows the picture of a dog

breed as a model or sample); the emission of a pairing response by a learner (e.g., the learner

selects a corresponding picture of another breed of dog from an array of different pictures); the

delivery of reinforcers by the interventionist [2, 3, 13].

As a result, in Cordeiro [14], all participants acquired the primary targets (simple tacts)through

differential reinforcement. Regarding the other targets according to class, they fully emerged

(without direct teaching) for one participant after only two primary target training sessions in

which criterion was reached. For a second participant, two of the new repertoires emerged

without errors (arbitrary visual pairing and intraverbal) and the other two emerged partially

(but many correct responses were emitted). One limitation was that additional training and

probe sessions were not implemented for this participant because his participation in the study

could no longer continue. In the case of a third participant, five probes indicated a very partial

emergence effect (performance, in fact, got worse along probes). For the last participant, no

emergence effect was demonstrated. Due to the result for this participant, the direct teaching

through differential reinforcement was established for one of the repertoires according to class

(tact according to class). Along this training, four additional probes for the remaining

repertoires according to class were conducted and they all emerged without errors.

Data for the last participant of the research by Cordeiro [14] is in accordance with the argument

by Cariveau et al. [8] that the delayed acquisition of new repertoires through IF should not be

expected. However, the new approach by Cordeiro was not very intrusive because only one of

the four secondary targets according to class was directly taught and emergence of the

remaining was demonstrated thereafter. Cordeiro [14] discussed these data in the sense that,

Page 8 of 25

8

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

if the IF procedure is not successful for a given learner, an alternative approach to produce

emergence of new repertoires should be considered. In fact, that was what happened to the last

participant. Plus, the emergence of one or more repertoires according to class because of

training others is consistent with the results of some learners from the previous literature

involving children with ASD as participants [16, 17]. Direct teaching of all repertoires

representing secondary targets should be planned and implemented as a last resort.

Tullis et al. [15], as Cordeiro [14], trained simple tacts (e.g., saying “calculator” in the presence

of the picture of calculator and the question “what is this?”). The IF information was delivered

in the consequent portion of the contingency (e.g., the experimenter said to the participant “a

calculator has batteries”). The secondary targets were the following: LRFFC (e.g., selecting the

picture of calculator under the instruction “touch the item that has batteries”); tact under

control of function, feature or class (e.g., saying “calculator” under the picture of calculator and

the instruction “say the name of the item that has batteries”); intraverbal (e.g., saying

“batteries” under control of the verbal antecedent “a calculator has...”); reverse intraverbal

(e.g., saying “calculator” under the verbal antecedent “something that has batteries is...”). A

probe for each of the secondary targets was conducted after each series of three primary target

training sessions with IF. The three participants (children diagnosed with ASD) successfully

acquired both the primary and secondary targets (considering three sets of stimuli). The

acquisition of the secondary targets was delayed since the emergence occurred in subsequent

tests along training with IF. Maintenance of the repertoires was demonstrated for two months.

In short, the recent literature on IF and its effects in children with ASD as participants showed

that new repertoires may be established without direct teaching. This was demonstrated both

through maintenance training sessions, in which primary targets were already established in

the learners’ repertoire [5-8], and by teaching unknown primary targets [14-15].

Some of these recent studies were concerned with measuring the potential of IF in producing

the acquisition of several new targets in children with ASD, increasing the efficiency of the

procedure [6, 7, 14, 15]. Nevertheless, in these cases, not all children demonstrated a fast and

robust emergence effect of new skills after IF (or any emergence at all). The research by

Frampton and Shillingsburg [6] was an exception, since the two participants showed a fast

emergence of all, or nearly all, new repertoires. The participants from Tullis et al. [15] also

showed emergence of several untaught skills, but it was delayed. In other words, it happened

after several probe sessions along more primary target training sessions with IF.

In Frampton and Shillingsburg [6], the primary target maintenance training with IF was very

efficient in the sense that, across three sets of stimuli, three series of three training sessions

sufficiently produced acquisition of several new skills without direct teaching. Regarding the

other studies focused on emergence of multiple unknown targets, the only cases with

participant who matched the efficiency of those from Frampton and Shillingsburg, were one

participant from Cordeiro [14] and another one from Laddaga Gavidia et al. [7] (third set of

stimuli). As to other participants from these studies and those from the research by Tullis et al.

[15], emergence of new targets, when occurred, was delayed. For some learners, the emergence

was not demonstrated.

Page 9 of 25

9

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

Since the recent investigations on IF in children with ASD as participants showed mixed results

(depending on the learner, the emergence of new repertoires was immediate, delayed or not

observed), new scientific research on variables, which may possibly influence the efficiency of

IF, is warranted. The literature showed that primary target maintenance training with IF, which

effectively produced emergence of new targets in some participants, requires less effort to

respond than the case in which primary targets are unknown. In this sense, the current study

sought to extend the investigation on primary target maintenance training (simple listener

responding) with IF and its effects in producing emergence of multiple new repertoires in

children with ASD as participants. An alternating treatments design was used to compare the

efficiency between two interventions with IF. In one case, attending responses to visual stimuli

in training were demanded, as in the study by Laddaga Gavidia et al. [7], as well as tact of these

stimuli. In the other case, these demands were not made. Comparisons between the two

teaching conditions were conducted to assess the possible role of additional variables

(attending responses and tact of visual stimuli) in increasing the efficiency of the intervention

with IF. It was assumed that the intervention, which produced a more robust (errorless) and

immediate emergence effect of multiple repertoires according to category as secondary targets

(and directly related to primary targets and IF information), would be the most efficient one.

METHOD

Participants

The participants were three children with ASD (P1, P2 and P3) aged between 4 and 7 years.

They all received specialized interventions on Applied Behavior Analysis (ABA) in a university- based laboratory where data collection for the current study occurred. The participants had the

following repertoires according to the VB-MAPP assessment protocol [9]: 1) tact responses of

over 200 nonverbal stimuli, such as objects, pictures and actions (they could also select these

stimuli as listeners); 2) generalized identity matching to sample (relate identical pictures and

objects); 3) generalized arbitrary matching to sample performance (relate similar pictures and

objects and relate objects to corresponding pictures); 4) over 70 selection responses of pictures

and objects under verbal instructions specifying function, feature and class to which they were

related to; over 70 intraverbal responses (fill in blank type and answering “what”, “who” and

“where” questions).

All participants were able to vocally communicate using full sentences (consisting in four or

more words) regarding different verbal functions (mand and tact). Intraverbals with sentences

in general were emitted under prompt control. Considering other complex intraverbal cases,

the children demonstrated multiple responses under divergent control (e.g., saying “dog, cat

and horse” under the verbal instruction “name some animals”). They also showed some

intraverbal responses under convergent control (e.g., saying “soup” in the presence of the

instruction “name a hot meal”), but had difficulty in presenting responses under both divergent

and convergent control (e.g., saying “soup, barbecue and roast chicken” under the instruction

“name some hot meals”). Since some of the repertoires defined in the research as dependent

variables (VD) required multiple control [11, 12, 18], being a learner of these types of skills

outside the context of research represented a selection criterion.

Materials, Environment, Interobserver Agreement (IOA) and Treatment Integrity (TI)

Stimuli used in assessment and intervention conditions were organized in picture cards

measuring 7 X 10 cm. Each of them contained an image related to a category/class. Overall, the

Page 10 of 25

10

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

stimuli consisted of nine pictures of animals regarding three subcategories (land animals,

marine animals, and flying animals) and nine pictures of transportation regarding three

subcategories (ground transportation, marine transportation, and air transportation). Data

collection occurred in a university-based research laboratory where behavioral interventions

based on ABA were provided to the participants once a week. The room where the study

procedures were undertaken contained a table and three chairs. Each participant and an

experimenter sat facing each other. The experimenter conducted assessment and intervention

tasks and systematically collected data on the participants’ performance using a data sheet. A

second observer, not familiar with the research objectives, in approximately 30% of the

sessions also took data.

IOA was determined for these sessions (two probe sessions and two intervention sessions)

regarding all participants. IOA consisted of agreements and disagreements using a trial-by-trial

basis. For each assessment or intervention trial, an agreement was verified if both

experimenter and second observer recorded the participant’s response in the same way. IOA

was calculated in the following manner: number of trials with agreement divided by the total

number of trials in each session. To determine a percentage, the result was multiplied by 100.

IOA for all participants was 100%.

In approximately 30% of sessions, the second observer took data on the experimenter’s TI. This

was also determined for two probe sessions and two intervention sessions. It was used a

checklist on the conclusion of the following components: 1) use of relevant stimuli during tasks;

2) appropriate administration of instructions; 3) waiting 5s for the emission of a response by

the participant in each trial; 4) appropriate use of reinforcers and IF information; 5)

appropriate use of correction procedures, if needed. For each session in which TI was

determined, the total number of components concluded correctly was divided by the total

number of components. To establish a percentage, the result was multiplied by 100. Mean TI,

considering sessions with the three children, ranged from 80% to 100%. Procedural errors,

when committed, referred to not waiting up to 5s for the emission of a response by the

participant.

Dependent Variables and Independent Variables

In this investigation, the dependent variables (DV) consisted of the following untaught

repertoires according to category or class (secondary targets): listener responding according

to category (e.g., touching the picture of shark under the instruction “show me a marine

animal”); tact according to category (e.g., saying “marine animal” under the picture of shark and

question “what is a shark?”); arbitrary visual pairing (e.g., relating the picture of shark to the

picture of whale); intraverbals of saying item names according to category (e.g., saying “shark,

whale and dolphin” in the presence of the verbal instruction “name some marine animals”). One

of the independent variables (IV) consisted of teaching simple listener responses (primary

targets) (e.g., touching the picture of shark under the instruction “touch shark”) and presenting

an IF information (e.g., “the shark is a marine animal”). Another IV consisted of teaching simple

listener responding, also demanding attending responses to the pictures used and the simple

tact of the pictures (e.g., touching the picture of airplane under the instruction “touch the

airplane”. Thereafter, after looking at the picture and upon the question “what is this?”, the child

says “airplane”) and these responses were followed by an IF information (e.g., “the airplane is

a transportation”).

Page 14 of 25

14

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

Figure 1: Percentage of Correct Responses According to Category for P1

Note. The upper graph represents the percentage of correct responses according to category

before (pre-intervention probe) and after (post-intervention probe) the implementation of IV

with animals: listener responding according to category (black bar), tact according to category

(grey bar), arbitrary visual pairing (white bar) and intraverbal of saying item names according

to category (crosshatched bar). The lower graph represents the same kind of data before and

after the implementation of IV with transportation. The second segmented vertical line

represents the insertion of the IV, that is, primary target maintenance training (data not

shown).

According to Figure 1, regarding the animal category, P1 already had the repertoire of listener

responding according to category before intervention. About arbitrary visual pairing

repertoire, 44% correct responses were demonstrated in the last pre-intervention probe

session. During the first post-intervention probe session, performance improved to 100%

correct responses. Although performance fell to 44% in the following session, no errors

occurred in the last two post-intervention probe sessions. Tacts and intraverbals according to

category were not demonstrated in pre-intervention probe sessions and very discrete

improvements (22% and 11%, respectively) occurred until the last post-intervention probe

session. As to the transportation category, there was an improvement in listener responding

according to category from 56% correct responses in the last pre-intervention probe session to

100% in the first post-intervention probe session. Although performance fell to 44% in the

following session, it improved again to 100% in the last two post-intervention probe sessions.

Regarding arbitrary visual pairing, there was an improvement from 44% in the last pre- intervention probe session to 100% in both penultimate and last post-intervention probe

session. Tact according to category was unaffected by intervention and no correct responses

were emitted. In the case of intraverbal according to category, there was only a slight increase

in performance to 11% in the penultimate post-intervention probe session. Figure 2 shows the

Page 15 of 25

15

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

percentage of correct untaught repertoires according to category (secondary targets) for P2

along pre-intervention probe sessions, and after implementation of primary target

maintenance training with delivery of IF information (post-intervention probe sessions).

Figure 2: Percentage of Correct Responses According to Category for P2

Note. The upper graph represents the percentage of correct responses according to category

before (pre-intervention probe) and after (post-intervention probe) the implementation of IV

with animals: listener responding according to category (black bar), tact according to category

(grey bar), arbitrary visual pairing (white bar) and intraverbal of saying item names according

to category (crosshatched bar). The lower graph represents the same kind of data before and

after the implementation of IV with transportation. The second segmented vertical line

represents the insertion of the IV, that is, primary target maintenance training (data not

shown).

Figure 2 for P2, regarding the animal category, shows that listener responding according to

category improved from 44% correct responses in the last pre-intervention probe session to

100% correct responses in the last post-intervention probe session. In the case of arbitrary

visual pairing, a high percentage of correct responses had been demonstrated until the last pre- intervention probe session (89%). Performance improved to 100% in the penultimate post- intervention probe session, but it fell to 78% during the last session. Regarding tact according

to category, there was a slight improvement from 0% in the last pre-intervention probe session

to 22% in the penultimate post-intervention probe session. However, no correct responses

were emitted during the last post-intervention probe session. As to intraverbal according to

category, there was an important improvement from 0% in the last pre-intervention probe

session to 67% in the penultimate post-intervention probe session. Nevertheless, performance

fell to 11% in the last post-intervention probe session. Considering the case of transportation

category, performance in listener responding according to category improved from 44% in the

last pre-intervention probe session to 78% in the first post-intervention probe session. In the

following session, performance fell to 44%.

Page 16 of 25

16

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

It was more robust in the penultimate session with 89% correct responses, and, in the last

session, 78% correct responses were emitted. Arbitrary visual pairing improved from 22% in

the last pre-intervention probe session to 100% during the second post-intervention probe

session. Although performance fell to 67% in the following session, it improved again to 100%

in the last session. Tact according to category improved slightly from 0% in the last pre- intervention probe session to 22% correct responses in the penultimate post-intervention

probe session, but performance dropped again to 0% in the last session. Intraverbal according

to category also only improved very slightly from 0% in the last pre-intervention probe session

to 11% correct responses in both penultimate and last post-intervention probe sessions. Figure

3 shows the percentage of correct untaught repertoires according to category (secondary

targets) for P3 along pre-intervention probe sessions, and after implementation of primary

target maintenance training with delivery of IF information (post-intervention probe sessions).

Figure 3: Percentage of Correct Responses According to Category for P3

Note. The upper graph represents the percentage of correct responses according to category

before (pre-intervention probe) and after (post-intervention probe) the implementation of IV

with animals: listener responding according to category (black bar), tact according to category

(grey bar), arbitrary visual pairing (white bar) and intraverbal of saying item names according

to category (crosshatched bar). The lower graph represents the same kind of data before and

after the implementation of IV with transportation. The second segmented vertical line

represents the insertion of the IV, that is, primary target maintenance training (data not

shown).

According to Figure 3, for the animal category, P3 showed an improvement in listener

responding according to category from 22% in the last pre-intervention probe session to 100%

correct responses in the last post-intervention probe session. In the case of arbitrary visual

pairing, in the last pre-intervention probe session correct responses were not emitted. After

intervention, the first post-intervention probe session showed 44% correct responses, but

performance fell to 0% in the following sessions. As to tact and intraverbal according to

category, performance was unaffected by intervention, that is, correct responses were never

emitted. In the case of the transportation category, there was an improvement from 44%

Page 17 of 25

17

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

correct responses in the last pre-intervention probe session to 100% in the last post- intervention probe session of listener responding according to category. As to arbitrary visual

pairing, after 56% correct responses in the last pre-intervention probe session, performance

was unaltered in the first post-intervention probe session. However, no correct responses were

demonstrated in the following probe sessions. Tact and intraverbal according to category were

not emitted at any time during data collection.

Table 2 shows the number of correct trials per secondary target demonstrated by all

participants along probe sessions (both pre-intervention and post-intervention probe

sessions) for each of the two defined categories (animals and transportation). It is important to

remember that, for each untaught repertoire according to category (secondary target), nine

trials were administered per session. Therefore, the maximum number of correct responses

possible by each participant for each repertoire corresponded to nine out of nine trials (which

represented 100% correct responses).

Table 2: Number of Correct Responses (Secondary Targets) in Probe Sessions

Pre-intervention probe sessions (animal

category)

Post-intervention probe sessions (animal

category)

Listener

responding

Arbitrary

visual

pairing

Tact Intravebal Listener

responding

Arbitrary

visual

pairing

Tact Intravebal

P1 9; 9; 9 2; 4; 4 0; 0;

0

0; 0; 0 9; 9; 7; 9 9; 4; 9; 9 0; 0;

1; 2

0; 0; 1; 1

P2 4; 4; 4 6; 8; 8 0; 0;

0

1; 0; 0 4; 8; 6; 9 6; 6; 9; 7 0; 1;

2; 0

0; 0; 6; 1

P3 3; 0; 2 4; 0; 0 0; 0;

0

0; 0; 0 3; 1; 2; 9 4; 0; 0; 0 0; 0;

0; 0

0; 0; 0; 0

Pre-intervention probe sessions (transportation

category)

Post-intervention probe sessions

(transportation category)

Listener

responding

Arbitrary

visual

pairing

Tact Intravebal Listener

responding

Arbitrary

visual

pairing

Tact Intravebal

P1 7; 5; 5 4; 4; 4 0; 0;

0

0; 0; 0 9; 4; 9; 9 2; 0; 9; 9 0; 0;

0; 0

0; 0; 1; 0

P2 1; 4; 4 6; 1; 1 0; 0;

0

0; 0; 0 7; 4; 8; 7 2; 9; 6; 9 0; 0;

2; 0

0; 0; 1; 1

P3 3; 0; 4 4; 0; 5 0; 0;

0

0; 0; 0 4; 2; 2; 9 5; 0; 0; 0 0; 0;

0; 0

0; 0; 0; 0

Note. The left portion of the table shows the number of correct responses, regarding four

secondary targets, by each participant in three pre-intervention probe sessions for the animal

(top) and transportation (bottom) categories. The right portion of the table shows the number

of correct responses, regarding four secondary targets, by each participant in four post- intervention probe sessions for the animal (top) and transportation (bottom) categories.

The data in Table 2, similarly to the cases in Figure 1, Figure 2, and Figure 3, shows that for the

animal category, immediate emergence of arbitrary visual pairing was demonstrated by P1

Page 18 of 25

18

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

(nine correct responses or 100% in the first post-intervention probe session). Both P2 and P3

showed a delayed emergence in listener responding according to category (nine correct

responses or 100% in the last post-intervention probe session). Regarding the transportation

category, P1 showed immediate emergence of listener responding according to category (nine

correct responses or 100% in the first post-intervention probe session) and delayed emergence

of arbitrary visual pairing (nine correct responses or 100% in the third post-intervention probe

session). P2 demonstrated delayed emergence of arbitrary visual pairing (nine correct

responses or 100% in the second post-intervention probe session). P3 demonstrated delayed

emergence of listener responding according to category (nine correct responses or 100% in the

last post-intervention probe session).

DISCUSSION

In this study, primary target maintenance teaching with delivery of IF information in the

consequent portion of the contingency was established for three participants (children with

ASD). Simple listener responding regarding stimuli from two categories, animals, and

transportation, were the primary targets. No incorrect responses were emitted along training

sessions by any participant. In the process, post-intervention probe sessions to verify the

emergence of new repertoires according to category (secondary targets) were administered

each time a learning criterion was achieved for simple listener responding (that is, after every

two training sessions without errors). Gains regarding untaught repertoires according to

category (listener responding, arbitrary visual pairing, tact and intraverbal according to

category) were partial for all participants. When robust(errorless)improvements were noticed

among participants, they were related to one or two skills only (listener responding according

to category and/or arbitrary visual pairing). A partial emergence effect of new repertoires, after

primary target training with IF, had also been demonstrated by other participants with ASD

from previous studies [6, 7, 14].

In the current investigation, the emergence of new skills was not always immediate. Besides,

two of the skills probed (tact according to category and intraverbal of saying item names

according to category) never emerged, or their emission was too discrete. The efficiency of

primary target (simple listener responses) maintenance training with IF was a function of the

extent of emergence of related untaught repertoires according to category, and whether

emergence was immediate or not. Maximum efficiency of training with IF would be verified for

each untaught skill assessed, if the first post-intervention probe session showed immediate

errorless performance. For only one participant (P1), some secondary targets emerged

immediately (arbitrary visual pairing in the case of animal category and listener responding

according to category in the case of transportation category). In previous literature, immediate

acquisition of new targets, after training with IF, had also been shown by some learners [6, 7,

14].

Still about P1 in the current study more specifically, for the animal category, it was said that

arbitrary visual pairing was the only secondary target that emerged immediately without

errors (during the first post-intervention probe session). Regarding this emergent repertoire,

training with IF seemed to be more efficient than the case with transportation category in which

attending, and tact responses to visual stimuli used were also required. In this case, the

errorless emergence of arbitrary visual pairing was delayed (third post-intervention probe

session). Also, for transportation category, listener responding according to category emerged

Page 19 of 25

19

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

fully and immediately. A comparison with the case of animal category was not possible because

the participant demonstrated listener responding according to category without errors in the

last pre-intervention probe session. This represented a limitation and future studies should

only select participants who do not fully show any of the repertoires defined as DV in before

the onset of intervention.

About P2 in this research, for the animal category, errorless emergence of arbitrary visual

pairing and listener responding was delayed (only in third and fourth post-intervention probe

session, respectively). As to transportation category, the full emergence of visual pairing was

delayed (second post-intervention probe session). In the case of listener responding according

to category, although errorless emergence did not happen, few errors occurred during the third

post-intervention probe session. For this participant, there seems to be no teaching procedure

with IF more efficient than the other. Finally, in the case of P3, only listener responding

according to category emerged without errors and it was delayed for both animal and

transportation categories (fourth post-intervention probe session). No training with IF was

more efficient than the other as well.

According to Cariveau et al. [8], if the acquisition of new repertoires through IF is delayed, a

more intrusive measure should be used. That is, the direct teaching of the skills through

differential reinforcement should be established. This logic seems applicable to the participants

of the current research, since delayed acquisition of some relations was verified for all of them.

Plus, important relations (tact according to category and intraverbals of saying item names

according to category) did not emerge for any participant at all. However, the literature

discusses the possibility of additional measures to produce emergent responding. In the study

by Cordeiro [14], as told previously, one of the participants did not show emergence of different

untaught repertoires according to category (secondary targets) after simple tact (primary

target) training with IF.

Thus, an additional IV was applied, that is, one of the secondary targets (tact according to

category) was directly taught. Along training sessions, all the remaining skills (listener

responding according to category, arbitrary visual pairing and intraverbals of saying item

names according to category) emerged after several post-intervention probe sessions.

Although the establishment of these remaining skills was not immediate, the new training may

be considered efficient in the sense that it resulted in acquisition of three new skills without

differential reinforcement. Since in ABA investigations on procedures which may produce

emergence of new repertoires is important for participants with ASD, their direct teaching

should be defined as last resort. Besides, previous studies have shown that teaching tact

according to category (and other repertoires), similarly to the case of one of the participants

from Cordeiro [14], possibly influenced the emergence of another skill as well. In a study by

Grannan and Rehfeldt [16] conducted with two participants (children with ASD), the teaching

of tact according to category was part of a more comprehensive approach.

The participants were also taught simple tacts (that is, tact responses according to noun) and

arbitrary visual pairing. The direct teaching of these skills produced emergence of intraverbals

of saying item names according to category for both participants. In Grannan and Rehfeldt [16],

the effects of the taught skills were not assessed separately, so determining the extent of

emergence (of the untaught intraverbal repertoire) possibly produced by each intervention

Page 20 of 25

20

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

was not possible. However, it was discussed that teaching tact according to category played an

important role on intraverbal emergence (and that an expressive repertoire of tacts is

important for the establishment of intraverbals).

Recently, Matos and Lima [17] conducted a similar investigation involving four participants

(children with learning disabilities). The participants were taught the following repertoires in

the following order: simple tact, tact according to category and listener responding according

to category (this one instead of arbitrary visual pairing). The effects of the interventions were

also assessed on the emergence of intraverbals according to category. In this study, the effects

of the interventions were measured separately, and it was noticed that, after teaching simple

tact and tact according to category, intraverbals largely emerged for a participant. In the case

of this learner, it is worth mentioning that, among all participants, he was the one with a very

expressive and generalized repertoire of tacts at the beginning of the research. The same

applied to the participant from Cordeiro [14] to whom teaching tact according to category

produced the emergence of other repertoires according to category, including intraverbals.

In this study, simple listener responding maintenance training with IF did not produce

emergence of tact according to category and intraverbal of saying item names according to

category for any of the three participants with ASD. Considering that all of them had a robust

tact repertoire (and they were all able to tact the stimuli used in the research according to noun

as well), tact teaching according to category could have been investigated as an additional IV in

possibly producing the emergence of a related repertoire (intraverbal), as in the case of some

participants from previous studies [14, 16, 17]. Intraverbal training could also be established

to measure the possible emergence of tact according to category as well. In this sense, two

teaching sequences (tact training – probing intraverbal; intraverbal teaching – probing tact)

could be compared to determine which would be the most efficient (in the sense of producing

a better emergence effect of the related untaught repertoire).

Another aspect of the investigation demands discussion. Some of the repertoires defined as DV

required multiple control [11, 12, 18]. Listener responding according to category was

demonstrated in post-intervention probe sessions. In each session trial, a participant had to

select a picture from an array under the control of two elements of an instruction (e.g., touching

the picture of airplane under the instruction “show me air transportation”). In the example, the

words “air” and “transportation” jointly need to control the selection response and only the

picture of airplane matched the subcategory mentioned. The DV regarding tact according to

category and intraverbal of saying item names according to category also required multiple

control.

Regarding tact according to category, two stimuli needed to exert joint control as well (e.g.,

saying “air transportation” in the presence of the picture of airplane and instruction “what is

airplane?”). In this example, it is expected that the question changes the evocative function of

the picture, providing the context for tacting according to category. During post-intervention

probe sessions, in most trials, the participants just repeated the name provided in the

instruction (e.g, “airplane”). This could also be related to a history of reinforcement for the

emission of simple tacts under the control of pictures (e.g., saying “car”, “dog”, whale” under

their corresponding pictures). This result suggests that direct teaching through differential

reinforcement should be defined (Cariveau et al. [8]), or related intraverbals according to

Page 21 of 25

21

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

category could be taught and new post-intervention probe sessions administered to verify

possible emergence of tacts according to category.

In the case of the intraverbal, the responses needed to be emitted under both divergent and

convergent control[11, 12](e.g., saying “whale, shark and dolphin” under the instruction “name

marine animals”). In the example, the words “marine” and “animals” jointly control the

mentioned multiple target responses. Regarding the current study, it was previously mentioned

that, before the onset of the investigation, the participants demonstrated intraverbals under

divergent control and under convergent control separately. However, they lacked the skill of

demonstrating intraverbal responses under both divergent and convergent control (as in the

example pointed out). This represented a limitation of the research, and it is important that

future studies select participants who already demonstrate at least some intraverbal repertoire

under both divergent and convergent control. Anyway, as in the case of previous studies [14,

16, 17], it would be important to verify if teaching tact according to category as an additional

IV produces the emergence of intraverbals.

The recent literature on IF has discussed the possible influence of echoing IF information on

the establishment of new repertoires without direct teaching. In Frampton and Shillingsburg

[6], the participants did not echo the IF information during primary target maintenance

teaching. However, the authors hypothesized the emission of covert echoic by the participants,

possibly influencing acquisition of new targets. In Laddaga Gavidia et al. [7], the participants

echoed the IF information in most primary target maintenance teaching trials and suggested

that overt echoic also influenced acquisition of new targets to a greater degree for one of the

participants.

In Cariveau et al. [8], for a single participant, there was an increase of overt echoic of the IF

information during a first assessment, but few overt echoics were demonstrated in a second

assessment. The authors also considered the possibility of emission of covert echoics and that

they may have influenced the emergence of targets from some sets. In the current study, the

participants did not overtly echo the IF information during primary target maintenance

teaching, but it is also hypothesized that they emitted covert echoics, possibly influencing

emergence of one or two of the four secondary targets (depending on the participant). Finally,

in the current study for a set of stimuli, the demand to attend to visual stimuli, as in Laddaga

Gavidia et al. [7], does not seen to have significantly increased the efficiency of teaching with IF

(as well as the demand to tact the pictures used).

This study had major limitations that need to be discussed. Although it was meant to be an

experimentally based investigation using a single case research design (adapted alternating

treatments design – AATD), the two treatments, whose effects were measured and compared

for each participant with ASD, aimed at teaching skills (simple listener responses) already

established in the participants’ repertoires. Considering the case of the design used (AATD), the

mentioned skills defined as primary targets could not be part of the participants’ repertoires

before the onset of the interventions, since functional relations between IV and DV would not

be feasible. Through an AATD design with two treatments conducted in an alternated fashion,

the demonstration of experimental control occurs if there is a differentiation in target

acquisition [20], which was not possible in this study, representing a threat to internal validity.

Page 22 of 25

22

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

Plus, important recommendations to increase the chances of demonstrating experimental

control were not followed.

Cariveau and Fetzner [21] conducted a review of studies on AATD with the purpose of analyzing

adopted measures to ensure or facilitate the establishment of experimental control. They

reviewed and coded 114 articles from 27 journals (five in Behavior Analysis and the others in

Special Education) published between 1985 and 2017. It was noticed that in more than half of

the studies, one or more important methods for demonstrating experimental control were

employed. A no-treatment control condition was used in less than one third of the studies.

However, in more than half of the studies with a control condition, correct performance was

demonstrated at a level above the initial baseline, representing a potential threat to internal

validity.

In 28.1% of the studies, the AATD design to address non-reversible behaviors was embedded

into another single case design (e.g., multiple baseline/MBL design or multiple probe/MP

design). The MBL and MP designs may be defined across behaviors, settings, or individuals at

staggered points in time. Of the articles that combined AATD with MBL or MP, 65.6% showed

that correct performance did not occur at a level beyond initial baseline along staggered panels.

In this sense, Cariveau and Fetzner [21] suggested that combining designs may be a better

method than using a control condition to prevent threats to internal validity. They also

suggested that both methods (no-control condition; additional designs like MBL or MP) should

be included in research on AATD to ensure the establishment of experimental control.

Cariveau et al. [22] conducted review research to report methods to equate target sets

regarding the difficulty in studies employing the AATD design. According to Cariveau et al.,

equating target sets is also a necessary measure to ensure the validity of the AATD. If the target

sets are not appropriately equated, changes in the DV may be a function of a difference in target

difficulty and not a function of the IV. The review involved a search in five journals in Behavior

Analysis, and the search was conducted in 2018. 65 articles using the AATD design were found.

In 36 of these articles, one or more methods of logical analysis were used to equate target sets.

They were equated based on (1) the number of syllables (e.g., targets across conditions should

have similar number of syllables); (2) overlapping sounds (assigning targets with similar first,

middle and end sound to different conditions; assigning targets that rhyme to separate

conditions); (3) novelty of the responses (by assessing whether other relations regarding

individual targets are acquired); (4) visual properties (when visual targets share common

aspects, the targets should be assigned to different conditions); (5) number of motor responses

(responses that share a similar number of steps); (6) number of letters in the targets. Among

all these methods, controlling for the number of syllables was the most used one. And the most

common combined methods in the studies were the number of syllables and overlapping

sounds. Cariveau et al. [22] concluded that, even though the ideal methods are still unknown, a

combination of those from the literature reviewed based on logical analyses should be used in

future research on AATD design.

In the current study, besides the fact that the primary targets were already part of the

participants’ repertoires, the recommended methods by the literature regarding the definition

of a no-treatment control condition, combining AATD with MBL or MP design and equating the

target sets [21, 22] were not addressed. This poses a threat to internal validity indeed.

Page 23 of 25

23

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

Nevertheless, four new repertoires according to category (secondary targets) were probed for

emergence along the process of conducting maintenance training of primary targets (simple

listener responses) with IF. Even though two of the new skills defined as secondary targets did

not emerge (tact and intraverbal according to category), errorless performance of other

relations was demonstrated to some extent by all participants (regarding arbitrary visual

pairing and/or listener responding according to category).

This suggests that maintenance training of primary targets with IF possibly played a role in the

emergence of new repertoires. It is important that the types of primary targets of this research

(simple listener responses with IF) be addressed in future studies as unknown skills by

participants with ASD. In one condition, attending and tact responses to visual stimuli in

training should be demanded. The target sets should be equated (regarding the difficulty) to

better assess the possible effects of attending and tact responses in increasing the efficiency of

the IF procedure. Plus, a no-treatment control condition should be defined by adding a target

set to which differential reinforcement, correction procedures and IF are not delivered. Finally,

a multiple probe design across participants should be combined with AATD. This way, the

demonstration of experimental control will possibly be addressed with the establishment of a

differentiation in primary target acquisition compared to baseline levels of responding, and the

possible emergence of untaught repertoires representing secondary targets.

DECLARATIONS

Ethical Approval

The study was approved by an ethics committee in research with humans (Research Ethics

Committee - Comitê de Ética em Pesquisa/CEP) (authorization No. 4.284.271) from Federal

University of Maranhão, São Luís-MA (Brazil). Those responsible for the participating children

(as well as the children themselves) signed an informed consent form for participation. All

personal information was kept confidential, and the procedures could be interrupted at any

time without prejudice to those involved. No form of financial compensation was foreseen for

participation, or for travel costs. All participants would be compensated for any expenses with

their participation in the research, and for any damages they could suffer for the same reason,

and resources would be guaranteed for these expenses. Those responsible for the children were

informed that the research results would always be presented to them and that these results

would also be disclosed in events and other productions of a scientific nature only, without any

identification of the participants. All mandatory terms of presentation have been delivered to

the ethics committee (CEP) and are in accordance with resolution 466/12 of the National Health

Council (Conselho Nacional de Saúde/CNS).

Competing Interests

The authors declare no competing interests.

Authors’ Contributions

D.C.M. and P.G. wrote the main manuscript text and reviewed the manuscript. D.J.R.L. and K.R.S

prepared figures and reviewed the manuscript.

Funding

Own funding (by the authors). The study involved no external funding.

Page 24 of 25

24

Advances in Social Sciences Research Journal (ASSRJ) Vol. 11, Issue 1, January-2024

Services for Science and Education – United Kingdom

Availability of Data and Materials

Raw data may be obtained by contacting the corresponding author.

References

1. Skinner BF. Verbal behavior. Prentice-Hall, Inc; 1992.

2. Greer RD, Ross DE. Verbal behavior analysis: Inducing and expanding complex communication in children

with severe language delays. Allyn & Bacon; 2008.

3. Sundberg ML, Partington JW. Teaching language to children with autism or other developmental disabilities.

Behavior Analysts, Inc; 1998.

4. Reichow B, Wolery M. Comparison of Progressive Prompt Delay with and without Instructive Feedback. J

Appl Behav Analysis. 2011; 44:327-340. http://dx.doi.org/10.1901/jaba.2011.44-327.

5. Tullis CA, Gibbs AR, Butzer M, Hansen SG. A comparison of secondary target location in instructive feedback

procedures. Advances in Neurodevelopmental Disorders. 2019; 3:45-53. https://doi.org/10.1007/s41252-

018-0090-4.

6. Frampton SE, Shillingsburg MA. Promoting the development of verbal responses using instructive feedback. J

Appl Behav Analysis. 2020; 53:1029-1041. https://doi.org/10.1002/jaba.659.

7. Laddaga Gavidia V, Bergmann S, Rader K.A. The use of instructive feedback to promote emergent tact and

intraverbal control: a replication. The Analysis of Verbal Behav. 2022; 38:95-120.

https://doi.org/10.1007/s40616-022-00171-y.

8. Cariveau T, Brown A, Platt D, Ellington P, Hurtado R. An evaluation of instructed feedback during mastered

demands. The Analysis of Verbal Behav. 2022; 38:179-189. https://doi.org/10.1007/s40616-022-00174-9.

9. Sundberg ML. The verbal behavior milestones assessment and placement program: The VB-MAPP. AVB

Press; 2008.

10. Miguel CF. Common and intraverbal bidirectional naming. The Analysis of Verbal Behav. 2016; 32:125-138.

https://doi.org/10.1007/s40616-016-0066-2.

11. DeSouza AA, Fisher WW, Rodriguez NM. Facilitating the emergence of convergent intraverbals in children

with autism. J Appl Behav Analysis. 2019; 52:28-49. https://doi.org/10.1002/jaba.520.

12. Sundberg ML. Verbal stimulus control and the intraverbal relation. The Analysis of Verbal Behav. 2016;

32:107-124. https://doi.org/10.1007/s40616-016-0065-3.

13. Matos DC. Análise do comportamento aplicada ao desenvolvimento atípico com ênfase em autismo [Applied

behavior analysis to atypical development with an emphasis on autism]. AICSA; 2016.

14. Cordeiro NCP. Ensino de repertórios por meio de feedback instrucional em crianças com transtorno do

espectro autista (tea) [Teaching repertoires through instructional feedback in children with autism

spectrum disorder (asd)] [Master’s thesis, Universidade Federal do Maranhão]. Repositório institucional da

Universidade Federal do Maranhão; 2020.

https://sigaa.ufma.br/sigaa/public/programa/noticias_desc_stricto.jsf?lc=es_ES&idPrograma=1157&noticia

=219561216.

15. Tullis CA, Gibbs AR, Priester J, Tillem A. Emergence of verbal responses using instructive feedback: A

replication and extension. Behavioral Interventions. 2022; 37:1-19. https://doi.org/10.1002/bin.1836.

Page 25 of 25

25

Matos, D. C., Loureiro, D. J. R., Silva, K. R., & Galvão, P. (2024). Comparison of Two Maintenance Training Arrangements with Instructive Feedback

in Children with Autism. Advances in Social Sciences Research Journal, 11(1). 01-25.

URL: http://dx.doi.org/10.14738/assrj.111.16172

16. Grannan L, Rehfeldt RA. Emergent intraverbal responses via tact and match-to-sample instruction. J Appl

Behav Analysis. 2012; 45:601-605. https://doi.org/10.1901/jaba.2012.45-601.

17. Matos DC, Lima ABR. Ensino de tatos e respostas de ouvinte e os efeitos sobre emergência de intraverbais.

[Teaching tact and listener responses and the effects on intraverbal emergence]. In: Vilas Boas DLO, Cassas F,

Gusso HL, Mayer PCM, editors. Comportamento em foco. Associação Brasileira de Psicologia e Medicina

Comportamental; 2018. pp. 140-153.

18. Michael J, Palmer DC, Sundberg ML. The multiple control of verbal behavior. The Analysis of Verbal Behav.

2011; 27:3-22. https://doi.org/10.1007/ BF033 93089.

19. Barlow DH, Hayes SC. Alternating treatments design: One strategy for comparing the effects of two

treatments in a single subject. J Appl Behav Analysis. 1979; 12:199-210.

https://doi.org/10.1901/jaba.1979.12-199.

20. Sindelar PT, Rosenberg MS, Wilson RJ. An adapted alternating treatments design for instructional research.

Education & Treatment of Child. 1985; 8:67-76.

21. Cariveau T, Fetzner D. Experimental control in the adapted alternating treatments design: a review of

procedures and outcomes. Behavioral Interventions. 2022; 37:805-818. https://doi.org/10.1002/bin.1865.

22. Cariveau T, Batchelder S, Ball S, Montilla ALC. Review of methods to equate target sets in the adapted

alternating treatments design. Behav Modification. 2021; 45:695-714.

https://doi.org/10.1177/0145445520903049.