Maintenance of Interventions: The Behavioral Research Supervisory System

Maintenance of Interventions: The Behavioral Research Supervisory System
Richard W. Malott

E-Mail From Nadia Mullen

Hello

You may recall my contacting you previously about your studies in JOBM but in case you don't, I am a student at the University of Otago in New Zealand, completing my masters thesis in the area of intervention maintenance.

I'd like to take this opportunity to thank you for your previous comments and apologize for contacting you a second time. I overlooked one of your studies and, having 'rediscovered' it today, would like to add it to my review. In order to do this however, I need to know the outcome.

The article I am referring to is "A Supervisory System for Accomplishing Long-Range Projects" which was published in JOBM in 1980. In this study the intervention was successful and I would like to know if, following the conclusion of the study, the intervention was maintained, either with the participants in the study, or with new MA candidates.

If the intervention was maintained, what characteristics of this study and intervention do you believe helped it to be maintained? If the intervention was not maintained, what characteristics of this study
and intervention do you believe deterred the university from implementing this successful intervention?

Thank you for your time.

Nadia Mullen

Reply from Dick Malott

Nadia: Nice to hear from you again. Looks to me like, if anyone's going to be able to maintain systems, it's you. I'm impressed with the way you're hanging in on this important topic.

First a brief review of the intervention. What you're talking about is our Behavioral Research Supervisory System (BRSS) that we use to manage the performance of students doing BA honors theses, MA theses, MA projects, Ph.D. dissertations, and assistantship tasks. It's a performance-management system where the researchers and assistants normally work about 13 hours per week. They are supervised by advanced grad students or by me, in the case of the Ph.D. students.

The students prepare weekly performance contracts consisting of recurring and nonrecurring tasks that lead to the completion of their research. They attend a weekly meeting with their research supervisor where they go over the work they've accomplished that week and discuss what they will do the next week. The system works well in terms of helping students complete high-quality research in a timely manner and actually graduate. Nationally, and according to our data, at WMU, only 60 % of the Ph.D. students ever complete their dissertations without such a system, and most who do, do not do so in a timely manner.

The article you refer to was Mike Dillon's Ph.D. dissertation. We published some follow up work that I'm pleased with, as well (Maria Malott's Ph.D. dissertation):

1. Garcia, M. E., & Malott, R. W. (1988) Una solucion al fenomeno "todo menos tesis y disertacion." Revista Intercontinental de Psicologia y Educacion, 1, 205-216.
2. Garcia, M. E., Malott, R. W., & Brethower, D. (1988). A system of thesis and dissertation supervision: Helping graduate students succeed. Teaching of Psychology, 15, 186-191.

(There are some references to a couple of other of our articles in the Garcia article, but I don't have the references with me at the JazzFest.)

At the time Maria was running the BRSS, I had about 23 MA and Ph.D. students. We finished almost all of the projects, theses, and dissertations, in 1987, before she and I took off for a year and a half to write the second edition of Elementary Principles of Behavior and travel around the world. Then I helped my few remaining students finish their dissertations etc. during a year or so after our return in 1989.

However, at the time of my return to campus, I had decided to stop taking graduate students because I was unable to do any significant writing and supervise the students at the same time, even with the BRSS. So I stopped using the BRSS. Then, over the next few years, I found myself drifting back into heavy-duty student supervision, with the result that last semester, I was supervising something like 2 Ph.D. students, 12 MA students, and 10 BA students, a fairly heavy load.

During the early transition years from essentially no grad students to my current group of students, I did not re-implement the BRSS, in a formal way. My supervision of those intervening students was, shall we say, informed by the BRSS, but it was a fairly pale copy; and those intervening students would have benefited greatly from the BRSS. Why didn't I re-implement the BRSS? Procrastination. I meant to. I'd do it tomorrow. The startup cost is fairly high for implementing any system, as elaborate as the BRSS. And maintaining it requires really having your feces together, which I don't, without the help of a lot of great grad and undergrad assistants. In the meantime, we just muddled through.

However, over the past 3 years, my grad and undergrad students and I have been reviving, revising and re-implementing the BRSS. Now it's in great shape and getting even better. Michelle Seymour, MA, and Rachel Bissi, undergrad, are presenting a poster on the BRSS at ABA 2000.

Now, Nadia, to your original questions:

What characteristics of the BRSS helped in its maintenance? Probably none. Of course, the system is effective. But I think that's not your point. I think your point is what contributes to the maintenance or lack of maintenance of systems, given that they are effective.

And what contributed to the lack of maintenance of the BRSS? Everything.

BRSS is a performance-management system. And essentially all PM systems that I know of are very effortful and have only small immediate benefits, though their cumulative benefits can be great, over a period of time. All of which spells failure of maintenance; which is why you're doing your thesis on this topic.

So why is the BRSS maintaining? Answering this takes us an issue we behavior analysts rarely discuss. The "personality, temperament, or belief structure" of the person responsible for the system. So that means, we need to discuss my favorite subject-me, tasteless though that may be. The reason the BRSS is again doing well in my overall system, which I call BATS (the Behavior Analysis Training System) has more to do with the characteristics of my orientation than it does with the characteristics of the BRSS.

First, I view almost everything I do from a systems perspective. As my former colleague, Neil Kent, knew and exploited, all he had to do was put the word, "system" after a task; and I'd willingly take it on, no matter how odious-"Malott, will you be in charge of the garbage-disposal system?" "Yes, sir."

But, what does "system" mean, in the context of the BRSS. It means, that when we consider an individual student's performing or failing to perform an individual research task, we don't consider this as the result of a unique, idiosyncratic circumstantial and personality quirk. Instead, we consider this performance or failure to be characteristic of many or most students in many or most circumstances. (In deed, 40% of all Ph.D. students in the United States fail to complete their dissertations.)

The traditional view is to blame the victim, which is what four out of four JABA reviewers did when they rejected the Garcia article on the BRSS, not arguing that the research design was poor or the data not significant, but that it was essentially immoral to establish a performance-management system that would help the weak sisters survive the rigors of graduate training, which the reviewers viewed as more of an IQ/moral-fitness test than an educational program.

So, part of the systems view of human performance is to consider performance to be a function of a set or system of external, controlling variables (mainly behavioral contingencies). Now most behavior analysts give lip service to this, but I observe few practicing it outside of their immediate area of expertise (e.g., children labeled "autistic"); they don't apply it to their students, their family, their colleagues, or themselves. Instead, if a person screws up, it's because the person is a screw up; the behavior analysts blame the victim of the pathological behavioral contingencies, almost as readily as everyone else does.

And a related part of this systems view is that the behavioral contingencies (typically pathological behavioral contingencies) are fairly common. So, if a research student fails to do a task today, there's a good chance he will fail to do one tomorrow (which gives rise to our most fundamental general rule: the best predictor of future behavior is past behavior [regardless of the behavor's rationalizations]). Furthermore, if this student failed to do this research task, many other students will probably fail to do similar tasks. Then, with that orientation, we keep our eyes open and maybe even collect a little "base-line" data, to see if we've identified a common problem area (e.g., student failure to complete research tasks). And, usually it turns out that, yes, we are dealing with common, frequent problems.

So this part of the systems view is that most human problems are performance problems, that most performance problems are a result of pathological contingencies, and that most performance problems and their causal pathological contingencies are common rather than unusual.

The next part of the systems view is that you can set up healthy, helpful PM contingencies that will support good performance (e.g., the BRSS); and, though expensive, it will usually be cost-effective to set up those PM contingencies.

Then, the system needs a prime mover (e.g., I'm the prime mover of the BRSS) and depending on how many systems the prime mover is moving, the system may need a systems manager (e.g., Mike Dillon and Maria Garcia were the original BRSS systems mangers and Michelle Seymour has been the most recent). All of this requires a fair amount of self-awareness (e.g., I don't have my act together enough to get myself to do single handedly everything needed for a successful BRSS). And this awareness of ones limitations often points to the need for considerable expertise at delegatory technology, the art of getting others to do reliably everything for which you are ultimately responsible, with between 80 and 200% of the quality that would result if you, yourself did those tasks.

So one of my belief-structure characteristics responsible for the survival of BRSS is my thorough-going systems orientation. Without that orientation, there would never have been nor would there now be a BRSS.

The other characteristic is the establishing operation, my fairly heavy guilt load, my fear of being a scum bag. I believe that, if I know what should be done and I don't do it, then I'm a scum bag (e.g., if there's a brother who needs keeping and I ain't keeping him, I'm a scum bag or maybe a sleaze ball). The extent to which I succeed as a systems prime mover is the extent to which I'm running from fear of guilt. The extent to which I fail is the extent to which I really am a scum-bag sleaze ball. I would argue that essentially all professionally successful people, including behavioral systems prime-movers/managers, come from a behavioral history that has set up fear of guilt as an effective establishing operation. I would argue that this is also true of us semi-successful people, as well.

I think prime-movers can be helped by PM contingencies (e.g., I make big personal use of externally monitored self-management contracts). However, I don't think we have the PM skills to generate really high-quality work (Gilbert's class work) and high quantity work from prime-movers, and maybe from anyone, if the PM contingencies aren't functioning hand in hand with fear/guilt trips that our parents set in motion before we entered grade school.

Jack Michael put it something like this: You've got a stack of articles you're supposed to write, a stack of quizzes you're supposed to grade, and a stack of letters you're supposed to answer; and you're behind on all of them. You whittle away at one stack until the guilt associated with one of the other stacks becomes so overwhelming that you shift over to it. Around and around, forever guilt driven from one stack to another.

Maria Malott put it somewhat like this: We're watching the Jugglers, Acrobats, and Magicians of the Peking Opera Company (or something like that) at Miller Auditorium. A cocky young man comes out in a white suit with a black, satin shirt and strikes a John Travolta, Saturday Night Fever pose. Then his lovely little assistant slides out a long, rectangular table and arranges four plates in a row. The dude starts each plate spinning on its edge, one at a time; then folds his arms and smiles complacently at the audience. But eventually, one of the plates slows down and is about to fall dormant to the table surface. The dude rushes over and sets it rapidly spinning again, repeating this with the remaining three in the same just-in-time manner. He loses his composure a bit, however when the assistant slides out another table with four more plates, which dude feels obligated to start spinning as well. Now, as he gets the last plate spinning, the first one verges on dormancy. So he breathlessly, rushes to the rescue, as assistant slides out another table and four plates. Etc., until poor dude is desperately keeping 20 plates spinning, breathlessly running from plate to plate. Maria suggested this was an excellent model of the way our professional lives work.

So, even though you may have a thorough-going systems view and plenty of guilt, you usually can't keep all the plates spinning.