Episode 69: What is Quality? with Dr. Bryant Silbaugh

https://link.springer.com/article/10.1007/s40617-021-00627-y
https://link.springer.com/article/10.1007/s40617-022-00750-4

Building Better Businesses in ABA is edited and produced by KJ Herodirt Productions

Intro/outro Music Credit: song "Tailor Made" by Yari and bensound.com

Give us a rating at Apple Music, Spotify or your favorite podcast channel:

Apple: https://podcasts.apple.com/za/podcast/building-better-businesses-in-aba/id1603909082

Spotify: https://open.spotify.com/show/0H5LzHYPKq5Qnmsue9HTwn

Check out Element RCM to learn more about billing & insurance support for Applied Behavior Analysis providers

Web: https://elementrcm.ai/

LinkedIn: https://www.linkedin.com/company/element-rcm

Instagram: https://www.instagram.com/elementrcm/

Follow the Pod:

Web: https://elementrcm.ai/building-better-businesses-in-aba/

LinkedIn: https://www.instagram.com/buildingbetterbusinessesaba/

Instagram: https://www.instagram.com/buildingbetterbusinessesa...

Transcript

Jonathan: 0:30

Dr. Bryant SBA has been in the ABA field since 2008. He specializes in the assessment and treatment of autism and pediatric feeding disorders. He's a former assistant professor of special education and past president of the Pediatric Feeding Disorder special interest group of the Texas Association for Behavior Analysis. He's now with New Beginnings Academy in San Antonio, Texas, and Bryant is also a Muay Thai practitioner. Bryant, welcome to the pod.

Dr. Bryant Silbaugh: 0:57

Hey, Jonathan, thank you for, inviting me. It's, a pleasure to speak with you today.

Jonathan: 1:01

It's a pleasure to have this conversation and I really want to dive deep on quality. This is like such a hot topic in our field now, but it's also like one of those things that just people don't really know what to talk about and there's lots of disagreement and you've literally written, um, and, and done research on this. Um, I mean, some of your background, Brian is, that research that focuses on variables that influence ABA service delivery quality. especially like the interactions between quality culture and clinical outcomes. And in fact, in, in 2021 you wrote a research paper with Dr. Robbie Al fatal on exploring quality and the applied behavior analysis service delivery interest industry. What did you learn about quality in our field?

Dr. Bryant Silbaugh: 1:42

Yeah. Well, I think one of the big things we learned from that paper, um, is that there's a lot of variability in, you know, how uh, ABA service delivery equality is conceptualized, both in the literature. popular, you know, textbooks that, new practitioners, contact in their graduate programs and, um, out in the field there's not, doesn't seem to be a consensus definition on Yeah, what ABA service delivery quality is. Um, how to operationally define it, how to measure it, variables to influence quality, and so forth. and without, you know, being able to define quality. Um, measure it, assess it. We can't apply our tools or tools even from other disciplines to experimentally evaluate or determine if you, what are the variables that are influencing service delivery quality, and how do you focus on those within your organization to, uh, promote quality service?

Jonathan: 2:44

Which, which is so crazy to me because, and you know, I've, I mean, I've only been in the field for, for a little over 10 years now, so I've got nothing on, on you and our field, our science has been around forever. One of the things I know we do so well are things like operational definitions, right? We get really clear and precise when we're talking about goals and behaviors and. And skills. So what has come back to, why is it so hard to conceptualize quality? Is this the idea, sort of the criticism from outside the field that, oh, you're not randomized clinical trials, you're single subject design. Is it something else? Like what's, what's going on? Hmm.

Dr. Bryant Silbaugh: 3:17

Yeah, interesting question. so from, uh, some of the research that Robbie and I did upfront in, uh, preparing for this manuscript, writing this paper, was to look at, definitions of quality outside the field of, of apply behavior analysis, you know, um, to understand how quality is viewed, you know, from other perspectives. So, for example, My wife Dorothy is, in quality. So she work for a medical device company here in San Antonio and they make, like stents and other medical devices for infants, and she's the manager of quality and so she's developed their quality systems. And before that, let's say like, I don't know, back in 2008 to 2013, I wanna say my wife worked at Abbott Vascular, and Abbott Vascular has, decades of time that they've put into quality systems is my understanding. So she had this knowledge of quality, you know, and then just through being with my wife, I've been cued into that, if you will. You know, we're aware. Okay. There's an entire field of quality outside the field of applied behavior analysis. It extends across many industries. There's, uh, international standards for quality. so anyway, so some of that was an influence as well. so, I'd say after the paper though, um, I, I've started to dive more into the, like, scholarly literature in the quality field, uh, written by some of the pioneers. And, one is, Joseph Juran. And so Juran is, I don't know if I'm overstating this, but he might be like the Skinner of the quality field, you know, like the father of managing for quality is the way I would phrase it. he actually wrote the, I believe, which is the authoritative text on the history of quality and traces, managing for quality all the way back to like 2,500 BC China in handy craft industries so he gives examples of like quality control back in, in those times. the way managing quality for quality has been conceptualized by Juran is you do quality planning, quality control and quality improvement, they call the Juran Trilogy or the Quality Trilogy. Okay. And the, the history book I mentioned walks you through different civilizations at different times in history and gives you examples of each of those things happening. Well, I, I think maybe the process or whatever quality is, is something fundamental that cultures do. It's all about planning to do something, doing that something, noticing the variability and trying to continuously improve to reduce the variability of whatever it is you're doing, and then trying to take it to the next level, But ultimately, you know, I think, because it may be something fundamental and it's been around for thousands of years, there's been so many different ways to talk about it, that defining it hasn't been hard for behavior analysts. It's been hard for everybody,

Jonathan: 6:17

this is enlightening. I'll drop a link in the show notes to, uh, to that book. the Joseph Juran father of Managing for Quality, but I love this idea of the trifecta, quality planning, quality control, quality improvement. Bryant, what comes to my mind? You know, if you think back to the sixties, seventies, eighties in like Japan, I think that's where the Six Sigma sort of manufacturing process, which essentially was like managing to minimize defect. So at a 99.99999%, uh, success rate, you are gonna produce this widget in the right way, right? Yes. So, like, that's really powerful. That's powerful in a manufacturing context, does that have a place in aba?

Dr. Bryant Silbaugh: 6:56

Ooh, excellent question. And Juran was one of the, uh, pioneers who went over, uh, to Japan and, and helped them initially developer or sort of occasioned a paradigm shift you will in, in how, Japanese manufacturing, overcame what was perceived as like really shock industrial, you know, products to being the world's leader in quality. And, um, absolutely, man. Yeah, I think it has a place, I, that's an excellent question. It's something that, I have so little time to work on this, so I really work hard to carve out in the middle of the night time to think about this question cuz it's a paper I wanna write and I think, yes, I think, in service delivery. You know, I think you can apply this trilogy. So, I'll relate it back to the, quality paper that Robbie and I, wrote. And so in there we defined, ABA service delivery quality. Okay. And we defined it at the organizational level. and so we defined it as the extent to which an organization's ABA products, services and outcomes meet standards determined by professionals and consumers over time in response to changes in a receiving system while maximizing the financial health of the organization. I talked about it as like a ASDQ or ABA service delivery quality, Framework is just a way of looking at the world to behave, let's say, more effectively with respect to certain problems, So the framework isn't right or wrong per se, it's just a way of thinking. And so within this framework, we're thinking of quality at the organizational level is are you meeting your professional, include consumers and theirs and benchmarks that your organization sets. Do you have strong financial health or organizational health that's not profit. It's much broader than that, and, are you able to maintain those standards over time in response to threats to the organization? Okay. Things that happen that compromise revenue streams and so forth, introduce variability in, talent acquisition or retention or revenue streams, et cetera. Things that the organization wants and needs to keep growing and thriving. Um, then we looked at it at the cultural. So we just say, Hey, just like behavior is selected by consequences, organizations are selected by cultural consequences. So let's bring it back to that trilogy thing. So imagine you have an, organization that's trying to meet these quality standards and by, by setting standards, right? You. Operationally, defining them, measuring them. You can use change initiatives within the organization to start objectively verifiably, improving quality over time. And gosh, maybe the way to do that is maybe applying this trilogy idea where it's like planning for quality might mean that you are designing processes, clinical or operational processes in advance. You're setting a standard, um, in a benchmark. And then once you've planned for a high quality process, operational or clinical process. Then you collect data on key performance indicators, some measurement of what is the result of this process, the process of supervision and intake process, uh, an assessment process, a scheduling process, I dunno, parent training process. We could go on and on, come up with all the processes you want, but if you can measure the outcome of those process, And now you have a quantity, you could compare to a benchmark, like a standard your organization set. Now you can start to control for quality by measuring that, looking at the variables that are impacting that operational or clinical process, trying to shrink the variability, get that KPI to exceed your benchmark. And once it exceeds your benchmark and you can maintain that, then it might be time to say, we have met our standard. Maybe someone around here has a really good idea for tweaking this in a way that's gonna take it next level, and we're gonna do this in a way that's better than ever before. That's the improvement part. I like this idea because you distinguish between, the maximal performance of a process is not quality control. That's quality improvement. Okay. Quality control is just, demonstrating that you implemented the quality plan successfully

Jonathan: 11:22

Bryant. Fuck dude. you're so singing from the same sheet of music right now as you're describing this quality framework and thinking about telescoping out and thinking about it really holistically. But our field, I'm sure you get pushback because when most people hear quality, what do they think? Clinical quality. Yeah. Yeah. So they'd say, why the heck are you including financial outcomes? Why are you thinking about it evolving over time? Why are you thinking about operations processes? That's stuff that happens in the backend. It's not important. What's most important is clinical quality, like how do you respond to that?

Dr. Bryant Silbaugh: 11:53

Yeah, there's, this concept of the total performance system that Brethower came up with several decades ago where it conceptualized the organization, and it's sort of the same way we think of the operant. So in an operant, you have an anticedent organism responds to the antecedent and produces a consequence that alters the future probability that response will reoccur or not reoccur now take that three term contingency, if you will, and take the organism out, put the organization in, and the organization receives input. So that's clients and that staff coming in and maybe other things I don't know. And then, There's products that come out of the organization, so it's like service delivery, um, treatment plans and progress reports. There's a few concrete examples. Now, the way you get from, the first touchpoint with the brand parent calls and says, my child has autism. I see on your website you provide ABA service. I have questions. Okay. All the way to six months after therapy, you're submitting a progress report, requesting a continuation or discontinuation, discharge, whatever services. Uh, everything that happens between there is a processing system where you have a, the organization's, a system, there's all these processes and they work together. Clinicians cannot deliver services independently of the schedule of scheduling process and, uh, treatment plan review process maybe at towards the end of the service period and I don't know, whatever else. Okay. It depends on your organization. There's gonna be variability in what the actual processes are across organizations and probably within an organization. You can break it up in ways that, in different ways. Uh, the point being is that these things work together so it doesn't make sense to me to think about quality, at the top organizational level as clinical, but it, it includes clinical because why do we exist? as far as I know, aba, autism service delivery organizations exist to improve lasting meaningful improvements in quality of life Achieved through behavior change and as indicated by outcome measures at the end. you have your, what I call proximal outcomes, which is child meeting targets or goals throughout the service period. And then you have your distal outcome, which is like your Vineland, you readminister every six months, right? And so, yeah, you can say, okay, so maybe those things are great, maybe your learners are learning. fantastic. The rate of skill acquisition's really high. The rate challenging, behavior reductions really good. And then at the end you have really big disparities between pre and post tests on outcome instruments. That sounds pretty good. All right. That sounds like really good clinical quality, but you can have all that and parents can hate their service or parts of the service, in which case, well you're not meeting consumer standards. You might be meeting some clinical standard within us as a subset, you could have consumer clinical standards, professional clinical standards. So from the BCBA's perspective, they may feel our clinical work is outstanding, but from the consumer perspective, they may say that clinical work, uh, some of it we really don't like, it doesn't work for us. Am I making sense? So it's like you gotta look at it comprehending. So that's why I say you gotta throw in the financial too because if the organization doesn't make enough money to maintain and invest in operational and clinical quality, it just won't exist. It'll, it'll die. or you'll be selling something else. You know, like, I don't know. Other than high quality services and organizations have to make money.

Jonathan: 15:37

Well, exactly. I mean, that's why this feels like such a, uh, no-brainer view of quality. Right that, um, I mean that there's the simple phrase of no money, no mission, but if you have to discharge clients because the organization was not able to evolve in response to certain external stimulus, right? Whether a payer cutting slashing rates or, um, or whatever else might happen, then you haven't done the appropriate job quality wise in service to your mission. So Where is there an interaction between, call it best practices and quality and, and part of where I'm coming from on. I don't think there's like silver bullet answers to this, but um, you know, there are accreditations that exist, for example, CASP and the ACQ or BHCOE accreditation, which, we are super passionate about, at Ascend and have been through voluntarily many times. And that's sort of like that checklist of, you know, best practices that you need to do. But how does accreditation slash best practices, interact with quality?

Dr. Bryant Silbaugh: 16:39

Man? Good, good question. Uh, so I'll admit that I am not an expert on these topics, but I am a student. I am learning. So I can share what my thoughts are on the topic. But, you ask about, accreditation and best practices. And what was the third one? It was interaction between best practices, accreditation, and, and. In quality. Okay. Of course. Um, so that's a good question. Let's maybe start with accreditation. in the paper, the 2021 paper, Robbie and I talked about, a little bit about the role of accreditation in. Ensuring quality and sort of the way we looked at it was like, in an organization, being accredited seems to, involves to some extent a focus on compliance and because I'm not an expert, I've never been through an audit for accreditation. Okay. So I lack that real world experience that, Robbie actually had brought, he brought to the, our paper, is like, it seems The organizations may be held to in an accreditation process are really about building the business and necessary things like do you have. treatment plans. You know, do you have a employee handbook? Do you have a parent handbook? Do you have some basic policies and procedures and so forth? And so there's a compliance aspect of it, and we've separated that, from quality where quality isn't really about complying with some policy or procedure Outside of the ABA field I think the focus is on meeting or exceeding customer's expectations. And so, I mean, you could probably be non-compliant with some things in an accreditation process, but really exceeding customer's expectations. I can imagine this happening, so you know, or you could be measuring things that you think are representative of clinical quality, but uh, you're not in compliance with other things in operations. I don't know. I'm just kind of making that up to make the point that there seems to be a distinction between compliance and uh, and quality in that way. In terms of best practices, the way I think about this, I'll say like clinical best practices. I think what the term best practices implied is, these are the things you should do to offer, the biggest clinical benefit. Right. so earlier I talked to how, you know, you take a client from intake to reassessment six months later in their first service period, and you have proximal measures of, uh, clinical benefit and improvement by the behavior change, right? So they're meeting goals, mastering targets, and generalizing them and so forth. and then if that happens, and that's really good and those are appropriate goals and targets corresponding to real delays and deficits, uh, measured by the outcome assessment like a Vineland or VB MAPP or something else, then there should be a causal relation right between behavior change and clinical improvement as indicated by those assessments. Now, then the question has to become, wait, how do you achieve those proximal gains over on a day-to-day basis, moment to moment, day-to-day? How are you in that Best practices? Okay. Supposed best clinical practices. I think it implies there's a cause of relationship between what the practitioner is doing moment to moment during discreet trial training or natural environment teaching and their progress in the short term and progress in the long term. I think there's an opportunity to advance some more empirical, understanding of ABA service delivery quality by um, for example, an, you know, an organization adopting certain best practices in a, let's say an instructional format, like discrete trial training. Okay. So, at New Beginnings Academy where I'm a clinical supervisor right now, we're developing a set of DTT fundamentals, we call'em, it's not even best practices. Okay. It's just fundamentals. If you're a new RBT, what do you need to be doing, to implement, high, discreet trial training. Okay. and then there's another layer, like if you could, if you master that then let's talk about best practices in discreet trial, which is a little bit more sophisticated and harder for an RBT to do, but I imagine you could promote quality by, let's say you have your fundamentals checklist, and you have your supervisors periodically on a weekly basis observing RBTs, running the checklist, giving RBT feedback and say, on this checklist, you scored 85% okay, on these DTT fundamentals, and here's where you can approve. Here's a suggestion. All right, let's try again. Run the checklist, see if there's improvement and so forth. I think that's quality control. I think quality planning is where you came up with the DTT fundamentals. You made a data sheet, you developed a process, you implement the process for assessing the implementation of DTT fundamentals and quality control is observing the variability in the scores on those checklists week to week. And then set a standard and you say, okay, so our standard is that we're going to reliably and consistently implement, discrete trial training with the fundamentals listed as A through whatever. Okay. And then a benchmark can be on average, on a weekly basis, R B T pods, if you will, or teams are going to score in 90% or better on this checklist. Okay, so there's your benchmark, I guess. And then week to week you can collect the data, average it and see if they meet it. And now you have, let's say you have your ASDQ framework. One of your professional clinical standards is this DTT Fundamentals. You have a quality dependent kpi, which is like in our paper we say it's a key performance indicator of an aspect of quality. So you're measuring that. Then over time you can intervene to try to, implement the quality plan, which is implemented 90%, from week to week. I think that's a way of thinking. Hey, okay, so there is a causal relationship between this best practice. I hear what I'm calling it, fundamentals, but whatever best practice, DT itself is the best practice. So this best practice in ABA with outcomes and our standards. I don't know man, that's what I'm thinking.

Jonathan: 23:01

I love this. Okay, so now I'm gonna start, I'm gonna start geeking out cuz it keeps coming back to this trinity, this, this triumvirate, right? A quality planning, quality control, quality improvement. Um, and so, and I'm gonna share, I think this is the first time I've ever publicly talked about this, but at Ascend we have, sort of evolving ways that we are thinking about quality. And I admit this is probably a little bit more quote, unquote, clinical quality. But you know, we've envisioned this idea of this Ascend quality index, which we have not published any research on, and this is not complete, so full disclosure, there are like six legs of the stool here. Are you? Okay, let's go through like a rapid fire. I'll give you each leg and give me thoughts or feedback. and at the end we'll talk about how well does this conform to this triumverent um, or what else? Why not be added? Sound good? Okay. So, so let's start. So the first one is, I think, pretty easy to conceptualize clinical process metrics, right? or just more broadly process metrics. Things like, you know, time it takes to get a kiddo into intake. Um, uh, treatment integrity, check scores, percentage of supervision. Alright. Okay. Yes, with me so far? Yes. Yes. Next leg of that stool is client rate of learning. And here's what's, and, and full disclosure. I think most people know I am not a behavior analyst. I am not nearly smart enough to be but the second is around client rate of learning. And that is, you know, based on a diagnosis and kids' cognitive abilities, et cetera, we have all different rates of learners. I mean, as human beings, people learn at different rates. And I know when we initially get a kid on a BCBA is gonna be like, oh my gosh, I can't keep up with like, in right enough programs cuz the kiddos mastering at'em so quickly. Or maybe there's a slower learner. So with the idea being that over several months, You can plot a line of sight, several data points that says these are the number of goals that they're mastering or that they have been mastering, which means we should be able to draw a line out of what that rate of learning is over time and whether they're staying within some tolerance of those. Thoughts on that client rate of learning metric?

Dr. Bryant Silbaugh: 25:04

Yeah, I think it's interesting. So it sounds like, you could have a standard, and a benchmark where you wanna see, maybe you have come up with some criteria where maybe you even have categories of learners where, let's say, based on levels, so we have level one autism, level two autism, level three autism, and then establish the standard is that we are going to evaluate clinical, you know, progress in terms of this rate of learning, if you will, or rate of goal mastery but we'll set a benchmark, maybe I'm just totally making this up on the spot. Maybe set up a benchmark proportional to those different levels because those, those learners might learn at different rates. That's totally speculation by me. Uh, maybe the presence of intellectual disability might even force you into like a fourth uh, benchmark, right? Yeah, totally. in an ASDQ framework, you can say, Hey, we have a professional clinical standard of that which we just described, and then we have benchmarks and whatever that metric is, in whatever process you're using to measure that rate of learning. Maybe it's, I don't know. when the final draft of the progress report comes in after six months, maybe there's a process where you, you know, somehow you calculate the rate of goal mastery you over that service period, and that's where you collect your data. Okay? Exactly. But that's but that's too long. By the way, it's six months Uhuh. if this is a key performance indicator, you gotta be able to react to it. Like on a weekly, biweekly, or monthly basis, because after six months is too late, six months happened, you know?

Jonathan: 26:40

Exactly, exactly. Well, you know, one of the things that on one hand, I think there's a valid criticism. On the other hand, I don't know how to think about this, but people say, oh, well, a BCBA can gain the system, right? Oh, they could just write a whole bunch of easy to master goals. And I get that on the one hand, on the other dude. Aren't we fucked? If, if that is legitimately a concern in our field?

Dr. Bryant Silbaugh: 27:01

Yes. Yes. Because that's true. Okay, here, here's a silly little concrete example. Okay? It's like, um, I can design a goal, I can write a goal in a treatment plan that says the learner will, this is totally made up, comply with a simple one step instruction to sit down at a table, across, varied sds, at least two therapists with 90% accuracy, minimum of 10 trials per session for two consecutive sessions, standing two feet from the chair. Okay. And so some learners are gonna be sensitive to the distance they have to travel on this goal. I believe that most learners are going to master that goal faster than if I said the learner can be anywhere in the room. Because then there can be distracting objects in the room. The travel distance can be too far. I haven't specified if the SDs gonna be delivered when they're engaged or not engaged in a preferred or not preferred activity. You know, and, and so there's all those different variables that influence how fast we're gonna meet that goal. Aside from all the operational other factors that can influence how much time you can actually spend working on that goal with that client on a regular basis. So yeah, I think it's totally possible to write goals in a way where you can inflate. The rate of, I don't know if it's actually an inflated the rate of learning because they are learning and they are mastering that goals and that's those goals. That's real. That's true. But some practitioners will be better at writing goals that can be met quickly. Another practitioner might write this treatment plan for the same client, write a similar goal, but yeah, not specify that two feet, in which case it could take them three months to meet the goal. I hope that makes sense. it all depends how you define it. so this may be, a threat to the validity of this measure of rate of goal mastery. But check it out. Nobody's really evaluates. I think like if you go to an I E P team in special education, no one is gonna walk in the room with one set of data and one assessment and say, we're gonna base all our decisions based on this metric or this assessment result. Right? So you take multiple sources of data, put them together, get a big picture, and make decisions. And I think that's the same case here, where that can be one metric. Um, And then you also consider that the distal outcome, the six month, every six months, the repeated, indirect assessments or direct, you know, direct assessments, just, like the Vineland VB MAPP and, and so forth. Um, does that make sense? It's all about the comprehensive view.

Jonathan: 29:46

I like how you helped me understand, this could be not like a, a fraudulent quote unquote, gaming of the goal. It's just different goals are written in different ways, so, okay. and your point is absolutely right. You have to bring a body of data, right? to show, progress for this kiddo against the quality index. So maybe that comes to leg of the stool number three. This quality index, I don't know the perfect term for this, but sort of client quality of life or developmental quotient, which I know that doesn't exactly exist, but as an example, the Vineland, which is not meant for kiddos with autism, so full disclosure. but it can help you indicate is this child making, uh, developmental progress faster than they are chronologically aging? Does that make sense? And, and thoughts on that?

Dr. Bryant Silbaugh: 30:30

Yeah, it does make sense. And so, um, this idea that you just brought up is something that, I'm trying to understand myself actually right now, literally with the Vineland, uh, I'm trying to, understand how to make sense,I mean, I'm just, you know, expressing how naive I am about some of this stuff. there's gaps to my knowledge Of course. And it's like, with the Vineland I'm trying to figure out how do you compare readministrations of that assessment over time, and look at what kind of different scores should you be looking at, you know? I'm doing that, literally doing that right now. Plotting it, and graphing it and trying to figure out like, can I make sense of this? where am I going wrong with interpreting these data? and when I see something that doesn't make sense, what does it mean? So for example, it's like the first administration of the Vineland, let's say at the beginning of a service year for one client is like maybe they were at, um, I don't know, 20 months on receptive communication. I don't know. I'm making this up. And then a year later we reassessed and they're like four years on receptive communication at that level. And that makes no sense. That's not real. I don't believe that nobody made four years gain in a year of therapy. No way. I mean, I just don't believe it. And so then the question is like, okay. There's gonna be variability, you know, in the assessment. And I guess you need a lot of kids to sort of a, I don't know if I, even if I'm going anywhere with this, but I think, uh, I, I would just say it's Comp Man and I don't have answers, but I think it makes sense to try to figure that out as a metric, you know, as the difference score. Cause we want bigger differences, but we want them to be real, right? We have other clients where it's like a two month difference, you know, in the pre and post for the Vineland after six months, and it's like, Oh, is that real? I don't know. It's hard to tell.

Jonathan: 32:24

It is. I mean, none of these are perfect measures, right? But one of the things, and, and I'm gonna put a pin in this in the moment, but you probably see me driving toward here, is how do we compare across kiddos, right? Because we have to have some type of, I mean, the way payers think about it, the way we think about overall organizational quality is like more of a population health level, right? At the population of the, the clients we're serving. How are we doing across them as opposed to just, yep. This kiddo made some good progress on the VB MAPP and this reassessment. This one made a little bit less this. There has to be some way to this, but let's put a pin there. Let me come to the next one. Ready, and this is one, I think that it's underrated, but I am super freaking passionate about. And that's family quality of life. So as an example, parenting Stress Index, we use the PSI four and administer that at certain points, and that's not perfect, right? Because there could be anything that's happening when you're administering it and the, you know, the, some of the parents going through that might make them more or less stress. But it feels like if we are not thinking about the family holistically and thinking about are we improving the parent's quality of life and as a proxy for that, their stress level, then what are we doing in treatment Thoughts?

Dr. Bryant Silbaugh: 33:30

Yeah. Yeah, that's super important and I think that's wonderful. And, I, I think, Uh, ICHOM I, I forget exactly what the acronym means, but ichom, is an organization that's, promoting standardization and like health outcomes measurement. And, Ivy Chong at the May Institute, led the development, I believe, a standard set, which I think included at least, at least one quality of life metric. I think that's super important and should inform, treatment as well. From quality perspective, let's say we, you know, try to fit it into the ASDQ framework, if you're defining quality the way, as we did in that paper, we, we can look at that as a aspect of consumer standards. So we can say within consumer standards, there are, uh, clinical standards. One of our clinical standards is that we, on a recurring basis indirectly measure quality of life using X, Y, or Z instruments. This pretty straightforward, I think for the quality dependent KPI there, where you're just, you know, collecting, the data from those assessments and then seeing a benchmark, and say, okay, so this is the outcome we really wanna see, uh, quantitatively on, on that quality of life. I don't have like recent familiarity with the items or questions on a quality of life, instrument. But I imagine it can also inform treatment. I like to do that where I look at all my assessments and I look at the individual items and see what parents scored, and see what other people scored, what did I score on a direct assessment and look at those items and really, really, really, really try to develop goals that hit that item, you know? Uh, so I think that's wonderful that you know that y'all are doing that where, you can not only assess it, but yeah, use it to, in, in inform treatment, which may be hard because that's super individualized. I have found that, many young BCBAs do sort of struggle with putting all your curriculum away, all that stuff away, and just look at the patient, look at the family, talk to the family, do the assessment, look at the item, and then brainstorm 10 goals that address that item that you've never read anywhere in your life. It's particular to person in their family, you know? So I think it's useful in that way.

Jonathan: 36:00

I like that and thanks for giving a shout out to the ICHOM framework and, and what Dr. Ivy Chung and, and many others have done it, that that's phenomenal. I'm gonna drop a link in the show notes to that. I know BHCOE has an, and outcomes and, framework and how to choose the right assessment tools. All right, well, let me laser through the last two of these. So the penultimate number five is, um, autism symptomatology. So that is, are the core symptoms of autism being addressed and, there's certain payers now, for example, Tricare that use, I guess the PDBDI. There's also maybe a separate one, the SRS, um, but that feels important if the analog is to other disciplines of medicine. How well are we treating the symptomatology of what we're experiencing? Thoughts on that? Yeah. And that's, that's like less ongoing maybe than a beginning and end of treatment? I'm not sure. But

Dr. Bryant Silbaugh: 36:51

yeah, I think that's really a good idea too. You know, I think in an ASDQ framework that would, you know, like fit in the professional slash standards where The standard is that we're gonna measure. You know, autism symptomology, the severity of autism, and use that to inform, not just as a clinical outcome measure, but as a tool for individualizing treatment plans and writing goals. I especially like it because, As you're listing all of these different components of what sounds like a composite, score or picture of clinical quality, you're hitting it at so many different critical, angles. I think it's really cool. It's really cool.

Jonathan: 37:32

We'll see. Well, but the last one, and this is evolving, but I recently spoke with a payer who made this phenomenally important point, and that is employee turnover is a proxy for continuity of care. So Bryant, think about the mind-blowing power of this that is, and especially that payers are thinking about when your employees turnover, that directly impacts clinical quality. I think a lot of ABA providers would think about that, oh, that's a turnover issue, right? And sure it is bad and maybe the kid. More progress, but there's like a one-to-one tie in gaps in service delivery. And so some type of measure, whether it's a, employee satisfaction scores or, employee net promoter score, I mean, there's lots of frameworks out there for it. Feel important for reducing that proxy, turnover, improving retention, and making sure that continuity of care is less of an issue.

Dr. Bryant Silbaugh: 38:25

Yeah, I that's an excellent point. Oh, for sure. Yeah. Yeah, because it can take, just someone who's never been in the ABA field and they, find an ABA agency and, they're learning, how to be a technician. That I always tell everybody the first three months of me being, a technician where Embarrassing because I was just terrible. Absolutely terrible. and luckily I was employed with an organization that was very committed to training me, and even though it was hard and to make sure I was good at my job, but for those three months, I probably was disrupting clinical quality on my cases. because I'm not delivering like someone else who's been in our fully trained and implementing those DTT fundamentals and dtt best practices, for example, with consistently high levels. And so yeah, you can expect that if you have a high turnover and you keep hiring new rbt, and maybe it's not such an issue if you're hiring seasoned rbt, experienced rbt, maybe turnover isn't such an issue, but if the turnover is, uh, resulting in you hiring a lot of new people and training them for the first time, then yeah, I could definitely see that as like adversely impacting clinical quality. But if you sort of do some of the, quality planning in advance, develop those fundamentals, best practices, not just screen travel, everything that the RBT needs to do clinically, and you have a, strong, robust training system in place and even like a metric, like a QD KPI for, how much time between when the R B T starts, let's say shadowing to when they demonstrate 90% on that checklist for two consecutive, supervision sessions. Maybe that's a metric you could used to control for the adverse impact of turnover on quality. Or maybe ooh, here's an idea. What if you grouped RBT s in terms of length of service to the organization, and then you looked at their best practice, data, and compared them to new hires. And then you saw that the new hires for a certain amount of time are scoring within, this range of the clinical quality checklist. But they need to be up here. Maybe that can inform a screening process where when you try to hire a new technician, you bring them in, run the checklist, see how high they are on the list and then all of this maybe is together, can control for adverse impacts on quality as turnover is happening. I don't know. I don't know why I went there, but that's where my mind went.

Jonathan: 41:08

Well, here's my strong conviction. I love where your mind just went on that. and I think it's these kinds of, look, there's, there's no perfect quality measure out there. But if we don't start like drawing a line in the sand and saying, we're gonna try things that we think generally are gonna lead to quality, then we're not operating part of this trinity. My conviction is We've gotta stop as a field. Just say, okay, well let's wait for someone else or some other group to do something or to follow along. Like that's not good enough as an individual ABA provider we need to say this is our view to quality. By the way, we should be able to say with data in one sentence why we are a high quality provider, but, we can't afford to let the rest of the field, or like committees or others figure this out. every provider has to take a stand for quality and make it happen. And a seminal moment in my career, this is like three or four or five years ago, I was at a conference and it was a panel of payers, Bryant, and they were describing, generally these types of topics. What does quality look like? And there's a question from the audience, from a provider that was like, well, payers, what quality measures should we use? and the payers were like, um, you, you don't really want us telling you what quality measures to use. And then another provider comes back and says, but you're paying for it. So like, you tell us what to use and we'll do it. So we, and they're like, no, we, and it was just this back and forth and finger pointing about who should do it. And I was like, stop. This is a circular firing squad again. I'm just like, I'm passionate that every organization is not gonna be perfect. But you have to make a stand for quality and say what you think quality means. And if that means following a certain, already defined. set of outcome indicators or framework, great. And if not, you start to develop what you think is going to lead in your experience and in what you know. In having already served many clients will lead to higher quality. That's our obligation.

Dr. Bryant Silbaugh: 42:51

I, I agree. And I, I think some steps that, organizations can take. I'd love to see this is more organizations, publishing, the work that they're doing internally to promote quality, you know, because, what Robbie and I found is in the literature's, nothing, there's nothing ABA Serivce delivery quality in terms of the empirical literature. And so one of the concerns I have even as carrying on this research line, trying to keep it alive is and I've shared this with other, you know, colleagues. I worry that like, wait a minute, am I just starting to figure out and understand some things that, other organizations already know and are already doing and this is old news, or they have more advanced understanding or systems in place to do quality delivery. But it's not out there in the literature. And so maybe it's not good for the field overall. I guess what I'm trying to say is this sort of happens in other industries like. I don't know, maybe neuroscientists will be studying something in behavioral pharmacology at some pharmaceutical company is like 15 years ahead of them. But the pharmaceutical company doesn't publish their work because they're making money on selling, uh, new pharmaceutical. So the same thing maybe is having aba, I don't know. Uh, and so I'd love to see folks publishing more of this and, um, Their work on quality. and then maybe even inviting, you know, researchers at universities to partner because for people in academia can do conceptual work on quality, sort of like what I'm doing. the work I've done is not empirical at all, it's conceptual, you know, uh, they can do that I think. But to do the empirical work and some of the real hard work of testing methods and procedures for promoting quality, they kind of gotta get in to the organization. You know, at, at every level. and partner, I don't know if that's true, but that's seems to be one way where we can advance this, is organizations are more willing to publish research on their quality, their approach to quality and partnering, with universities to advance the science, of behavioral quality, if you will. Like how do you apply principles of behavior to, evaluating variables that impact ABA service delivery quality, at least in the Autism industry.

Jonathan: 45:14

Oh, I love it, Bryant. Okay, so I'm going to co-op. My next question is gonna be, what's every one thing every ABA organization should start, stop doing? Is it okay if I co-op start publishing, whether it's just internal, right, or that's okay or external. And seek out partnerships with universities to publish on quality to add to the body of literature. Does that sound like a worthwhile start doing?

Dr. Bryant Silbaugh: 45:35

Absolutely.

Jonathan: 45:37

I love it. Okay. What should ABA provider stop doing?

Dr. Bryant Silbaugh: 45:41

Oh gosh. Um, closely related to this is if organizations are able to invest a lot of time and resources into developing their quality systems, to start, publicizing their quality metrics, their data on quality, and compete with other organizations on quality, not, patients and talent acquisition? like financially, stop competing financially with other organizations trying to pay clinical supervisors more than your competition or trying to, I don't know, offer things that are more attractive to families and instead compete on quality. Where I imagine a world that's like this, it's like if you're a bcba, Interviewing for a clinical supervisor job, right. A lot of the questions are focused on. Uh, your knowledge, skills, and abilities, right as you're trying to get that job. And maybe you'll ask a few questions on organizational culture or the job requirements or something like that. But here's the future. I'd love to see this. Okay. Is that and an interview, BCBAs go in and they say, um, can you tell me about the systems and processes you have in place that are gonna enable me to deliver really high quality services? Cuz that's what I want to do here. And so then it's a, you flip it and it's like the organization needs to compete and say, I'm not gonna pay you a hundred thousand dollars, but I'm gonna right now tell you three systems that we use and here's three metrics and our BCBAs are crushing it. And look how awesome their clients are doing. You can do that too. Come join us.

Jonathan: 47:19

Oh, that is so beautiful. That is, let your quality data speak for itself and compete on quality. Not on paying the extra buck or on doing something different and fancy to acquire the next client. That is powerful dude. Revolutionary, I might say, Bryant. Bryant, where can people find you online?

Dr. Bryant Silbaugh: 47:41

Um, easiest to find me on LinkedIn or Research Gate

Jonathan: 47:46

love it. All right, you ready for a hot take questions, dude?

Dr. Bryant Silbaugh: 47:49

Yes.

Jonathan: 47:51

Bryant, you're on your deathbed. What's the one thing you wanna be remembered for?

Dr. Bryant Silbaugh: 47:55

Oof. For, being loyal and committed to my family.

Jonathan: 48:02

Love it. What's your most important self-care practice?

Dr. Bryant Silbaugh: 48:05

Ooh, Muay Thai Muay Thais like kickboxing. And, it's pretty intense. You know, there's a lot of cardio involved. You get your butt kicked and that may not sound relaxing, but for me to go in and do some super high intensity Muay Thai training, down the street, and then walk away with bruises and, uh, I don't know, a bruised rib and I'm limping. I'll tell you that. The problem is my technique. So I need to improve my technique and I won't walk away harmed. But if I'm like sweating and you know, like bruised, I had a good time at Muay Thai and I feel relaxed and I'm ready for the next day. That may sound weird, but I feel it feels good.

Jonathan: 48:47

Not weird at all. It puts you in your flow state, dude. I get it. I get it. What's your favorite song?

Dr. Bryant Silbaugh: 48:54

Ooh. Um, ooh. Uh, oh gosh, I'm gonna say it's changes all the time. It depends on what my kids are having fun singing at the moment. So I'm gonna say it's like, uh, I like to move it, move it. It's a song from, I think Madagascar or something like that. Uh, it's what my kids sing in the morning, so that's my current favorite song.

Jonathan: 49:16

I love it. I've got a good buddy from college and, uh, a number of years ago we were hanging out and his little boy was singing that song, but he couldn't say that the v is a hard word, right. So he would just run around the house going I like to Mus It Mus It. He had the dance oh. It was just, it was unbelievable. So shout out Dan Seif wherever you are. That's awesome. He lives in Texas. Not far from you. Actually. Uh, Bryant, what's one thing you tell your 18 year old.

Dr. Bryant Silbaugh: 49:43

My 18 year old self, I would say, um, uh, try not to bite off more than you can chew. When you have goals in mind, projects you wanna pursue, make sure you know you take the time to break them down. Into their own parts and take one step at a time. I've, I've been frustrated in my life in a few situations where I tried to take on too much all at once and didn't get the result I wanted, like in my career, you know? So that's what I would say. 18 year old Bryant, take it easy, slow it down, step by step, man.

Jonathan: 50:18

If you could only wear one style of footwear, what would it be?

Dr. Bryant Silbaugh: 50:22

Um, it would be, Oh, in ideal world, flip flops all day, I'd be at the beach in flip flops. I can't wear them on a regular basis in my life now, but if I could only wear one, you know, pair of shoes, flip flops.

Jonathan: 50:39

Well, Brian, thank you so much. This has been such a far-ranging and fun and mind expanding conversation. High five, dude.

Dr. Bryant Silbaugh: 50:47

High five.