0:00:05.8 VB: Welcome to the Inclusive Education Project. I'm Vicki Brett.

0:00:12.0 AS: I'm Amanda Slohe. We're two civil rights lawyers on a mission to change the conversation about education, civil rights, and modern activism.

0:00:19.8 VB: Each week we're gonna explore new topics, which are going to educate and empower others. And give them a platform to enact change in education and level the playing field.

0:00:33.8 VB: Welcome back listeners. Some of you may.

0:00:36.6 AS: Hi, friends.

0:00:37.5 VB: Had your last week, last week of school, end of May. But we know that there are plenty of people that go well into the third week of June.

0:00:47.9 AS: Oh yeah, that's us.

0:00:50.0 VB: So, yeah.

0:00:51.6 AS: I go back and forth on like, is it a good thing or a bad thing? It's to a certain extent, like consistent schooling is good. Summer tends to be crazy, but sometimes in a good way, sometimes wild. So I don't know. I don't know if we're lucky or not.

0:01:08.1 VB: Yeah, I mean obviously like if they started the third week of September, a lot of the school districts, like we were talking about the other day, Amanda know we're trying to align so that the AP kids don't have two months of school where they're not doing anything. And we know that a lot of districts, especially here in California, are moving towards that. And so then it just means that school starts August 12th or something, right? So [laughter] with that, we kind of wanted to shift the focus back for a part two with Dr. Tan regarding the integrity of the assessments that we see when there is an English learner that is being assessed. So, Dr. Tan we just kind of wanna jump right into it with you. Welcome back. Thank you for coming back on.

0:01:50.4 AT: Thanks for having me.

0:01:51.0 VB: So where we kind of left off, we got into a couple of different topics in the realm of when we're having a child that English is their second language. And then before we started recording, we got into the discussion about the integrity of the numbers, right? So once an evaluation is completed, what is the process for you in your private practice? Just so that parents have something to compare it to when the school district completes an assessment? So what happens when somebody comes to you and gets an assessment?

0:02:26.7 AT: Yeah, sure. So my evaluations, I assess comprehensively across domains, across brain functions, and a lot of the decisions that we are making either diagnostically or in terms of our conceptualization is based on the scores that we get from our assessments. And so I actually always schedule a feedback session either sometimes same day or just because sometimes I think parents are very anxiously awaiting the results. I like to try to give them an idea of kind of what we looked at in the evaluation, what the main findings were, and to kind of prepare them for what they're going to see once they get the report.

0:03:09.4 AT: And so I always follow with a comprehensive report as well. And so just to talk to them about kind the overall patterns, maybe explaining some of the different scores that were notable, kind of explaining why I am going to give a certain diagnosis. And I find it's just very helpful to have that in-person conversation to help equip them with the knowledge and information and to give them the chance to ask all the questions they have about the assessment process. And so they can better understand once families are using those reports to help advocate in the school districts, et cetera. They're equipped with the information of what everything actually means.

0:03:46.6 VB: That's really helpful for parents because when we show up to an IEP meeting, so the district has completed the assessment. Amanda and I have talked about this ad nauseum, but we always try to get a copy of the draft assessment, at least a week before the IEP meeting, so that parents that were able to kind of read it beforehand and digest and or come prepared with questions. We don't need the school psychologist to read us the report. We need to be given a little bit more context. And it sounds like in the meetings that you have, that's your opportunity to kind of explain a little bit of your reasoning. Even though it's technically there in the report, it's presented in a different way when it's a conversation. And I think that's what a lot of IEP teams miss, is the conversational aspect of it.

0:04:37.5 VB: If you provide a copy of the draft of the IEP with the expectation that the parent is able to read it and digest it beforehand and or come prepared with questions, then you're able to kind of let loose and really have a conversation of like, now this is what it means. I think that a lot of times districts wanna have a part one where it's like, we read the assessment and a part two where we discuss the IEP placement and services, and I find that parents are way too anxious and they either kind of understand, for the most part their child qualifies, but they wanna know like the meat and potatoes, like, how are you gonna support my child? And I think that's probably what you see is missing from a lot of teams, right? Is like how their, that explanation component to parents, especially in the English language realm, right? Or English as a second language realm where potentially a translator is used. Is there an impact for some of the testing that use or see where the evaluator is not speaking the primary language of the child?

0:05:43.3 AT: Yeah, of course. And I think this is where we start getting into, you mentioned this phrase earlier, the integrity of our tests and our test scores. We really have to, this last episode part one, we talked about how do we provide these culturally and linguistically responsible evaluations, right? Part of that is the responsible administration of our tests and our awareness of where our tests come from, and importantly how they were standardized and on what samples and normative data is based off of. And I find that in some of our more diverse groups if they're speaking a different language, these considerations start to fall off the window and we get these scores and we just interpret them the same way we would interpret any score and then just maybe put an asterisk and say, oh this was given in an unstandardized way, but are we actually considering how that affects our interpretation?

0:06:37.8 AT: Right? And so, even at baseline most of our measures are developed primarily in the United States and are kind of very western based assessments. And so just at baseline, let's say we are just using an interpreter to translate kind the instructions and maybe to administer items. Already we are deviating from the standardization, right? You're talking about a third party in the room you're talking about a method of administration that might not be as engaging of the child's attention. You're talking about these phrases and words that get lost in translation that might not mean the same thing. And so then you give this measure and then you kind of have these scores and can we truly say that the numbers represent what they're supposed to mean? Always. That's not always a given. And there's... We can sometimes give our measures and use an interpreter. We can sometimes say, oh look, this measure has been formally translated, right? There's many measures that do have translations, especially into Spanish, but what about the normative samples? They may be translated, but they numbers might not be actually coming from sample of people that represents the demographic of the...

0:07:55.9 AS: Yeah. Yeah, yeah. Yeah. And then when you look at the analysis, like you were saying, we always have like, on every assessment, there's always like a little asterisk or a little section that assessors tend to skim by that says oh, these results are not affected by cultural language deficits, dah, dah, dah, dah, dah. And I see those in a lot of these evaluations that are with an English learner. And I'm like, wait a minute. Like, we're just glossing over this. And it was interesting, I just did a training with a panel earlier this week, and we were talking about assessing in and identifying the need for assessment in early education. So like the preschool settings. And we had a speaker that was talking specifically about the cultural implications. And she brought up a good point that I feel like assessment teams at school districts with young kiddos, like they don't think about.

0:08:46.4 AS: And what she said was, in her culture, in the Vietnamese culture, a lot of the emphasis is on the grandparents taking care of the kids. So she said while I work, my parents are taking care of my son, picking him up from school, doing the feeding. And in that culture, it's a lot of spoon feeding rather than having them use the utensils. And so when a child like that is evaluated for their fine motor skills or their feeding skills, the scores are gonna be lower. And are we really taking that into consideration to say, is there a developmental delay or is it a cultural norm that we need to take into consideration? And then the same goes for different cultures who place a lot of emphasis on independent skills versus others that focus a lot more on communication. And it's been very, very rare that I've seen an assessment team talk through this, like an analysis of what do these numbers actually say for this child based on these all these other implications.

0:09:48.6 AT: You mentioned, [laughter], there's so many different examples, right? Of just how culture and language differences can come in and really impact the integrity. I mean, even very face value level, you mentioned the Vietnamese population. The Vietnamese alphabet does not have all the same letters as the English alphabet. And there are measures that we use right, to assess verbal fluency, phonemic fluency how many words can you generate that start with these letters? And some of those letters that we ask them are not actually present in the Vietnamese alphabet. And so it just even on a very obvious level sometimes we just don't have the numbers might not actually be that valid much less some of these more nuanced considerations. And we've had, we're starting to see some progress in the field.

0:10:37.8 AT: There are now measures beyond just we're administering them with an interpreter beyond just, oh, we're just translating it. Even if it's formal translations. We now have measures that are starting to be adapted to other cultures and other populations and using samples for their normative data are accurate. So for example, we have some measures now that are looking at English and Spanish language fluency vocabulary, for example, and that were validated and their normative data taken from bilingual children in the United States, right? And so that's the measure I'm going to use when I'm looking at a child that maybe English language learner, I'm trying to establish language dominance that is the measure that I think is the most appropriate as opposed to taking one of our typical measures and then just accepting answers in Spanish or having an interpreter translate. It's just not the same integrity in terms of the data.

0:11:32.1 VB: What have you kind of seen when, or this is like something that we've seen, let me take it back a second. So we'll have the numbers show up, right? And it'll be like low average, low average in like three out of the five categories, but then the school psychologists and their like little analysis will say, but overall they're average in this area. We always when they're like pretty specific, we're like, no goals still needs to be provided for this area. And anytime I try to ask like, how did you get average when the majority [laughter] is low average? Is that something that you've kind of encountered or do you have any thoughts on the, I know it's difficult because you don't know what they're, but I was just wondering if you kind of encountered that or you yourself, like know how to get to average when a couple of the areas are low average?

0:12:26.7 AT: Yeah. It's funny, and this almost ties back to what we were just talking about in terms of the feedback sessions. I work with fellows and residents. I always tell them the feedback session is the most important part of the evaluation. Because what is the point of everything we just did? If parents don't understand the results, how are they going to be motivated to follow our recommendations and our treatment plan if we are not on the same page in terms of the information, right? And a secondary thing that I always say is a robot should never be able to do our job. Right? The robot can look at all these different test scores, label them as average, low, average impaired, whatever. And then maybe use some kind of rubric or some kind of a formula to determine if something is, if a domain is overall in fact average or if it's an area that needs support, right?

0:13:25.7 AT: That's something a robot can do, right? Or software. I'm not so techy, so [laughter] I don't know what it would take. [laughter] Imagine that it's possible to write a program that could do that. What a robot cannot do is consider holistically the overall patterns in the scores, understand when those scores might actually have other contributing factors and understand how to interpret this cluster of scores that may have some variability or may illustrate some inconsistency even within the domain. And so, yeah, I think a lot of it, you know what I, for me, what it comes down to is am I telling the accurate or a fair story of this child, you know? So if I know this is a child that is struggling in a specific area maybe it's their attention in executive functioning and I know that they're struggling in the real world environment.

0:14:28.2 AT: I know that parents have a lot of concerns. I know it's affecting their academics, but maybe some of my scores, a lot of them are average, right? Am I just gonna now say, oh, this child has no executive functioning difficulties or am I going to consider the fact that what does the literature say about how useful our executive functioning measures are in the data compared to real world observation? Or how kind of artificial are our executive functioning measures and how much do we actually do the executive functioning for the child in those measures versus what they're faced with in their real world environment with no support, expectations of independence?

0:15:04.3 AS: Oh, and a very completely different environment than often testing on a one-on-one many times in a quiet room versus in a large, loud classroom with a lot of other students. And that's the disconnect that we see a lot is the numbers say one thing on standardized measures in a very specific environment because that's how they're norm. That's how they have to be standardized. But that way the child operates within the classroom, within the home setting out on the playground is completely different. And if we're not looking at the whole picture and trying to analyze, well, why is it, is it that they have the capability of doing something but they're not able to demonstrate it because of the setting? Or is it because they need more support or X, Y, and Z? That's that missing link that I feel like rarely gets talked about.

0:15:55.0 AT: Yeah. And that's unfortunate because it just again, paints this unfair representation of this child, or it kind of doesn't present an accurate picture. And this is where parents start getting frustrated, right? Because it's like parents are the expert on their children, they know their child. It's kind of like, I don't it's like, I mean it's kinda almost like, I don't know what your numbers say, but I know my child is struggling and so are we as evaluators helping to bridge that disconnect between what the numbers say and what the parents know to be a struggle for their child. So we have to make the decisions, right? Again, and this is something that I don't feel a robot should ever be able to do. We have to make those decisions on how to tie the data together to really present a narrative that fairly and accurately represents the child.

0:16:45.4 AT: And especially for an English language learner where there are so many other confounding factors on our measures, this becomes, I think, a particular point that just doesn't get emphasized enough when we're talking about interpreting data. Does the data even mean anything at a certain point if you're giving a measure that's only for I've seen, for example, I've seen young children get evaluated using the WPPSI, right? And the WPPSI-IV, that's the common measure for preschool children. And then, okay, now it's a Spanish speaking one, and you kind of have three options. You give a English WPPSI, but you kind of use an interpreter and just have items administered in Spanish, or you use an English WPPSI and just ignore their ability to respond in Spanish or understand items in Spanish.

0:17:37.5 AT: Or number three, there is now a WPPSI in Spanish, but it's been validated in Spain, right. Which is a very different culture and language. Mexico or Latin America. So it's kind of like you don't have really a good option either way. I would say probably the worst thing to do is to give it, and not provide any credit in Spanish and then say, and then call the the child impaired intellectually, which I've seen happen. And why does that happen? It's because of the scores, right? It's because the test in what they call the most standardized way, which is no accommodation for English language learning, no interpreter, et cetera. And then you get these scores, and of course do they actually represent the child's cognitive level or do they just represent the fact that the test wasn't accessible to the child?

0:18:30.7 VB: Exactly right. Yeah. And I think that's like the biggest component is that we have to understand the social construct of the culture and literally even the culture of the classroom, right? And not just the neighborhood that the child comes from or where their parents are from, but we find the better IEPs not only rely on the data, which we always say data is the fidelity of the data is everything, but with Andrew F it really sheds light on the fact that there are circumstances and whatever that might be if the child is without a home, if the child is a foster care child if they have autism, right? Like it does not like it's supposed to encompass so much more. And I think that's why your work is really important, because you are able to kind of be that kind of missing puzzle piece because you understand the cultural impact of some of these tests where the child may have just needed some intervention, right?

0:19:30.1 VB: That other general education students can get, or they actually need specialized academic instruction, but it needs to be provided to them in the way that they're able to access their curriculum. And I think those two things are so different and more often than not, I don't know if it's just because it's the way that districts have done things, but they just don't take culture into account at all. And then they're either acknowledging that the child needs special education services, but they really don't, or they do give the child special education eligibility, but then don't even provide them the right type of services and intervention. So we just we're so grateful that you were able to come on and kind of explain again, for our listeners the impact with which a child, especially those that are learning English as a second language, and how these tests are really important for the parents to kind of pay attention to.

0:20:27.4 VB: And then ask those questions and not just have the school psychologist read the report, but actually explain, well, what was your reasoning, Right? Or for lack of a better term, right? Like, how did you get here my child has this and to provide that information there. Is there anything that you would like the parents to kind of have in the back of their mind as they go into the end of the assessment season? For some of them?

0:20:51.0 AT: Yeah.

0:20:52.2 VB: Or provide your contact info again.

0:20:54.8 AT: For sure, for sure. I think I'd like to maybe end on, on this, and I hope everyone's listening. I hope parents are listening, but even more so I hope evaluators are listening. The American Education Research Association actually came out with four major guidelines for are we ensuring test fairness and evaluation fairness for our diverse populations. And so I just wanna those four aspects. Number one is, are we maximizing comfort with the testing situation? Again, evaluations are such a westernized form of how we do things and how we measure things. And not every culture is comfortable with such a testing environment. So are we creating an appropriate linguistic environment? Are we decreasing anxiety? Are we setting things up in a way that minimizes the interference of cultural response to testing, et cetera? Factor number two is minimizing the biases in our tests and test items. Understanding the limitations of western tests and norms, understanding how some of our items even when they're translated may not be accurate to a certain dialect or a certain culture.

0:22:07.0 AT: Factor number three is maximizing the accessibility of our tests. So are we ensuring that they have even adequate language proficiency to respond. Are we making sure we're finding either the best translation or even better a measure that's been validated in a certain sample in a certain population and has normative data. And then finally, number four, and I think this one is especially key, is maximizing the validity of our interpretations. Are we using the most relevant normative data? Are we considering the cultural context? Are we actually doing what we say when we say the impacts on validity have been considered [laughter] and the the possible impact on our scores have been considered. Are we actually considering that? And sometimes I find that it's, I use the numbers less and more. I use my clinical judgment in terms of the patterns of strengths and weaknesses what we expect given their cultural background and their language background and how the data, how the patterns in the data might support our conclusions.

0:23:08.7 AT: That to me, is more important than just looking at the numbers and whether or not they're average, low average areas, et cetera. So those are kinda the four considerations for test fairness that have been established. And I hope that parents are empowered to ask those questions. Were these things considered, and I hope that examiners hear that and realize that have a responsibility to do their due diligence. We may have the WPPSI example earlier, we may not always have a good option. But are we using the option for the situation? Are we doing what we can to maximize the validity here? So just wanted to end on that point.

0:23:46.0 VB: I love, love that because I think overall it kind of shows just be transparent, as transparent as possible, right? I mean we are a team that's coming together to try to provide the correct and appropriate services for a child's unique learning style, right? So we thank you so much for your time again. Dr. Tan and listeners, we will talk to you next week.

0:24:09.1 AS: Yep. Bye.

0:24:10.4 VB: Bye.

0:24:10.6 AT: Bye.