Reflections from the Spencer SCD Conference at Vanderbilt Part 1
This was important enough to me that it seemed worth posting my first real blog about. This first part of my reflections are more about how the experience at Vanderbilt made me feel and the general impressions I got from being there than it is direct thoughts or questions about single-case designs. Apologies if this is not very well-organized, I’m adapting some stream of consciousness notes.
This past Wednesday through Friday I attended a small single-case design conference organized by Jennifer Ledford and held at Vanderbilt, funded by the Spencer foundation. I’ll say up front that I hope this is the first of these conferences rather than the only conference. I attended for a lot of reasons, but the biggest reason was that while my work and research has focused heavily on SCDs, I have very little opportunity to interact with researchers who use SCDs. As a methodologist, it’s really easy to become wrapped up in problems that don’t really relate to the things that applied researchers are thinking about. It’s also really easy to miss important areas of research that you wouldn’t otherwise know exist.
The conference was small, but I got to meet a lot of people whose names I know and whose work is very important to my understanding of SCDs. I also learned, to my surprise, that people knew who I was. Perhaps I am not an idiot. I got to talk with a lot of people who are thinking very critically about the work they do. I believe subtle misunderstandings that I had about the design and about the people doing this work were corrected by spending several days with researchers and faculty. I’ll talk about one of these in a later post, but I’m sure there are changes in my thinking I’m not even aware of.
After the conference ended at noon on Friday, I begged off any more face time with all the new colleagues I met and went and sat in the shade some trees on the Vanderbilt campus and just listened to the sound of the leaves and the wind and tried to let my brain settle. Then I went to Chuy’s (a comforting reminder of time in Austin) for a quiet lunch by myself. Afterwards, I went to the lobby of the hotel where I had been staying and spent almost two hours writing down my thoughts. Then I went to the airport, and spent another few hours writing more thoughts. Saturday morning I got up and puttered around, and then spent another hour or so writing more thoughts. In all, I’ve got about nine pages of stream of consciousness that feels like it’s best as a series of blog posts. Anyway, here are some vague impressions I got:
Applied SCD researchers think very hard about measurement
I don’t know that this should be surprising, it just had never occurred to me. It’s just that so many grad students (and to a lesser extent early career researchers) who I worked with in group designs didn’t really worry so much about measurement. They tend to select a measure that exists and is commonly used, without a lot of consideration for how appropriate it is for their purposes. If you select a measure intended to capture a relatively steady feature of personality or behavior, it’s going to be hard to move that behavior. (Ann Kaiser and Wendy Machalicek both independently made similar points about this).
Sometimes when some group design researchers do think about measurement, maybe they do in the wrong way, trying to make up their own measure without worrying about validity issues because they don’t think anything existing is suitable. In any case, it’s clear that they think & worry a lot about measurement and the shortcoming in their measures. I heard a lot of offhand discussion about the problem of the quality of social validity measures.
Grad students (and faculty) are very engaged with complicated methodological work that they may view as difficult but take seriously.
(Also some of the grad students know my name?? aka possibly I am not an idiot[!!])
Ok, so, when I first emailed Jen about attending the conference her response to me she was immediately welcoming and she wrote that she had “read [my] work.” Which was nice, but I kind of assumed wasn’t about me per say. I’ve got a couple of first author publications, but the papers I’m on that I think get cited most are all the papers where James Pustejovsky is first author. This is understandable, James is casually brilliant whereas I am merely sometimes clever. I’m comfortable with that.
However, during the first round table session on the very first day there were a few grad students sitting near me and I was chatting with them. One of them looked at my name tag and asked excitedly “Wait, are you the Swan from statistics??” That made my brain glitch out for a second because I’m not really a statistician. Statisticians are mathematicians with a perverse interest in empirical things. James is a statistician. I’m a methodologist, which means I only (badly) do calculus and linear algebra when absolutely required rather than recreationally.
So, look, pendatic aside aside, yes I was the Swan from statistics. It turns out Jen teaches an Advanced SCD course where they read my first author work. This was the first time I’d ever met anyone who had read my work outside methodologists. Corey Peltier told me once by e-mail, but it’s another thing entirely for a grad student to come over very politely and ask “Dr. Swan” if they can ask a question about a project they are working on. It was a meta-analysis question, so to hide my incredulity I made Man Chen come over and help me answer it because she truly knows way more about meta-analysis than I do, despite not being “Dr. Chen” yet.
More broadly, though, it was clear to me that despite many of these researchers viewing the work my colleagues and I do with some trepidation, they were working hard to take advantage of the advances that methodologists have offered them over the years. Not only did they want to use all the tools they had, there was an active interest in the tools they didn’t yet have (for instance, a between-case standardized mean difference for alternating treatment designs) in a way that I hadn’t really anticipated. Methodologists interested in SCDs who aren’t looking at applied researchers as eager partners in their work are probably missing out badly.
Behaviorism is alive and well
My undergraduate background in psychology kind of treated behaviorism as a dead tradition. I knew in an abstract way that SCDs arose out of the behaviorist tradition and that ABA and SCD had a close link. But I wasn’t really ready for how committed a lot of folks were. Certainly, none of the folks I met here were (or at least professed to be) hardcore Skinner-type behaviorists. But most of them were ABA certified or on their way there, and that behaviorist orientation is a profound part of their understanding of how to help children, students, and sometimes adults.
There was also a keen understanding among some of the people I talked to of the epistemological grounding of their research tradition, particularly compared to what I see from many group design researchers. I worry about the epistemology of quantitative research these days, so it was interesting to hear. I suspect it might be consequential in ways that aren’t clear to me yet, so I’ll have to keep it in mind.
Future posts in this series will cover more substantive topics:
- Schools of Analysis and Research Design Choices (Thanks John Ferron)
- Possible consequences of attention to therapeutic trends in baseline
- I have misunderstood the “functional relation” and its implications for meta-analysis
- I still don’t know what stability is or how important it is
- Do we need a central repository for SCD data?
Possibly some of these topics will be covered together. I don’t know exactly yet.
Daniel M. Swan
Research Associate, Applied Research Methods and Statistics Lab
I am a methodologist whose work has primarily focused on single-case designs.