3-optimal-design-20061122a.doc
Study Design Principles
Overview
Information Control
The total activity of the brain during an experiment produces a set of mathematical (digital) information. Call this I, for the set of all information:
I
This set is much more than what the fMRI sees, which is changes in protons. It comprises the quintillions of bits of information embodied by the quantum state of each elementary particle in the brain over the course of the scan. It is the mathematical equivalent of the sum total of thoughts and feelings and processes and physiology of the brain, and it is known only to God.
In an ideal world the investigator would be able to learn I. But this will never happen. The investigator captures only a subset of the total information – partial information
I(p)
One key insight into fMRI study design is that you want I to be low – you want the brain to produce as little information as possible. A second key is that you want I(p) to be high. You want to know everything about a small amount of information, not a small amount about a large amount of information. Maximizing your knowledge of I is the key to a good study.
To maximize I(p) a good experimenter does two things:
1. They MINIMIZE the amount of information coming from the brain during the experiment.
a. To do this badly, recruit a heterogenous group of subjects, poorly characterize their baseline psychological tendencies, train them insufficiently for the task in the scanner, give them vague instructions (“think of something sad”) that allow them great latitude to creatively think about whatever they like, and ensure that your stimuli are subtle and only mildly arousing so as to ensure that subjects plenty of reserve attention to think about whatever the heck they like in the back of their minds while absentmindedly performing your task. Afterwards, or maybe before, give these subjects a large number of measures that have little to do with the task at hand ‘just in case’ they show something interesting later. Viola: a huge excess of useless information.
b. To do this well, do the opposite. Be a control freak. You thereby minimize the amount of information coming out of the subject’s brain. This is good.
2. They MAXIMIZE their ability to record all the relevant data produced by the subject.
a. To do this badly, assume that even though your subjects were poorly trained, differed from one another, and were thinking about other things while completing your task, the only thing you need to model in your fMRI analysis is the task that you happen to be interested in. Pretend that the only thing going on the brain of someone comparing X’s to O’s is 100% attention on the X’s and O’s. Ignore things like physiology and emotion. Don’t bother to record heart rate, or eye position, or fatigue, or mood. Don’t try to learn if some of your subjects are depressed, or on medication. Don’t interview them afterwads to see if anything unusual happened in the scan – for example, did they fall asleep? Get anxious? Assume whatever you like. In all you do, miminize what you know about your subjects, focusing only on what you are interested in, and just pretend that that’s all that actually happened.
b. To do this well, do the opposite. Think of every possible explanation for the subject’s behavior, experience, and physiology. Explain all the variance you can. What is left will be a much stronger signal.
Signal versus Noise
Much of the information (I) produced by the brain during a scan is ‘noise’. Of course this noise is vitally important to the brain – its what allows it to do what it needs to do. But to you, the scientist, it is noise. It is information about the entropy of the system, it is information about non-psychological matters. It doesn’t matter to your experiment. A smaller amount of the information produced by the brain during a scan is ‘signal’. A crucial question to ask yourself is
“How am I going to distinguish signal from noise? How do I ensue that I don’t misinterpret signal as noise? How do I ensure that I don’t misinterpret noise as signal?”
The basic answer is conceptual – and much of this primer is devoted to the details of making it a reality. The basic answer is that you need to:
1. Minimize noise as much as possible, but MODEL THE NOISE YOU CANNOT ELIMINATE. Fail to model noise, and you are doomed to incorrectly interpret (at least some of) it as signal.
2. Maximize signal. Maximize signal. Maximize signal. Present long, strong stimuli to sensitive subjects.
In short, you want to create a highly sensitiveand highly specific study design.
Subtractions
As you will soon see, your obsessive motto when designing a study must be ‘what are my subtractions?’ The more elegant your answer to this question, the better your study. What does this mean?
fMRI images are the product of subtraction. At the most elemental, a single pixel has a certain numerical value, say from 1 to 10, at a given moment in time. The value of that pixel (say 5) at a time point in which a mental condition is not present (no flashing checkerboard) is subtracted from the value of that pixel (say 10) at a time point in which the mental condition is present (yes flashing checkerboard). The difference (10-5 = 5) supposedly reflects the brain correlate of the mental condition.
There are roughly one million philosophical problems with this formulation of the connection between brain behavior and mental experience. Nevertheless this is how fMRI is done; get used to it.
In an actual study, multiple pixels are studied over periods of time, requiring manipulations such as averaging and accounting for drift and signal spread. Yet at the core, subtractions are essentially subtractions of a pixel’s value when a mental state is off from that when the state is on.
For this reason, any study design MUST be organized in such a way that subtractions will be possible. You will need a baseline to subtract from your experimental condition, just as in clinical studies you need a placebo control group to subtract from your experimental group.
As will be discussed below, there are a variety of rules of thumb about how to ensure that you have two very different conditions, baseline and experimental, the most basic of which is to have multiple examples, randomly interspersed, of each condition type. You do not, for example, first aquire a baseline and then acquire experimental data; rather as at a buffet you take a little baseline, then a little experimental, then back for a little more baseline, then perhaps a bit more experiment, and so forth. This makes you subtractions more powerful.
This requirement of fMRI runs up against normal human psychology. People typically do one thing for a while, and then another; they do not typically go back and forth. In the study of emotion, for example, subtraction requires subjects to get sad, then feel neutral, then feel sad again, then neutral again, and so forth. But human psychology has to take a back seat to the needs of the fMRI. The machine simply cannot ‘see’ human psychology in any other way than through subtraction.
The way that you do all of this involves regressors, which will be explained below. But remember, you use regressors like vacuum cleaners, to ‘suck up variance’. Regressors that suck up the most variance produce the most robust images of brain activity, because they accurately predict where signal is emanating from in the brain.
Regressors (AKA Explanatory Variables – EVs): What makes Subtraction Possible
Your motto for regressors will be ‘what does my three-column text file specify?’ As you will soon see, a regressor = a three column text file. They are the same thing, though regressor is a concept and a text file is part of a computer program. You will be asking yourself this question almost as much as you ask yourself ‘what are my subtractions.’ The difference is that your three-column text files are much more flexible, and can be fiddled with much more, you’re your subtractions.
Practically speaking, what makes subtractions possible? Then answer is regressors. The MRI acquires scans of the brain for a period of time – say a 10 minute scan. At the time of acquisition the machine does not ‘know’ that the subject is sad one moment and neutral the next. It simply objectively acquires image upon image.
You, the experimenter, must know what the subject is thinking/exerpiencing at each time point during the scan. You will then construct regressors to tell your data analysis program (for the sake of this writeup we will assume you are using FSL, but SPM and BrainVoyager are two other common commercial programs) how to divide the single scan into at least two separate conditions, one of which can be subtracted from the other.
A regressor is defined by three columns. The first tells FSL when a condition started. The second tells it how long the condition lasted. The third, which is optional, tells FSL the relative intensity of the expected signal. This third column is explained below, under ‘parametric modulation.’ For the purpose of this section, just assume a regressor has two columns: when a condition starts, and how long it lasts.
Parametric Modulation: What makes a regressor dynamic
The big picture overview on design is this: you want an orthogonal, parametric design. Orthogonal means that you have two factors – let’s say recentness of loss (near to distant) and degree of attachment (high to low). These are potentially orthogonal. You can give someone a stimulus that is:
* Near/high
* Near/low
* Distant/high
* Distant/low
Who cares? You care. Now you can do your analysis showing that attachment is a coherent independent predictor of sadness intensity, and that recentness of loss is, and that they are independent of one another. Contrast this with a study in which people get sad over a recent loss of a loved one and only the loss of a loved one. What is making them sad – the recentness of the loss or the level of love? You don’t know. You are forced to do a between-group analysis, showing that people mourning distant losses get less sad than people mourning recent losses. This doubles your N to get the same result, and doing a mixed rather than fixed effects analysis loses you power.
Now here’s what parametric means: you can dose the stimulus and see if the response is dose. Here is a list of parametric ideas:
* The redder the square, the stronger the shock; is the galvanic skin response proportional? That’s parametric modulation
* The more recent the death, the stronger the sadness, the larger the ACC activation
* The longer the word, the longer the reaction time to the response
Putting these things together, ideally you have orthogonal stimuli that are parametrically modulated. That was kind of implicit in the example above. If instead of just having different types of attachment, you have a set of attachments which are ranked by intensity, and instead of types of loss (far and distant) you have near medium and far, or near, near-medium, far medium, and far, you are in business.
Of course, the more parameters and factors you have, the more stimuli you need, and this begins to push you towards an event related design.
1. Orthogonal design. One way to do this is using a variation on the Hariri task. They could see both pictures and words. Picture of Living loved one, dead loved one, living stranger, dead stranger Word name of living loved one, dead loved one, living stranger, dead stranger. I wonder if the strangers could be further subdivided into familiar (eg John Kennedy) versus unfamiliar. Remember to have people rate familiarity and other affective criteria for each of these.
2. Parametric design. As Jack says, the best studies are parametric and orthogonal. Its a very powerful and convincing result, if it works. So concretely for my study, I might want to have a recent death, a mid-level death, a very distant death. Or I might want to have a very close loved one, a moderate loved one, a stranger. That kind of thing – varying the attachment.
Orthogonal Design: What makes multiple regressors independent
If subjects show a certain region activation (region X) in response to words, it may be attributable to either the semantic meaning of the word, or some property of the word itself – eg its length, the fact that it is a word, and so forth. However if you expose subjects to a completely different class of stimulus – say photographs – and only those photographs with the particular semantic meaning common to the words activate region X, you have produced convergent evidence that region X responds to semantic meaning across stimulus categories. This is a far more impressive finding. Therefore optimal experiments will employ more than one type stimulus to elicit a single dependent variable.
Ecological Validity
Your motto for ecological validity is “so do people really work this way?”
It is one of the small tragedies of modern neuroscience that fMRI is in no way analogous to film or digital photography. Or rather, that it takes enormous skill and luck to make fMRI like film photography. You cannot look at people in their natural environments, and even in the lab, you cannot allow them to simply think and then image this thinking. You HAVE to tell them what to think when. You have to manipulate them. And not only that, but as the foregoing shows, you have to manipulate them in a highly idiosyncratic, and dare I say, unnatural way.
The ideal way to study sadness is to have someone get intermittently angry, multiple times, at random intervals, during a predetermined set of time. “Get mad… now be neutral… now get sad…. Now mad again… now neutral… neutral again…. Now mad! Now sad!”
So the question is: does your study design have ecological validity? Are you imaging what you think you are imaging, or are you inadvertently imaging something else – like the su
Brass Tacks for Increasing Signal: Noise for Dependent Variables
1. Maximize the maximal intensity of the dependent variable(s), then minimize the minimal intensity of the dependent variable. My study was on sadness. My goal was to get people as sad as they could possibly get. A study on anxiety should get people very anxious; a study on pain should cause extreme pain (within ethical guidelines); a study of depression should study very depressed people, and so forth. The more intense the dependent variable, the stronger the sign
2. Minimize the duration of the dependent variable (s
3. Increase the frequency of the dependent variable (s)
4. Randomize the occurrence of the dependent variable(s)
Other Fundmental Considerations
On ‘neutral’: What is it? Does it even exist?
In a purely cognitive study, eg of a flashing checkerboard, you can have a blank screen that is truly ‘neutral’ to compare to what the brain does when it sees an image. But what about in the study of emotion?
On design implications of fixation
Putting aside the question of ‘what are people thinking during fixation’ – and assuming that their minds go blank – fixation STILL is a very tricky concept to get one’s head around from the standpoint of study design. The problem of fixation is subtle, and the more you think about it the more subtle it becomes.
Consider a simple block design comparing perception of fearful and neutral faces: FNFN. One can imagine several permutations of this design, with the ‘+’ sign signifying fixation. Assume blocks are of equal length.
A. FNFN
B. +FNFN+
C. +FN+FN+
D. +F+N+F+N+
There are several things to notice
1. Fixation adds minutes to the study. This reduces signal by robbing you, per unit time, of … well…. signal. So all things being equal, you want to minimize fixation.
2. Take case A. Here, there is no fixation baseline to compare both F and N to. In this case, N becomes your ‘baseline’ and your only regressor is F. What’s the drawback of this method? An important one: you don’t know whether signal in F is the product of
1) Reductions in BOLD signal during N (less likely, but possible)
2) Increases in BOLD signal during F (more likely, but not certain)
3. Now take case D. Given +, you CAN tell whether F represents an increase over baseline and N, or just over N. EG, N may show deactivations to baseline, and F no changes from baseline. This appears to be the superior design – but it takes TWICE AS LONG as A. This represents a serious loss of signal, in addition to the fact that subjects may become more bored.
4. Now take cases B and C – these give you some baseline, but it may be more annoying than useful.
5. Other options include having very short but frequent +.
Can fMRI show deactivations?
References: Amir Shmuel, Mark Augath, Axel Oeltermann & Nikos K Logothetis. (2006) Negative functional MRI response correlates with decreases in neuronal activity in monkey visual area V1. Nature Neuroscience 9, 569 – 577
The term ‘deactivation trends’ is actually misleading for fMRI. In PET studies, if you do an arterial sample telling you how many units there are per mL blood, then the ABSOLUTE level of activation of a brain area can be determined by comparing levels of a radiotracer in a region during a task to levels in a control condition. When the levels are higher, the absolute blood flow is higher (in FDG PET). fMRI does not look at absolute levels of blood flow. It looks at relative blood flow. This is a problem for interpretation. Let us say that you do a simple subtraction; in condition A you have a person think about an alive dog, and in condition B they think about a dead dog. you instruct them to ‘feel their feelings’ in both conditions and they say they are 10/10 sad thinking about their dead dog, but 0/10 sad thinking about their alive dog. Since imagery etc is held constant, B minus A should reveal a portrait of sadness activations. So far it’s like PET. BUT….. what if you want to know regions that ‘deactivate’ in sadness, eg that get less blood flow during sadness (for example, if there is evidence from PET studies that ventral cortex is ‘shut off’ during sadness). Naturally you do an A minus B subtraction. This is your only mathematical option. Now in PET this would show you areas that had an absolute level of blood flow that was less during sadness than during neutral emotion. But what does it show you in fMRI? It’s hard to interpret. Or rather, it is impossible to interpret. Any statistically significant regions could EITHER be areas that are more active in neutral than in sadness, or that are less active in sadness than in neutral. While mathematically these are equivalent differences, biologically they are not equivalent. EG, let’s say that the ventral cortex shows a ‘deactivation’. All you reallly know is that the ventral cortex is less active in sadness than in neutral. But this could be because in ‘neutral’ the ventral cortex is having a higher-than-baseline blood flow, and in sadness is simply having a baseline blood flow.
The solution to this problem? Possibly it is to have a fixaton period in your experiment that you can subtract from both the neutral and the sadness periods. But you can never really address the problem, because what are they thinking about in the neutral period, and are there ‘activations’ or deactivations there? In the end you are stuck only knowing about relative blood flow.
Regressors Are King: A Case Study
In a simple study design comparing neutral to affective words in an emotional stroop, subjects reported the color of words by button press. The amount of time they took to respond was their reaction time, or RT.
The study design was this, in blocks of 30s: +NANANANA+
Several regressors were possible:
1. Fixation: +________+
2. Neutral: _N_N_N_N__
3. Affective: __A_A_A_A_
4. Reaction time: _xxxxxxxx _
Remarkably, which regressors were chosen had a massive effect on the apparent neural activation during reaction time and affect, the two regressors of interest
fMRI Primer 
Leave a comment