Understanding Correlation, Sampling, and Experimental Bias in Research

the topic for this lecture is analyzing our findings so as we were talking about at the end

of the previous lecture we were talking about the idea of correlation and the fact that prediction uh just because two

things are correlated that you can predict one thing from the other doesn't mean that one thing causes the other

so for example higher correlation coefficients lead to an increased ability to predict one variable based

off of the other so SAT and ACT scores are moderately correlated with first-year college gpa approximately 0.5

which is a a good modest correlation so a perfect correlation would be a correlation of 1.0 which means that two

things are perfectly positively correlated with each other if two things have a zero correlation it means that

there's no systematic association between two variables as one variable changes it doesn't tell you anything

about what's changed what about any sort of change in the other variable if two variables have a perfect negative

correlation it means that as one variable goes up you can perfectly predict that the other variable will go

down most things don't have perfect correlation so like we're seeing with

SAT and ACT scores and their correlation with first year college GPA is there's an association so in general when people

tend to do better on the SAT and the act before starting College in general they tend to do better in the first year of

college with regard to GPA right and now what's important though is it doesn't mean that one thing causes the other

necessarily right just because two variables may be highly correlated like we're seeing with

SAT scores and ACT scores in first-year college gpa it doesn't mean that one thing causes the other right and for a

moment if you think about the s-a-t-a-c-t and first year college gpa connection

that makes sense that there isn't necessarily A causal Association it isn't a case that because you're good at

because you did well on the SAT it doesn't mean that you then are going to do well in college it's not that somehow

doing well on the SAT makes you a better student right now likely what they're both tapping into is some other variable

that may be explaining both SAT scores as well as GPA things like intelligence work ethic and so on

so the example we're going to talk about just to make sure that we're all perfectly clear about the idea that

correlation does not equal causation as we're going to talk about the fact that murder rates tend to be associated with

ice cream sales right they're positively correlated with each other so let's think for a moment about this

connection between murder rates and ice cream sales and by the way this is a robust connection

when murder rates are higher ice cream sales also tend to be higher right they're positively correlated mean they

meaning they tend to change in the same direction now if we think about the possible

reasons why murder rates and ice cream sales may be correlated with each other or sorry why they are correlated with

each other there are at least three possibilities that we want to consider possibility one is that X causes y right

X in this case is going to be murder rates Y is going to be our ice cream sales

so it is possible right that maybe murder rates cause ice cream sales right so maybe for example after someone's

been out on a killing spree maybe they decide they want some more dairy products right that that is a

possibility not a very likely one but a a possibility nonetheless right the second possibility is that maybe our y

variable ice cream sales actually causes our X variable murder rates right so for example maybe there's something about as

increasing ice cream sales go as ice cream sales go up maybe that's causing murder rates

for example I'm lactose intolerant right if I were to eat a lot of ice cream maybe that would really upset me so

maybe I would uh decide to go out on a killing spree right maybe that's a possible explanation for the connection

we murder rates and ice cream sales doesn't seem terribly likely to me right I doubt if lactose intolerance or any

other sort of issue or anything connected with ice cream sales is doing anything to drive murder rates right

that just doesn't seem like a very likely proposition now this third possibility is something

we should carefully consider because it's actually going to be the explanation for why murder rates and ice

cream sales are connected it's possible that some third variable Z right that we haven't even talked about may actually

be responsible for both X and Y right so think for a moment why do you think that Z variable might be that might be

responsible for both increases in murder rates and increases in ice cream sales right if you were thinking about

temperature right you would be right it's about it about the season of the year right when things are hotter right

when the world is hotter right people are more likely to eat ice cream they're also more likely to hurt each other and

kill each other and so the reason why there's a connection between murder rates and ice cream sales isn't because

murder rates and ice cream cells cause each other rather it's because both of them are impacted by something else

right they're both impacted by temperature right when it's hot outside we're more likely to eat ice cream we're

also more likely to kill other people or be killed by other people right so here the correlation isn't directly about the

connection between murder rates and ice cream cells rather there's some third variable Z involved

right this is why it's so important right for us to think about situations where when we're told that two things

are correlated right to not automatically jump to the assumption that ah one of them must cause the other

one because in some cases the world may be a bit more complicated than that we also want to talk about some of the

other issues that impact our ability to conduct research and how we evaluate research

one is going to be we have to think carefully about the sample that we're using so a sample is going to be the

collection of participants right who are selected for observation and some sort of empirical study whether it's a

correlational study or an experiment right the sample is going to be the participants that we actually include in

our study the sample is going to represent the population right the population is a

much larger collection of individuals from which the sample is drawn researchers are going to use the sample

in order to make generalizations about the population right so for example what we would do is

if we were interested in trying to understand something about the students of Oakland University right Oakland

University students all 20 000 or so of them right they they would be our population if we didn't have the time or

resources to actually uh chat with or interview each of the 20 000 students at Oakland what we may do is choose a

representative sample right from those students so since there are 20 000 or so students in Oakland maybe we select 500

of them to use as our sample and our study right so if we were interested in knowing do Oakland students really like

the Golden Grizzlies mascot maybe we don't have the time or energy to contact all twenty thousand maybe instead we we

sample 500 of them to ask their thoughts about the Golden Grizzlies mascot now sampling bias right it exists when a

sample isn't very representative of the population from which it's drawn right so for example going back to our Golden

Grizzlies example if we have our 20 000 Oakland students right with uh Majors from different departments from the

College of Arts and Sciences within Health within business right all these different Majors within different

schools or colleges we would want our sample to represent adequately right the diversity of the student body at Oakland

If instead if we decided well we're just going to ask one classroom full of students maybe 25 business students from

a graduate course that's taught on Tuesday nights right if we decide to interview those folks right there may be

some sampling bias that would be creeping in because our sample may not be very representative of the population

from which it's strong right if those business students are all in a masters of Business Administration

program right it's likely that they're going to be older than the typical Oakland University student they're

already going to have an undergraduate degree which is not going to be the case for most students at Oakland right so

this sample may not do a great job of representing the diversity of the student body

there's another approach we can use to sampling which is random sampling this is a wonderful approach that's kind of

the gold standard of sampling however it's difficult to actually use it in practice but the basic idea behind

random sampling is that every member of the population should have an equal chance of being selected now there are a

lot of qualifiers that go into this so for example even though every member of the

population may have an equal chance of being selected right they may not actually want to participate and so you

can start getting some some issues creeping into whether or not your random sample actually represents the

population perfectly but the basic idea behind random sampling is a very good one which is

let's go back to our Golden Grizzlies mascot example so if we had 20 000 students at Oakland and we were going we

had enough time and energy and resources to interview 500 of them let's say we could maybe reach out to the office of

the registrar or some other entity on campus and try to get a list of the the students at Oakland and then use some

sort of random procedure right to try to sample adequately right from each of the students at Oakland right so maybe we

just go through and we use like a random number generator to uh to students from a list of some sort right something of

that kind might do a nice job of trying to get a relatively representative population of students right that would

be a much better approach if we were trying to get a broad representation than doing something like taking uh

students from a a a from a relatively uh low-level nursing course for example which those nursing students may be

really nice representatives of nursing students but they may not represent the broader University population same way

we wouldn't want to use like psychology Majors because they may not be very representative of the rest of the

University this is the connection between our population and our sample so as you can

see here we have a population of some sort so let's imagine for a moment this represents Oakland University students

and the different colors represent different uh Majors within the university let's say

so if we have a representative sample then what we should be seeing is the sample should resemble right the

population that it's being drawn from in contrast if we have an unrepresentative sample what we see is

that the proportions right don't really align terribly well with the underlying population right so for example here in

the population this blue group is about maybe a little more than a quarter right but down here in our sample now it's a

little more than 50 percent right that wouldn't be ideal because now we have a sample that doesn't look very much like

the population that it's being drawn from right so for example in our population we had some purple folks here

who don't show up in our sample at all right so again what we want is a sample that represents the population that it's

being drawn from the population that it's being used to represent another issue that we want to address

we've talked about this previously but I want to make sure that we're explicit about it are Placebo effects right we've

talked about placebos and some of our examples previously but Placebo effects occur when participants expectations

right lead them to experience some change even though the treatment right that they actually received was

ineffectual right so we talked about this in the context of people who are depressed receiving either an actual

dose of a drug that increases their levels of Serotonin or a placebo which might be a sugar pill that doesn't

actually have any impact right on the level of Serotonin the reason why we include these sorts of

placebos as control conditions is because of placebo effects in some cases if people are told that they're taking a

pill that will help them feel better make them feel less depressed for example they may actually feel less

depressed not because the drug is or the pill is having any sort of real impact but because their expectations lead them

to feel a little bit better right and that's what we're talking about with the placebo effect is when nothing real is

happening right but we feel a little bit better or at least feel a little bit differently because we're expecting to

feel differently right some types of folk medicine right may have effects through this Placebo

mechanism for example when my mother was alive uh she had some mild arthritic symptoms and so she started wearing

these magnetic bracelets that she thought would help with her arthritis um I don't believe there's any empirical

support showing that these magnetic bracelets are actually helpful at all with arthritis I'm not up to speed on

arthritis research so forgive me if there's any evidence I would be deeply skeptical that there is but she felt a

little bit better for a little while because she was expecting them to be helpful right and that's the that's the

very idea behind placebo effect is our expectations can actually shape our experiences to some extent

there are also some distortions and self-report data that are important for us to think about in terms of evaluating

and analyzing our data so for example when we talked about our depression example from previous lecture we talked

about using the back depression inventory a self-report measure of depression symptoms as our outcome for

how we're going to operationalize depression now if we do that since we're relying on

self-reports we would have to think carefully about issues of social desirability which is the tendency for

people to give socially approved answers to questions about oneself so for example if we were asking people about

various depressive symptoms right it's possible right that they might actually give us inaccurate responses because

they know that it's socially undesirable right to admit to having certain types of symptoms for example right so we

would have to be aware of the fact that participants aren't always perfectly honest right so sometimes they give

answers that are designed to avoid violating social expectations because they want to be approved of by people

around them they're also response sets which are Tendencies for people to respond to

questions or items in a particular way that isn't really related to the content of the question right so if you're going

through and doing a self-report survey so for example um if you if you go to uh your family

doctor and periodically they'll ask you to do things like update like your your basic medical history so they'll give

you a long checklist of various conditions or problems you may have experienced and they'll ask you to check

either yes or no and if you're relatively healthy and if you're going through Page after page of these

different health conditions and you're saying no to each one at some point you may just start kind of clicking no for

everything without even really carefully reading the items anymore right and that's the sort of response set that

we're talking about here where at some point you're just saying no no no no no and you're not even really thinking

about the content of the question which again is not great for self-report data because now your answers don't

necessarily the connect to your actual experiences if you're simply saying no to everything

same thing happens by the way if you're if you have a lot of health issues and you're saying yes to everything without

actually reading it that that's also at least is problematic finally we want to talk about some

issues of experimental bias that can influence studies and how we analyze results

one of these issues with experimenter bias is going to be whether or not a researcher's expectations or preferences

about the outcome of the study influence the results that are obtained right now in some cases experimental bias can be

very pronounced and intentional by the researcher however usually that's not the case usually what happens is that an

experiment or when experimenter bias creeps into a study usually it's unintentional meaning that the

experimenter May unintentionally communicate through things like body language or facial cues or something of

that sort what they're expecting the person to do in response so let's say for example that let's go

back to our depression example from previous lectures so we have two groups of individuals an experimental group in

a control group the experimental groups given a dose of a drug that we think is going to increase serotonin which as a

consequence will then lower depression the control group it's a placebo they're going to take a sugar pill we don't

expect the sugar pill to do anything for the person's suppression so what we're expecting is for people in the

experimental group to report lower levels of depression than people in the control group

if you're the experimenter for this and you've designed the study so the re so you're the researcher you know what's

going on what may happen is as you are asking people to fill out the back depression

inventory right even though it's a self-report measure right you may somehow unintentionally right

communicates people in the experimental group that you're expecting them to feel a little bit better right maybe kind of

subtle things in your mannerisms the way in which you give them the questionnaire right little things that you just kind

of mentioned to them whereas people in the control condition since you're expecting them to be more depressed you

may unintentionally influence their responses just in the way that you're interacting with them right and these

are kind of the subtle ways in which experimental bias can influence results one of the safeguards against

experimental bias is to use what's called a double-blind procedure a double blind procedure is a research strategy

in which neither the participants nor the experimenters know which participants are in the experimental or

control groups right so for example if if I was a researcher in charge of our depression study that we were talking

about right I would assign participants to the experimental group and control group

right but then I may have an experimenter right maybe one of my graduate students for example actually

go in and interact with the participants to give them the medications to give them the self-report measure at the end

and I may tell the research the research assistant right my graduate student that this is participant one give them this

medication here's participant two give them this medication but they don't know which medications are the real ones and

which ones are the placebos right in that case neither the participant knows if they're in the real drug group the

experimental group worth in the placebo or control group in this case if we're using a double

blind procedure the person actually interacting with them doesn't know which group they're in either right now since

we're keeping careful records we know who participant one is who participant two is right I as the researcher can

then still later go back in and make connections between ah these participants were in the experimental

group these were in the control group but the individuals who are actually interacting with them directly they

don't know right so the idea is that's going to reduce the possible impact of any sort of experimental bias

now double blind or sorry single blind procedures are also fairly common and a single blind procedure is when the

participants don't know what group they're in right but the experimenter actually does know right and single

buying procedures are are fairly common in research now with the sort of drug study that we were talking about we

could do a double-blind procedure right there isn't any reason why the experimenter is handing the people the

pills and explaining the procedure has to know whether the pills in the bottle are actually the active drug and for the

experimental group or the placebo for the control group right we just have to we just have to have some researchers to

keep records right but the actual experiment who's interactive person doesn't have to know

now in some other procedures other types of studies that may be harder right for the experimenter not to know right if

there's some sort let's say for example if we move away from our depression example let's say we're doing something

uh where we're manipulating the type of feedback that a participant gets uh for their behavior on on some sort of task

right so they uh they write an essay telling us about their career goals and so on and in one group The experimenter

gives them positive kind of affirming feedback that's supportive in the other feedback condition maybe the participant

gets negative feedback they're criticized for the way in which they talk about their career goals

in that case if the experimenter is one providing the feedback then they obviously have to know what

condition the participants are in right because they're going to be the ones administering the feedback so that would

be a single blind procedure where the participants wouldn't know if they were in the experimental or control condition

all they know is they were asked to write an essay they wrote the essay they gave it to the experimenter and then the

experimenter gives them feedback right they would then obviously be aware of the feedback they would know if the

experimenters said nice things are critical things but they wouldn't know that that was something that was being

manipulated right so that sort of procedure would be a single blind procedure