?

Log in

Chameleon
deadlychameleon
... .::::: .:. ... ....:: ....: .. .:::. .:::.:
Chameleon [userpic]
Bad research and fandom: Surveyfail

In regards to surveyfail: http://www.fanhistory.com/wiki/SurveyFail

I called the Boston University IRB office. The direct approach works.

They've gotten a lot of emails regarding Dr. Ogas. He is no longer in any way affiliated with Boston University, except as a recent graduate. They have asked him to stop using his official Boston University email address in connection with this project, or his website. He is officially on his own, and this project is NOT IRB APPROVED.

That is the official status as stated by the Boston University IRB office.

The problem with this is threefold:

1. The researcher has no expertise in the area he is researching, nor has he recruited anyone to give him guidance.

2. The researcher has substantial profit motivation to produce work in this area (book contract with Penguin) which may lead to unethical conduct/a tendency to misrepresent his results.

3. The research is in no way overseen by any external body which can examine it for potential unethical conduct.

In addition to all of these, the researchers have now alienated their participant population, who are now very likely to become unreliable participants.

The only way to salvage the study at this point, I believe, would be for them to change it to an observational one.

ETA: if you want to share, please repost with attribution, that way people don't have to click through to get the information. If people need to contact me, they're certainly free to do so, but I'd like the discussion to spread out.

Also, all comments here are based on the information I had at the time of posting - if it turns out to be inaccurate, my deepest apologies. My statements regarding the problems with this research are based on my personal opinions, and should be taken as such.

ETA 2: I need to do other things for awhile, so I apologize if comments go unanswered. I should add that the IRB stated they contacted Dr. Ogas, and confirmed that this was indeed his project (they also gave him some basic methodological suggestions, despite the fact that he's no longer affiliated). They claim he's been very receptive to their suggestions. I say this just to confirm that YES THIS IS HIS PROJECT, and probably not a case of stolen identity.

ETA 3: Oh look who just f-locked his entries. Anyone with screencaps? Anyone? Anyone? Bueller?

ETA 4: Hopefully the last ETA I'm going to need to make.

I don't even know offhand where the line between unethical and illegal begins here, because they're so far past anything I'd ever do.

To start with:

1. Lack of proper anonymity/privacy protections for participants potentially admitting to illegal activity.

2. Lack of proper anonymity/privacy protections for participants potentially admitting to things that could negatively impact them at work and in their communities were they to become known.

3. Lack of any kind of age statement - this is methods 101 people, you always ask if a person is over 18, otherwise you need parental consent for any research study.

4. Lack of ANY kind of informed consent. No statement that you may withdraw from the research at any time. No statement saying what the research will be used for. No statement of risk vs. benefit. No statement regarding deception. No statement giving the affiliations of the researchers.

This is BAD RESEARCH, not to belabor the point. As one commenter pointed out, it's one thing when it's a survey about ice cream flavors. But the risk level here is fairly high. It's possible for someone to lose their job were this information to become public and identified with a given person.

ETA 5. 11/11/09
It's like Zombie Research. We study brains! We eat them and never die!

I have no confirmation that these are the same folks as the ones above, but I'm suspicious.

You know. Fans aren't actually stupid. Someone needs to start an IP spreadsheet for these folks. Show them how we do it in fandom.

Signal bump time: http://genericresearch.livejournal.com/323.html

My screened comment to these idiots:


Seriously guys, if you're carrying out IRB approved research you're generally REQUIRED to state your University, IRB approval number, and supervising faculty member if you're students.

The whole "knowing you're from a real university might bias results" idea is stupid. Also, livejournal is in no way a good way to collect the kind of survey questions you're asking, you should be using a data collection service (e.g. surveymonkey). Even if you were students, you should know that.

If this is not IRB approved research, get the heck out of the community, thanks.

NO ONE REPLY TO THESE IDIOTS UNTIL THEY ADDRESS THE ABOVE ISSUES. THANKS!

Comments
Page 1 of 2[1][2]

Do you mind if I repost this in its entirety on my LJ?

Thank you for this information! Will be signal-boosting.

I'd prefer folks re-post with attribution, so that people don't need to click through. But yes, it's important information, and the wiki article has been updated.

I wish I could have been there when the officials at BU came calling and starting telling him "We don't know you. Stop using our name. YOU ARE NOTHING TO US." I wish I'd seen him the moment he realized that his professional reputation was in the toilet, and it was all because of a group of bizarre porn-reading internet freaks didn't just let him pull out his microscope and lab coat and do whatever the hell he wanted to them. [giggles]

I bet his face was hilarious.

*sigh* I feel for the guy, you know? It sucks to be an unemployed Ph.D. But he needs to suck it up and maintain ethical conduct nonetheless.

I hope it's all right to share this. Cookies for you for calling BU!

I'd prefer folks re-post with attribution, so that people don't need to click through. But yes, it's important information, and the wiki article has been updated.

Thank you, and linked.

Thanks, though I'd prefer you re-post with attribution, that way folks don't have to click-through to get to the important info.

Reposted in entirety and attributed. Now I finish making the altar to you. *G*

Altars are unnecessary! I did it as a fellow researcher who was fairly disgusted by the whole thing.

Awesome, thank you -- have referred this to the linkspam round-up at DW: http://linkspam.dreamwidth.org/6763.html?view=87147&style=mine#cmt87147

It's like Zombie Research. We study brains! We eat them and never die!

You know. Fans aren't actually stupid. Someone needs to start an IP spreadsheet for these folks. Show them how we do it in fandom.

Signal bump time: http://genericresearch.livejournal.com/323.html

My screened comment to these idiots:


Seriously guys, if you're carrying out IRB approved research you're generally REQUIRED to state your University, IRB approval number, and supervising faculty member if you're students.

The whole "knowing you're from a real university might bias results" idea is stupid. Also, livejournal is in no way a good way to collect the kind of survey questions you're asking, you should be using a data collection service (e.g. surveymonkey). Even if you were students, you should know that.

If this is not IRB approved research, get the heck out of the community, thanks.

NO ONE REPLY TO THESE IDIOTS UNTIL THEY ADDRESS THE ABOVE ISSUES. THANKS!

Thank you for the information (and if it was you, thank you for updating the wiki article with this as well. I've been trying to keep track of events there but it's been exploding so quickly it's hard to keep up with it all.)

Edited at 2009-09-02 05:10 pm (UTC)

Yeah, it exploded really, really fast. Kudos to fandom, most groups wouldn't have figured things out so quickly. Heck, I was willing to give benefit of the doubt at first, but that quickly waned.

(Anonymous)

Did you ask about Sai Gaddam? Because Ogi may not be affiliated, but some things about the BU website seemed to indicate that Gaddam doesn't actually have his PhD yet. Which would mean he's still firmly affiliated with the Uni.

Even if he was, the IRB office clearly indicated the study itself was NOT in any way affiliated.

However, Sai Gaddam has graduated according to his official webpage. I do not know if he is still affiliated for sure, but the project is not.

http://cns.bu.edu/~gsc/

Reposted with attribution!

It's like Zombie Research. We study brains! We eat them and never die!

You know. Fans aren't actually stupid. Someone needs to start an IP spreadsheet for these folks. Show them how we do it in fandom.

Signal bump time: http://genericresearch.livejournal.com/323.html

My screened comment to these idiots:


Seriously guys, if you're carrying out IRB approved research you're generally REQUIRED to state your University, IRB approval number, and supervising faculty member if you're students.

The whole "knowing you're from a real university might bias results" idea is stupid. Also, livejournal is in no way a good way to collect the kind of survey questions you're asking, you should be using a data collection service (e.g. surveymonkey). Even if you were students, you should know that.

If this is not IRB approved research, get the heck out of the community, thanks.

NO ONE REPLY TO THESE IDIOTS UNTIL THEY ADDRESS THE ABOVE ISSUES. THANKS!

Posted in my own journals with you referenced. Thank you so much.

Reposting on my own LJ as well. :)

(With attribution, of course.)

Edited at 2009-09-02 05:26 pm (UTC)

I've reposted this in its entirety [ETA: with attribution, of course!] on my DW, LJ, and IJ journals. I've also reposted it as a first-level comment (direct link) in Ogi's latest journal entry, in hopes that people commenting there may see it as well. Though it's kinda lost under the FAIL MACROS!

(Also, Ogi is editing that post under people's feet, and who knows how long he'll leave it up...)

Edited at 2009-09-02 05:41 pm (UTC)

Ah well. Kind wish you hadn't done that - I'd rather not get that level of exposure, you know? I'm really only posting at all because this sort of information is meaningless without a source. *shakes head* But, everything I say here is true.

I'll be reposting later today. You = rule.

reposted on my journal, with attribution.

thank you!

(I sent an emamil to the heads of IRB -- the poor guys must have had a heck of a day yesterday!)

Yeah. Their exact words "I had a feeling it would be about that."

As someone who just put her own Master's project proposal through the IRB gauntlet... i THANK YOU for contacting the IRB there about this.

I am late to this (Grad School... it takes over your life! ::grin::) but was disgusted by the level of shoddy science/survey work i was seeing just in PASSING.

There's a reason why marginalized communities look at academic researchers with (understandable) suspicion - people like THIS! People who have little understanding of the community they're (supposedly) studying, talk DOWN to members of the community, and wish to use the community for their own ends - regardless of the impact *on* the community. ::Sigh::

So, on behalf of one of those academics following the rules... thank you.

peace,
the kendermouse (who HOPES this is at least half-way coherent)

Yeah, this. Seriously.

Has anyone contacted Penguin about this? He has been rather obnoxious in his correspondence. The publisher might be interested to know his credentials are questionable, let alone his research methodology.

(Anonymous)

Depends how he's talked himself up to Dutton.

Dutton isn't a scientific publisher. They might welcome publicity for this new book, no matter of what kind.

But polite letters pointing out:
- that *at no point* should the "research" done for this project be in any way described as done with the support of Boston University,
- that the researchers managed to offend the group being researched so badly that their survey had to be withdrawn after 3 days, while subjects of the survey were deliberately going back and changing their answers to be misleading/confusing (and other subjects were running a Greasemonkey script to clear their answers out of the survey);
- and that the "researchers" failed to use any ethics whatsoever in asking a group that included minors intrusive sexual questions,

those three points might make Dutton/Penguin Books think about whether they can get out of their book contract. Especially, the last.

(no subject) - (Anonymous)   Expand  
(Anonymous)

Updated my post at FeministSF with this information, attribution, and link.
http://blogs.feministsf.net/?p=1259

signal-boosted with attribution. Thank you so much.

Is it just me or did they just flocked their lj?

Oh they did. See ETA 3. Waiting for the sock puppet commences.

Is that why we can't see them? F-lock? Bastard. I was about to post someone the Fandom/Ogi pegging-sans-lube porn they'd requested. >:D

Has anyone mentioned this study to Marc Abrams, over at Improbable Research? It sounds like a runner-up for next year's Ig Nobel awards.

I strongly second this. They certainly do deserve an IgNobel prize for Bad Science.

lol, beat me to it! I also have screencaps of the various lj posts. http://www.mediafire.com/fail_machine

Thanks for posting this and the link to the fandom history wiki on SurveyFail. I was able to access Ogi's post of earlier today that by the time I signed on was down. Glad you called the university and found out this information.

We have closed ranks together and will not permit the uninformed to use us.

Page 1 of 2[1][2]