Doomed Relationships: Why you should not fall in love with your model

Back when I first started in my lab, I started with a continuation of a former grad student’s project.  I hadn’t yet gained the maturity to come up with my own experiments and I was sort of blithely moving along, doing experiments that my advisor suggested.  These experiments were testing a particular model of how we (and by “we” I mean my advisor, really, because I hadn’t put that much thought into it) thought our system functioned.  It was a tough experiment for various reasons and it took a long time before I actually had results (1.5 years), but when I got the results they did not fit the model.  I was crushed.  I thought there might be something wrong with my results.  Then, I desperately tried to make the data fit the model.

My advisor, however, was not too bothered by my results and said we simply needed to revise the model.  I did not want to revise the model.  I liked the model.  The model explained everything I thought to be true but hadn’t proved yet.  And now, I had this bit of data that was fucking it up.

Eventually, I learned to let go of that model and trust my data.  Since that time, we’ve been through two or three revisions of the model and when I’m gone I’m sure the lab will go through many, many more revisions of the model.  My attitude towards models has changed so much that I’m actually excited when I get new data that blows apart the previous model because I know that I’ve made a big leap forward and I’ve really contributed something worthwhile to the field.  It’s also scary, though, because I’m trying to wrap things up and publish and graduate and I can’t do that if I’ve just discovered the way I’ve been approaching your problem is wrong and I’m going to have to do ten more experiments.  Sometimes, when a labmate proposes an experiment to me in lab meeting, and I argue against it, I have to ask myself why I’m so against doing the experiment.  Often, it’s because I’m afraid that the result won’t be what I want it to be and my model will be in big trouble.  And those are the experiments I know that I have to do.

Last week, at lab meeting, I had one of those moments which I later discussed with my advisor.  And he agreed that if a possible result from an experiment scares the crap out of me, then it’s an experiment I should do.  This is maybe the best thing that came out of lab meeting.

At that same lab meeting another student, Problem Child, presented.  She has a model for a different system.  Like any good cell biologist knows, a good way to learn about a system is to try to screw it up.  So, Problem knocked out a gene that she thought should have a significant effect on the system.  It took her a very long time to get this knock-out strain of yeast.  Then, she looked at her cells via immunofluorescence.  If she had significantly screwed up this pathway, it would be obvious by IF.  But, there was nothing obvious there.  She thought maybe, if you squinted, and turned your head so you were looking out of the corner of your eye while jumping up and down on one foot, there was a phenotype.  So, she looked by EM and did some quantitation, but there was nothing there (although she has yet to show us those results in lab meeting, she has told me there is no discernible phenotype).  THEN, she did live cell imaging and took multiple movies of her strain which is what she showed us at lab meeting.

My friends, there was nothing in them worth noticing.

We said as much.  And, at the same meeting, she talked about other ways she could screw up the system.  We thought those were really fine ways and actually much more promising.  But, she wants to continue with this knockout strain.

So, I asked her, “Let’s say, in the best possible scenario, there is a very subtle phenotype in this strain.  What does that tell you?  Aren’t you looking for something much more dramatic?  Wouldn’t it be a better use of your time to pursue these other possibilities?”  Advisor agreed.  Everyone else in lab agreed.  But, she still thought it was worth making more movies and taking more EM images.  “In my spare time,” she said, “while waiting for the other strains to grow up.”  But, she doesn’t spend that many hours in lab and I know that she’s spending a ridiculous amount of time making movies and so on because I have seen her analyzing them.

So, after lab meeting, in a private conversation, I suggested to her that it seemed as though there were two ways she could take her project.  One was a characterization of the protein for the gene she knocked out.  Nobody else has done much work with it.  There are a number of things she could do with it that would be publishable and she could even publish some of the data she has collected.  OR, she could go full force trying to get the phenotype she wants by knocking out her other candidates one by one or in combinations and spend 6 months doing that and if she’s able to get her phenotype, then great.  From what I can tell, this is what Advisor is telling her she should do.

She insists that she really is working toward making those knock-outs but she really believes there is something in this first knock-out strain.  She will not be persuaded–not by her labmates, not by her advisor, and not by her data.  She is in love with her model and she is bound and determined to make her data fit it.  This is dangerous for many reasons.  First, and foremost, I think this sort of blindness to what the data are telling you is how people end up falsifying data.  If I were Advisor, I would demand to always see her raw data and not just her analysis of it (which I believe he does; Advisor loves looking at raw data–it’s what he lives for).  Second, this devotion to a strain with no phenotype is not getting her anywhere.  It’s wasting lab resources and it’s wasting her time.  She is never going to graduate if she doesn’t move forward.

It reminds of talking to a teenager, trying to convince her that the guy she is dating is bad for her.  No matter what you say, she is going to remain with this guy.  Emotion has blinded her to the truth.  Parental guidance holds no sway and she will sneak out in the middle of the night to be with her guy.  Likewise, Problem Child ignores all advice and comes in–literally in the middle of the night–to work on this strain that she insists proves her model.  And there is nothing to do but to sit and watch it happen.

Advertisements

8 thoughts on “Doomed Relationships: Why you should not fall in love with your model

  1. Excellent post! I think we’ve all been in that situation where we all want so badly for our model to be right. It sounds like you’ve done all you can for Problem Child. This might just be something she has to learn for herself.

  2. Great post!
    The point about checking the raw data and falsification is a good point. I never thought so much about model love before but I’ll try to keep that in mind from now on

  3. I totally got to use “that’s what happens when you fall in love with a model’ in journal club right after you published this.

    Took everyone by surprise. It was essentially a published paper that did a ridiculous amount of work to prove one small insignificant piece of data: it had like 200 monkeys in the study!

Comments are closed.