How I Read Papers

By Neuroskeptic | January 30, 2011 6:55 pm

Last year I blogged about how I write blog posts. I don’t really have anything to add to that, so here’s some advice on how I read scientific papers – both the ones I read for my day job, and the ones I blog about.


Software:
If you read papers you need PubCrawler. It’s free, and it’s the best thing since PubMed, because it automatically searches PubMed for you and emails you the results. Second, you need a reference manager program. I use EndNote, but there are others, including various free ones. They’re indispensable.

PubCrawler sends you lists of new papers you might want to read. A reference manager lets you to keep track of what you’ve read, and what you need to read in future; it lets you make notes on papers (see below), search them etc. and best of all it lets you insert them into Word or whatever and automatically generates a References list. If you’re not using these tools, you’re making life much harder than it should be.

Deciding What To Read: There are a lot of papers out there. My PubCrawler includes a search term for “antidepressants”, which nets about 10 per day; one for “autism”, about 5 per day; one for various brain regions I’m interested in, up to 50 per day, another for neurotransmitters I’m into, also 50…

So you need a triage system. I mentally put papers into 3 categories, based purely on the titles:

  1. Irrelevant – don’t even click on it. I’d say about 80% of PubCrawler hits fall into this category.
  2. Somewhat interesting – read the abstract. 15%.
  3. Very interesting – read the whole thing. 5%.

Reading papers: Start with the abstract. Then read the Introduction, as it’s usually a pretty good summary of previous work. I’ll skip this only if I know all the existing literature (very rare). Then, head to the first paragraph of the Discussion: this typically contains a summary of the main results in non-technical language.

Finally, I’ll skim the Methods and the Results. If something seems unusual, dodgy, or especially interesting, I’ll go back and read these fully, but most of the time I don’t bother. The remainder of the Discussion is generally just speculation, and rarely worth reading.

All that applies to original experimental articles. For review papers, if I read them at all I read them straight through; a well-written review should all be useful. A bad review is no use at all. If you start reading a review, and by the end of the first page you’re wondering “But what’s the point of all this?”, it’s probably the latter.

Making notes: This is the key to memory, for me at least. If I just read something, I barely remember it the next day let alone next month. Making notes forces you to actually understand it, and then it sticks. I make notes in EndNote for every paper, and even every abstract, I read. Once you get into the swing of it it’s a natural part of reading and doesn’t take much time.

Here’s my notes on one recent paper:

Abstract. NRG1 –> ErbB4 promotes the formation of glutamatergic –> GABA interneuron synapses via stabilizing the PSD-95 at these synapses, but NOT at other synapses i.e. glut –> glut. Therefore, NRG1 contributes to the development of inhibitory signalling. The authors say this is interesting re: SCZ [but I think it’s interesting re: autism as well!]

This makes sense, if you’re me. Actually, though, I rarely ever read these notes. The point is to make them. You could scribble them on toilet paper and flush them once you’re finished and they’d still do their job of boosting your memory.

Here’s an uncensored extract from my notes on a paper I didn’t like:

Less “medication resistant” patients did better [well that’s AWESOME for a treatment that’s meant to be an alternative to meds isn’t it, you fuck]. They admit that the actual performance was crap NNT=12, but say it would be better if concomitant meds allowed [….well yeah either that or the effect would DISAPPEAR] and that it is equivalent to what would be expected if you gave a new drug or augmentation to this population [but you DIDN’T did you, you are referring to the literature, which is shit]. There’s so many conflicts of interest it’s almost tragic.

It deserved it, seriously. My comments are [in brackets], obviously.

Again – when I wrote these, I didn’t expect to ever read them. The point is that by writing down my comments, I forced myself to make them coherent, and hence made myself remember them. This is crucial: if you only remember what the paper said, and not the fact that when you read it, you burst out laughing in disbelief, you’ll go away thinking that the paper must have been fine.

CATEGORIZED UNDER: papers, science
  • Anonymous

    thanks this is really helpful.. may help me to keep up with the reading list a bit better! will definitely get pub crawler and EndNote. Foxit reader is also quite good for highlighting and making notes on papers.

  • Anonymous

    What search terms do you use for pubcrawler?

  • http://culturalneuroscience.wordpress.com/ Ada

    Thanks for sharing this! I was glad to find that I was intuitively doing some things right :) I am now giving PubCrawler a shot, sounds really interesting. Am using EndNote but Papers is a really great alternative for Mac users.

    I find making notes and diagrams (for visualization) very helpful. I also like to make comments and mark words in the pdf directly. I make notes on paper as well, but it often happens that I can't understand my own scribble (too abstract and many abbreviations) two days later :)

  • reasonsformoving

    Ugh, Endnote and the various versions, Refman, (they're all owned by the same company)have so many shortcomings. They're better than nothing, but so is a band-aid when you've been shot in the leg. They often format refs improperly and are just a simple pain to use.

  • http://www.blogger.com/profile/09013133419305271189 Michelle Greene

    Hmm… I actually think that it's easiest to read the introduction last (if at all). Go straight for the results, then the discussion. If the claims from discussion don't really match the results, that's a problem.

  • Anonymous

    If I read this correctly, that means reading somewhere north of fifteen abstracts and two full articles a day, and that's before you get around to grants, articles for review, articles you're working on, world news, etc. I doubt I could even make it through the 100 titles a day. Where do you find the time?

  • http://www.blogger.com/profile/16924185873732166984 orcasha

    Pubcrawler is a god send. Thank you.

  • http://www.blogger.com/profile/05660407099521700995 petrossa

    Great minds think alike :) It's exactly my system for reading papers. I don't use pubcrawler however.To static. Having all the time i want i just do the searching myself as takes my whim of the day.

    I just think you are being kind with the 80% crap. To me it's more like 90% crap, 8% interesting, 2% meaningful.

    There's a distinct trend over the years however. When i first began reading online somewhere in the 80's papers were erudite, looking to enhance knowledge. Scientific as it were.

    Since uni's were forced to look for commercial grant money, papers that 'confirmed' an assumption were becoming more prevalent. Today 'confirmation' papers far outweigh scientific papers.

    Luckily they are easy to spot so you can speedread them.

  • http://gehirniskraemerei.wordpress.com Michael

    Very helpful, thanks :-) People might also want to try Zotero instead of Endnote because it integrates itself very well in Firefox and you can save papers “on the fly” and share a citation-database with others.

  • Wombi

    Thanks for this post! Ill definitely have a try on PubCrawler. As mentioned in other comments above I am amazed about your daily reading dose, maybe PubCrawler will help me to increase mine a little.

    You mentioned there are some free alternatives to EndNote, can you or anyone recommend me one?

  • http://www.blogger.com/profile/14316501618006695088 JoeDuncan

    This is pretty much exactly how I read papers as well.

    The only thing I do differently, is that while skimming the method and results, I make sure that I understand every graph or figure.

    Do the units make sense? Are the axes off or deceptive? Do comparable graphs use the same scale? etc…

    If I can't easily figure out what a figure or graph is trying to say – or what it's purpose is – that's a point against the paper. Figures should only ever make things clearer, if they make things more confusing you know something's not right with the paper.

    @Michelle Greene:
    The point of reading the abstract & intro first is to give you a grounding perspective from which to understand the paper.

    You don't really need to read the results, because they are basically endless recitations of stats and numbers. Any important results are summarized in plain English in the discussion. If some of the discussion summaries don't make sense or raise some questions, then you can go to the results and piece it out.

    Trying to fully read the results/methods sections of most papers tends to bog down your reading so much that you simply can't get enough papers read. If you don't understand a paper after reading the discussion and reviewing the figures – it's a badly written paper.

    The whole point of this technique is to read papers efficiently and maximize throughput – without it I would not have gotten through grad school.

  • http://www.blogger.com/profile/00905981056679713525 SteelWolf

    @Wombi:

    I try to use Zotero myself even though my uni gives out EndNote for free. It's a great reference manager and free to boot.

    I'm still trying to figure out the best system for taking notes on papers.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Ada: Visualizations are extremely important. I'd say that I don't really understand something until I have a mental picture of it – but that's another story. This is the one disadvantage of writing notes in EndNote , you can't do diagrams. if something's really important I'll get out Paint and make a picture of it.

    Wombi: I'm afraid I can't recommend any free alternatives to EndNote as it's the only one I've used.

    As for the number of papers I read, it is a lot, but it doesn't take that long. Abstracts take about 1-2 minutes each. Most papers take 5-10 minutes. Every now and then one will take much longer, if there's something really interesting going on, but that's rare. I reckon I spend maybe 90 minutes a day reading, on average. This is still more than most people, but the payoff is when I come to writing stuff, I write very fast because I've already done all the preperation. plus, writing notes is great writing practice. So overall it all evens out.

  • Anonymous

    Reading a paper in 5-10 minutes is quite impressive, it takes me at least an order of magnitude longer (and I've been at this for 20 years).

    I should add that it's quite helpful to use citation alerts for key articles in your area of interest. I often find them more helpful than pubmed searches.

  • http://www.blogger.com/profile/15825120191635055759 nostalgia

    Yes! this is incredibly useful! Google scholar isn't getting me anywhere LOL!

  • http://www.blogger.com/profile/09558506776197746442 BrianW

    I think reading the entire discussion can potentially be important for a few reasons:

    1) It can give you an idea where other labs in the field (potential collaborators or competitors) are headed.

    2) If the author is a big name in your field, their speculation can be influential, which can be especially important when you're submitting a grant.

    3) If you're new in the field, it can give you ideas for experiments.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    BrianW: That's a good point, but I think it only applies to a small % of papers which are directly related to the research you're doing.

  • Vince

    There is also a program you should check into called I, Librarian which has some neat sharing characteristics for a research group.

    http://www.bioinformatics.org/librarian/

  • http://www.blogger.com/profile/11523279614928219759 Princess Ojiaku

    This is such an awesome post! Thank you for posting. Pubcrawler and Zotero are going to be invaluable to my research and writing!

  • http://www.blogger.com/profile/09517291869386626120 Katy-Ann

    Skim the methods? That's the most important part! If you're going to poke holes in someone's study, that's where to go to try to find an alternative explanation for the resulting data!

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Katy-Ann: Right, but you don't need to do that for every paper. “Background reading” and so forth doesn't usually require it; a skim-read to catch any obvious flaws should be enough. If you're planning on basing a new experiment directly on the methods or results of someone else's paper, of course, you should read it in extreme detail.

  • http://petrossa.wordpress.com/ petrossa

    You struck out on the last paper you twittered about emotions though :) What a mess that was. That qualified as WTF? Still wonder why you thought it worth mentioning, the whole issue of that rag was filled with nonsensical 'studies' and even worse conclusions.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »