The New York Times has published another report by Gina Kolata on the Forty Years’ War against cancer. In it you’ll find part of the answer to the question I’ve posed.
"Dr. Barnett Kramer, associate director for disease prevention at the National Institutes of Health, recently discovered a paper that startled him. It was published in the medical journal The Lancet in 1962, about a decade before the war on cancer was announced by President Richard M. Nixon. In it, Dr. D. W. Smithers, then at Royal Marsden Hospital in London, argued that cancer was not a disease caused by a rogue cell that divides and multiplies until it destroys its host. Instead, he said, cancer may be a disorder of cellular organization.
‘Cancer is no more a disease of cells than a traffic jam is a disease of cars,’ Dr. Smithers wrote. ‘A lifetime of study of the internal-combustion engine would not help anyone understand our traffic problems.’
Dr. Kramer said: ‘I only wish I had read this paper early in my career. Here we are, 46 years later, still struggling with issues this author predicted we’d be struggling with.’"
There may be lots of science going on these days but review any journal and you’ll quickly see that there’s often not a lot of deep thinking behind it. Most science is derivative and most of it is false.
What people think of as science nowadays is in fact a vast jobs program, the main purpose of which, after the employment of academics, is to maintain and expand the pardigms on which its various parts rest. Accordingly, research outside the prevailing paradigms is typically starved for cash and efforts to falsify dominant theories don’t just go unrewarded, often they are punished. I suspect therefore that the prayer of many if not most scientists today is, to paraphrase St. Augustine, "Lord grant me critical thinking and skepticism, but not yet."
Reading an old journal article is one way to step back from the minutiae of microarrays and data dredging and to consider big ideas from a time when no one had the ability to sequence the genes of a malignant cell or unleash sophisticated software to find never before noticed confirmatory associations among mountains of numbers. A time when ideas were, perhaps, more likely the spark of sudden insight rather than the product of of a self-replicating system.