Page 1 of 1

#Article: A comparative study of Q/Q Research

Posted: Fri May 20, 2011 7:44 am
by Boyd Baumgartner
http://www.wired.com/wiredscience/2011/ ... e-testing/


Image

With several companies on the verge of mass-marketing genetic tests that claim to read aging’s cellular clock, some researchers say the science isn’t yet ready for prime-time use.

The tests measure telomeres, or protein sheaths that prevent the tips of chromosomes from fraying. As time passes, they grow shorter, a process hastened by stress, environmental insult and disease. If they get too short, cell breakdown follows. A large body of research links telomere deterioration to deteriorating health.

That makes telomeres an alluring target for quantifying the ravages of aging, which have proved surprisingly difficult to measure. But the clinical use of telomeres has yet to be determined.

“Aging is extremely complex. It’s going to involve many behavioral and genetic factors,” said Boston University gerontologist Thomas Perls, who runs the world’s largest study of centenarian health. “It’s going to be unwise to try and pin it down on one particular marker. There are going to be many different factors.”
Telomere tests came to public notice this week after an article in The Independent described a test being developed by Life Length, a Madrid startup company. In the article, University of Texas Southwestern geneticist and Life Length consultant Jerry Shay extolled the test’s potential.

“Telomere length is actually a pretty good representation of your biological age,” he said. “People might say ‘If I know I’m going to die in 10 years, I’ll spend all my money now.’ Or, ‘If I’m going to live for 40 more years I’ll be more conservative in my lifestyle.’”

Shay’s comments seemed to verge on touting Life Length’s $700 test as a fairly precise predictor of longevity, something researchers generally agree isn’t now possible. The company officially describes the test as a general measure of health. That description is in keeping with a $290 test offered by Houston’s SpectraCell Laboratories and another in development by the California firm Telome Health.

The latter company was co-founded by Elizabeth Blackburn, one of three researchers who in 2009 shared a Nobel Prize for their work on telomeres. But whereas Life Length plans to offer its test directly to the general public, SpectraCell requires that tests be provided by physicians on behalf of their patients. That requirement that will be shared by Telome Health, who currently offer tests only to researchers.

“We do have a plan to address consumers through their physicians, but where we see the largest long-term value is with properly validated clinical studies in the area of case management with severe illnesses,” said Telome Health chief scientific officer and co-founder Calvin Harley. According to Harley, telomere length can’t predict longevity, but “there is a risk factor associated with having shorter telomeres. People should be able to utilize telomere length into their personal health management through their doctors.”

Yet for Johns Hopkins University geneticist Carol Greider, who also shared that 2009 Nobel prize, such mainstream testing is premature.

Except for 1 percent people whose telomeres are spectacularly short, and are thus likely to have family histories of telomere-linked diseases like pulmonary fibrosis and bone marrow failure, “We don’t know for any individual what this would mean,” said Greider. “There are no clinical studies that tell us anything about the other 99 percent.”

The science is indeed complicated, if not downright murky. Whereas tests in cell cultures, animals and those 1 percent of human outliers have established the importance of telomeres, population-level effects are hard to define. Some researchers still say the general link to longevity is “equivocal.” Even when telomeres are linked to health, the results can be complicated.

Image
A graph of the relationship between telomere length and age in 1,122 women. Lancet

“The question is, how much can you tell people right now? And with what confidence?” Greider said, calling attention to a 2005 Lancet study on telomeres, obesity and cigarette smoking in women (.pdf). On average, telomere length and age were linked in the study’s 1,122 subjects — but on a person-by-person basis, the numbers were widely scattered (see graph at right.)

“Look at someone who is 40 years old. Your telomere length can be at 5 kilobases or 9 kilobases,” said Greider. “In an epidemiological study like this, when you’re looking at a whole bunch of people, you conclude that shorter telomeres are bad. But that doesn’t mean you can take an individual and tell them their risk.”

For now, the best indicator of longevity and aging comes from not from telomeres or any other genetic measure, but a common-sense mix of family history and lifestyle, said Perls. He offers an online questionnaire for people interested in evaluating their health. (According to Perls, the service is non-commercial, with advertisements defraying server costs.)

“If people want a prediction, it’s their health-related behaviors that are going to be by far the most important and powerful predictor,” he said. “And they need to pay attention to that. You can’t pick your parents, but you can pick whether you smoke, exercise and heave a healthy diet. It’s great to get markers, but it’s much more powerful if predictions are based on something you can affect.”

Image: Telomeres (illuminated with fluorescent tags) at the tips of chromosomes (NIH).

Re: #Article: A comparative study of Q/Q Research

Posted: Fri May 20, 2011 7:48 am
by Tazman
Also see "Hayflick Limit."

Re: #Article: A comparative study of Q/Q Research

Posted: Fri May 20, 2011 7:51 am
by Boyd Baumgartner
I find this article to parallel the sentiments in our discipline regarding statistics.

This quote summed it up for me:
‘When you’re looking at a whole bunch of people, you conclude that shorter telomeres are bad. But that doesn’t mean you can take an individual and tell them their risk.’
Same sentiment holds true in my estimations for likelihood ratios. Great, likelihood ratios tell you something, but that something is disconnected from the case at hand. The variables are too complex to be winnowed down into counting points, and put in a box plot based upon someone elses judgement.

In this study the ultimate findings were:
For now, the best indicator of longevity and aging comes from not from telomeres or any other genetic measure, but a common-sense mix of family history and lifestyle, said Perls. He offers an online questionnaire for people interested in evaluating their health.
So, the findings were that qualitative indicators ultimately trumped the quantitative estimations.

Re: #Article: A comparative study of Q/Q Research

Posted: Sun May 22, 2011 11:07 pm
by g.
Fascinating take on the article as it applies to FRS/Q&Q...

I got that the researchers found predicting life expectancy is not as simple as a single genetically determined variable...but rather a complex set of factors which take precedence (environment/nurture over nature). These factors being life style choices, healthy habits-exercise etc.

Presumably, they know that many of these other more imporant factors "trump" this one single telemore factor from tons and tons of data, research, and statistics from medical studies, CDC, and lots of government funded research. In other words, tons of statistics showing that one who eats lots of crap food, doesn't exercises, smokes, drinks, and makes poor lifestyle choices is more likely to die earlier than a healthy person. Not always...but on average, more likely. In other words, don't "bet on living long" with that lifestyle (no matter how good your genes are to start off). Sounds like a practical way to use statistics to me.

In any event, why not conduct a FRS examiner study comparing purely quantitative approaches against: 1. purely qualitative approaches and 2. a mix of Q/Q. Easy to criticize until you do a study yourself.

g.

Re: #Article: A comparative study of Q/Q Research

Posted: Mon May 23, 2011 7:22 am
by Boyd Baumgartner
Easy to criticize until you do a study yourself.
This statement seems to imply a self defense on your part, Glenn.

Wasn't it you who extolled the virtues of Popper in the Fingerprint Sourcebook, Chapter 14, "Scientific Research Supporting the Foundations of Friction Ridge Examinations "

http://ncjrs.gov/pdffiles1/nij/225334.pdf
Sir Karl Popper (1902–1994) recognized the difficulty of defining science. Popper, perhaps one of the most respect-ed and widely known philosophers of science, separated science from nonscience with one simple principle: falsifiability.
Regardless of the fact that there is a whole body of Philosophy of Science literature out there that would disagree with this sentiment, (Google 'The Grue Paradox' or read Imre Lakatos or Paul Feyerabend), Popper's brand of Philsophy of Science is called 'Critical Rationalism', which according to Wikipedia (yeah yeah, I know, it's just due to time constraints) is characterized as:
Critical rationalists hold that scientific theories, and any other claims to knowledge, can and should be rationally criticized...
http://en.wikipedia.org/wiki/Critical_rationalism

So, instead of playing a semantics game, lets just say that rational criticizing can be called scrutiny and move forward. If scrutiny is a virtue by your own account, why is it being likened to the old guys who sit in the balcony on the Muppets?

If we can't scrutinize studies in our own discipline, we've really moved no further past the claims of absolutism that have plagued our history in recent past. It just becomes a fallacious appeal to the authority of the credentials of our researchers or to the sub-discipline in which they're enshrined. (aka Scientism).

Re: #Article: A comparative study of Q/Q Research

Posted: Mon May 23, 2011 3:38 pm
by g.
=) Fair point. I walked into that one...

So do nothing. Fine.

g.

Re: #Article: A comparative study of Q/Q Research

Posted: Tue May 24, 2011 12:52 pm
by Neville
I know a guy who is a Fingerprint Expert here in NZ who reads palms, I am sure he could give an equally accurate indication of your life span for a lot less dollars. He may even throw in some other information as a bonus. And he has a science degree so I am sure it must be very scientific.

Re: #Article: A comparative study of Q/Q Research

Posted: Wed May 25, 2011 12:09 pm
by Michele
So do nothing
This is going to be part of the presentation Boyd and I are giving at the IAI Conference.

It’s easy to categorize people into two groups, those who do research and those who do nothing, but there’s another group that should be highly valued, those who scrutinize research (or the lack of research). This could also apply to basic assumptions/ideas and conclusions in everyday casework. Looking at ideas critically should be done by everyone, it should be a basic expectation in a scientific culture.

Here’s the value of those who scrutinize. If I conduct a study (or arrive at a conclusion) then Boyd has three options:
a) Look at enough of it to confirm the conclusion (confirmation)
b) Re-do it himself (reproducibility)
c) Look at it critically and question every aspect of it (scrutiny)

Even if my conclusion is correct, Boyd’s scrutiny may improve my conclusion, the articulation of my conclusion, or may improve the support behind my conclusion. Accepting the results or reproducing the conclusion gives weight to the research (or conclusion) but rigorous scrutiny can improve the results or the supporting data. If I want my work to be the best possible, then I should want Boyd to use option c.

There are several reasons people conduct research or arrive at conclusion. If we are trying to arrive at the best possible conclusion then we should value scrutiny of our work as a way to improve it. It’s a way of checking that we are as thorough as possible.

Not long ago we had an examiner in our office preparing for testimony. He prepared his answers and then asked other examiners, “Where would they attack this answer? What comebacks should I prepare for?” He didn’t just ask them what they would say (reproducibility) or ask them if his answer was good enough (confirmation), he was looking at anything possible that would improve his answers. If our work holds up to the most rigorous scrutiny then we should be happy that that we made our best effort.

In a previous post, people asked what kind of cases agencies verified. In our agency, we verify everything but it’s not as simple as that. We don’t use verification as confirmation or reproducing a conclusion. We use verification as a full case review. The verifier is being asked to find anything that would improve the case. One of our examiners always says, “if I didn’t find something to improve then I wasn’t doing my job”.

I’ve seen many people say that human errors can’t be avoided. Perhaps they can never be stopped but they can be diminished if certain QA measures are used. Skepticism and scrutiny are QA measures above confirmation and reproducibility.
Easy to criticize until you do a study yourself.
Maybe the culture needs to change. Maybe scrutiny should be valued instead of taken as a criticism?

Re: #Article: A comparative study of Q/Q Research

Posted: Wed May 25, 2011 3:54 pm
by g.
Again, I stand corrected. Please, by all means, continue to "scrutinize" away...

And let me offer the following suggestions:

1. Why not take this well-meaning scrutiny and publish it in a journal article? If the concerns are valid, then publish them in a professional journal where hopefully, there is a decent vetting process. I might suggest even taking them to the journals where the statistical models/error rate studies are being published. Seems to me that 'in science' that's a good place to lay out professional criticisms and 'scrutiny'.

2. I might also consider the scrutiny to be helpful if it comes from a place of knowledge on the topic. Attending courses, conferences, workshops, reading the papers, understanding the terms, etc. helps me to be adequately informed on the topic to enter a discussion and accept the scrutiny from others who have done the same. Don't "blame the trainee" for not providing an adequate summary of the research (see previous Twitter Length Rant thread).

3. Offer solutions. I'm mighty tired of lots of 'scrutiny' without offering DEMONSTRABLY effective alternatives. (by demonstrable, I prefer data, usually resulting from research, but accept there are other forms of informative evidence). For example, some previous threads questioned the validity and appropriateness of the army crime lab's procedures. That's pretty bold. Show me that one agency's policy is better than another agency's with data. I might buy the argument then. I might adopt the alternative when I look at the "risk/cost/benefit" data.

I know where you are coming from Michele. I do. I am no stranger to scrutiny and critique. Having published and worked on a lot of research in the last 10 years, I am used to scrutiny. It does improve the process but only when it comes from an informed source with helpful suggestions to improve the work.

g.

Re: #Article: A comparative study of Q/Q Research

Posted: Wed May 25, 2011 6:51 pm
by Michele
There are lots of papers written on the limitations of scrutiny (or peer review) in scientific journals. It can work but doesn’t guarantee good research or research.

Opening scrutiny to all does mean that you’ll have to listen and respond to a lot of crap but limiting scrutiny to a certain group could mean that real concerns goes unheard.

All I’m saying is that if people really want improvement, sometimes asking the right questions to the right people can help.

Have you ever heard the saying that an expert is someone that is 50 miles away? Its meaning is that sometimes the answers are right under our noses but we discount the value of those close at hand (maybe our own employees) and value the opinions of those outside of the situation. It’s my opinion that the value should be in the information, not in who says it, their credentials, if it’s published and where it’s published.

Re: #Article: A comparative study of Q/Q Research

Posted: Mon May 30, 2011 3:48 pm
by Neville
Hi Boyd

I was surprised to see this Telomere article being discussed on one of our National TV programmes last night in prime time. Some guy in the USA is starting a company selling this technology to the public so the reporters went to two professors at two NZ universities asking their opinions on the issues of selling this to the public. The one from Auckland University appeared to think it was not accurate enough for making life decisions on; he was some kind of expert in this field. The other was From the University of Dunedin and he was head of some medical ethics community for medical practitioners his opinion appeared to be less black and white he felt that any information so gained could be use in conjunction other information to form an opinion........
the link is
http://tvnz.co.nz/close-up/predicting-y ... eo-4199530