If you’ve ever seen an old toothpaste commercial, then you’ll know exactly where we’re going with this article. For the first part of the commercial, you see all the actors going about their day, flashing big, broad smiles so that viewers can clearly see their gleaming white teeth. At the end of the commercial, there’s an up-close shot of one of the models while the narrator concludes that the toothpaste being advertised is “recommended by nine out of ten dentists.” And then, watching the commercial, you can’t help but think to yourself, “Wow! That toothpaste must really be good if 90% of all dentists recommend it to their patients!”
So, what’s going on here? Well, first of all, the advertisers are using an “appeal to authority.” If anyone can be trusted about the quality of a product meant to care for your teeth, it’s going to be a dentist, right? So, the advertisers want viewers to know that the most authoritative people around are vouching for their product. More importantly, though, is that iconic phrase, “nine out of ten recommend.” Making this claim leaves the impression with viewers that, of all the dentists in the entire profession, only a small minority would not recommend the product. Is this really what’s happening?
Well, we can’t actually speak to the dental care industry specifically, but a lot of companies that use this sort of claim are probably not relying on much of a scientific methodology in their research. Suppose a furniture company is promoting a new patented technology in its recliners. To justify the technology to their customers, they decide to survey 100 chiropractors—and one of the questions they ask is, “Would you recommend our recliners to your customers?” Let’s say that, of the chiropractors sampled, only about 2/3 of them say they would recommend the product. The question for the furniture company then becomes, “Do we throw out the research or improve the technology?”
When they look closer at the responses, though, they start to rationalize. Well, this particular chiropractor isn’t very well-reviewed by his patients and that particular chiropractor may have gotten a bad sample of our recliners. Little by little, they start throwing out surveys from their sample. Of course, it’s only a coincidence that the surveys discarded all happen to be from chiropractors that did not recommend the product. When they’re finished cleaning up the data set, they’re left with 78 surveys, 67 of which recommend the product. As a percentage, that’s 85.9%. Now, all they have to do is round up and they’ve got their proof.
“Nine of ten chiropractors recommend our new, patented technology.”
So, what does all this have to do with investing? We thought you would never ask! When you go to talk with a financial advisor, he/she may offer you a variant of the “nine of ten recommend” claim. They may show you a collection of funds that, by and large, have performed very well as proof that they have special insight into the market. But here’s the thing: you only get to see the winners.
Most mutual fund companies know many, if not most, of their managers, will produce below market returns and only a handful will beat the market through random luck. So what do they do with the funds that fail? They take a closer look at them. Whether or not they do it intentionally, they start finding various justifications for discarding the funds that haven’t performed well enough. They make them disappear by closing them or merging them with more successful funds. The lucky funds that survive are the “nine of ten” that are paraded out in marketing campaigns to lure more investors into the trap.
Here’s a basic guideline for interpreting statistics. Never trust them completely when the people using them to sell you something are also in control of the dataset on which they’re based. Are you tired of being fed cherry-picked information? Do you want someone to just lay all their cards on the table and give it to you straight? Feel free to reach out to us for a complimentary consultation. We have nothing to sell you but the truth. Because our only goal is your success, and we know that you can’t achieve that until you have a clear and accurate picture of what’s in front of you.