As a researcher, it is important to me that you know the difference between anecdotes and data (the stuff produced through primary research).
What is Research?
This is a broad question that has many different facets, I realize. And I could bore you with the specifics of randomized controlled trials vs. observational studies, but our purposes here, I consider data/research to be something that has been tested. Many subjects have been involved and conclusions have been drawn. Research is testing whether defaulting workers into contributing to retirement funds actually increases earnings and then finding out that it does.
The conclusions drawn from research mean that they’ve been tried and tested. Many, many people save way more money into retirement plans if you default them into contributing in the first place. What do we do with this information? If you default your contributions, you may not end up clawing them back at all!
What is an Anecdote?
An anecdote is a personal story of something working for someone. An anecdote is me saying that making one simple change greatly impacted our finances.
The problem with anecdotes is that it is just one (or a handful) of personal stories. There was no controlled test environment. Maybe that one change I bragged about wasn’t even the driver of us upping our savings rate. Maybe something else happened that week that subconsciously forced us to save more money, but maybe I didn’t realize it. Anecdotes are at the crux of all the crazy stories you hear. (“My aunt blew up a jug of milk in her yard and hasn’t had a cold since!”)
Anecdotes aren’t all bad. Stories are great. Anecdotes should be used for ideas or inspiration. Trying to jumpstart your finances? An awesome story about how one family paid off $100k in debt in 3 years is motivating, and maybe you’ll get some great ideas as to what might help you. Want to become an entrepreneur? Hearing success stories can help you get motivated to start, and maybe you’ll get ideas on how to get started.
Anecdotes and Research are not interchangeable
Be careful when trying to use anecdotes in the place of research. You cannot tell me the success story of your cousin’s mortgage payoff and assume the same thing will work for me. Finances are relative. That is an anecdote. Maybe it would work, but you don’t have any research to back that up. And it working for two people just becomes two anecdotes.
On the flip side, me telling you the results of a large-scale study on saving money can be informative, but not necessarily motivating. A good plan of action combines research and anecdotes. First, find out what the data says about your situation so you know what things research has shown to help. Second, read success stories of people that have done it. Then you are armed with knowledge and inspiration!
ChooseBetterLife
And even with research, the results are only as good as the study design and the integrity of the researchers. Half of what we practice in medicine is not helpful and sometimes even harmful, but we don’t know which half for years.
I’m skeptical about new drugs and procedures just like I’m skeptical about new investment products and get-rich-quick schemes. Let someone else try them out first, and we’ll let time and experience be the judge.
MaggieBanks
Yes! Time and experience. Preach it sister! (I really wanted to write a 20 page diatribe about study design, sample size, statistical significance, etc etc, but this was the best I could come up with without sounding too angry). 🙂
Full Time Finance
Just remember there are lies, damnable lies, and statistics. In the same way an anecdote doesn’t show the answer, often neither does the data. It’s often all about how you ask the question and the assumptions that led you to the study design in the first place.
MaggieBanks
And correlation is not the same as causation.
Matt @ Optimize Your Life
When I started writing I would based entirely on research. I was frustrated that much of the writing in the personal finance space was anecdotes and I wanted to be research-based. The main problem is that nobody reads anything that is just research-based. I’ve since been working on combining an illustrative anecdote with piles of research so that readers actually care, but I don’t lose any of my accuracy. It is a tough balance to find.
MaggieBanks
Agreed. Those are the best posts. And yes, so time consuming! 🙂
Ms. Frugal Asian Finance
Great post. I’m always interested in an anecdote that illustrates a generalization backed by research.
MaggieBanks
Agreed. Those are the best.