What Do Experts Really Know? Embracing the Unknown

As published in CFA Institute Enterprising Investor on July 29, 2024.

How confident are you in what you know? In 2002, Donald Rumsfeld, U.S. Secretary of Defense spoke these, now-famous, words during a Pentagon briefing:

“As we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know.”  

Professional experts are assumed to know a lot about their areas of knowledge whether in national security or investing, or something else. But, as Rumsfeld’s comment shows, ‘metaknowledge’ or awareness of the limits of your knowledge is just as important as knowing what you know.

Do professional experts have an edge over non-experts by having higher levels metaknowledge? A new study sought to answer that question by conducting research with experts in the fields of climate science, psychological statistics, and investment. The researchers concluded that experts did tend to have higher metaknowledge than non-experts. For example, they were less overconfident overall but had more conviction in their correct answers than non-experts. However, experts were also more likely to exhibit greater confidence in their wrong answers compared to non-experts.

Previous studies found cognitive biases among finance and medicine experts. For example, economists display overconfidence in their theories despite a long history of incorrect forecasts.

Investment managers, while touting the importance of decision analysis in general, often fail to do so in practice yet maintain strong conviction in their sub-optimal conclusions. Alas, years of experience does not seem to ameliorate these tendencies. Similar patterns have been noted among medical professionals. In one study, physicians’ confidence in a diagnosis remained at 70%, even when they correctly diagnosed difficult cases only 5.8% of the time. Just as misjudgement can harm a medical patient, sub-optimal decision analysis can harm a client’s investment returns.

Given the durability of certain cognitive biases, how can advisors de-risk decision-making by raising their metaknowledge? One way to do this is by leveraging individual investing talents within a structured team environment. This gives an organizational edge.

Organizational edge is not merely about the sum of individual talents but also how these talents are structured, integrated, and leveraged. A well-designed organization optimizes team dynamics, encourages effective communication, and fosters a culture that supports decision-making aligned with its strategic objectives. Having the right environment and processes in place can amplify individual capabilities which are as essential to success as are market strategies.

Bigger is not always better when it comes to investment teams. Having a large research investment team does not guarantee good decision making or sound judgement. In fact, it can add unnecessary complexity and inefficiencies into the investment process. Flatter organizations tend to do better. This may be due to more simplified structures.

Leveraging the insights of research analysts alongside those of portfolio managers is the mark of skilled leadership and a supportive environment. Diverse teams along the lines of education, experience, skills, and information can add value when there are shared goals and open communication. Studies also show that gender balanced investment teams may have an increased potential to achieve superior risk-adjusted returns.

Confidence is a necessary but insufficient factor in long-term investing success. Raising the metaknowledge quotient of the investment team can help protect against the surprises that lurk in left-tail events that remain are unknown—until they’re known!