Why I Hire for Depth (as a VP of Engineering).
And why I don't like coding tests.
In a nutshell
During interviews, I meticulously assess how deeply a candidate can understand and discuss their technical decisions.
It can range from "because my tech lead said so" to a nuanced understanding of possible alternatives, their tradeoffs, and the consequences of the choice.
I prioritize depth over direct skills for several reasons, which I discuss in the post.
I don't like typical coding tests because they don't reveal depth.
How I interview
My interviews start with an open-ended conversation about the candidate's experience, interests, and preferences. This usually reveals areas they are passionate about, have extensive experience in, or where they have had to make significant decisions.
And then I drill deep.
Suppose a candidate mentioned they chose MongoDB as the data store for a new project.
I'll then dig:
What led you to choose a document-oriented database over other types like relational or key-value?
What led you to choose MongoDB over other document-oriented databases?
Is MongoDB your default preference, or was it the best fit for this particular project? Why?
What tradeoffs were involved? What did you have to sacrifice? Why do you believe the benefits outweigh these drawbacks?
Why were these specific benefits important for the project?
Could they have been achieved through other means, not involving MongoDB?
How did you mitigate the drawbacks?
Would it make sense to use multiple data stores? What would it solve? What would it complicate?
Is this solution short-term or future-proof? How will it scale with the load? How will it scale with increasing load, team size, and data complexity?
And so on... You get the gist.
And then, based on the candidate's responses, I'll probe even deeper.
For example, if the candidate mentioned choosing MongoDB over other document-oriented databases because it is easier to self-host, I might explore:
Why do you prefer a self-hosted solution over a managed cloud service?
What are the risks and benefits of self-hosting?
How have you addressed these risks?
And so forth...
While interview time is limited and it's not possible to explore every angle, my goal is to assess depth rather than breadth. Therefore, it's ok to select just a few paths and gauge how deeply the candidate can delve into them instead of trying to cover as many paths as possible.
This approach effectively reveals the depth at which a candidate operates.
The 5 levels of depth
Depth is a continuous spectrum, but to illustrate how different candidates operate, it's convenient to group it into five broad levels:
Knows just one way. The candidate lacks personal opinions and is unaware of alternative approaches. Asked about their choices, they typically respond "Because we used this in my previous company" or "Because my tech lead recommended it".
Blindly follows best practices. The candidate keeps up with industry news and is aware of multiple alternatives. However, they follow the best practices blindly. Asked about their choices, they typically respond "Because such and such book or blog said so" or "Because Google and Netflix do this". However, they struggle to discuss the tradeoffs, especially the ones specific to their project.
Makes own decisions, but shallow. The candidate knows the alternatives and makes intentional choices among them. Their reasoning, however, is shallow and they typically talk in generic terms. For example, they might say they chose MongoDB because document-oriented databases are more flexible than relational ones, but they won't be able to discuss the tradeoffs, limitations, edge cases, or potential issues.
Makes deep and nuanced decisions. The candidate not only knows the alternatives but understands their strengths, limitations, ideal use cases, and boundaries. They make well-informed choices with comprehensive consideration of all pros and cons, tradeoffs, and consequences—and know how to mitigate the drawbacks.
Understands also the full context, and second- and third-order effects. The candidate grasps all nuances of their choices, including the origins of the technology and the problems it was designed to solve—and how well they translate to the particular project's context. They consider not only the direct impact of their choices but also the potential long-term effects on the codebase, team, processes, and subsequent decisions.
From a seasoned senior developer, I expect at least a Level 4 discussion.
It’s important to note: I don't expect such depth on every topic.
Some topics are straightforward. If a candidate prefers function brackets on the same line rather than a new line, saying that they find it more readable or aesthetically pleasing is more than enough.
Some topics are well-established. I don't expect a candidate to know the entire history of HTML or to provide an in-depth explanation of why it is used in web applications.
Finally, I don't expect a candidate to know in-depth and be opinionated about every technology ever made. If a candidate says their team chose MongoDB but they weren’t involved in the decision because they lack interest in databases—fine with me. The greatest strength of an outstanding developer lies in their ability to effectively research and learn as needed, not being a know-it-all.
However, if a candidate claims to be a proponent of a specific flavor of microservices, and has influenced their team to adopt it—then we'll go Mariana Trench-deep on this topic.
And while I don't expect equal depth in every area, a senior developer should be experienced or interested that deep in at least a couple.
The quality of the discussion matters, too
An important side note: As a professional developer, you'll be involved not only in making decisions—but also in explaining, discussing, selling, evangelizing, documenting, and arguing about them.
That's why not only the quality of the decision (and the considerations that went into it) matters. The quality, flow, and clarity of the discussion itself are equally important.
Why is depth so important?
Depth is strongly correlated with several desirable traits in a developer:
Better decision-making. Their choices are more thought through, better tailored to a specific use case, well balanced, and consider all the subtle but critical details.
Safer decisions. They think not only about the here and now but understand second- and third-order effects and the long-term consequences of their choices.
Resistance to hype. Their decisions are based on thorough analysis and merit, not on current fads or some influencer's opinion. They also better understand the long-term risks of getting locked into a wrong choice.
Ability to solve new, unique problems. Which is what software development is all about—and is impossible with only superficial knowledge. A high level of depth proves curiosity, diligence, exploratory skills, and ability to get to the root of the problem.
Positive impact on the whole team. Ability to understand problems deeply—and then explain them simply and clearly—makes them a fantastic mentor. They will expand the team's horizons and bring it to a new level.
Enjoyable to work with. Deeper discussions and more nuanced points of view are more interesting.
No zealotry. A high level of depth and nuance indicates open-mindedness and a lack of bias. This means making the best decisions for the company, not trying to force their way. It also translates to being more open toward their teammates' diverse worldviews, which makes them more pleasant to work with.
Future-proofness. Depth confirms an affinity for learning (which, together with problem-solving, is the second pillar of what software development is about). This makes them more adaptable and better equipped for inevitable change.
Aptitude for working with "business" and customers. Their affinity for depth makes them better at understanding requirements and addressing customer needs. And their communication skills help them discuss and "sell" their ideas to non-technical stakeholders.
Aptitude for teamwork. Their open mind and appreciation for ambiguity and nuance, make them better at considering and reconciling multiple opinions. And their ability to synthesize and clearly explain complex ideas enhances their impact on the team even further.
Bottom line
Depth is a strong indicator of a great developer. That's why it's one of my primary recruitment criteria—and why I probe so hard for it during interviews.
It's also why I don't like coding tests. They showcase the final result but not the research, learning, reasoning, synthesizing, discussing, and negotiating that would go into it in a real-life project. They test ability but not depth.
Under the tight time constraints of an interview, depth can only be assessed through a deep discussion.
P.S. In the next post, I'll share a few techniques for building deep knowledge on a technical topic. Stay tuned!