Ch6 04: You Don’t Need to Read Code to Pick the Right CTO#
Here’s a fear that paralyzes non-technical founders: “I can’t evaluate a technical co-founder because I don’t understand technology.” They whisper it like a confession at the border without a passport.
Let me cut through that right now: you don’t need to understand code to pick a CTO. You don’t need to know Python from Rust, microservices from monoliths, or SQL from NoSQL. What you need is the right questions and the ability to read evidence.
Evaluating someone in a domain you don’t understand isn’t a technology problem. It’s a judgment problem. And judgment has a method.
The Behavioral Verification Method#
Most people evaluate others through self-assessment. “How good are you at backend development?” “Rate your leadership on a scale of 1 to 10.” “What are your strengths?”
This is worthless. Research on self-assessment bias (Dunning-Kruger, 1999) shows everyone rates themselves above average. Self-assessment is a mirror with a built-in Instagram filter — it only shows what the person wants you to see.
The alternative: behavioral verification. Instead of asking what people can do, ask what they did. Use the STAR framework — Situation, Task, Action, Result.
Situation. What was the context? “Tell me about a time your team faced a critical technical deadline.” You want specifics — a real project, real company, real timeframe. Vague answers (“we often had tight deadlines”) are a red flag. People who did the work remember the details.
Task. What was their specific responsibility? Not the team’s — theirs. “What were you personally responsible for delivering?” This separates leaders from passengers. Genuine owners describe their scope with precision. Passengers blur their contribution into the team’s achievement.
Action. What did they actually do? “Walk me through your actions, step by step.” This is the gold mine. Push for detail. When someone says “I architected the system,” follow up: “What were the key design decisions? Why that approach over alternatives? What tradeoffs did you accept?” People who truly did the work can go three or four levels deep. Those who didn’t get vague or defensive after level one.
Result. What happened? “What was the outcome? How did you measure success?” Concrete results matter. “The system handled 10x the traffic” is useful. “It went well” is not. And the crucial follow-up: “What would you do differently?” People who owned the outcome have thought about this. Those who didn’t will give you a rehearsed line about “better communication.”
The STAR method works because behavior is the best predictor of future behavior — not credentials, not self-assessment, not references. What someone did in a specific situation, under real pressure, with real stakes — that’s your data.
Output Verification: Trust but Check#
Behavioral interviews reveal how someone thinks. But you also need to verify their actual output — even though you can’t evaluate it yourself.
Three approaches, ranked by reliability:
Look at their work. Ask to see things they’ve built — open-source contributions, side projects, shipped products. You don’t need to understand the code. You need to see that code exists and that they can point to concrete things they’ve created. A CTO candidate with nothing public isn’t automatically disqualified, but it’s a data gap you must fill through other means.
Get a third-party evaluation. Find someone technical you trust and ask them to review the candidate’s work. A friend, advisor, or consultant you pay for two hours. Give them a specific question: “Based on this code/product/architecture, would you trust this person to lead a technical team at a startup?” A competent evaluator can give you a clear signal in an afternoon.
Run a paid trial project. Before committing to a co-founder relationship, work together on a small, defined project — two weeks, clear scope, concrete deliverable. This is the best data you’ll ever get. You see how they communicate, handle ambiguity, make tradeoffs, and what they actually produce. A bad CTO hire costs months. A two-week trial costs two weeks.
Behavioral interviews plus output verification give you a solid judgment foundation — even in a domain you don’t understand.
The Values Alignment Check#
Technical skill is necessary but not sufficient. The most common reason CTO relationships explode isn’t competence — it’s values mismatch.
Three dimensions where misalignment kills:
Speed vs. quality. You want to ship an MVP in six weeks. Your CTO wants a “proper” architecture that takes six months. Neither is wrong — but in a startup, speed almost always wins. If your CTO can’t stomach shipping something imperfect, you’ll burn your runway building a cathedral nobody visits.
How to test: Ask about a time they shipped something they weren’t proud of. If they can’t think of one, or the story is packed with justifications about why they insisted on quality — proceed with caution.
Autonomy vs. direction. Some technical leaders want clear specifications and well-defined tasks. Others want to own the problem space and figure out solutions themselves. In a startup, you need the second type. Ask: “If I give you a problem and no specification, what do you do?” The right answer involves talking to users and making decisions. The wrong answer involves asking you for a document.
Long-term vs. short-term. Some people optimize for the next quarter. Others for the next five years. Startups need someone who toggles between both — build for today, design for tomorrow. Pure short-term thinking produces technical debt that kills you in year two. Pure long-term thinking produces nothing usable in year one. If they have a rigid philosophy in either direction, they’ll struggle with the constant context-switching a startup demands.
The Pitfalls of Cross-Domain Evaluation#
Even with solid methods, cross-domain evaluation has traps.
The confidence bias. Confident people sound competent, especially in domains you don’t understand. A CTO candidate who speaks with authority about “scalable distributed systems” and “event-driven architecture” can sound like a genius to a non-technical founder — even if they’re stringing buzzwords together. Counter this by always pushing for specifics. Confident generalities are cheap. Specific details about real decisions in real projects — those are expensive to fake.
The halo effect. Big-name companies on a resume create a glow that blinds. “She worked at Google for five years” tells you almost nothing about what she personally accomplished, how she handles startup chaos, or whether she can build from zero. Google has 180,000 employees. Some are extraordinary. Some are average people in an extraordinary system. The resume shows where someone worked, not what they did.
The technical intimidation trap. Some candidates deliberately use jargon to create information asymmetry. If you don’t understand something, ask them to explain in plain language. If they can’t — or won’t — that’s a signal. The best technical leaders explain complex concepts simply. The mediocre ones hide behind complexity.
The “I’ll learn it later” trap. You tell yourself you’ll eventually understand enough tech to evaluate your CTO properly. You won’t — not because you’re not smart enough, but because you have a hundred other things to do. Build your evaluation system now, while making the decision. Don’t defer verification to some imaginary future where you have more time.
The Team Audit#
Here’s your exercise — applicable to every key team member, not just your CTO.
Pick anyone on your team whose domain you don’t deeply understand. Your head of marketing, lead designer, sales director. Run the STAR method on their track record since joining.
Take their most important project:
- Situation: What was the context when they started?
- Task: What were they specifically responsible for?
- Action: What did they personally do? (Not what the team did.)
- Result: What was the measurable outcome?
Compare their actions to their claimed capabilities. Do they match? Is there a gap between what they say they can do and what they’ve actually done?
If you find a mismatch — someone who talks about “leading” but whose actions show “participating,” someone who claims “strategic thinking” but whose results show “tactical execution” — you have a calibration problem. You’ve been operating with inaccurate data about your team’s real capabilities.
Do this for every key role. It takes an afternoon. The clarity it produces is worth weeks of guesswork.
You don’t need to become a technical expert to lead a technical team. You don’t need to become a marketing expert to evaluate your head of marketing. You need a method for extracting real evidence from real behavior — and the discipline to use it even when the answers are uncomfortable.
The method exists. The question is whether you’ll use it or keep relying on gut feel and impressive resumes.
Your gut has its uses. But it shouldn’t be your primary instrument for the most consequential hiring decision you’ll ever make.