The SignalFire take
Today, we’re welcoming Vitaly Gordon, CEO and co-founder of Faros AI, as a guest author on the SignalFire blog to share strategies for measuring engineering productivity.
Ever since Fred Brooks published "The Mythical Man-Month," there's been a debate around the right metrics to apply to software engineering productivity. A long sequence of flawed metrics followed, such as IBM's "KLoCs" (as in thousands of lines of code), which has led to skepticism of metrics in the industry. The reality is that software engineering is a complex endeavor, and these complexities can't be captured in a few neat numbers—they require context. We believe the right way forward includes assembling that context and letting teams use it as input for understanding productivity. That’s why we invested in Faros AI.
Why engineering productivity matters
Engineering is one of the most critical and costly functions for a startup. This hefty investment demands that engineering leaders justify their budgets in quarterly reviews, annual planning, and board meetings. Simply put, they must be able to conduct business-oriented conversations about very technical things—making explicit connections between engineering efforts and business outcomes and providing insights into where resources are needed most.
For small teams, tracking and communicating the output of a handful of engineers may be manageable, but as your team expands, you need more sophisticated systems in place. That’s when startups typically establish engineering productivity programs to create a line of sight for leaders while, in parallel, empowering line managers to make informed decisions that align with company goals.
Context is king, even for metrics
Engineering productivity programs should not be one-size-fits-all. What works for a Series A startup focused on rapid product innovation will differ from a Series C company that is refining its technical strategy to support a larger codebase and growing customer base.
Your engineering productivity program should adapt to your goals, operating model, and company culture. As your startup evolves, so will the metrics that matter most to you.
The SPACE framework is the most comprehensive framework for measuring engineering productivity today. It advocates for a holistic view of productivity that measures multiple dimensions: Satisfaction and well-being, Performance, Activity, Communication and collaboration, and Efficiency and flow.
While very comprehensive, SPACE isn’t prescriptive about what precisely to measure for each dimension—the details are up to the organization. For startups, especially in the early stages, selecting the SPACE metrics that align with your specific goals and challenges is key.
Identify what matters to you
Before selecting the metrics to track, it’s essential to step back and clarify your goals. What does success look like for your startup? How do you define productivity? The metrics you choose should reflect these answers.
Consider these three key elements when selecting what to measure: your goals, operating model, and engineering culture.
Tailoring metrics to your growth stage
Your goals will differ depending on your stage of growth. A startup optimizing for product-market fit will have different priorities than a scaling company balancing growth with stability.
Regardless of size, world-class engineering organizations operate on the foundation of five essential pillars: budgets, talent, productivity, delivery, and outcomes.
The five pillars ensure that operations are efficient, strategic, and aligned with the company’s goals. Each pillar is reinforced by specific recurring processes and cadences that facilitate sustained performance and growth. When these meetings are fueled with high-quality data, decisions are made more quickly and confidently.
The table below highlights common metrics relevant to different stages and their goals. Incorporate these metrics into recurring cadences like monthly operational reviews, product reviews, quarterly business reviews, annual and quarterly planning, and periodic talent reviews.
Common metrics aligned to goals per company stage
Often, change management will be necessary to modify existing meeting protocols and practices to include the presentation and discussion of integrated data. Here are some tips for implementing the transition to data-driven engineering operations.
Tips for transitioning to data-driven engineering operations
Tailoring analysis to your operating model
Is your team remote or hybrid? Do you outsource development? Are you structured with regional hubs or geo-concentrated? Do you have a centralized software development lifecycle (SDLC)? Each of these factors introduces nuances in measuring productivity and analyzing and comparing results. The table below highlights the analysis dimensions per operating model.
Common metrics and analysis dimensions per operating model
Be consistent with your engineering culture
Your company’s values and DNA will influence how you measure productivity. Some organizations are comfortable with individual metrics, while others focus on team-level metrics. The table below provides a few examples of engineering cultures and how they impact common metrics.
Common metrics and analysis dimensions per engineering culture
Conclusion
Having an adaptable engineering productivity program is critical for early-stage startups and those scaling rapidly. By aligning your metrics with your unique context—your goals, how you work, and your culture—you can ensure that you’re not only tracking productivity but also driving meaningful improvements. Discover how Faros AI can accelerate your engineering teams at www.faros.ai.
Faros AI helps companies like Discord, Vimeo, and Coursera improve engineering productivity and optimize the developer experience. Founded by the engineering team that built AI at Salesforce, Faros AI creates instant visibility into productivity, delivery, outcomes, budgets, and talent— customized to your unique context and needs. Powered by the cutting-edge Lighthouse AI engine, our platform delivers real-time insights, uncovers bottlenecks and their root causes, and offers personalized recommendations to boost both velocity and quality.
*Portfolio company founders listed above have not received any compensation for this feedback and may or may not have invested in a SignalFire fund. These founders may or may not serve as Affiliate Advisors, Retained Advisors, or consultants to provide their expertise on a formal or ad hoc basis. They are not employed by SignalFire and do not provide investment advisory services to clients on behalf of SignalFire. Please refer to our disclosures page for additional disclosures.
Related posts
