Principal Site Reliability Engineer

Ready to be a Titan?

We're looking for a Principal Site Reliability Engineer to join our Site Reliability & Infrastructure Engineering team. We run entirely on the cloud, and this team owns the reliability and health of the systems running on top of it — designing the signals that tell us when something's wrong, and building the systems that keep ServiceTitan running better, faster, and cheaper as we scale.
 

At Principal level, you won't just operate the platform — you'll define how we think about reliability across the entire engineering organization. You'll set the technical direction, establish the standards, and be the person other engineers turn to when the hardest infrastructure problems need to be solved.
 

We make a huge impact on thousands of companies in the U.S. and abroad by enabling them to be more efficient and effective at running their business. We have a cultural foundation built on diversity, inclusion, and innovation, and we want you and your ideas to thrive at ServiceTitan. Come join us.
 

What You'll Do

• Define the technical vision and long-term architecture for ServiceTitan's reliability and infrastructure platform — setting the standard for how we operate at scale.

• Lead the design and delivery of major platform initiatives: Kubernetes infrastructure, observability frameworks, SLO programs, and incident management systems.

• Partner with Engineering Managers, Staff Engineers, and product engineering leadership to review architecture and infrastructure decisions before they ship — and hold the bar on non-functional requirements across the org.

• Identify systemic reliability risks before they become incidents, and drive remediation across teams with org-wide impact.

• Define and operationalize SLIs, SLOs, and error budgets across ServiceTitan's platform — not just within the SRE team, but as a standard every engineering team adopts.

• Drive adoption of reliability and observability best practices across engineering — through documentation, design reviews, and direct partnership with product teams.

• Leverage and champion AI-assisted engineering tools (Claude Code, GitHub Copilot, MCP-based agents) to automate investigations, accelerate incident resolution, and build the next generation of SRE tooling.

• Mentor Staff and Senior SREs, raising the technical ceiling of the team through code reviews, architecture feedback, and direct pairing on complex problems.

• Contribute to the technical hiring bar — participate in system design interviews and help define what Principal-level looks like at ServiceTitan.
   

What You'll Bring

• Kubernetes (must-have): deep, expert-level understanding of Kubernetes as a system — not just operations, but internals, failure modes, capacity planning, and large-scale cluster management.

• SRE principles: proven experience defining and operationalizing SLIs, SLOs, and error budgets across an engineering organization — able to speak to how you drove adoption, not just how you implemented them on your own team.

• Observability at scale: deep expertise with a modern observability stack (OpenTelemetry, Prometheus, Grafana, Datadog, or Elasticsearch) and the ability to design observability frameworks that work org-wide, not just per-service.

• Cloud engineering & networking: expert grounding in AWS, GCP, or Azure — including networking fundamentals, security, and cost optimization at scale.

• Distributed systems depth: able to reason through complex failure modes (retry storms, cascading failures, thundering herds, timeout propagation) and architect systems that degrade gracefully under pressure.

• Incident management: track record of leading major incident response, driving blameless postmortems, and implementing systemic fixes — not just resolving individual incidents.

• CI/CD: deep experience with CI/CD systems and the ability to improve pipeline reliability and safety at org scale.

• AI tooling: hands-on experience building or deploying AI-assisted SRE capabilities — root cause analysis agents, automated investigation tools, or AI-augmented runbooks — in production systems.

• 12+ years of relevant hands-on experience, with a clear track record of technical leadership on complex, cross-team infrastructure initiatives at product companies operating at scale.
  
   

About You

You've operated at Staff or Principal level before and know what it means to own a technical domain — not just execute within it. You're the person Engineering Managers and VPs call when a system needs to be rethought, not just fixed. You can set direction with incomplete information, make architectural trade-offs explicit, and build consensus across teams without needing organizational authority to do it. You care about reliability the way a product engineer cares about user experience — it's not overhead, it's the product. And you measure your success not by what you personally shipped, but by how much better the entire engineering organization runs because of the standards and systems you put in place.

‌

Be Human With Us: 

Being human isn’t about checking every box on a list. It’s about the experiences we have, people we meet, and the perspectives we share. So, if you have the skills but are hesitant to apply because of your background, apply anyway. We need amazing people like you to help us challenge the conventional and think differently about the problems that we’re solving. We’re in this together. Come be human, with us. 

Use of AI Technology:

We use technology, including automated and AI-assisted tools, to support certain aspects of our recruitment process. These tools are designed to improve efficiency and enhance the candidate experience. AI tools are not used to make hiring decisions; all hiring decisions are made by our hiring teams.

At ServiceTitan, we celebrate individuality and uniqueness. We believe that the convergence of fresh perspectives and experiences from all walks of life is what makes our product and culture so great. We do not discriminate against employees based on race, color, religion, sex, national origin, gender identity or expression, age, disability, sexual orientation, or any other characteristic protected by applicable laws.