Principal Software Engineer

Red Hat is seeking a Principal Software Engineer to join the CI/CD and delivery engineering team within Red Hat OpenShift Service on AWS (ROSA). ROSA is Red Hat's fully managed Kubernetes platform on AWS, built on a large-scale multi-tenant architecture where Red Hat operates shared control plane infrastructure while customers securely run workloads in their own AWS accounts.

About the job

This team owns the systems and practices that define how ROSA software moves from commit to production — across global AWS regions, safely and at speed. You will design, build, and operate the CI/CD pipelines, testing infrastructure, release automation, and delivery tooling that the broader ROSA engineering organization depends on every day. The stack includes Prow, ci-operator, Argo CD, Terraform, Konflux, and app-interface/qontract, with a strong emphasis on pipeline reliability, progressive delivery, automated quality gates, and infrastructure-as-code validation.. There is no separate operations team: what you build, you run — including operating across a multi-region fleet of management clusters and service clusters via OCM APIs.

This is not a role where CI/CD is a side responsibility. It is the job. You will own the end-to-end delivery pipeline for a complex, multi-region managed Kubernetes service — designing systems that catch problems before they reach production, enable safe and frequent releases, and give engineers fast, reliable feedback on every change. Success requires deep expertise in CI/CD architecture, build and release engineering, automated testing strategy, and production delivery, combined with strong Kubernetes and AWS fluency and the judgment to make sound trade-offs in a high-stakes production environment.

As Red Hat continues evolving toward a more AI-enabled software development lifecycle, you will also help define how AI-powered tooling and automation are integrated into CI/CD — from intelligent test selection and AI-assisted code review to agentic pipeline automation and LLM-powered release validation. Our engineering culture values strong ownership, technical depth, open collaboration, and continuous improvement.

What you will do

• Own and evolve ROSA's CI/CD platform. Design, build, and operate the CI/CD pipelines and delivery infrastructure that ROSA engineering depends on to ship software safely and frequently. Own the full pipeline lifecycle — from source integration and build automation through testing, artifact management, promotion, and production deployment. Ensure pipelines are fast, reliable, observable, and secure. You are accountable for the health and reliability of the delivery path the same way a platform engineer is accountable for uptime.

• Design and implement automated testing and quality gates. Build testing infrastructure and automated quality gates that catch defects, regressions, security issues, and configuration drift before they reach production. Define and enforce testing strategies across unit, integration, end-to-end, and infrastructure validation stages. Ensure that test results are fast, trustworthy, and actionable — flaky tests and slow feedback loops are problems you own and fix.

• Drive progressive delivery and release engineering. Implement and operate progressive delivery strategies — canary deployments, sector-based staged rollouts, and progressive promotion through environment gates — that allow ROSA to release changes safely across global AWS regions. Build release automation that is repeatable, auditable, and resilient. Own the tooling and processes that determine how and when software reaches production.

• Build and maintain infrastructure-as-code and configuration validation. Ensure that Terraform, Kubernetes manifests, and other infrastructure-as-code artifacts are validated, linted, and tested as part of every pipeline run. Integrate policy-as-code and security scanning into the delivery pipeline so that infrastructure changes meet compliance and security requirements before they are applied..

• Leverage AI to accelerate CI/CD and delivery. Explore and integrate AI-powered tooling into the CI/CD lifecycle — intelligent test selection, automated root cause analysis for pipeline failures, and agentic CI triage. Help shape how the broader engineering team adopts AI-assisted delivery practices.

• Operate what you build with a high reliability bar. The CI/CD platform is production infrastructure. Own its reliability with clear SLOs, monitoring, alerting, and incident response. When a pipeline is broken or a release is blocked, you are in the room driving resolution. Continuously measure and improve pipeline performance — build times, test cycle times, deployment frequency, change failure rate, and mean time to recovery.

• Collaborate across engineering, product, and SRE. Work closely with platform, SRE, security, and product teams to understand their delivery needs and ensure the CI/CD platform supports them effectively. Surface risks, bottlenecks, and improvement opportunities proactively. Translate delivery engineering best practices into tooling and automation that the whole organization benefits from.

• Champion delivery engineering practices and mentor peers. Lead by example in code reviews, design discussions, and architectural decisions related to CI/CD and delivery. Share deep domain knowledge through design documents, runbooks, and hands-on collaboration. Help engineers across the organization understand and adopt CI/CD best practices, and mentor less experienced engineers on build, test, and release engineering.

What you will bring

We're looking for a strong individual contributor who combines deep CI/CD and delivery engineering expertise with sound judgment and production accountability. You think in pipelines and feedback loops, write automation that operates reliably at scale, and treat the delivery path as critical infrastructure.

• Bachelor's degree in Computer Science, Engineering, or related STEM field — or equivalent practical experience.

• 7+ years of software engineering experience, with a significant focus on CI/CD, build and release engineering, or developer tooling in production environments.

• Deep CI/CD expertise. You have designed, built, and operated CI/CD pipelines end-to-end for complex software systems — including build automation, automated testing at multiple stages, artifact management, progressive delivery, pipeline security, and quality gates. You understand CI/CD not as a checkbox but as a discipline: pipelines should be fast, reliable, observable, and secure. Hands-on experience with tools such as Prow, Argo CD, Tekton, GitHub Actions, Jenkins, or similar CI/CD platforms in production..

• Strong programming skills in Go, Python, or similar languages, with experience building delivery tooling, test infrastructure, and automation systems.

• Solid Kubernetes and AWS fluency. You understand the platform your pipelines deliver to — including EKS, IAM, networking, Terraform, GitOps, and infrastructure-as-code patterns. You don't need to be an infrastructure architect, but you need to understand how delivery interacts with the platform and how infrastructure changes should be validated and promoted.

• Experience with or strong interest in AI-assisted engineering. You've used or are eager to integrate AI-powered tooling into engineering and delivery workflows — coding agents, automated code review, intelligent test selection, or agentic CI/CD — and you actively look for new ways to apply AI to accelerate delivery and improve quality.

• Strong production ownership. Experience operating CI/CD infrastructure as production systems with real SLOs, participating in incident response when the delivery path is impacted, and taking full accountability for pipeline reliability and performance.

• Expertise in automated testing strategy. You understand how to design testing pyramids, manage test environments, reduce flakiness, and build quality gates that engineering teams trust and rely on.

• Strong communication and collaboration skills. Able to clearly articulate delivery engineering designs, risks, and trade-offs in code reviews, design documents, and cross-team discussions. You write clearly and review thoroughly.

• Strong SDLC discipline. You understand the full software development lifecycle and continuously improve how software moves from development through testing, staging, and production. You follow defined processes, identify inefficiencies, and advocate for improvements that make delivery faster, safer, and more predictable.

Nice to have:

• Experience with HyperShift, OpenShift, or managed Kubernetes control plane technologies in production environments

• Experience building CI/CD systems for multi-region, multi-tenant managed services on AWS with strong availability, security, and compliance requirements

• Experience with DORA metrics and using delivery performance data to drive engineering improvements

• Track record of adopting AI-assisted delivery workflows, including intelligent test selection, automated code review, or agentic CI/CD tooling

• Experience with policy-as-code frameworks (OPA, Kyverno) integrated into delivery pipelines

• Experience with Prow, ci-operator, or OpenShift CI infrastructure

• Contributions to upstream open source communities, particularly within Kubernetes, CI/CD tooling, or delivery engineering projects

About Red Hat

Red Hat is the world’s leading provider of enterprise open source software solutions, using a community-powered approach to deliver high-performing Linux, cloud, container, and Kubernetes technologies. Spread across 40+ countries, our associates work flexibly across work environments, from in-office, to office-flex, to fully remote, depending on the requirements of their role. Red Hatters are encouraged to bring their best ideas, no matter their title or tenure. We're a leader in open source because of our open and inclusive environment. We hire creative, passionate people ready to contribute their ideas, help solve complex problems, and make an impact.

Inclusion at Red Hat
Red Hat’s culture is built on the open source principles of transparency, collaboration, and inclusion, where the best ideas can come from anywhere and anyone. When this is realized, it empowers people from different backgrounds, perspectives, and experiences to come together to share ideas, challenge the status quo, and drive innovation. Our aspiration is that everyone experiences this culture with equal opportunity and access, and that all voices are not only heard but also celebrated. We hope you will join our celebration, and we welcome and encourage applicants from all the beautiful dimensions that compose our global village.

Equal Opportunity Policy (EEO)
Red Hat is proud to be an equal opportunity workplace and an affirmative action employer. We review applications for employment without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other basis prohibited by law.

Red Hat does not seek or accept unsolicited resumes or CVs from recruitment agencies. We are not responsible for, and will not pay, any fees, commissions, or any other payment related to unsolicited resumes or CVs except as required in a written contract between Red Hat and the recruitment agency or party requesting payment of a fee.

Red Hat supports individuals with disabilities and provides reasonable accommodations to job applicants. If you need assistance completing our online job application, email application-assistance@redhat.com. General inquiries, such as those regarding the status of a job application, will not receive a reply.