Senior AI Software Engineer

Job Description:

Cityblock is putting AI directly into the hands of the care teams and members we serve. You are a full-stack product engineer who takes an AI-native idea from a rough prototype to a feature running in production, in front of real clinical users, and you contribute to the prototyping loop that lets product and design move at that speed.

Most of the value of AI in our product is still undiscovered. Your job is to shorten the distance between an idea and something people can actually use, and then to productionize effective use of AI into features that hold up in a regulated healthcare product.

Reporting into the Product Engineering org, this role covers system design, including the LLM/SDK integration underneath, the evals and golden datasets that tell you whether it's working, and the UX. You embed with product, design, and work with clinical operations. Because these features run in a regulated healthcare product in front of real members and care teams, you help decide where AI acts on its own, where it stays advisory with a human in the loop, and how we prove a feature is safe to trust, designing for compliance from the start.

Responsibilities

• Implement full-stack feature work that integrates LLM APIs and SDKs into member- and care-team-facing product surfaces.

• Own features end to end: system design, model/SDK integration, evals, and the UX that determines whether the feature is actually trusted and used.

• Build observability and evals alongside the feature, so quality can be measured and then improved.

• Apply and help improve the team's prototyping loop, eval discipline, and quality bar through the work you ship.

• Put prototypes in front of real users early, learn from what breaks, and harden the ones worth keeping into production.

• Bring an honest read of what's feasible back to product and design, so the roadmap reflects reality.

• Work with product management, design, and clinical operations to find and understand the high-value AI opportunities behind the features you build.

• Partner with the data organization for the grounded, trustworthy context your features depend on.

Requirements for the Role

• 4+ years of experience as an engineer

• You are a strong full-stack product engineer who ships. You solve problems by building working software.

• Experience integrating LLM APIs or SDKs into a shipped product, or closely related production experience you can build on.

• Strong product sense: you care about whether people use your products and whether they provide measurable value.

• Comfort owning ambiguity by taking a well-scoped, high-value idea and turning it into something real, with decreasing guidance over time.

• Experience building evals or quality measurement for AI features, or the clear instinct that you need to.

• Fluent daily use of AI assistants for engineering, with a clear sense of when to trust them and when not to.

• Drawn to mission-oriented work, and comfortable partnering with clinical and design experts whose expertise differs from yours.

Nice to Have

• A CS degree or equivalent.

• Prior work in a regulated or compliance-heavy environment (HIPAA, SOC 2, or similar).

• Experience building AI-native product features from scratch, including the evals around them.

• Design sensibility strong enough to prototype UX yourself.

• Exposure to clinical operations or working alongside clinical staff as end users.

• Hands-on with retrieval-augmented generation (RAG): embeddings, vector search, and grounding LLM output in trusted context.

• Familiarity with agentic patterns including tool/function calling, structured outputs, and multi-step orchestration.

• Experience with LLM eval and observability tooling (e.g., Braintrust, LangSmith) and LLM-as-judge techniques.

• Comfort in a modern TypeScript/Node/React stack, including streaming and latency-sensitive LLM UX.

We take into account an individual’s qualifications, skillset, and experience in determining final salary. This role is eligible for health insurance, life insurance, retirement benefits, participation in the company’s equity program, paid time off, including vacation and sick leave. The actual offer will be at the company’s sole discretion and determined by relevant business considerations, including the final candidate’s qualifications, years of experience, skillset, and geographic location. The expected salary range for this position is: 

$139,000.00 - $183,000.00 Annual

Cityblock values diversity as a core tenet of the work we do and the populations we serve. We are an equal opportunity employer, indiscriminate of race, religion, ethnicity, national origin, citizenship, gender, gender identity, sexual orientation, age, veteran status, disability, genetic information, or any other protected characteristic.

We do not accept unsolicited resumes from outside recruiters/placement agencies. Cityblock will not pay fees associated with resumes presented through unsolicited means.