Senior Product Manager - LLM Fleet Management

To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts.

Job Category

Job Details

About Salesforce

Salesforce is the #1 AI CRM, where humans with agents drive customer success together. Here, ambition meets action. Tech meets trust. And innovation isn’t a buzzword — it’s a way of life. The world of work as we know it is changing and we're looking for Trailblazers who are passionate about bettering business and the world through AI, driving innovation, and keeping Salesforce's core values at the heart of it all.

Ready to level-up your career at the company leading workforce transformation in the agentic era? You’re in the right place! Agentforce is the future of AI, and you are the future of Salesforce.

Salesforce is building the infrastructure that powers 1,000+ AI features across the enterprise, spanning 25+ regions including Gov Cloud, and a growing fleet of 100+ 1st party and 3rd party models.

As Product Manager for AI Infrastructure, you will own the capacity posture of Salesforce’s LLM fleet from signal to decision to execution. This role sits at the intersection of AI infrastructure, data, and automation. As we scale to enterprise anchor customers with billion-token workloads, we need a new kind of operator.

This is not a traditional Ops PM role. You will write code, build automation pipelines, and instrument systems instead of just tracking tickets and writing PRDs. We call it Fleet Management because, at this scale, running LLM capacity across proprietary and external models is closer to operating a distributed infrastructure fleet than managing a software feature. 

You will work deeply with infra engineering, vendor partners, and data teams to ensure our AI infrastructure is resilient, optimized, and scalable.

What You’ll Own

• Own the capacity intelligence layer: Build & operate automated systems that monitor TPM/RPM utilization across our entire fleet of 1st party and 3rd party models, regions, and providers in real time, surfacing signals before they become incidents.

• Forecast and model capacity demand: Apply strong mathematical and statistical rigor to project trillion token workloads. You will build the financial and operational models that dictate our hyperscaler commitments and internal cluster sizing, balancing strict latency SLAs against complex cost structures.

• Automate capacity decisions: Write and maintain scripts and pipelines that trigger PTU reservations, rate limit adjustments, PayGo spillover rules, and provider-level failover based on usage patterns.

• Manage the fleet economics: Track model spend across our internal 1st party infrastructure and 3rd party providers (Azure/OpenAI, Google Gemini, Anthropic, and others), model by model, region by region. You will build tooling that dynamically optimizes reservations versus PayGo tradeoffs.

• Drive anchor customer capacity: Manage capacity plans for our largest and most complex customers by analyzing usage data, stress-testing projections, and building capacity buffers that prevent hard-kill scenarios at launch.

• Build the operating playbook: Define and automate the end-to-end model capacity lifecycle from new model onboarding to burst handling to deprecation, working in coordination with infra engineering and vendor partners.

• Be the system of record for capacity: Own dashboards, data pipelines, and alert systems so the team always has an accurate, real-time view of fleet health, rather than a spreadsheet that is two days stale.

You’re a Strong Fit If

• You have 10 to 12 years of experience in a technical PM, infrastructure PM, or equivalent role where you were expected to build, not just specify.

• You have hands-on coding and AI agent fluency. Python is the minimum bar, but how you write it matters. You are highly capable of leveraging coding agents (like Claude Code, Codex to multiply your output by writing automation scripts, querying APIs, parsing logs, and building lightweight dashboards without waiting for an engineer.

• You possess strong data and mathematical fluency. You can pull and interpret usage telemetry independently, use tools like SQL & Splunk, and apply solid math skills to build accurate forecasting models from complex, noisy traffic patterns.

• You have a deep understanding of LLM infrastructure fundamentals, including tokens, TPM/RPM rate limits, PTU versus PayGo economics, context window tradeoffs, and model latency characteristics across both proprietary and API-based models.

• You have operational instincts at scale, having operated or supported production systems under pressure, and you understand the difference between a p99 incident and a capacity misallocation.

• You excel in vendor and partner engagement, comfortable navigating commercial and technical conversations with 3rd party model providers (Azure, Google, Anthropic) and translating usage data into procurement decisions.

Even Better If

• You have experience with multi-provider LLM routing such as round-robin, latency-based, cost-optimized, or failover routing patterns.

• You are familiar with Provisioned Throughput versus PayGo tradeoffs at a hyperscaler (e.g., Azure OpenAI, Google Vertex, AWS Bedrock).

• You have experience building capacity management tooling for ML inference or other stateful, rate-sensitive workloads.

• You have a background in SRE, MLOps, or platform engineering and share a daily working vocabulary with infra engineers.

• You have a demonstrated ability to build internal tools such as Slack bots, operational dashboards, or lightweight automation apps that measurably reduced manual toil.

Unleash Your Potential

When you join Salesforce, you’ll be limitless in all areas of your life. Our benefits and resources support you to find balance and be your best, and our AI agents accelerate your impact so you can do your best. Together, we’ll bring the power of Agentforce to organizations of all sizes and deliver amazing experiences that customers love. Apply today to not only shape the future — but to redefine what’s possible — for yourself, for AI, and the world.

Accommodations

If you need a reasonable accommodation during the application or the recruiting process, please submit a request via this Accommodations Request Form.

Please note that Salesforce uses artificial intelligence (AI) tools to help our recruiters assess and evaluate candidates’ resumes and qualifications throughout the recruiting process. Humans will always make any candidate selection and hiring decisions. Please see our Candidate Privacy Statement for more information about how we use your personal data and your rights, including with regard to use of AI tools and opt out options.

Posting Statement

Salesforce is an equal opportunity employer and maintains a policy of non-discrimination with all employees and applicants for employment. What does that mean exactly? It means that at Salesforce, we believe in equality for all. And we believe we can lead the path to equality in part by creating a workplace that’s inclusive, and free from discrimination. Know your rights: workplace discrimination is illegal. Any employee or potential employee will be assessed on the basis of merit, competence and qualifications – without regard to race, religion, color, national origin, sex, sexual orientation, gender expression or identity, transgender status, age, disability, veteran or marital status, political viewpoint, or other classifications protected by law. This policy applies to current and prospective employees, no matter where they are in their Salesforce employment journey. It also applies to recruiting, hiring, job assignment, compensation, promotion, benefits, training, assessment of job performance, discipline, termination, and everything in between. Recruiting, hiring, and promotion decisions at Salesforce are fair and based on merit. The same goes for compensation, benefits, promotions, transfers, reduction in workforce, recall, training, and education.