Senior ML Engineer - Client Experimentation

Cupertino, California, United StatesSenior$184,700 - $324,80010+ years

At Apple, we work every day to create products that enrich people’s lives. Our Advertising Platforms group makes it possible for people around the world to easily access informative and imaginative content on their devices while helping publishers and developers promote and monetize their work. Today, our technology and services power advertising in Search Ads in the App Store and Apple News. Our platforms are highly-performant, deployed at scale, and setting new standards for enabling effective advertising while protecting user privacy. Ads ML Experimentation team is looking for a Senior ML Engineer to build the next generation of software platforms and systems to advance experimentation that connect millions of global users to content from publishers and developers. Your work will drive the experimentation framework development of machine learned algorithms at scale. You will build sophisticated systems essential in the end-end development of predictive algorithms. You will design and engineer tools for managing AB test experiments, for performing exploratory analysis, and for real-time monitoring of experiment behavior, while preserving privacy of our customers.

Partner with business leaders and stakeholders to design, implement and evolve the features of a multivariate experimentation platform. Extend the experimentation platform to the client edge enabling safe execution and measurement of experiments on-device. Promote experimentation best practices across the organization. Execute on the experimentation roadmap and future integrations with data-driven prioritization. Exercise strong judgment and make thoughtful decisions with ambiguous situations. Striking the right balance between short-term wins vs. long-term success based on various constraints. You will join and contribute to a culture that emphasizes observability and understandability, reliability, resiliency, simplicity, reusability, extensibility, scalability, velocity and productivity. We are one team, nurturing each other’s growth and supporting each other in delivering for our customers and Apple.

Strong experience in Experimentation frameworks, A/B testing and Machine Learning methodologies
Familiarity with statistics and ability to use data analysis techniques to understand data quality, relationships between business metrics and technical performance
Experience building client-side experimentation including SDK-based feature flagging, remote configuration and on-device experiment assignment (deterministic bucketing/consistent hashing) on iOS or comparable native platforms
Proven track record instrumenting client-side exposure and metric logging (trigger-based exposure events, telemetry pipelines from device to backend) and reasoning about their correctness under real-world network and lifecycle conditions
Hands-on experience operating staged/phased rollouts, holdbacks, kill-switches and forced-treatment overrides for client features, including safe ramp-up and fast rollback
Fluency in programming languages like Java, Scala, Python and SQL
Excellent communication, social and presentation skills
Degree in Computer Science, Statistics, Applied Math or related field.
10+ years of industry experience designing, building, maintaining, and extending web-scale production systems.

Familiarity with privacy-preserving client-side measurement (on-device aggregation, differential privacy, minimizing data leaving the device)
Experience designing experiments on native UI/UX and app features
Experience ensuring client/server allocation consistency, app-release cadence vs. server-controlled config, backward/forward compatibility across app versions, offline behavior, config caching and cold-start
Prior experience in advertising industry
Ability to condense complex concepts and analysis into clear and concise takeaways that drive action
Curious business attitude with a proven ability to seek projects with a sense of ownership
Familiarity with ML applications in Ads, recommender systems, information retrieval or related domains.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $184,700 and $324,800, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant

At Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.

Learn about accessibility in Apple’s workplace

Learn about reasonable accommodations for job applicants

Apple accepts applications to this posting on an ongoing basis.

Retail Finance Analyst - Process, Analytics, Reporting and Technology (PART)Cupertino, California, United States

$114,900 - $172,700

US-Operations SpecialistEaston Town Center, Columbus, Ohio, United States

Tactile Simulation Research Engineer - Special Projects (Robotics)Santa Clara, California, United States

$184,700 - $324,800

Tactile AI Research Engineer - Special Projects (Robotics)Santa Clara, California, United States

$184,700 - $324,800

Community Specialist, Channel RetailStockton, California, United States

Senior ML Engineer - Client Experimentation

Similar Jobs