Data Engineer, SCOT Fulfillment Optimization

When you order items on Amazon, there are practically thousands of ways we can fulfill that order. Which Fulfillment Center do we ship from, what carriers do we use, what boxes do we combine items in — now scale that question to billions of items shipped annually worldwide. At Amazon's Supply Chain Optimization Technologies (SCOT), we are tasked with fulfilling every customer order in the most intelligent way possible while making sure customers get their orders on time.

Within SCOT, the Fulfillment Optimization organization owns the artificial intelligence, optimization, and simulation systems that decide how Amazon's outbound network is planned and operated. Our team is a new horizontal team responsible for the data foundations that power every Fulfillment Optimization product — from long-range planning to day-of execution. Our customers are the Applied Scientists, Business Intelligence Engineers, Data Scientists, and Software Development Engineers building those products, and the leaders who depend on trustworthy data to make multi-billion-dollar planning decisions.

We are looking for a Data Engineer to help us build that foundation. You will work alongside more senior Data Engineers, Software Development Engineers, and Applied Scientists to design, build, and operate the data pipelines, models, and warehouses that our products and analytics depend on. You will take well-defined requirements, build a solution, and deliver it on schedule. You will learn the team's data architecture deeply, contribute to its evolution, and grow into an autonomous owner of significant pieces of it.

This role is a great fit for someone who is excited about working with large-scale, complex data, who cares about getting the details right, and who wants to grow under the mentorship of an experienced data engineering team.

Key job responsibilities
Build and optimize physical data models and ETL pipelines for datasets that serve our products and analytics, using Amazon's data platforms (Redshift, EMR, Spark, S3, Hive, etc.)
- Take well-defined requirements from Data Scientists, Business Intelligence Engineers, and Software Development Engineers, and deliver tested, documented, and maintainable data solutions on schedule
- Troubleshoot, root-cause, and resolve issues in existing datasets and pipelines, leaving them better and easier to maintain than you found them
- Measure and improve dataset quality — completeness, freshness, correctness — and contribute to the team's monitoring and SLA framework
- Write secure, stable, testable, and well-documented code in SQL and Python (or Scala), and submit it for code review
- Classify, store, and handle data in accordance with Amazon's security and privacy policies
- Participate in team design, scoping, and prioritization discussions; learn the business context behind the team's data architecture and the products it supports
- Collaborate with peers across Fulfillment Optimization and partner teams to integrate data sources and unblock downstream consumers

A day in the life
You will spend your day writing SQL and Python, designing tables and pipelines, reviewing teammates' code, and pairing with data scientists and engineers to understand what they need from the data. You will start with well-scoped pieces of work — a new dataset, an optimization to an existing pipeline, an investigation into a data quality issue — and grow your scope as you build context. You will be supported by a team that values mentorship, code review, and writing things down.

About the team
The Fulfillment Optimization organization owns and operates the artificial intelligence, optimization, and simulation systems that plan and run Amazon's outbound fulfillment network. The organization is spread across the United States and Europe and is multi-disciplinary — Research Science, Applied Science, Business Intelligence, Product Management, Data Engineering, and Software Development. Our team is a newly formed horizontal team responsible for the data foundations that all Fulfillment Optimization products and analytics depend on. Come join us as we build that foundation from the ground up.

Basic Qualifications

- Experience in data engineering
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with one or more query language (e.g., SQL, PL/SQL, DDL, MDX, HiveQL, SparkSQL, Scala)
- Experience with one or more scripting language (e.g., Python, KornShell)

Preferred Qualifications

- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
- Experience with any ETL tool like, Informatica, ODI, SSIS, BODI, Datastage, etc.

Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions based on your experience and skills. We value your passion to discover, invent, simplify and build. Protecting your privacy and the security of your data is a longstanding top priority for Amazon. Please consult our Privacy Notice (https://www.amazon.jobs/en/privacy_page) to know more about how we collect, use and transfer the personal data of our candidates.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.