Cloud Data Platform Engineer (Snowflake + Databricks)

 

Innovation Hub Overview 

Jefferies is creating a Technology Innovation Hub in Pune, a greenfield opportunity to build the systems that power global markets. As our first India technology center, this hub brings together hands on builders who engineer the platforms behind Jefferies’ growth across capital markets, investment banking, and institutional securities. We’re scaling toward an elite team of 500 engineers while maintaining the agility, ownership, and meritocratic spirit that defines Jefferies. From cloud and data to AI, risk, and core business technologies, teams in Pune will lead high impact work with a global mandate.

Team Overview

IT Infrastructure Technology 

Jefferies’ IT Infrastructure Technology team builds and runs the core technology backbone that enables the firm to operate globally. It covers enterprise infrastructure engineering and operations while driving modernization initiatives such as Cloud Adoption and Network Resiliency. The group supports critical platforms including networking, end-user computing, cloud, databases, server/Unix engineering, communications, and global data centers (including low-latency colocations near exchanges). 

Role summary

Build and operate a modern, scalable data platform leveraging Snowflake and Databricks to enable analytics, data science, and machine learning at scale. As a Data Platform Engineer, you will design platform architecture, implement data infrastructure, optimize performance and cost, and ensure security and governance. You will partner with Data Engineers, Analytics Engineers, ML Engineers, and business stakeholders to deliver a world-class data platform that powers data-driven decision-making across the organization.

You will collaborate closely with Data Engineers, Analytics Engineers, ML Engineers, Cloud Architecture, Cloud Platform, Cloud Security, Cloud Networking, and business stakeholders across Equities, Investment Banking, and other divisions. Strong teamwork, customer service orientation, and the ability to balance platform stability with innovation are essential. Experience working in Agile teams using Jira and Confluence is expected

 Key responsibilities 

• Design, implement, and operate data platform infrastructure on Snowflake and Databricks; ensure scalability, performance, reliability, and cost efficiency
• Build and maintain platform integrations with AWS services (S3, Glue, Lambda, Kinesis, IAM); establish seamless data flows between cloud storage, data lakes, and analytics platforms
• Implement Infrastructure as Code (Terraform) for Snowflake and Databricks resources; follow team standards for modules, state management, and Terraform Enterprise workflows
• Establish platform governance and security controls: role-based access control (RBAC), data encryption, network security, audit logging, and compliance with regulatory requirements
• Optimize Snowflake and Databricks configurations for cost and performance: warehouse/cluster sizing, auto-scaling, caching, partitioning, and query optimization
• Build CI/CD pipelines for platform infrastructure and data workflows using GitHub, Bamboo, GitLab, or similar tools 
• Develop self-service capabilities and "golden path" patterns for data teams to onboard workloads; provide tooling, documentation, and training
• Monitor platform health, performance, and usage; implement alerting and proactive remediation for issues
• Collaborate with Data Engineers, Analytics Engineers, ML Engineers, Cloud Architecture, and Cloud Security teams to define platform requirements and best practices
• Drive adoption of DataOps practices: version control, testing, deployment automation, and observability for data workflows
• Troubleshoot complex platform issues spanning Snowflake, Databricks, AWS, networking, and identity
• Lead cost optimization initiatives; implement chargeback/showback models and cost visibility dashboards
• Proficiency using GenAI assistants (ChatGPT, Claude, GitHub Copilot) for platform design, SQL/Python code generation, troubleshooting, and documentation
• Ability to implement AI agent-driven automation (agentic workflows) for platform operations (e.g., cost optimization, performance tuning, anomaly detection, user provisioning) with enterprise safety controls: human-in-the-loop approvals, comprehensive logging and auditability, guardrails to prevent service disruption, rollback mechanisms, and secure credential management
• Proactive mindset to identify platform operational toil; ship automation that measurably reduces manual work and improves platform reliability
• Monitor platform health, performance, and SLAs; respond to incidents promptly
• Participate in on-call rotation as needed for critical platform support
• Conduct post-incident reviews for platform issues; track corrective actions to completion
• Maintain runbooks and operational documentation for Snowflake and Databricks
• Continuously optimize platform cost, performance, and user experience

 Requirements

• 7+ years in data platform engineering, data infrastructure, or cloud engineering roles
• Deep hands-on experience with Snowflake: architecture, warehouses, databases, schemas, stages, pipes, tasks, streams, RBAC, and security
• Strong experience with Databricks: clusters, notebooks, jobs, Delta Lake, Unity Catalog, and MLflow
• Proficiency with AWS data services: S3, Glue, EMR, Redshift, Lambda, Kinesis, and IAM
• Expert-level Terraform skills for data platform infrastructure; experience with Terraform Enterprise preferred
• Strong understanding of data engineering concepts: ETL/ELT, data pipelines, data lakes, data warehouses, and data modeling
• Proficiency in Python and SQL; experience with PySpark or similar big data frameworks
• Strong understanding of data governance, security, and compliance best practices
• Experience with CI/CD pipelines and DevOps/DataOps practices 
• Excellent troubleshooting and problem-solving skills
• Strong communication and collaboration skills; ability to work with technical and business stakeholders

Preferred

• Snowflake SnowPro Core or Advanced certification
• Databricks Certified Data Engineer or Platform Architect certification
• AWS Certified Solutions Architect or Data Analytics Specialty certification
• Experience with dbt (data build tool) for transformation orchestration
• Familiarity with streaming platforms (Kafka, Kinesis, MSK)
• Background in software engineering or site reliability engineering
• Experience with data catalog and lineage tools (Collibra, Alation, Monte Carlo)s

We have been made aware of bad actors falsely claiming to be associated with Jefferies Group soliciting individuals to attend virtual job interviews, complete online tests or courses and sending fictitious employment offer letters.

Please note that any email contact with Jefferies personnel will come from an “@jefferies.com” email address. Further, Jefferies will not notify shortlisted candidates through social media platforms (e.g. WhatsApp or Telegram) or ask candidates to make payment to participate in the hiring process.

#LI-MF1