Software Engineer III: Data Engineer

Job responsibilities

• Supports review of controls to ensure sufficient protection of enterprise data
• Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems .
• Creates secure and high-quality production code and maintains algorithms that run synchronously with appropriate systems.
• Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development.
• Gathers, analyzes, synthesizes, and develops visualizations and reporting from large, diverse data sets in service of continuous improvement of software applications and systems.
• Proactively identifies hidden problems, patterns in data, and uses these insights to drive improvements to coding hygiene and system architecture.
• Contributes to software engineering communities of practice and events that explore new and emerging technologies
• Updates logical or physical data models based on new use cases
• Frequently uses SQL and understands NoSQL databases and their niche in the marketplace

Required qualifications, capabilities, and skills

• Formal training or certification on Software Engineering concepts and 3+ years applied experience
• Working understanding of NoSQL databases with Advanced at SQL (e.g., joins and aggregations)
• Experience with statistical data analysis and ability to determine appropriate tools and data patterns to perform analysis
• Experience in ETL process/Advance concepts
• Hands-on practical experience in system design, application development, testing, and operational stability
• Experience in AWS, design, implementation, and maintenance of data pipelines using Python and PySpark (secondary alternative: Java)
• Experience in performance and tuning to ensure jobs are running at optimal levels and no performance bottleneck
• Proficiency in Unix scripting, data structures, data serialization formats such as JSON, AVRO, Protobuf, or similar, big-data storage formats such as Parquet, Iceberg, or similar, data processing methodologies such as batch, micro-batching, or stream, one or more data modelling techniques such as Dimensional, Data Vault, Kimball, Inmon, etc., Agile methodology, TDD or BDD and CI/CD tools

• Python Advance development skills  / Kafka & S3 integration in Performance optimization, Lambda, ECS, EKS, Kinesis, CloudWatch
• Experience in carrying out data analysis to support business insights
• Strong in PySpark, AWS, Terraform & Snowflake, GitHub Copilot, Airflow, Kubernetes