Senior Data Engineer

Job Title: Senior Data Engineer

• Delivery & Ownership
• Own the end-to-end delivery of data pipelines and data products, from design through deployment and production support
• Develop, maintain, and optimise scalable data pipelines using Azure Databricks, DBT, and Python
• Ensure solutions are production-ready, reliable, and meet defined SLAs
• Migration & Platform Engineering
• Execute and support migration of data from Snowflake (AWS) to Azure using Databricks, DBT, and Lakebridge
• Build and manage infrastructure-as-code using Terraform to enable scalable and repeatable deployments
• Ensure seamless integration between legacy systems and the Azure data platform
• Data Quality, Observability & Governance
• Implement data quality frameworks (e.g., DBT testing, validation rules) to ensure trusted data pipelines
• Enable observability through logging, monitoring, and alerting of pipelines and data workflows
• Support data governance standards including data lineage, consistency, and compliance requirements
• Architecture & Engineering Excellence
• Contribute to the design and architecture of scalable data solutions, aligned with lakehouse principles
• Apply best practices in data modelling, performance optimisation, and cost efficiency
• Follow and promote engineering standards, including modular design and reusable components
• Production Support & Reliability
• Monitor and maintain data pipelines in production, ensuring high availability and performance
• Troubleshoot issues and perform root cause analysis (RCA) for incidents
• Continuously improve system stability, resilience, and operational processes
• Collaboration & Technical Leadership
• Work closely with cross-functional teams (data, product, architecture, business) to translate requirements into scalable technical solutions
• Provide guidance and mentorship to junior engineers through code reviews and best practices
• Contribute to documentation of architecture, pipelines, and operational processes

Required Skills & Qualifications

• Strong hands-on expertise with: Azure Databricks, DBT, Snowflake, Python and SQL
• Proven experience building and managing large-scale data pipelines and transformations
• Experience working with high-volume datasets (e.g., terabytes, complex data models, multiple pipelines)
• Solid understanding of: Data modelling and ETL/ELT frameworks, Distributed data systems and lakehouse architectures, Experience with Terraform and infrastructure-as-code principles
• Strong experience with CI/CD pipelines and Git-based version control
• Experience supporting production systems, including monitoring, alerting, and incident resolution
• Knowledge of data quality, testing, and observability frameworks
• Strong problem-solving and analytical skills
• Excellent communication skills with the ability to work across technical and non-technical stakeholders
• Experience in insurance or financial services domains is highly desirable

Preferred Skills:

• Experience with real-time/streaming technologies (Kafka, Event Hubs, Delta Live Tables)
• Knowledge of security, compliance, and data protection standards in regulated industries
• Experience with cost optimisation and performance tuning in cloud environments
• Exposure to multi-cloud or hybrid architectures

Posted On: 14-07-2026