Lead Data Engineer
UK Visa Sponsorship Analytics
Occupation Type
IT business analysts, architects and systems designers
Occupation Code Skill LevelHigher Skilled
Sponsorship Salary Threshold
£54,900
(£28.15 per hour)
Occupation rate applies
Above analytics are generated algorithmically based on job titles and may not always be the same as the company's job classification. You can also check detailed occupation eligibility, and salary criteria on our UK Visa Eligible Occupations & Salary Thresholds page.
Disclaimer: Hunt UK Visa Sponsors aggregates job listings from publicly available sources, such as search engines, to assist with your job hunting. We do not claim affiliation with Cushman & Wakefield. For the most up-to-date job details, please visit the official website by clicking "Apply Now."
Description
Job Title
Lead Data Engineer
Job Description Summary
Job Description
The Lead Data Engineer helps architect and lead the development of enterprise-scale data platforms and advanced analytics solutions across multiple business units and subject areas. In addition, they assist the project delivery team with scoping and planning integration projects and contribute to the design, implementation, and support of the data artifacts in the data lakehouse and databases. This role combines deep technical expertise with leadership and mentoring responsibilities, driving innovation and best practices across the data engineering function.
Key Responsibilities
Lead Data Engineer
Job Description Summary
Job Description
The Lead Data Engineer helps architect and lead the development of enterprise-scale data platforms and advanced analytics solutions across multiple business units and subject areas. In addition, they assist the project delivery team with scoping and planning integration projects and contribute to the design, implementation, and support of the data artifacts in the data lakehouse and databases. This role combines deep technical expertise with leadership and mentoring responsibilities, driving innovation and best practices across the data engineering function.
Key Responsibilities
- Support the Solution and Data Architect(s) to help lead the design, development, and optimization of scalable data platforms and analytics solutions that support global data strategy.
- Architect and implement robust, scalable, metadata-driven pipelines in Databricks using Delta Lake, Unity Catalog, and declarative workflows.
- Oversee the development and maintenance of Azure SQL database objects using advanced T-SQL for data quality and insight generation.
- Extensive hands-on experience (6+ years) in cloud-native data engineering, including 3+ years with Databricks and Delta Lake in Azure, combined with good (2+ years) in a technical leadership or architecture role.
- Deep practical experience of the Databricks ecosystem including Delta Lake, Unity Catalog, Declarative Pipelines, Spark Structured Streaming, Change Data Feed, Serverless SQL, and Cluster management & optimization.
- Expert-level SQL skills, with a track record of designing performant queries, optimizing data models, and driving insight generation from large datasets.
- Proficiency in multiple programming languages used in data engineering and analytics (e.g., PySpark, Scala, R, Python), with experience mentoring others in their use.
- Deep understanding of cloud-native data architecture, including Lakehouse design principles, data integration patterns, and automation strategies.
- Proven ability to design and deliver scalable, production-grade data pipelines, with a focus on reliability, maintainability, and performance.
- Strong experience with CI/CD, version control (Git), ideally including Databricks Asset Bundles, and Infrastructure as Code, leading DevOps practices in data engineering teams.
- Hands-on experience with machine learning pipelines or integrating data engineering with advanced analytics and AI workflows.
- Experience designing and implementing data mesh or domain-oriented data architectures in large-scale environments.
- Test-driven development using common testing frameworks (e.g. pytest, nUnit, tSQLt, nBI, etc.)
- Exposure to real-time integrations or IoT solutions e.g. Event Hubs, Service Bus, IoT and Event Grid in Azure; IoT Core, PubSub in Google Cloud; etc.
- Strong communicator in English, both verbal and written, with the ability to engage effectively across global teams and all levels of the organisation.
- Analytical and curious mindset, driven to solve complex problems and understand intricate technical systems and architectures.
- Organised and detail-oriented, with a diligent, quality-focused approach and a strong sense of ownership.
- Collaborative and proactive, able to build relationships, resolve conflicts, and contribute meaningfully within cross-functional teams.
- Adaptable and dependable, capable of managing multiple priorities, meeting deadlines, and working independently or as part of a team.
- Continuous learner, demonstrating initiative in technical self-improvement and staying current with evolving tools and practices.