SC Cleared Python Data Engineer – Azure Cloud

CompanyMontash

LocationUnited Kingdom

Posted At1/9/2026

UK Visa Sponsorship Analytics

Analytics are greyed out due to low classification confidence (37.0%).

Occupation Type

IT business analysts, architects and systems designers

Occupation Code Skill LevelHigher Skilled

Sponsorship Salary Threshold

£54,900 (£28.15 per hour)

Occupation rate applies

Above analytics are generated algorithmically based on job titles and may not always be the same as the company's job classification. You can also check detailed occupation eligibility, and salary criteria on our UK Visa Eligible Occupations & Salary Thresholds page.

Disclaimer: Hunt UK Visa Sponsors aggregates job listings from publicly available sources, such as search engines, to assist with your job hunting. We do not claim affiliation with Montash. For the most up-to-date job details, please visit the official website by clicking "Apply Now."

Description

Job Title: SC Cleared Python Data Engineer – Azure Cloud

Contract: 12 months

Rate: Up to £410/day (Inside IR35)

Location: UK Based - Remote

Start: January 2026

Clearance: Active SC Clearance required

Role Overview

We are seeking an SC Cleared Python Data Engineer with strong hands-on experience in PySpark, Delta Lake, and Azure cloud services. The role focuses on designing and delivering scalable, well-tested data pipelines, with particular importance placed on the ability to understand, explain, and design PySpark architectures and demonstrate deep, production-grade Python expertise.

You will work in a containerised, cloud-native environment, delivering maintainable, configurable, and test-driven solutions as part of a multi-disciplinary engineering team.

Key Responsibilities

Design, develop, and maintain data ingestion and transformation pipelines using Python and PySpark.
Clearly articulate PySpark architecture, execution models, and performance considerations to both technical and non-technical stakeholders.

Implement unit and BDD testing (Behave or similar), including effective mocking and dependency management.

Design and optimise Delta Lake tables to support ACID transactions, schema evolution, and incremental processing.

Build and manage Docker-based environments for development, testing, and deployment.

Develop configuration-driven, reusable codebases suitable for multiple environments.

Integrate Azure services including Azure Functions, Key Vault, and Blob/Data Lake Storage.

Optimise Spark jobs for performance, scalability, and reliability in production.

Collaborate with Cloud, DevOps, and Data teams to support CI/CD pipelines and environment consistency.

Produce clear technical documentation and follow cloud security and data governance best practices.

Required Skills & Experience

Strong Python expertise, with a demonstrable depth of experience in designing modular, testable, production-quality code.
Proven experience explaining and designing PySpark architectures, including distributed processing and performance tuning.
Hands-on experience with Behave or similar BDD frameworks, including mocking and patching techniques.

Solid understanding of Delta Lake concepts, transactional guarantees, and optimisation strategies.

Experience using Docker across development and deployment workflows.

Practical experience with Azure services (Functions, Key Vault, Blob Storage, ADLS Gen2).

Experience building configuration-driven applications.

Strong problem-solving skills and ability to work independently in agile environments.

Desirable Experience

Databricks or Synapse with Delta Lake.
CI/CD pipelines (Azure DevOps or similar) and infrastructure-as-code.
Knowledge of Azure data security and governance best practices.
Experience working in distributed or multi-team environments.