Principal Data Engineer
Disclaimer: Hunt UK Visa Sponsors aggregates job listings from publicly available sources, such as search engines, to assist with your job hunting. We do not claim affiliation with Oritain. For the most up-to-date job details, please visit the official website by clicking "Apply Now."
Description
Oritain is the global leader in product verification, with locations in Auckland, Dunedin, London, Singapore and Washington D.C.
Our Mission: Harnessing our data to protect people & planet
Our mission is to protect people and the planet by harnessing science, technology and services to build a community of origin verified suppliers and buyers.
We are looking for a Principal Data Engineer to lead on the transformation of our entire data platform. This is a critical leadership role responsible for defining, building and running the scalable, robust, and trustworthy data infrastructure that will underpin all future product development, scientific analysis and business operations.
The Opportunity: Founding the Data Platform
Reporting to the Head of Engineering, you will be the most senior technical voice for data platforms within the organisation. You will own the strategy, design, and initial implementation of the pipelines and architecture required to integrate complex scientific data with our commercial software applications.
You will act as a technical leader and mentor to the wider engineering team, ensuring that all data-related systems meet the highest standards of reliability, performance, and security.
Key Responsibilities
Data Architecture & Strategy
Our Mission: Harnessing our data to protect people & planet
Our mission is to protect people and the planet by harnessing science, technology and services to build a community of origin verified suppliers and buyers.
We are looking for a Principal Data Engineer to lead on the transformation of our entire data platform. This is a critical leadership role responsible for defining, building and running the scalable, robust, and trustworthy data infrastructure that will underpin all future product development, scientific analysis and business operations.
The Opportunity: Founding the Data Platform
Reporting to the Head of Engineering, you will be the most senior technical voice for data platforms within the organisation. You will own the strategy, design, and initial implementation of the pipelines and architecture required to integrate complex scientific data with our commercial software applications.
You will act as a technical leader and mentor to the wider engineering team, ensuring that all data-related systems meet the highest standards of reliability, performance, and security.
Key Responsibilities
Data Architecture & Strategy
- Platform Leadership: Define and own the technical strategy and architecture for our entire data platform covering ingestion, storage, processing, governance, and consumption. To include use-cases in support of Operations, Data Science, Customer-Facing Portals and Business Intelligence.
- Pipeline Design: Design and implement highly scalable, performant, and reliable ETL/ELT data pipelines to handle diverse data sources, including complex scientific datasets and supply chain inputs alongside business information.
- Technology Selection: Evaluate, recommend, and drive the adoption of new data services and modern data tools to ensure we have a future-proof data ecosystem.
- Hands-on Development: Serve as the most senior, hands-on developer, writing high-quality, production-grade code (primarily Python and/or Scala/Spark) to build initial pipelines and core data services.
- Data Governance & Security: Architect data security and governance policies, ensuring compliance and best practices around data access, masking, and retention, particularly for sensitive origin data.
- Data Quality: Implement automated deduplication, conflict resolution and anomaly detection to maintain data integrity across ingestion sources.
- Operational Health: Implement robust monitoring, logging, and alerting for all data pipelines and infrastructure, ensuring high data reliability and performance.
- Infrastructure as Code (IaC): Work closely with the Infrastructure team to define and automate the provisioning of all Azure data resources using Terraform or similar IaC tools.
- Scientific Collaboration: Partner closely with the Science teams to understand the structure, complexity, and requirements of raw scientific data, ensuring accurate data translation and ingestion.
- Mentorship: Provide technical guidance and mentorship to software engineers on best practices for interacting with and consuming data services.
- Product Partnership: Collaborate with the Product Director to understand commercial and user-facing data requirements, translating these needs into actionable data infrastructure features.
The Engineering Environment
- Technology: We currently make extensive use of Microsoft Azure and related data services and are moving to Databricks. This role will be an authority across both.
- Collaboration: You will be the technical data expert, integrating with the Software Engineering, Data Science and Product teams.
- Work Style: London office, with a minimum requirement of three days per week on-site to facilitate strategic planning and hands-on collaboration.
- Principal/Lead Expertise: Extensive experience (typically 7+ years) focused on data engineering, including significant time spent in a Principal, Lead, or Architect role defining data strategy from the ground up.
- Databricks: Deep, practical, and architectural experience of the Databricks platform.
- Azure Data Stack: Operational experience of building and running within the Microsoft Azure data ecosystem (e.g., Azure Data Factory, Azure Data Lake, Azure Synapse Analytics, Azure SQL/Cosmos DB).
- Coding Proficiency: Expert-level proficiency in Python (or Scala) and SQL, with a strong focus on writing clean, tested, and highly performant data processing code.
- Data Warehouse Design: Proven track record designing and implementing scalable data warehouses/data marts for analytical and operational use cases.
- Pipeline Automation: Strong experience with workflow orchestration tools and implementing CI/CD for data pipelines.
- Cloud Infrastructure: Familiarity with Infrastructure as Code (Terraform) and containerisation.
- Experience processing scientific, geospatial, or time-series data.
- Experience in the governance or compliance sector where data integrity is paramount.
- Familiarity with streaming data technologies
- Paid Leave- 35 days (inclusive of public holidays)
Transparency is one way of holding ourselves accountable, but for it to be effective it must be underpinned by proven traceability.
Our scientific traceability does just that. Working with Mother Nature, we help brands across fashion, food and pharmaceutical industries verify the origin and authenticity of their products and raw materials. With this verified truth, brands are empowered to make changes across their supply chain to operate more sustainably, and pass those assurances onto their consumers