Data Architect

Role: Data Architect

Location: London (Hybrid Working)

Department: Data Services

Contract: Permanent

Join NatCen’s digital transformation and shape the future of reporting architecture.

We are seeking an experienced Data Platform Architect to design and lead the implementation of a scalable, secure, and modern data platform using Databricks. This platform will underpin both our Business Intelligence (BI) reporting and research data environments, which will operate distinctly but be supported by a shared infrastructure.

You will be the technical lead responsible for establishing the architectural foundations of our data capabilities — ensuring that the platform is robust, flexible, and future-proof to support both real-time and longitudinal data needs across the organisation. You will have full architectural autonomy to shape our platform.

Key Responsibilities

• Architect and deliver a data platform using Databricks and Azure supporting:
• BI and operational reporting
• Research data workflows (e.g. surveys, longitudinal studies, modelling)
• Data science and AI developments
• Design for multi-tenancy and workload isolation between BI and research environments.BI and operational reporting
• Establish data architecture patterns for bronze/silver/gold layers, metadata management, and schema evolution.
• Build and maintain CI/CD workflows, job orchestration (e.g. Databricks Workflows or Airflow), and cluster management policies.
• Lead performance tuning, cost optimisation, and observability across the platform.
• Develop and maintain platform documentation and reusable architecture assets.
• Collaborate with business stakeholders and technical SMEs to align solutions with strategic goals.
• Drive innovation by staying current with emerging trends in cloud, data, and AI technologies.
• Provide technical guidance and upskilling for analysts, researchers, and data professionals across the organisation — ensuring they understand how to access, use, and trust the data platform.
• Develop and deliver training materials, workshops, and onboarding sessions for platform users with varying levels of technical experience.
• Act as a champion for platform adoption, promoting best practices, self-service, and data literacy across teams.

Skills, Knowledge and Expertise

Essential:

• 3+ years’ experience architecting and delivering data platforms on Databricks (3+ years minimum).
• Strong experience with cloud-based data solutions, particularly in Azure and Databricks ecosystems.
• Proven ability to design and deliver enterprise-scale data warehouses in regulated or complex environments.
• Expertise in ETL/ELT, and reporting system architectures.
• Strong technical skills in SQL, Python, PySpark, Apache Spark.
• Hands-on background as a data engineer or platform engineer — you can design and build.
• Excellent communication and relationship-building skills across technical and non-technical audiences.
• Demonstrated success in leading cross-functional delivery teams and influencing architectural decisions.

Nice to Have:

• Experience working in or with non-profits, research institutions, or public sector organisations.
• Familiarity with tools like Power BI, dbt, Airflow, or MLflow
• Understanding of data cataloguing, stewardship, and quality frameworks
• Exposure to survey data, longitudinal studies, or research ethics/data protection.

Benefits

As well as a competitive salary, an excellent working environment (including a home/office hybrid working environment), you will be working for the largest independent social research organisation in the UK. We are proud of the benefits we offer our employees which include:

• 25 days holiday (plus 8 bank holidays) and flexible working
• Generous company pension scheme
• Life insurance and health cash plan
• Cycle to Work scheme and season ticket loan
• Career development, professional subscriptions support and mentoring support from industry-leading experts.