Azure Data Engineer

CSA are a unique and multi-skilled team of cybersecurity scientists, data and analytics professionals, and engineers. Our mission is to harness the power of data, analytics, AI/ML, and cybersecurity science to innovate and advance BANK’s cybersecurity capabilities.

The Lead Cybersecurity Analytics Engineer will form part of the CSA Platform & Data Engineering Team, joining a global team of data technology professionals to deliver critical analytics engineering requirements for the strategic cybersecurity data lake and analytics platform.

The position is a mid-senior technical, hands-on delivery role, requiring knowledge of data engineering, cloud infrastructure and platform engineering, platform operations and production support.

The Lead Cybersecurity Analytics Engineer role will work closely with Principal engineers within the Platform & Data Engineering Team, contributing to and coordinating data engineering, data acquisition, cloud infrastructure and platform engineering, platform operations, and production support activities using ground-breaking cloud and big data technologies. There is ample room for growth in the role, with opportunities to develop skills across multiple domains, starting from a strong foundation in one or more core technical areas e.g., data engineering or cloud infrastructure engineering. For this role, cybersecurity-specific knowledge is preferred but not essential.

The ideal candidate will possess strong technical skills, an eagerness to learn, a keen interest in Cybersecurity, the ability to work collaboratively in a fast-paced environment, and an aptitude for picking up new tools and techniques on the job, building on existing skillsets as a foundation.

Principal Accountabilities:

Data Engineering & Data Acquisition

§ Ingestion and provisioning of raw datasets, enriched tables, and/or curated, re-usable data assets to enable Cybersecurity use cases.

§ Driving improvements in the reliability and frequency of data ingestion including increasing real-time coverage.

§ Support and enhancement of data ingestion infrastructure and pipelines.

§ Designing and implementing data pipelines that will collect data from disparate sources across the enterprise, and from external sources, transport said data, and deliver it to our data platform.

§ Extract Translate and Load (ETL) workflows, using both advanced data manipulation tools and programmatically manipulating data throughout our data flows, ensuring data is available at each stage in the data flow, and in the form needed for each system, service, and customer along said data flow.

§ Identifying and onboarding data sources using existing schemas and, where required, conducting exploratory data analysis to investigate and determine new schemas.

Required Skills

The successful candidate will be a student of the Google Site Reliability Engineering (SRE) philosophy as applied to managing large-scale cloud infrastructure, possess skills and experience within one or more of the following areas, and demonstrate a willingness to learn additional skills via certification and/or on-the-job learning where required. 

Programming, Software & Network Principles

§ Experience with SRE and Azure DevOps

§ Ability to script (Bash/PowerShell, Azure CLI), code (Python, C#, Java), query (SQL, Kusto query language) coupled with experience with software versioning control systems (e.g., GitHub) and CI/CD systems.

§ Programming experience in the following languages: PowerShell, Terraform, Python Windows command prompt and object orientated programming languages.

§ Demonstrable experience of Linux administration and scripting (preferably Red Hat Systems)

§ Understanding of hardware and software principles and storage technologies (SSD, HDD, NVMe), CPU architectures, and Memory & Operating system principles (especially network stack fundamentals)

§ Understanding of network protocols and network design

Data Engineering & Data Acquisition

§ Data Acquisition, Cloud-based Data Pipelines (Azure preferred)

§ Data Transport and Data Cleaning

§ Data Engineering pipeline automation, productionisation, and optimisation

§ Designing, building, and maintaining data pipelines and ETL workflows across disparate datasets

§ Cloud Cost optimisation

§ Dataset and Data Asset Curation

§ Data Modelling and Cataloguing

§ Database Architecture and Design

§ Data Warehousing and Data Integration

§ Real-Time Analytics Deployment for Large-Scale Datasets

§ Applying data engineering methods to the cyber security domain

Technology Stack

§ Technical knowledge and breadth of Azure technology services (Identity, Networking, Compute, Storage, Web, Containers, Databases)

§ Cloud & Big Data Technologies such as Azure Cloud, Azure IAM, Azure Active Directory (Azure AD), Azure Data Factory, Azure Databricks, Unity Catalog, Azure Functions, Azure, Kubernetes, Service, Azure Logic App, Azure Monitor, Azure Log Analytics, Azure Compute, Azure Storage, Azure Data Lake Store, S3, Synapse Analytics and/or PowerBI

§ Experience with server, operating system, and infrastructure technologies such as Nginx/Apache, CosmosDB, Linux, Bash, PowerShell, Prometheus, Grafana, Elasticsearch)

§ Experience with Infrastructure-as-Code and Automation tools such as Terraform, Chef, Ansible, CloudFormation/Azure Resource Manager (ARM)

§ Streaming platforms such as Azure Event Hubs or Kafka, and stream processing services such as Spark streaming

§ Experience with Security Information & Event Management (SIEM) and Security Orchestration, Automation & Response (SOAR) technologies, especially cloud based, is a significant asset

Education and Experience:

The successful candidate will possess:

§ Positive attitude, strong work ethic and passion for learning.

§ Bachelor’s degree in Computer Science, Software Engineering, Data Science, or a related field advantageous.

§ Bachelor’s degree in any Science, Technology, Engineering, or Medical (STEM) field accompanied with data, technology and/or programming experience also considered.

§ Applicants without a Bachelor’s degree will be considered if they have demonstrable career experience within one or more of the required areas and/or have relevant industry certifications in Cloud, Data, or Cybersecurity.

§ CSA are a supportive team who welcome candidates from diverse skills and educational backgrounds. Thus, candidates with transferrable skills gained through software development or any other technology background will also be considered.

§ Proficiency in programming languages such as Python, Java, C#, or similar preferred.

§ Experience operating in highly regulated industry, e.g., Financial Services advantageous.

§ Understanding of cybersecurity principles, global financial services business models, regional compliance regulations and applicable laws advantageous.

§ Understanding and knowledge of common industry cybersecurity frameworks, standards, and methodologies, including OWASP, ISO2700x series, PCI DSS, GLBA, Global data security and privacy acts, FFIEC guidelines, CIS, and NIST standards advantageous.

Governance and Committee Memberships

Programme boards; daily/weekly management reporting and governance meetings.

• Supports the development of the Cybersecurity Science & Analytics Team, making sustainable decisions that protect and enhance BANK’s values, reputation, and stakeholder value.

• Builds rapport and mutual understanding to communicate and create opportunities for cross-business and/or international working, encouraging debate and open discussion. 

• Encourages people to build sustainable relationships beyond transactional levels and use empathy and insight to build better understanding of mutual benefits.