Data Engineer

About us: Turning the fantasy of analytics, data and A.I. into reality…

​

In a world where vast amounts of data are being created in a multitude of different ways, Lumilinks exist to help companies collate and use data in an automated and compliant way, creating live and actionable insights.​

We help businesses across the entire data journey, eliminating silos and creating data transparency. This allows our clients to be data confident in making strategic and tactical decisions that will further their business and create automation that improves processes, compliance, capability and reduces costs.​

The Role

As a Data Engineer at Lumilinks, you will play a pivotal role in designing, building, and maintaining our data architecture and pipelines. You will work closely with data scientists and analysts to ensure that high-quality, reliable data is readily available for analysis and decision-making. Your responsibilities will include developing scalable ETL processes, optimising data storage solutions, and implementing best practices for data management.

In this fast-paced environment, you will have the opportunity to innovate and experiment with cutting-edge technologies, contributing directly to the success of our data-driven initiatives. Your expertise will help us turn complex data into actionable insights, driving the growth and impact of our company.

This is an exciting opportunity to contribute to the expansion of our data science company and make a significant impact in the field.

Please speak to us if you have …..

…..the following professional aspirations

• Skill Enhancement: Aspires to deepen technical expertise in data engineering practices, including mastering tools and technologies like Apache Spark, Kafka, cloud platforms (AWS, Azure, Google Cloud), and data warehousing solutions.
• Career Progression: Aims to advance to a senior data engineer or data architect role, with long-term goals of leading data engineering teams or projects and influencing architectural decisions.
• Building Scalable Systems: Motivated to design and implement scalable, efficient, and robust data pipelines that can handle increasing data volumes and complexity as the company grows.
• Cross-Functional Collaboration: Seeks to work closely with data scientists, data analysts, and other stakeholders to understand their data needs and deliver solutions that empower data-driven insights and decision-making.
• Innovative Solutions: Eager to contribute to innovative projects that leverage emerging technologies and methodologies, ensuring the company stays competitive in the data landscape.
• Continuous Learning: Committed to lifelong learning through formal education, certifications, workshops, and staying current with industry trends and best practices in data engineering.
• Data Governance and Quality: Aspires to play a key role in establishing data governance frameworks and ensuring data quality and integrity across Lumilinks.
• Mentorship and Leadership: Interested in mentoring junior data engineers or interns, sharing knowledge and best practices, and contributing to a collaborative team culture.
• Impact on Business Outcomes: Aims to align data engineering initiatives with business goals, ensuring that the data infrastructure supports the company’s strategic direction and contributes to overall success.

….. the following personal attributes

• Analytical Mindset: Possesses strong analytical skills, with the ability to evaluate complex problems, identify patterns, and derive actionable insights from data.
• Detail-Oriented: Meticulous and thorough in work, ensuring high standards of data accuracy, consistency, and integrity in all engineering processes.
• Proactive Approach: Takes the initiative to identify areas for improvement in data systems and processes, actively seeking solutions to enhance efficiency and performance.
• Effective Communicator: Strong verbal and written communication skills, capable of conveying technical concepts to non-technical stakeholders clearly and effectively.
• Team-Oriented: Collaborative and able to work well in a team environment, valuing diverse perspectives and creating a positive and supportive atmosphere.
• Adaptable: Comfortable with change and ambiguity, able to pivot quickly when priorities shift in a dynamic start-up environment.
• Time Management: Skilled at managing multiple projects and deadlines, prioritising tasks effectively to ensure timely delivery of data solutions.
• Curiosity and Lifelong Learning: Naturally curious and motivated to continuously learn about new technologies, methodologies, and industry trends to stay at the forefront of data engineering.
• Resilient: Demonstrates resilience in the face of challenges, maintaining a positive attitude and persistence in solving complex problems.
• Integrity and Accountability: Upholds high ethical standards and takes responsibility for the quality of work, ensuring reliability and trustworthiness in data management practices.

…the following technical skills and knowledge

• Programming Languages: Proficient in languages commonly used in data engineering, such as Python and SQL. Familiarity with languages like Scala or Go can also be beneficial.
• Database Management: Strong knowledge of relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra) for efficient data storage and retrieval.
• Data Warehousing: Experience with data warehousing solutions, such as Amazon Redshift, Google BigQuery, Snowflake, or Azure Synapse Analytics, including data modelling and ETL processes.
• ETL Processes: Proficient in designing and implementing ETL (Extract, Transform, Load) processes using tools like Apache NiFi, Talend, or custom scripts. Familiarity with ELT (Extract, Load, Transform) processes is a plus.
• Big Data Technologies: Familiarity with big data frameworks such as Apache Hadoop and Apache Spark, including experience with distributed computing and data processing.
• Cloud Platforms: Proficient in using cloud platforms (e.g., AWS, Google Cloud Platform, Microsoft Azure) for data storage, processing, and deployment of data solutions.
• Data Pipeline Orchestration: Experience with workflow orchestration tools such as Apache Airflow or Prefect to manage and schedule data pipelines.
• Data Modelling: Strong understanding of data modelling concepts (e.g., star schema, snowflake schema) and best practices for designing efficient and scalable data architectures.
• Data Quality and Governance: Knowledge of data quality principles and experience implementing data governance practices to ensure data integrity and compliance.
• Monitoring and Performance Tuning: Experience with monitoring tools and techniques for optimising database and data pipeline performance, including query optimisation and resource management.
• APIs and Data Integration: Familiarity with RESTful APIs and experience in integrating data from various sources, including third-party services and internal systems.

…and the following experience, accreditations, and qualifications

• Educational Background: Bachelor’s degree in computer science, information technology, data engineering, data science, or a related field. Advanced degrees (e.g., Master's) can be an advantage but are not always required.
• Work Experience: Relevant professional experience as a data engineer or in a related role, typically ranging from 1 to 3 years. Experience in a startup environment is a plus, as it demonstrates adaptability and familiarity with fast-paced workflows.
• Internships or apprenticeships: Practical experience through internships or apprenticeships that involve data engineering tasks, data analysis, or software development.
• Certifications: Relevant certifications can enhance credibility and demonstrate expertise, such as: Google Cloud Professional Data Engineer, AWS Certified Data Analytics – Specialty, Microsoft Certified: Azure Data Engineer Associate, Cloudera Certified Professional (CCP) Data Engineer
• Projects Portfolio: A portfolio showcasing relevant work, such as projects completed during previous employment, internships, or personal initiatives that highlight data engineering capabilities.
• Familiarity with Agile Methodologies: Experience with Agile development practices (e.g., Scrum or Kanban), as many startups leverage these methodologies for project management.
• Soft Skills: Strong communication and collaboration skills to work effectively with cross-functional teams, including data scientists, analysts, and product managers.
• Continuous Learning: A commitment to staying current with industry trends, emerging technologies, and best practices in data engineering through self-study, workshops, or online courses.