AWS Data Engineer

HCLTech is a global technology company, home to more than 220,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion.

Job Description:

We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy.

Skills Must have :

• Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance optimization.
• Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats.
• Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library.
• Experience implementing data retention and archival strategies in cloud environments.
• Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured data processing.
• Advanced programming skills in Python, PySpark, and SQL, with the ability to build modular, efficient, and scalable data pipelines.
• Deep expertise in data modeling for both relational databases and data warehouses, including Star and Snowflake schema designs.
• Extensive experience working with AWS Redshift and Aurora for data warehousing and transactional workloads.
• Experience using dbt (Data Build Tool) for building modular, version-controlled, and testable data transformation workflows, with a strong understanding of modeling best practices and deployment strategies.
• Solid knowledge of infrastructure as code using AWS CloudFormation and Terraform to automate and manage cloud resources.
• Experience developing ETL workflows using AWS Glue, querying data with Athena, and managing access and governance with Lake Formation.
• Strong command of AWS Lambda for serverless data processing and Boto3 for programmatic interaction with AWS services.
• Demonstrated experience working with GDPR-compliant architectures and handling sensitive data, ensuring data privacy, encryption, and access control in accordance with regulatory standards.
• Familiarity with data anonymization and masking techniques for handling sensitive datasets.
• Proficiency in containerization using Docker for packaging and deploying data applications in local and cloud environments.
• Experience writing unit tests and integrating data pipelines into CI/CD workflows using AWS CodeBuild, CodePipeline, and GitHub, including signed commits and tagging strategies.
• Ability to collaborate effectively with both technical and non-technical stakeholders, translating business requirements into technical solutions.
• Strong documentation skills to clearly articulate data flows, architecture decisions, and technical findings.
• Experience with pair programming, conducting code reviews, and mentoring team members.
• Proven ability to reconcile data across systems and ensure data quality and integrity.
• Experience with data cataloging and metadata management using tools like AWS Glue Data Catalog.
• Demonstrated self-sufficiency in exploring new tools, troubleshooting issues, and continuously improving processes.
• Hands-on experience with Apache Airflow for orchestrating complex data workflows and ensuring reliable execution.
• Understanding of cloud security and governance practices including IAM, KMS, and data access policies.
• Experience with monitoring and observability tools such as CloudWatch.
• Experience working in Agile/Scrum environments, participating in sprint planning, retrospectives, and backlog grooming.

Good to Have:

• Exposure to Azure data services such as Azure Data Factory, Synapse, or Blob Storage.
• Familiarity with AWS Lambda Powertools for structured logging, tracing, and metrics in serverless applications.
• Experience with cost optimization strategies in AWS, including resource tagging, budgeting, and usage analysis.