Job Description

Position summary
Industry: FMCG & Supply Management
Job category: FMCG, Retail, Wholesale and Supply Chain
Location: Western Cape
Contract: Permanent
Remuneration: Market Related
EE position: No
Introduction
The Engineer Technical Manager: Cloud leads a team focused on designing, developing, and optimizing cloud-based data products that support analytics, reporting, and AI/ML workloads. The primary responsibility is to build scalable and high-performance data pipelines, ensuring that structured and unstructured data is ingested, processed, and made available for business insights. This role works closely with platform and cloud architecture teams to align data solutions with enterprise cloud strategies, security policies, and governance best practices while providing high-quality, well-structured data for analytics and AI/ML use cases.
Job description

• Develop and optimise cloud-based data products that power analytics, AI/ML, and operational reporting.
• Design and implement scalable, high-performance data pipelines that transform and enrich raw data into structured, reusable datasets.
• Ensure efficient batch and real-time data processing, enabling seamless integration across engineering teams.
• Optimise data pipeline performance to reduce latency and enhance reliability for downstream analytics and AI workloads.
• Develop, implement, and maintain cloud-native data pipelines using AWS services like Glue, Lambda, Step Functions, Kinesis, and EMR.
• Automate data ingestion, transformation, and delivery into data lakes, data warehouses, and analytics platforms.
• Implement change data capture (CDC) and event-driven architectures to support real-time data needs.
• Ensure seamless data integration across structured and unstructured sources (databases, streaming platforms, APIs, etc.).
• Ensure high-quality, structured data availability for AI/ML teams, enabling efficient model training and inference.
• Develop reusable, well-governed datasets for engineering teams, supporting self-service access and automation.
• Work with AI/ML engineers to enable optimised data flows for feature engineering and machine learning pipelines.
• Support advanced data transformations, ensuring data products are aligned with engineering-driven insights and automation.
• Develop CI/CD pipelines for automated testing, deployment, and monitoring of data pipelines.
• Use Infrastructure as Code (IaC) tools like Terraform and CloudFormation to automate provisioning and scaling.
• Implement version control, testing, and rollback strategies to ensure data integrity and reliability.
• Ensure compliance with data governance frameworks, metadata management, and lineage tracking.
• Work closely with platform and cloud architecture teams to align with security and compliance standards.
• Implement automated data quality checks and observability tools to maintain high data accuracy.
• Lead and mentor a team of cloud data engineers, ensuring best practices in cloud-based data product development.
• Collaborate with engineering teams across data, AI/ML, and platform functions to ensure seamless integration.
• Align with platform and cloud architects to leverage cloud infrastructure effectively for data engineering needs.

• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field.
• AWS Certifications (AWS Data Analytics, AWS Solutions Architect) are highly advantageous.
• 5 years of experience in data engineering, with at least 2 years focused on cloud-based data product development.
• Experience in building scalable data products for analytics, reporting, and AI/ML workloads.
• Proven expertise in developing and managing cloud-based data pipelines, with a strong focus on automation, scalability, and optimisation.
• Experience working in a cross-functional engineering environment, collaborating with data engineers, AI/ML teams, and platform architects.
• Expertise in building and optimising cloud-based data pipelines using AWS services such as Glue, Lambda, Step Functions, and EMR.
• Strong knowledge of AWS data services, including S3, Redshift, Snowflake, DynamoDB, and Athena.
• Experience working with Kafka, Kinesis, and CDC-based data flows.
• Expertise in Git, Jenkins, Terraform, and CloudFormation for deploying automated data pipelines.
• Proficiency in Python, SQL, and PySpark for data transformation and engineering workflows.
• Ability to monitor, troubleshoot, and optimise data pipelines using AWS CloudWatch.

• Collaboration & Communication: Strong ability to work with engineering, AI/ML, and platform teams.
• Problem-Solving: Ability to troubleshoot and optimise data pipelines for large-scale, cloud-based workloads.
• Continuous Learning: Passion for staying up to date with emerging trends in data engineering and cloud computing.
• Leadership & Mentorship: Proven ability to mentor and grow a team of cloud data engineers.
• Adaptability: Thrives in a fast-moving, evolving technology landscape, adjusting strategies as needed.
• Cultural Fit: Demonstrates integrity, accountability, and a commitment to fostering a collaborative and data-driven organisation.