Job Description

We seeking a Data Engineer to reports to the National Manager: Data and BI, and is responsible for designing, building, and maintaining scalable data pipelines and architecture. The role includes supporting both batch and real-time data processing to ensure data is readily available for analytics and reporting, and the development of AI-enabled predictive analytics capabilities. The Data Engineer will collaborate with BI developers, analysts, and business stakeholders to translate business requirements into scalable data solutions.

Duties and Responsibilities:

Design, develop, and maintain data pipelines using Azure Data Factory, Synapse Pipelines, and Microsoft Fabric Dataflows. Integrate data from Dynamics 365 (Sales, F&O, Higher Ed) and external systems such as Celcat, Moodle, and Microsoft Graph API. Design and maintain star/snowflake schemas and data models for both?

data warehouses

?and?

Lakehouse

?architectures. Build and manage?

Lakehouse environments

?using Microsoft Fabric Lakehouse or Synapse Analytics to support scalable analytics. Implement?

event streaming

?solutions for real-time data ingestion and processing. Develop and maintain

reflexes

?(automated responses to data events) within Microsoft Fabric to support dynamic data workflows. Ensure data governance, security, lineage, and metadata management. Implement CI/CD pipelines for data workflows using Microsoft Fabric and tools like Azure DevOps or GitHub Actions. Develop data transformation and automation scripts using Python or Scala. Collaborate with BI developers, analysts, and business stakeholders to understand data requirements. Ensure data privacy and compliance with regulations such as POPIA and GDPR. Utilise Microsoft Fabric Data Science tools to build and deploy machine learning models.

Minimum Requirements for the job:

Qualifications / Education

*

A Bachelor's degree or equivalent NQF-7 qualification (Advanced Diploma or Bachelor of Technology) in Computer Science, Information Systems, or a related field. Microsoft Certified: Fabric Analytics Engineer Associate DP600. : A post-graduate qualification in the above (NQF Level 8 and above). Microsoft Certified: Fabric Data Engineer Associate (DP700) Microsoft Certified: Power BI Data Analyst Associate (PL300)

Experience:

A minimum of?

5 years of experience

?in data engineering, supporting both batch and real-time data processing. At least?

2 years in a senior data engineering capacity

, with responsibility for designing scalable data solutions. A proven track record of?

delivering robust data pipelines and architectures

?using Azure Data Factory, Synapse Pipelines, or Microsoft Fabric. Experience integrating data from?

CRM and ERP systems

?such as Dynamics 365 and learning platforms like Moodle. Demonstrated ability to implement?

data governance, lineage, and metadata management

?practices in compliance with POPIA and GDPR. Experience in?

establishing data engineering best practices

, including CI/CD workflows and automation using Azure DevOps or GitHub Actions. Exposure to?

machine learning workflows

?using Microsoft Fabric Data Science tools and Python-based frameworks (e.g., scikit-learn, TensorFlow) is

highly desirable

.

Technical / Functional Competencies:

Data pipeline development and orchestration. Data modelling and schema design. Data governance and metadata management. Machine learning model development and deployment. API integration and automation scripting. Microsoft Fabric Data Science tools for machine learning

Behavioural Competencies

Excellent communication and interpersonal skills, with the ability to build relationships and interact effectively with diverse stakeholders. Strong leadership and facilitation skills. Analytical thinking and problem-solving skills. Good organising and time management skills Ability to work independently and collaboratively in a team Adaptability, Innovation and Change Management.

Employee Benefits:

A competitive, market-related salary with opportunities to earn incentive performance bonuses A supportive, high-performing environment where innovation is encouraged A Hybrid remote working model - two days in office / three days working remotely. Subsidised academic studies through Eduvos for staff, their spouse/life partner, and dependent children, after 6 months of service. Provident Fund
Job Types: Full-time, Permanent

Work Location: In person