Job Description

Overall Purpose of the Job (Please provide a summary of why the job was created and what the main responsibilities are)

The data engineer is responsible for ingesting and transforming a wide range of data across the business into formats that can be used by end-users to develop differentiating business insights. By structuring data requirements into logical data constructs, the Data Engineer enables optimal, physical implementation of the data structure by generating data models. A key deliverable is the development of visual reports, dashboards and KPI scorecards.

The role will involve collaborating with internal and external resources to ensure the data analytics team have the required data from the source systems as well as be responsible for the accuracy of data loaded into the Data Warehouse

Education (Formal Qualification required)

Diploma in IT or related field

Bachelor's degree in (Information Systems, Informatics, Computer Systems or Information Technology) - NQF 7 Or National Diploma / Diploma in Software Development /IT- NQF 6

SQL/ Data engineering certification

Azure Data Certification

Experience (Minimum Experience required - type and number of years- experience can run concurrently)

Job Title / Function

Minimum time spent in job

SQL Developer


Year(s)



5

BI Developer


Year(s)



5

Data Engineering


Year(s)



3

Synapse pipelines / Data factory / SSIS


Year(s)



3

Data analysis / data modelling/ report building

Creates data model designs in accordance with recommended best practice Collaborates with team members and business to translate business requirements into data requirements Analyzes and profiles the source data to understand data quality issues, relationships, patterns, and rules in the data Structures data requirements into logical data constructs based on recommended best practice including ERD models and dimensional models to ensure optimal implementation Compiles Source to Target Mapping Specifications with the Business Intelligence Business Analyst (BIBA) Ensures complete and correct data by identifying definitive or authoritative sources of data; analyzing source data; and identifying gaps Enables physical implementation of the data structure by generating the physical data model from the logical data model Cleans and maintains databases by restructuring and deleting old data/stored procedures Develops visual reports, dashboards and KPI scorecards using Power BI desktop. Connects to data sources, importing and transforming data for Business Intelligence Implements row level security on data, demonstrating an understanding of application security layer Writes DAX queries in Power BI.

ETL and pipeline development

Assists in designing and implementing scalable and robust processes for ingesting and transforming complex datasets. Develops the work packages as per the design to deliver elements of the data warehouse architecture (data ingestion, ETL, etc.,) Provides technical support, troubleshooting and upgrades setup or support on ETL and data warehouse related issues Analyzes and maps data from source systems to target data warehouse Participates in all aspects of quality assurance Monitors and maintains ETL pipelines and data warehouse Develops the data dictionary

Task tracking and Support Resolution

Makes use of DevOps to track tasks, effort estimates, effort completed and repos Responds timeously to support calls and tracks issues to completion

Continuous improvement

Owns the development of elements of centralised, automated data improvement initiatives under the guidance of the Team Lead Makes increased contributions by broadening and enhancing individual skills. Keeps abreast of industry trends and best practice and makes recommendations to management Investigates new technologies to improve and optimize standard reporting.

Skill

Weighting Importance

(Please note that this should add up to 100%)

Data modelling


15%

Excellent communication skills


5%

Analytical and problem solving


15%

Microsoft Power BI


15%

Azure Synapse SQL and pipelines


20%

Microsoft SQL Server Technologies


30%

Behavioural Attribute

Assertive Timeous Decisiveness Team player * Innovation