Job Description

About the Role
We are seeking a highly skilled Senior Data Engineer to join our dynamic data team. You will be responsible for designing, building, and maintaining robust and scalable data pipelines and architectures that enable analytics and business intelligence at enterprise scale. This role is ideal for professionals who thrive in complex data environments, value clean, efficient, and secure data engineering practices, and enjoy collaborating with cross-functional teams to deliver high-quality, reliable data solutions.
Key Responsibilities * Data Architecture & Pipeline Development

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes
• Implement data modeling and architecture solutions including Data Vault, dimensional modeling, and normalized structures
• Optimize data storage and retrieval to support analytics, reporting, and machine learning workloads
• Integrate structured and unstructured data from multiple sources, including cloud and on-premises systems
• Develop and maintain data warehouses, data lakes, and cloud-native data platforms
• Cloud & Big Data Engineering

• Work with cloud platforms such as AWS, Azure, or GCP for data storage, processing, and orchestration
• Implement and manage data processing frameworks such as Spark, Databricks, or Hadoop
• Develop scalable, automated, and reliable batch and real-time data workflows
• Ensure data pipelines meet performance, security, and compliance standards
• Collaboration & Process Improvement

• Collaborate with data analysts, data scientists, software engineers, and business stakeholders to understand requirements and deliver solutions
• Mentor junior and intermediate data engineers, promoting best practices and code quality standards
• Participate in Agile/Scrum ceremonies, including sprint planning, stand-ups, and retrospectives
• Continuously research and recommend new tools, technologies, and approaches to improve data engineering efficiency, reliability, and scalability

Requirements & Qualifications
Must-Have Skills:

• 6+ years of professional experience in data engineering or software engineering with a focus on data
• Strong expertise in SQL, relational databases (PostgreSQL, MSSQL, MySQL), and NoSQL databases (MongoDB, Cassandra, DynamoDB)
• Experience with Data Vault modeling and implementation
• Hands-on experience with ETL/ELT tools and processes
• Proficiency in programming/scripting languages such as Python, Scala, or Java
• Experience building data pipelines and architectures in cloud platforms (AWS, Azure, GCP)
• Strong understanding of data warehousing, data lakes, and cloud-native storage solutions
• Experience with CI/CD for data pipelines and automation of data workflows

Highly Desirable Skills:

• Experience with big data frameworks: Spark, Hadoop, Kafka
• Familiarity with orchestration tools such as Airflow, Prefect, or Azure Data Factory
• Knowledge of machine learning pipelines and integrating data for analytics
• Understanding of data governance, security, and compliance best practices
• Exposure to containerization (Docker, Kubernetes) in data workloads
• Strong communication skills to present complex data concepts to technical and business stakeholders

Education

• Bachelor's degree, higher certificate, or diploma in Data Engineering, Computer Science, Information Systems, or a related field

Additional Information

• Contract Duration: 12 Months (with possibility of renewal)
• Salary: Negotiable
• Location Options: Johannesburg / Cape Town / Remote
• Must be located in South Africa or have a valid work permit for South Africa

For a more comprehensive list of opportunities, please visit our careers page: https://www.idbasesoftware.com/

Skills Required