Job Description

Big Data Engineer (Python, Java, Analytics, Storage)
Role Overview:
We're looking for a highly skilled Big Data Engineer to help architect and implement large-scale data storage and analytics platforms as part of a global client implementation. You'll be responsible for managing data flow from ingestion through to storage and insight generation, using both open-source and enterprise technologies.
Key Responsibilities:

• Design and build end-to-end data pipelines from ingestion to analytics-ready format.
• Leverage Python and Java to process large datasets, build transformation logic, and optimize performance.
• Integrate real-time and batch data flows using Kafka, RabbitMQ, and NiFi.
• Model and manage scalable data storage solutions for analytics using HDFS, S3, or data lake tools.
• Support business and client teams with accessible, structured datasets and metadata tagging.
• Enable searchability and observability using ElasticSearch and visual tools.

• 4+ years' experience in big data engineering, with proven projects in implementation settings.
• Proficient in Python and Java for large-scale ETL and transformation.
• Experience with Kafka, RabbitMQ, NiFi, and streaming architecture design.
• Deep understanding of data lake / warehouse design, data partitioning, and indexing.
• Experience with ElasticSearch and Zookeeper in a production context.
• Familiarity with security and compliance practices in multi-tenant data environments.

• Experience integrating with BI tools or data visualization platforms.
• Knowledge of ML model deployment pipelines.

• Eligibility for Top-Level Security Clearance:

• Candidates must be eligible to obtain and maintain security clearance at the highest level, in accordance with applicable national security regulations.
• On-Site Work Requirement:

• This role requires full-time, on-site presence at the client's premises located in Pretoria. Remote or hybrid work arrangements are not applicable.