Senior Data Engineer Full Time NEW

Hire Resolve’s client is seeking a highly skilled Senior Data Engineer to join their dynamic team within the telecommunications domain. The ideal candidate will be responsible for designing, developing, and maintaining scalable and reliable data pipelines to support the ingestion, processing, and analysis of Call Detail Records (CDRs) and real-time data streams.

This role demands advanced technical expertise in stream processing frameworks such as Apache Kafka, Apache NiFi, Apache Flink, and Apache Druid, along with strong collaboration skills and a passion for data-driven solutions.

Responsibilities: 

• Design and implement robust ETL/ELT pipelines for ingesting and transforming CDRs and telecom log data across multiple network types (2G/3G/4G/5G).
• Develop and maintain real-time data pipelines using Apache Kafka, NiFi, and Flink.
• Structure and integrate data for analytics platforms such as Apache Druid, Hudi, and Superset.
• Manage large-scale distributed file systems and ensure efficient batch/stream data processing.
• Build and manage CI/CD pipelines for data workflow deployments.
• Optimize data performance, cost-efficiency, and reliability through continuous monitoring and enhancement.
• Enforce data governance best practices including schema validation, access control, and regulatory compliance (e.g. GDPR).
• Collaborate with development, DevOps, and analytics teams to deliver query-ready datasets for machine learning models and BI dashboards.
• Implement role-based access control, data encryption, and secure identity management.
• Design and maintain metadata repositories, data lineage, and asset discovery frameworks.
• Build quality assurance and anomaly detection tools to monitor data integrity across all pipeline stages.
• Support analytical modelling using OLAP systems and columnar databases.

 Requirements: 

• Bachelor’s Degree in Computer Science, Software Engineering, Computer Engineering, or related field.
• Minimum of 5–8 years of experience in data engineering, preferably in telecom or large-scale distributed systems.
• Strong expertise in Apache stack tools: Kafka, NiFi, Flink, Spark, Hive, Druid, Hudi.
• Proficiency in Python, SQL, Shell scripting, and Linux-based systems.
• Experience in database management: PostgreSQL, MySQL, and analytical querying.
• Familiarity with data serialization formats (JSON, Avro, etc.) and secure data transfer protocols (SFTP).
• In-depth knowledge of data quality control, lineage tracking, and data discovery tools.
• Experience in full-text search and indexing of structured/unstructured data using Apache or similar tools.
• Excellent problem-solving abilities, multitasking, and a fast-learning attitude.

How To Apply:

• Contact Hire Resolve today for your next career-changing move
• Our client is offering a highly competitive salary for this role based on experience.
• Send your CV to: itcareers@hireresolve.za.com or connect with Mischa Bornman via LinkedIn.
• Alternatively, you can also contact me directly at Hire Resolve mischa.bornman@hireresolve.us 

We will contact you telephonically in 3 days should you be suitable for this vacancy. If you are not suitable, we will put your CV on file and contact you regarding any future vacancies that arise.