- Terraform
- Python 3.x
- SQL (Oracle/PostgreSQL)
- PySpark
- Boto3
- ETL
- Docker
- Linux / Unix
- Big Data
- PowerShell / Bash
- Enterprise Collaboration tools (e.g., Confluence, JIRA)
- Experience with data formats: Parquet, AVRO, JSON, XML, CSV
- Experience with Data Quality Tools (e.g., Great Expectations)
- Experience with REST APIs
- Basic Networking and troubleshooting skills
- Expertise in data modeling (Oracle SQL)
- Strong analytical skills for large data sets
- Testing and data validation experience
- Precise documentation skills
- Self-driven with ability to work independently and in teams
- Experience with AWS Glue, Data Pipeline, or similar
- Familiarity with AWS S3, AWS RDS, DynamoDB
- Knowledge of software design patterns
- Specification preparation for programming
- Organizational skills
- AWS Components knowledge: Glue, CloudWatch, SNS, Athena, S3, Kinesis Streams, Lambda, DynamoDB, Step Function, Param Store, Secrets Manager, CodeBuild/Pipeline, CloudFormation, BI Experience, technical data modeling, Kafka, AWS EMR, Redshift
- Relevant IT/Business/Engineering Degree
- Certifications: AWS Certified Cloud Practitioner, AWS Certified SysOps Associate, AWS Certified Developer Associate, AWS Certified Architect Associate, AWS Certified Architect Professional, HashiCorp Certified Terraform Associate
-
Junior Data Engineer
14 hours ago
Qualifications: IT Degree · Microsoft Certified: Data Engineer · Requirements: 3+ years in a similar role · Azure, SQL, Python, ETL · Data Formats: JSON, CSV, XLSM, XML · Stored procedures · KPAs: Build and maintain data pipelines and ETL processes · Profile data for quality, com ...
-
Senior Manager Data Engineer
1 week ago
RESPONSIBILITIES · Create and execute a long-term strategy for leveraging data and analytics as a key business enabler and competitive differentiator across the business. · Take responsibility for developing and delivering a key element of the organisation's data management syste ...
-
Data Engineer
4 days ago
E-Merge Sandton, South AfricaA Software company with premier custom software and product development, delivering custom-made solutions for large enterprises as well as dynamic start-ups is looking for a Data Engineer to join an environment of like-minded engineers who are able to express themselves freely an ...
-
Data Engineer
1 week ago
RPO's client is seeking an experienced Data Engineer to join their innovative team at a consultant or manager level. This role offers a unique opportunity to work on groundbreaking projects within the financial services industry, collaborate with top-tier professionals, and gain ...
-
Data Engineer
3 days ago
Description · To provide required support within guidance of set processes and procedures to ensure effective movement, collection, integration, storage and provisioning of data to build solutions and meet business objectives through understanding of business unit requirements a ...
-
Data Engineer
3 days ago
Description · Hello Future Data Engineer III (Generative AI/NLP) · Welcome to FNB, the home of the #changeables. We design for the shapeshifters and deliver products and services that make us incredibly proud of people that make it happen. · As part of our talent team, you will ...
-
Data Engineer
1 week ago
Datonomy Solutions Johannesburg North, South AfricaThe Group Data and Analytics Office is on the lookout for a skilled Ab Initio Data Engineer. The ideal candidate will have extensive experience in data engineering, particularly with Ab Initio, and will be responsible for designing, developing, and maintaining the data infrastruc ...
-
Data Engineer
4 days ago
A technology and analytics driven consulting company empowering clients by solving real-world business challenges that deliver tangible value and count their core capabilities as spanning the disciplines of Data Science, Data Engineering and Software Engineering have an opening ...
-
Data Engineer
1 week ago
A leader in the Banking and Financial space is on the lookout for a Data Engineer to join their team. · The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company's Data Architecture Roadmap, to advance technical ...
-
Data Engineer
1 week ago
We are seeking a skilled Data Engineer with strong expertise in business intelligence (BI) and Oracle systems to join our dynamic team on a contract role within the Financial Services Industry. The successful candidate will work on designing, developing, and maintaining data pipe ...
-
Data Engineer
1 week ago
Main Purpose: · Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. · Ensure the data lake contains accurate, up-to-date, and high-quality data, enabling data scientists t ...
-
Data Engineer
1 week ago
Job Description · Make an impact with NTT DATA · Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and i ...
-
Data Engineer
4 days ago
Preferred Qualifications:Qualifications: Education: Bachelors or Masters degree in Computer Science, Engineering, Information Technology, or a related field · Experience: 5+ years of experience in data engineering and ETL development · Extensive experience with Ab Initio, includi ...
-
Data Engineer
1 week ago
We currently have numerous vacancies for Data Engineers skilled in tools such as the Microsoft Stack (SSIS, SSRS, SSAS), Power BI, AWS, Azure, DataStage, SAS etc. · Duties: · Design, develop, test and deploy ETL for ODS and data mart projects, as well as application and managemen ...
-
Data Engineer
4 hours ago
We are seeking a skilled Data Engineer with strong expertise in business intelligence (BI) and Oracle systems to join our dynamic team on a contract bases for a company within the Financial Services. The successful candidate will work on designing, developing, and maintaining dat ...
-
Data Engineer
1 week ago
We help our clients understand the rich data we collect from the traditional retail market in South Africa and Africa broadly. The ideal candidate will have a strong analytical and mathematical background to help drive value from our data. This person will wear many hats in the r ...
-
Data Engineer
1 week ago
An IT solution company dedicated to transforming data into actionable insights that drive business decisions are looking for a talented Data Engineer to join our dynamic team and play a pivotal role in leveraging cutting-edge technologies to optimize our data processes. · Respons ...
-
Data Engineer
5 days ago
A leading telecommunications company, with innovative solutions from designing, building, deploying, and managing high performance computing infrastructure and an innovative approach with a focus on innovation topics such as Development and Operations for are looking for an Azure ...
-
Data Engineer
5 days ago
Role overview: · We are looking for an experienced Data Engineer to design and manage a resilient data pipeline that effectively processes and integrates both structured and unstructured data into data lakes, warehouses, databases, and other data sources. In this role, you will t ...
-
Data Engineer
3 days ago
Overview: · Our client, a well-established hedge fund in South Africa with a long history of delivering excellent returns, is seeking a talented Data Engineer to join their data analytics team. They are committed to growing their business and technology capabilities, utilizing t ...
AWS Data Engineer - Midrand, South Africa - Liyema Consulting
Description
Data Engineers build and maintain Big Data Pipelines using Data Platforms, ensuring data integrity and security aligned with information classification requirements.
Essential Skills
Advantageous Skills
Qualifications/Experience
Role and Responsibilities
Data Engineers at our IT Hub are responsible for building and maintaining Big Data Pipelines using Data Platforms. They ensure data integrity and security aligned with information classification requirements, enabling secure data sharing on a need-to-know basis.
