Jobs
>
Cape Town

    AWS Data Engineer - Cape Town, South Africa - PBT Group

    PBT Group
    PBT Group Cape Town, South Africa

    Found in: Talent ZA C2 - 4 days ago

    PBT Group background
    Full time
    Description

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.

    The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources, ensuring its conformity to data quality standards, and enabling downstream users to access data promptly. This position is an integral part of an agile team.

    These professionals are entrusted with the responsibility of establishing the infrastructure required to derive insights from raw data, integrating data from various sources seamlessly. They empower solutions by efficiently managing substantial volumes of data, both in batch and real-time, utilizing cutting-edge technologies from the realms of big data and cloud computing. Additional responsibilities encompass the development of proof-of-concepts and the implementation of intricate big data solutions, with a primary focus on collecting, parsing, managing, analyzing, and visualizing extensive datasets. They are adept at employing technologies to resolve challenges associated with handling vast amounts of data in diverse formats, thereby delivering innovative solutions.

    Data Engineering is a technically demanding role that necessitates a broad spectrum of expertise in software development and programming. These professionals possess knowledge in data analysis, understanding end-user and business requirements, and have the ability to translate these needs into technical solutions. They exhibit a strong grasp of physical database design and the systems development lifecycle. Collaboration within a team environment is essential for success in this role.

    Key Responsibilities:

    • Architecting Data analytics framework.
    • Translating complex functional and technical requirements into detailed architecture, design, and high-performance software.
    • Leading the development of data and batch/real-time analytical solutions by leveraging transformative technologies.
    • Engaging in multiple projects as a technical lead, overseeing user story analysis, design, software development, testing, and automation tool creation.

    Duties: Primary Job Objectives:

    • Development and Operations
    • Database Development and Operations
    • Establishment and Adherence to Policies, Standards, and Procedures
    • Communication
    • Business Continuity and Disaster Recovery Planning
    • Research and Evaluation
    • Coaching and Mentoring

    Required Skills, Knowledge, and Experience:

    • A minimum of 5 years of experience in Data Engineering or Software Engineering.
    • Demonstrated leadership experience, managing teams of engineers for 3-5 years.
    • A minimum of 2 years of experience in Big Data.
    • At least 5 years of experience with Extract, Transform, and Load (ETL) processes.
    • A minimum of 2 years of experience with AWS (Amazon Web Services).
    • Demonstrated experience with agile or other rapid application development methodologies for at least 2 years (e.g., Agile, Kanban, Scrum).
    • 5 years of proven expertise in object-oriented design, coding, testing patterns, and working with commercial or open-source software platforms and large-scale data infrastructures.
    • Proficiency in creating data feeds from on-premise to AWS Cloud (2 years).
    • Support experience for data feeds in production on a break-fix basis (2 years).
    • A minimum of 4 years of experience in creating data marts using Talend or similar ETL development tools.
    • Proficiency in data manipulation using Python and PySpark (2 years).
    • Experience in processing data using the Hadoop paradigm, particularly with EMR, AWS's distribution of Hadoop (2 years).
    • DevOps experience in Big Data and Business Intelligence, including automated testing and deployment (2 years).
    • Extensive knowledge of various programming or scripting languages.
    • Expertise in data modeling and an understanding of different data structures and their suitability for specific use cases.

    Additional Technical Skills Required:

    • The ability to design highly scalable distributed systems using various open-source tools.
    • Proficiency in both batch and streaming Big Data tools.
    • Experience with Talend for at least 1 year.
    • Familiarity with AWS services such as EMR, EC2, and S3 for at least 1 year.
    • Proficiency in Python for at least 1 year.
    • Familiarity with PySpark or Spark (desirable for at least 1 year).
    • Experience in Business Intelligence data modeling for 3 years.
    • Proficiency in SQL for 3 years.

    Desirable:

    • 5+ years of experience in Retail Operations.

    Qualifications/Certifications:

    • A Bachelor's degree in computer science, computer engineering, or equivalent work experience for a minimum of 4 years.
    • AWS Certification, at least at the associate level.

    * In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

  • PBT Group

    Data Engineer

    Found in: Executive Placements ZA C2 - 2 days ago

    PBT Group Cape Town, South Africa

    We currently have numerous vacancies for Data Engineers skilled in tools such as the Microsoft Stack (SSIS, SSRS, SSAS), Power BI, AWS, Azure, DataStage, SAS etc. · Duties: · Design, develop, test and deploy ETL for ODS and data mart projects, as well as application and managemen ...

  • IOCO

    Data Engineer

    Found in: Executive Placements ZA C2 - 2 days ago

    IOCO Cape Town, South Africa

    We are seeking an experienced Data Migration Engineer to join our team on a contract basis to facilitate this migration process. The ideal candidates will have strong expertise in data engineering, particularly with Azure Synapse services, SSIS, and API integration. · What you'll ...

  • Dimension Data

    Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    Dimension Data Cape Town, South Africa temporary

    As a Data Engineer, you'll design and build the data architecture of our products. You will use your technical knowledge to help us develop new products by taking an idea from concept to reality. You'll work on projects across multiple teams in order to achieve company goals and ...

  • DLK Group

    Data Engineer

    Found in: Talent ZA C2 - 3 days ago

    DLK Group Cape Town, South Africa Contract

    Job Description · Modifying the database structure, asnecessary, from information given by application developers. · Enrolling users and maintaining systemsecurity. · Controlling and monitoring user accessto the database. · Monitoring and optimizing the performanceof the databas ...

  • 60degrees

    Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    60degrees Cape Town, South Africa Permanent Full time

    Job Description · Data Engineer (Gauteng) · Compensation: R – R per annum · Working model: Hybrid · The opportunity that awaits you: · Our client, a solutions energy engineering solution provider with projects mainly in Sub-Saharan Africa seeks to expand their team. Our cl ...

  • Armstrong Appointments

    IT Data Engineer

    Found in: Talent ZA 2A C2 - 5 days ago

    Armstrong Appointments Cape Town, South Africa

    Our client based in Cape Town are currently seeking the skills of an IT Data Engineer to assist in designing and implementing scalable and robust processes to support the data engineering capability. This role will be responsible for extracting and transforming massive amounts of ...

  • Isilumko Staffing

    Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    Isilumko Staffing Cape Town, South Africa Full time

    Degree - Computer Science / Engineering / Informatics / Applied Mathematics / Statistics / Information Systems · Min 3-5 years with Microsoft BI Stack - SSIS, SSRS and T-SQL · MS SQL Server and Cloud Data Systems · Relational Database Management · Data modelling · Systems analysi ...

  • iSanqa

    Data Engineer

    Found in: Talent ZA C2 - 2 days ago

    iSanqa Cape Town, South Africa Full time

    Requirements · Solid 5+ years of experience with Azure Data Factory, Databricks, and Power BI. · Degree in Computer Science, IT, or a related field with a strong analytical focus. · Proficiency in Python, particularly in the context of data manipulation and notebook-based analyti ...

  • PBT Group

    Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    PBT Group Cape Town, South Africa Full time

    PBT Group has a requirement for an intermediate Data Engineers skilled in AWS, SQL, Oracle, legacy systems, infrastructure, Cobol. · Duties: · Design, develop, test and deploy ETL for ODS and data mart projects, as well as application and management reports. · Provide technical s ...

  • Cletech Recruitment

    Data Engineer

    Found in: Executive Placements ZA C2 - 2 days ago

    Cletech Recruitment Cape Town, South Africa

    Responsibilities:Configure and implement software, meeting asset managers research & investment process needs. · Develop end-to-end data service solutions, ensuring client data needs in financial markets are met. · Collaborate closely clients to provide support, enhance service ...

  • PBT Group

    DevOps Data Engineer

    Found in: Executive Placements ZA C2 - 2 days ago

    PBT Group Cape Town, South Africa

    PBT Group has a requirement for an intermediate DevOps Data Engineer. · Duties: · Strong experience with AWS, Terraform, CI/CD, Git · Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in co ...

  • PBT Group

    DevOps Data Engineer

    Found in: Executive Placements ZA C2 - 3 days ago

    PBT Group Cape Town, South Africa

    PBT Group has a requirement for an intermediate DevOps Data Engineer. · Duties: · Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, K ...

  • PBT Group

    AWS Data Engineer

    Found in: Executive Placements ZA C2 - 2 days ago

    PBT Group Cape Town, South Africa

    PBT Group has an opportunity for an AWS Data Engineer, preferably with Hadoop experience. · Duties: · Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spa ...

  • IO Global

    Senior Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    IO Global Cape Town, South Africa Full time

    Summary · IO Global (IOG), the creator of the Cardano blockchain platform, is searching for a Senior Data Engineer, preferably with a DeFi background. IOG is creating a spinout venture that combines crypto/Web3/DeFi innovations with traditional finance (TradFi) to make borrowing ...

  • PBT Group

    Data Streaming Engineer

    Found in: Talent ZA C2 - 4 days ago

    PBT Group Cape Town, South Africa Full time

    PBT Group has an opportunity for a Data Streaming Engineer to build, enhance, and maintain real-time data pipeline. Work with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be respons ...

  • Hire Resolve

    Senior Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    Hire Resolve Cape Town, South Africa Full time

    Hire Resolve is currently seeking a highly skilled Data Engineer to join their client's internationally based team. Their Data Engineer will be based in Cape Town with the Data Team, who work on a hybrid work model. · Requirements · Bachelor's degree in Computer Science or a rela ...

  • Boldr

    Data Integration Engineer

    Found in: Talent ZA C2 - 3 hours ago

    Boldr Cape Town, South Africa Full time

    A LITTLE BIT ABOUT Boldr · Boldr is the first global B-Corp dedicated to delivering world-class Client experiences while creating access to dignified, meaningful work in communities around the world. · We are a global team, united by our desire to connect diverse people with comm ...

  • Crypto Banter

    Data & QA Engineer

    Found in: beBee S2 ZA - 6 days ago

    Crypto Banter Western Cape, Cape Town, South Africa Full time

    Crypto Banter's job vacancy, Career and Recruitment · Job title : Data & QA Engineer jobs in Western Cape · Job Location : Western Cape, Cape Town · Deadline : May 01, 2024 · Quick Recommended Links · Jobs by Location · Job by industries · Role Overview: · We're seeking a multif ...

  • PBT Group

    SQL Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    PBT Group Cape Town, South Africa Full time

    PBT Group, leaders in Business Intelligence, has a vacancy for a Microsoft SQL Data Engineer. · Duties: · Plan and analyse complex business requirements and implement technology enabled solutions to address multi-discipline business opportunities/problems. · Conduct planning, ana ...

  • PBT Group

    Kafka Big Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    PBT Group Cape Town, South Africa Full time

    PBT Group has an opportunity for a Kafka Big Data Engineer. As a Kafka Engineer, you will be responsible for the building, improvement and scaling of our streaming data platform. This role requires a combination of strong technical skills, deep understanding of distributed system ...