Jobs
>
Cape Town

    AWS Data Engineer - Cape Town, South Africa - PBT Group

    PBT Group
    PBT Group Cape Town, South Africa

    Found in: Executive Placements ZA C2 - 6 days ago

    PBT Group background
    Description

    PBT Group has an opportunity for an AWS Data Engineer, preferably with Hadoop experience.

    Duties:

    • Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.
    • Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
    • Design and build production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala.
    • Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
    • Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
    • Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
    • Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
    • Creation and support of real-time data pipelines built on AWS technologies including Glue, Redshift/Spectrum, Kinesis, EMR and Athena
    • Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency
    • Working closely with team members to drive real-time model implementations for monitoring and alerting of risk systems.
    • Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning
    • Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers

    Experience:

    • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
    • Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
    • Experience working with distributed systems as it pertains to data storage and computing
    • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
    • Strong analytic skills related to working with unstructured datasets.
    • Build processes supporting data transformation, data structures, meta data, dependency and workload management.
    • A successful history of manipulating, processing and extracting value from large disconnected data sets.
    • Working knowledge of message queuing, stream processing, and highly scalable Big Data, data stores.
    • Strong project management and organizational skills.
    • Experience supporting and working with cross-functional teams in a dynamic environment.
    • Experience in a Data Engineer or similar roles
    • Experience with big data tools is a must: Hadoop, Spark, Kafka, etc.
    • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
    • Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
    • Experience with AWS cloud services: EC2, EMR, RDS, Redshift
    • Experience with stream-processing systems: Storm, Spark-Streaming, etc.
    • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc

    Qualifications/ Certification:

    • Bachelor's Degree in Computer Science, Information Technology or other relevant fields
    • Has experience in any of the following AWS Athena and Glue Pyspark, EMR, DynamoDB, Redshift, Kinesis, Lambda, Snowflake
    • Proficient in AWS Redshift, S3, Glue, Athena, DynamoDB, EMR
    • Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations

    * In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

  • IOCO

    Data Engineer

    Found in: Executive Placements ZA C2 - 4 days ago

    IOCO Cape Town, South Africa

    We are seeking an experienced Data Migration Engineer to join our team on a contract basis to facilitate this migration process. The ideal candidates will have strong expertise in data engineering, particularly with Azure Synapse services, SSIS, and API integration. · What you'll ...

  • Dynamic Outsourced Solutions

    Data Engineer

    Found in: Trabajos Diarios ZA C2 - 1 day ago

    Dynamic Outsourced Solutions Cape Town, South Africa Permanent

    The data engineer is ultimately accountable for preparing and manipulating data for further analysis, including ETL and will support testing across the development lifecycle The Data Engineer is responsible for extracting and transforming data from source systems, understanding h ...

  • Creative CFO

    Data Engineer

    Found in: beBee S2 ZA - 3 days ago

    Creative CFO Western Cape, Cape Town, South Africa Remote

    Creative CFO's job vacancy, Career and Recruitment · Job title : Data Engineer – Cape Town jobs in Western Cape · Job Location : Western Cape, Cape Town · Deadline : April 28, 2024 · Quick Recommended Links · Jobs by Location · Job by industries · The position: · As a Data Engin ...

  • DLK Group (PTY) Ltd

    Data Engineer

    Found in: Talent ZA C2 - 6 days ago

    DLK Group (PTY) Ltd Cape Town, South Africa

    · Modifying the database structure, as necessary, from information given by application developers. · Enrolling users and maintaining system security. · Controlling and monitoring user access to the database. · Monitoring and optimizing the performance of the database. · Maintai ...

  • DLK Group

    Data Engineer

    Found in: Talent ZA C2 - 6 days ago

    DLK Group Cape Town, South Africa Contract

    Job Description · Modifying the database structure, asnecessary, from information given by application developers. · Enrolling users and maintaining systemsecurity. · Controlling and monitoring user accessto the database. · Monitoring and optimizing the performanceof the databas ...

  • Armstrong Appointments

    IT Data Engineer

    Found in: Talent ZA 2A C2 - 22 hours ago

    Armstrong Appointments Cape Town, South Africa

    Our client based in Cape Town are currently seeking the skills of an IT Data Engineer to assist in designing and implementing scalable and robust processes to support the data engineering capability. This role will be responsible for extracting and transforming massive amounts of ...

  • Cyberpro Consulting

    Data Engineer

    Found in: Talent ZA 2A C2 - 1 day ago

    Cyberpro Consulting Cape Town, South Africa

    Duties and Responsibilities: · Data Factory Maestro: Leverage Azure Data Factory to orchestrate and automate data workflows, ensuring seamless ETL processes. · Databricks Developer: Utilize Databricks to perform complex data analytics, employing both Azure and Python notebooks fo ...

  • Cletech Recruitment

    Data Engineer

    Found in: Executive Placements ZA C2 - 5 days ago

    Cletech Recruitment Cape Town, South Africa

    Responsibilities:Configure and implement software, meeting asset managers research & investment process needs. · Develop end-to-end data service solutions, ensuring client data needs in financial markets are met. · Collaborate closely clients to provide support, enhance service ...

  • Old Mutual

    Data Engineering Lead

    Found in: Talent ZA C2 - 5 days ago

    Old Mutual Cape Town, South Africa Full time

    Description · The Data Engineering Lead will be responsible for overseeing the design, development, and maintenance of our data infrastructure. This role involves leadership responsibilities, including managing a team of data engineers, setting technical direction, and collabora ...

  • Kuda

    Intern Data Engineer

    Found in: beBee S2 ZA - 1 day ago

    Kuda Western Cape, Cape Town, South Africa Full time

    Kuda's job vacancy, Career and Recruitment · Job title : Intern Data Engineer (6 Months) jobs in Western Cape · Job Location : Western Cape, Cape Town · Deadline : May 16, 2024 · Quick Recommended Links · Jobs by Location · Job by industries · Learning Opportunities · Direct exp ...

  • Crypto Banter

    Data & QA Engineer

    Found in: beBee S2 ZA - 1 day ago

    Crypto Banter Western Cape, Cape Town, South Africa Full time

    Crypto Banter's job vacancy, Career and Recruitment · Job title : Data & QA Engineer jobs in Western Cape · Job Location : Western Cape, Cape Town · Deadline : May 01, 2024 · Quick Recommended Links · Jobs by Location · Job by industries · Role Overview: · We're seeking a multif ...

  • Boldr

    Data Integration Engineer

    Found in: beBee S2 ZA - 5 days ago

    Boldr Western Cape, Cape Town, South Africa Full time

    Boldr's job vacancy, Career and Recruitment · Job title : Data Integration Engineer – Cape Town jobs in Western Cape · Job Location : Western Cape, Cape Town · Deadline : May 04, 2024 · Quick Recommended Links · Jobs by Location · Job by industries · WHAT WILL YOU DO · Understan ...

  • LexisNexis

    Senior Data Engineer I

    Found in: Talent ZA C2 - 6 days ago

    LexisNexis Cape Town, South Africa Full time

    Do you have excellent experience building data pipelines? · Do you enjoy collaborating cross-functionally to deliver on common goals? · Join us in shaping a more just world. · About Us · LexisNexis, a part of RELX, is a leading global provider of legal, regulatory, and business i ...

  • The Legends Agency

    Part-time Data Engineer

    Found in: Talent ZA C2 - 1 week ago

    The Legends Agency Cape Town, South Africa Part time

    Data Engineer (20hrs per week) · About the UX Design Institute · UX Design Institute is the global leader in online UX education and certification. We help · individuals and teams throughout the world become better UX professionals. Were looking for · ambitious people to help us ...

  • IOCO

    Senior BI Data Engineer

    Found in: Talent ZA 2A C2 - 1 day ago

    IOCO Cape Town, South Africa

    We are currently in search of a versatile and experienced Senior BI Data Engineer who possesses strong technical skills along with the ability to perform Business Analyst tasks. The role combines deep data engineering expertise with the capability to interact closely with busines ...

  • Amazon Development Centre (South Africa) (Proprietary) Limited

    Data Insights Analytics Cloud Support Engineer, Support Engineering

    Found in: Talent ZA C2 - 2 days ago

    Amazon Development Centre (South Africa) (Proprietary) Limited Cape Town, South Africa Full time

    ABOUT US · Amazon Web Services is the market leader and technology forerunner in the Cloud business. As a member of the AWS Support team you will be at the forefront of this transformational technology, assisting a global list of companies and developers that are taking advantage ...

  • Acuity Consultants

    Senior Data Engineer – Drive the scalability of the data management environment for the fastest growing tech company in logistics – Cape Town, R1.3M

    Found in: Talent ZA C2 - 1 day ago

    Acuity Consultants Cape Town, South Africa Permanent, Full time

    Job Description · This is an excellent opportunity for a Senior Data Engineer to drive scalability data management environment for the fastest growing tech company in logistics. You will play a pivotal role in evolving the data strategy, driving innovation and competitive advant ...

  • Capitec Bank

    Product Data Engineer II

    Found in: beBee S2 ZA - 3 days ago

    Capitec Bank Western Cape, Stellenbosch, South Africa Full time

    Capitec Bank's job vacancy, Career and Recruitment · Job title : Product Data Engineer II jobs in Western Cape · Job Location : Western Cape, Stellenbosch · Deadline : May 05, 2024 · Quick Recommended Links · Jobs by Location · Job by industries · Purpose Statement · To contribu ...

  • RTT

    DevOps Engineer

    Found in: Executive Placements ZA C2 - 1 day ago

    RTT Cape Town, South Africa

    The DevOps engineer is responsible for deploying product updates, identifying production issues, and implementing integrations that meet our customers' needs. You will work with developers and engineers to ensure that software development follows established processes and works a ...

  • HR Genie

    Plant Engineer

    Found in: Job Placements ZA C2 - 1 day ago

    HR Genie Cape Town, South Africa

    Our client is looking for a Plant Engineer. The Plant Engineer will use engineering knowledge and skill to continuously optimize the operations within the manufacturing facility. The role will be responsible for designing and implementing all modifications as well as engineering ...