Jobs
>
Johannesburg

    AWS Data Engineer with DataBricks - Johannesburg, South Africa - PBT Group

    PBT Group
    PBT Group Johannesburg, South Africa

    1 day ago

    PBT Group background
    Description

    PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience.

    Duties:

    • Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.
    • Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
    • Design and build production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala.
    • Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
    • Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
    • Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
    • Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
    • Creation and support of real-time data pipelines built on AWS technologies including Glue, Redshift/Spectrum, Kinesis, EMR and Athena
    • Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency
    • Working closely with team members to drive real-time model implementations for monitoring and alerting of risk systems.
    • Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning
    • Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers

    Experience:

    • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
    • Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
    • Experience working with distributed systems as it pertains to data storage and computing
    • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
    • Strong analytic skills related to working with unstructured datasets.
    • Build processes supporting data transformation, data structures, meta data, dependency and workload management.
    • A successful history of manipulating, processing and extracting value from large disconnected data sets.
    • Working knowledge of message queuing, stream processing, and highly scalable Big Data, data stores.
    • Strong project management and organizational skills.
    • Experience supporting and working with cross-functional teams in a dynamic environment.
    • Experience in a Data Engineer or similar roles
    • Experience with big data tools is a must: Hadoop, Spark, Kafka, etc.
    • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
    • Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
    • Experience with AWS cloud services: EC2, EMR, RDS, Redshift
    • Experience with stream-processing systems: Storm, Spark-Streaming, etc.
    • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc

    Qualifications/ Certification:

    • Bachelor's Degree in Computer Science, Information Technology or other relevant fields
    • Has experience in any of the following AWS Athena and Glue Pyspark, EMR, DynamoDB, Redshift, Kinesis, Lambda, Snowflake
    • Proficient in AWS Redshift, S3, Glue, Athena, DynamoDB, EMR
    • Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations

    * In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

  • Tych Business Solutions

    Senior Specialist Insights and Reporting

    5 days ago

    Tych Business Solutions Johannesburg, South Africa

    **Job ID**: 83683**Date Posted**: Posted 48 mins ago**Location**: Johannesburg**Job Title**: SENIOR SPECIALIST INSIGHTS AND REPORTING (12 MONTHS) FTC - JOHANNESBURG - GAUTENG · **KEY OBJECTIVES** · Compliance · - Ensure adherence to departmental and company policies and rules · - ...

  • WorldRemit

    Treasury Data Analyst

    1 week ago

    WorldRemit Johannesburg, South Africa

    **About Zepz**: · Zepz is the group powering two leading global remittance brands: WorldRemit and Sendwave. Since 2010, we have been disrupting an industry previously dominated by offline legacy players with our relentless focus on reducing the cost of remittances and increasing ...

  • Recruitment and Consulting specialists Pty Ltd

    Data Analytics

    5 days ago

    Recruitment and Consulting specialists Pty Ltd Johannesburg, South Africa

    **Skills**: · Advanced analytical, problem solving and research skills · Previous experience and demonstrated capability in managing multiple systems and analytic output · Advanced Excel skills and experience in utilising multiple systems · Knowledge of relational databases · Pre ...

  • THE SKILLS MINE (PTY) LTD

    Data Engineering Specialist

    5 days ago

    THE SKILLS MINE (PTY) LTD Johannesburg, South Africa

    **Requirements**: · - BS degree, preferably in Information Technology, Management Information Systems, Computer Science, or · similar discipline, Master's Degree · - 8-10 years' experience developing and maintaining data architecture aligned to · best practice data warehouse desi ...

  • PBT Group

    Data Architect

    1 day ago

    PBT Group Johannesburg, South Africa

    PBT Group has an opportunity for a Data Architect for a project in Sandton, Johannesburg. · The scope for Data Architecture is to design and implement a well-structured and efficient data architecture that supports an organization's data needs and aligns with its overall business ...

  • H2R Africa

    Azure Synapse Microsoft Architect/ Enginee

    21 hours ago

    H2R Africa Johannesburg, South Africa

    Our client, an IT Solutions Company, has a contract vacancy for an Azure Synape Microsoft Arthitect/Engineer. The position is mostly remote work. The Azure Synapse Architect is expected to: · - function as a productive member of a team, working and communicating proactively with ...

  • Moving Heads Personnel

    Data and Analytics Manager

    2 days ago

    Moving Heads Personnel Johannesburg, South Africa

    Lead and mentor a team of data analysts, data engineers, AI engineers, and BI analysts to ensure high performance, collaboration, and professional development. · - Develop and execute strategies for collecting, analyzing, and interpreting data to drive business insights and decis ...

  • Virago Recruitment

    Microsoft Data and Analyst Consultant

    6 days ago

    Virago Recruitment Johannesburg, South Africa

    Collaborate with clients to identify their data and analytics needs, challenges, and objectives. · Design and develop end-to-end data and analytics solutions using Microsoft technologies such as Azure Data Platform, Power BI, SQL Server, Azure Machine Learning, and others. · Lead ...

  • Boardroom Appointments

    Senior Specialist Insight and Reporting

    3 days ago

    Boardroom Appointments Johannesburg, South Africa Full time

    **CONTRACT ROLE: 12 MONTHS (ON-SITE)**** · **Key Roles and Responsibilities**: · - Ensure adherence to departmental and company policies and rules · - Ensure compliance to all South African IT legislation and governance · - Ensure continuous improvement of standard operating proc ...

  • Multi Search

    Senior Microsoft Data and Analyst Consultant

    2 days ago

    Multi Search Johannesburg, South Africa

    **Responsibilities**: · Collaborate with clients to identify their data and analytics needs, challenges, and objectives. · Design and develop end-to-end data and analytics solutions using Microsoft technologies such as Azure Data · Platform, Power BI, SQL Server, Azure Machine Le ...

  • The Coca-Cola Company

    Data & Analytics Manager

    2 days ago

    The Coca-Cola Company Johannesburg, South Africa

    **What You'll Do for Us**: · The key responsibilities are: · - Analyze business analytical requirements (e.g., functional and technical requirements, data need, business processes) by conducting information-gathering sessions with stakeholders, analyzing the information and prior ...

  • Sindzisa Consulting

    Practice Lead

    2 weeks ago

    Sindzisa Consulting Johannesburg, South Africa

    Responding to client RFI, RFP documents with proper solution design including cost estimates · - Understanding customer requirements and create technical proposition · - Creating proactive proposals by understanding customer business priorities, and technology landscape · - Contr ...

  • RecruiTech

    Senior Microsoft Data Analyst

    4 days ago

    RecruiTech Johannesburg, South Africa

    **R **R Depending on experience)** · **Johannesburg - Hybrid** · An exciting opportunity to join a well-known Software Consulting house as a Microsoft Data and Analytics Consultant (Hybrid). You will work closely with clients to understand their unique requirements, design and im ...

  • Heineken

    Data Scientist X2

    3 days ago

    Heineken Johannesburg, South Africa

    **We Go Places How about you?** · Immediate Superior: Chapter Lead - Data Science · Location: Johannesburg (Hybrid) · Function: Africa & Middle East (AME) Analytics Hub · Type of Contract: Permanent · Reference Number: 102718 · Closing Date: 02/02/2024 · The Africa & Middle East ...

  • Juru Holding (Pty) Ltd

    Google Cloud Platform

    1 week ago

    Juru Holding (Pty) Ltd Johannesburg, South Africa

    Are you a cloud enthusiast with a passion for exploring new horizons? Do you love the thrill of leading teams to new heights of technology and efficiency? If so, we want to hear from you · **Duties & Responsibilities** · Participates in end-to-end cycle from opportunity identific ...

  • Planet10

    Data Scientist

    1 week ago

    Planet10 Johannesburg, South Africa

    Purpose of the Position · - Extract and gather data from various sources, both real time event streams and batch, develop technical architectural solutions in the cloud utilizing this data for intelligent automation and data driven insights. · Qualifications Essential · - Degree ...

  • Microsoft

    Cloud Solution Architect

    2 weeks ago

    Microsoft Johannesburg, South Africa

    With over 18,000 employees worldwide, the Microsoft Customer Experience & Success (CE&S) organization is responsible for the strategy, design, and implementation of Microsoft's end-to-end customer experience. Come join CE&S and help us build a future where customers come to us no ...

  • TransUnion

    Customer Success Consultant

    1 week ago

    TransUnion Johannesburg, South Africa

    TransUnion's Job Applicant Privacy Notice · **What We'll Bring**: · About TransUnion: · TransUnion is a global information and insights company which provides solutions that help create economic opportunity, great experiences and personal empowerment for hundreds of millions of p ...

  • NTT

    Senior Specialist Insight and Reporting

    5 days ago

    NTT Johannesburg, South Africa

    NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. · In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-cla ...

  • De Beers Group of Companies

    Data Scientist

    1 week ago

    De Beers Group of Companies Johannesburg, South Africa

    Company Description · - We're re-imagining mining to improve people's lives. That includes the lives of everyone who works for Anglo American. We strive to be a great company where employees are happy, inspired to do their best work. A place where every colleague can grow, develo ...