Jobs
>
Johannesburg

    AWS Data Engineer with DataBricks - Johannesburg, South Africa - PBT Group

    PBT Group
    PBT Group Johannesburg, South Africa

    Found in: Executive Placements ZA C2 - 3 days ago

    PBT Group background
    Contract
    Description

    PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience.

    Duties:

    • Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.
    • Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
    • Design and build production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala.
    • Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
    • Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
    • Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
    • Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
    • Creation and support of real-time data pipelines built on AWS technologies including Glue, Redshift/Spectrum, Kinesis, EMR and Athena
    • Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency
    • Working closely with team members to drive real-time model implementations for monitoring and alerting of risk systems.
    • Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning
    • Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers

    Experience:

    • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
    • Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
    • Experience working with distributed systems as it pertains to data storage and computing
    • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
    • Strong analytic skills related to working with unstructured datasets.
    • Build processes supporting data transformation, data structures, meta data, dependency and workload management.
    • A successful history of manipulating, processing and extracting value from large disconnected data sets.
    • Working knowledge of message queuing, stream processing, and highly scalable Big Data, data stores.
    • Strong project management and organizational skills.
    • Experience supporting and working with cross-functional teams in a dynamic environment.
    • Experience in a Data Engineer or similar roles
    • Experience with big data tools is a must: Hadoop, Spark, Kafka, etc.
    • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
    • Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
    • Experience with AWS cloud services: EC2, EMR, RDS, Redshift
    • Experience with stream-processing systems: Storm, Spark-Streaming, etc.
    • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc

    Qualifications/ Certification:

    • Bachelor's Degree in Computer Science, Information Technology or other relevant fields
    • Has experience in any of the following AWS Athena and Glue Pyspark, EMR, DynamoDB, Redshift, Kinesis, Lambda, Snowflake
    • Proficient in AWS Redshift, S3, Glue, Athena, DynamoDB, EMR
    • Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations

    * In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

  • Job Masters

    IT Senior Data Scientist

    Found in: Executive Placements ZA C2 - 2 days ago

    Job Masters Johannesburg, South Africa

    IT Senior Data Scientist - JHB North · Minimum requirements: · Bachelor''s or higher degree in Computer Science (preferred), will consider Engineering or related. · o Maths: Statistical techniques and hypothesis testing, University level mathematics o required) · Working Experien ...

  • Puma Energy

    Data Engineer

    Found in: Talent ZA C2 - 4 days ago

    Puma Energy Johannesburg, South Africa Full time

    Main Purpose: · Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. · Ensure the data lake contains accurate, up-to-date, and high-quality data, enabling data scientists t ...

  • PBT Group

    Azure Data Architect

    Found in: Talent ZA C2 - 4 days ago

    PBT Group Johannesburg, South Africa Full time

    PBT Group has an opportunity for a Data Architect for a project in Sandton, Johannesburg. · The scope for Data Architecture is to design and implement a well-structured and efficient data architecture that supports an organization's data needs and aligns with its overall business ...

  • E-Merge

    Junior Data Engineer

    Found in: Talent ZA 2A C2 - 2 days ago

    E-Merge Johannesburg, South Africa

    A consulting company with a good blend of experienced leadership and freethinking youth with a proven track record of realising business values by delivering quality technology solutions and facilitating business change while providing extended Business and Information Management ...

  • NTT

    Business Intelligence Developer

    Found in: beBee S2 ZA - 5 days ago

    NTT Johannesburg, South Africa Full time

    NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. · In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-cla ...

  • IT Ridge Technologies

    Data Engineering

    Found in: Talent ZA C2 - 4 days ago

    IT Ridge Technologies Johannesburg, South Africa Full time

    As a Data Engineering & Analytics vertical market lead you will be responsible for providing advisory and thought leadership on the provision of analytics environments leveraging Cloud based platforms, big data technologies, including integration with existing data and analytics ...

  • Dimension Data

    Senior Data Engineer-

    Found in: Talent ZA C2 - 4 days ago

    Dimension Data Sandton, South Africa

    Functieomschrijving · About Dimension Data · At Dimension Data we empowering our people to deliver client-centric solutions. We want to see our people make the most of every opportunity, and together we can achieve the extraordinary. Our diverse and inspiring people are experts a ...

  • Dimension Data

    Intermediate Data Engineer-

    Found in: Talent ZA C2 - 4 days ago

    Dimension Data Sandton, South Africa

    Functieomschrijving · Intermediate Data Engineer, Data Processing and Archiving, SQL, Big Data Analytics, Big Data This is a great opportunity to work with an industry leading company in the finance sector. The role will be based at our headquarters in London. The ideal candidate ...

  • Experian

    Senior Data Analyst

    Found in: Talent ZA C2 - 4 days ago

    Experian Sandton, South Africa Full time

    Job Description · An experienced Senior Data Analyst who will be responsible for processing, analysing, and interpreting large amounts of data to identify best practices, insights and patterns that will add value and improve our data platform and data offering. To support and ma ...

  • HR Genie

    Senior Data Engineer

    Found in: Executive Placements ZA C2 - 3 days ago

    HR Genie Randburg, South Africa

    Our Client a Global tech firm is seeking a Senior Data Engineer (Junior, Intermediate and Senior positions available) to join their team in Randburg on a contract basis. They offer stability, growth, attractive rates and a great working environment. · Candidates will be working f ...

  • Clarkhouse Human Capital

    Senior Data Scientist

    Found in: Talent ZA C2 - 17 hours ago

    Clarkhouse Human Capital Randburg, South Africa

    Our client, a Global Business is looking to onboard a Senior Data Scientist. · To be successful in this role you need to excel in machine learning, software development, cross-functional and cross-domain collaboration and share a passion for enabling scalable solutions that deliv ...

  • HR Genie

    Senior Specialist Insight and reporting/ Data Analyst

    Found in: Executive Placements ZA C2 - 3 days ago

    HR Genie Randburg, South Africa

    Our Client a Global Tech firm is seeking a Senior Specialist Insight and reporting/ Data analyst (Senior, Intermediate and Junior) to join their team in Randburg. They offer stability, growth, attractive rates and a great working environment. · The Senior Specialist Analytics and ...

  • Planet10

    AWS Senior Data Engineer

    Found in: Executive Placements ZA C2 - 3 days ago

    Planet10 Randburg, South Africa

    Purpose of the Position · To collect and validate business information (which can then be used by data scientists and business stakeholders). · To design and implement data models and architectures that enable business stakeholders to gain insights from large-scale datasets. · To ...

  • HR Genie

    Data Modeller

    Found in: Executive Placements ZA C2 - 3 days ago

    HR Genie Randburg, South Africa

    Our Client a Global Tech firm is seeking a Data Modeller to join their team in Randburg. They offer stability, growth, attractive rates and a great working environment. · The Data Modeller is responsible for providing expert-level support in the development and management of repo ...

  • Dimension Data

    Data Scientist

    Found in: Talent ZA C2 - 4 days ago

    Dimension Data Randburg, South Africa

    Functieomschrijving · Do you want to work with the latest technology? Do you want to be part of a team that is constantly improving their product and customer experience? Would you like to work with cutting edge algorithms, machine learning and data mining tools? Are you looking ...

  • Datacentrix

    Senior Analyst In Data

    Found in: beBee S2 ZA - 6 days ago

    Datacentrix Gauteng, Johannesburg, South Africa Contract

    Datacentrix's job vacancy, Career and Recruitment · Job title : Senior Analyst In Data & Analytics jobs in Gauteng · Job Location : Gauteng, Johannesburg · Deadline : May 16, 2024 · Quick Recommended Links · Jobs by Location · Job by industries · Managed Talents Solutions' clien ...

  • Job Masters

    IT Senior Data Analyst

    Found in: Executive Placements ZA C2 - 3 days ago

    Job Masters Centurion, South Africa

    IT Senior Data Analyst AA/EE Centurion · Minimum requirements: · Bachelor's degree in a relevant quantitative field · Azure Certifications · Minimum 4 years experience in data analytics, or other : data science, data engineering, machine learning or BI development · Minimum 2 yea ...

  • Deka Minas (Pty) Ltd

    Senior Manager Data Engineer

    Found in: Talent ZA C2 - 2 days ago

    Deka Minas (Pty) Ltd Midrand, South Africa Full time

    RESPONSIBILITIES · Create and execute a long-term strategy for leveraging data and analytics as a key business enabler and competitive differentiator across the business. · Take responsibility for developing and delivering a key element of the organisation's data management syste ...