Log In

Don't have an account? Sign up now

Lost Password?

Sign Up

Private Jobs / Private Sector

Python Pyspark and AWS TL Position Full Time

Do you love a career where you Experience Grow & Contribute at the same time while earning at least 10% above the market If so we are excited to have bumped onto you.
If you are a Python Pyspark and AWS TL Position looking for excitement challenge and stability in your work then you would be glad to come across this page.

We are an IT Solutions Integrator/Consulting Firm helping our clients hire the right professional for an exciting long term project. Here are a few details.

Check if you are up for maximizing your earning/growth potential leveraging our Disruptive Talent Solution.
Role:Python Pyspark and AWS TL Position
Location: Hyderabad/Bangalore/Gurgaon/Mumbai/Pune
Hybrid Mode Position
Exp: 5 6 years


Requirements

We are seeking a Python Pyspark and AWS TL Position highly skilled with expertise in Python PySpark and AWS cloud services. The ideal candidate will have a strong background in Business Intelligence (BI) and data warehousing technologies with a focus on ETL processes and RDBMS. You will work closely with the application development teams to design build and optimize data pipelines and workflows throughout the development lifecycle. Additionally you will lead a small team of engineers to ensure the successful delivery of datadriven solutions.

Key Responsibilities:

  • Design and Develop Data Pipelines:

    • Create and maintain robust scalable data pipelines using Python PySpark and Spark SQL.
    • Build Airflow DAG pipelines for seamless data ingestion and workflow automation.

  • Data Management and Processing:

    • Work with large datasets processing them into usable formats using PySpark and DataFrames.
    • Leverage your expertise in SQL for data extraction transformation and loading (ETL) ensuring optimal performance of queries and data models.

  • Cloud Services and Architecture:

    • Implement and maintain data solutions on AWS using services like S3 Glue Redshift Lambda IAM EMR Athena and Spectrum.
    • Ensure data security scalability and costeffectiveness within the AWS environment.

  • Collaboration and Support:

    • Work closely with application development teams to align data engineering solutions with business requirements and development lifecycles.
    • Gather requirements from key business stakeholders and translate them into actionable data solutions.
    • Mentor and lead a team of 4 or more engineers ensuring best practices in coding testing and deployment.

  • Performance Optimization and Monitoring:

    • Conduct performance tuning and optimization of SQL queries to ensure efficient data access and transformation processes.
    • Monitor and troubleshoot issues within data pipelines ensuring data integrity and operational efficiency.

  • Leadership and Ownership:

    • Take full ownership of tasks and drive them to successful completion showing initiative in problemsolving and improving existing processes.
    • Contribute to continuous improvement efforts by identifying and implementing automation opportunities and enhancing existing workflows.

Required Skills and Qualifications:

  • Technical Skills:

    • Proficiency in Python PySpark Spark SQL and working with DataFrames.
    • Strong understanding of ETL processes data warehousing concepts and relational databases (RDBMS).
    • Handson experience with Airflow for managing and automating data pipelines.
    • Extensive knowledge of AWS cloud services such as S3 Glue Redshift Lambda IAM EMR Athena and Spectrum.
    • Solid experience in SQL query optimization and performance tuning.

  • Experience:

    • Minimum 56 years of experience in data engineering or related fields.
    • Proven track record of leading teams of at least 4 members with handson involvement in project delivery.
    • Experience working within agile development environments collaborating with crossfunctional teams.

  • Soft Skills:

    • Strong written and verbal communication skills capable of translating technical concepts for nontechnical stakeholders.
    • Excellent problemsolving skills with a proactive attitude and ability to take initiative.
    • Skilled at gathering and understanding business requirements and aligning them with technical solutions.


Benefits

CEO Message: Click Here
Clients Testimonial: Click Here

Job Overview

  • Date Posted:
  • Location:
  • Job Title: Python Pyspark and AWS TL Position
Apply for job

Apply For This Job

Upload your CV/resume or any other relevant file. Max. file size: 2 GB.

You can apply to this job and others using your online resume. Click the link below to submit your online resume and email your application to this employer.