Data Engineer (PySpark Lead)- Lightning job by Cutshort ⚡ at Pune, Bengaluru (Bangalore), Mumbai Full Time

Lightning job by Cutshort ⚡

As part of this feature, you can expect status updates about your application and replies within 72 hours (once the screening questions are answered)

About the company:

Vithamas offers end to end design services in the areas of embedded firmware, custom PCB design and mobile applications for clients seeking to create reliable, innovative products while keeping the process cost-effective giving utmost importance to quality and trust.

Responsibilities:

• Candidate will be Interacting with client to understand and gather requirements.
• Candidate will be responsible for technical design, development, and system/integration testing using oracle methodologies.
• Implement Data Pipelines using Delta lake framework
• Develop Spark APIs for Data Services from Data lake

Skillsets:

• Minimum of 4 to 8 years of overall experience with minimum 2 years project implementation experience in Python Programming Language, PySpark, Linux Shell Scripting, Data Integration experience preferably on OCI (ODI/OCI-DI/ OCI Data Flow), Oracle Database basics with ANSI SQL.
• Good knowledge of SQL for data analysis
• Good knowledge of Shell Scripting for creating code wrappers
• Configuration and Scheduling of Spark Pipelines using Apache Airflow and Cron jobs
• Good Understanding of DataLakehouse architecture and ability to create and design as per DataLakehouse Design Standards
• Knowledge and understanding of Data lake formats to efficiently design Data lake data stores
• Experience in developing event driven data pipelines
• Ability to develop re-usable code components
• Source code management using Git
• Data Integration experience using platforms like ODI/OCI-DI/ OCI Data Flow on Oracle Cloud Infrastructure (OCI)
• Ready to travel as and when required by project Desired Skills or Experience
• Oracle Cloud Infrastructure Experience or Certification will be a plus
• Healthcare Industry experience or standard certifications will be a plus
• Hadoop, Hive, Hue, Kafka, Kubernetes, Ansible, Terraform, BitBucket experience, Apache Airflow experience would be a plus
• Creativity, Personal Drive, Influencing and Negotiating, Problem Solving Building Effective Relationships, Customer Focus, Effective Communication