Urgent Opening|| Senior Data Engineer|| Bangalore

Skills required:

• Experience of working on public clouds (AWS / Azure)
• Good Coding skills in Python, Pyspark and SQL is must.
• Experience of writing/maintaining Spark code base in a production environment.
• Big Data Experience - Spark, Relational Databases (Postgres, etc.)
• Worked with orchestration technologies like Airflow, Kubeflow etc.
• Build & support scalable data engineering pipelines (ETLs etc.). This will entail extract, load and transform of big data’ from a wide variety of sources, both batch & streaming, using latest data frameworks and technologies, along with real time monitoring dashboards and alerting.
• Essential experience of working with distributed systems software development.
• Demonstrated experience of production experience in big data infrastructure and data modelling.
• Critical experience performance optimization for both data loading and data ingestion.
• Know-how of critical tools, in a developer’s toolkit such as GitHub, Dockers, VSCode
• Ability to work in a fast-paced and deadline driven environment.

Responsibilities:

• Build & support scalable data engineering pipelines (load, extract, cleaning, transform, ingest etc.) for our SaaS AI Applications and Predictions-as-a-Service Implementations and Product.
• Build and Deploy the Data Engineering pipelines for scale using Kubeflow
• Have ownership of multiple data pipelines.
• Review, audit and improve the Data Engineering pipelines
• Document and communicate technical and functional design (HLD, LLD), timelines and plans
• Work closely with AI/ML, DevOps, MLE, other engineering and product team
• Work with the engineering and implementation team to understand business problems and implement efficient software solutions
• Follow configurable programming practices (config files, modular programming, parameterization, etc.)
• Support Maintenance/enhancement of deployed pipelines. This includes bug-fixes and performance improvement.
• Participate in organization building activities (cross team training, hiring, etc.) to further the Company mission and culture

Experience:- 3years to 5years

Location:- Bangalore

Education:- B.Tech/BCA/B.E/M.Tech/MCA