- Experience of working on public clouds (AWS / Azure)
- Good Coding skills in Python, Pyspark and SQL is must.
- Experience of writing/maintaining Spark code base in a production environment.
- Big Data Experience - Spark, Relational Databases (Postgres, etc.)
- Worked with orchestration technologies like Airflow, Kubeflow etc.
- Build & support scalable data engineering pipelines (ETLs etc.). This will entail extract, load and transform of big data’ from a wide variety of sources, both batch & streaming, using latest data frameworks and technologies, along with real time monitoring dashboards and alerting.
- Essential experience of working with distributed systems software development.
- Demonstrated experience of production experience in big data infrastructure and data modelling.
- Critical experience performance optimization for both data loading and data ingestion.
- Know-how of critical tools, in a developer’s toolkit such as GitHub, Dockers, VSCode
- Ability to work in a fast-paced and deadline driven environment.
- Build & support scalable data engineering pipelines (load, extract, cleaning, transform, ingest etc.) for our SaaS AI Applications and Predictions-as-a-Service Implementations and Product.
- Build and Deploy the Data Engineering pipelines for scale using Kubeflow
- Have ownership of multiple data pipelines.
- Review, audit and improve the Data Engineering pipelines
- Document and communicate technical and functional design (HLD, LLD), timelines and plans
- Work closely with AI/ML, DevOps, MLE, other engineering and product team
- Work with the engineering and implementation team to understand business problems and implement efficient software solutions
- Follow configurable programming practices (config files, modular programming, parameterization, etc.)
- Support Maintenance/enhancement of deployed pipelines. This includes bug-fixes and performance improvement.
- Participate in organization building activities (cross team training, hiring, etc.) to further the Company mission and culture
Experience:- 3years to 5years
Salary: 12,00,000 - 22,00,000 P.A.
Industry:IT Services & Consulting
Functional Area:Engineering - Software & QA
Role Category:Software Development
Employment Type:Full Time, Permanent
UG:BCA in Any Specialization,B.Tech/B.E. in Any Specialization
PG:M.Tech in Any Specialization,MCA in Any Specialization
Mount Talent Consulting Private Limited
Contact Company:Mount Talent Consulting Private Limited