Hadoop Developer

Location: Pune

Required Skills : Scala, Python, Data Engineering, AWS, Cassandra/AstraDB, Athena, EMR, Spark/Snowflake

Level of skills and experience:

5 years of hands-on experience in using Python, Spark,Sql.

Experienced in AWS Cloud usage and management.

Experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow).

Experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch.

Experience with orchestrators such as Airflow and Kubeflow.

Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).

Fundamental understanding of Parquet, Delta Lake and other data file formats.

Proficiency on an IaC tool such as Terraform, CDK or CloudFormation.

Strong written and verbal English communication skill and proficient in communication with non-technical stakeholderst

Requirements

• Extensive and expert programming experience in at least one general programming language (e. g.

Java, C, C++) & tech stack to write maintainable, scalable, unit-tested code.

• Experience with multi-threading and concurrency programming.

• Extensive experience in object oriented design skills, knowledge of design patterns, and a huge passion

and ability to design intuitive modules and class-level interfaces.

• Excellent coding skills - should be able to convert design into code fluently.

• Knowledge of Test Driven Development.

• Good understanding of databases (e. g. MySQL) and NoSQL (e. g. HBase, Elasticsearch, Aerospike etc).

• Strong desire to solve complex and interesting real world problems.

• Experience with full life cycle development in any programming language on a Linux platform.

• Go-getter attitude that reflects in energy and intent behind assigned tasks.

• Worked in a startup-like environment with high levels of ownership and commitment.

• BTech, MTech or Ph. D. in Computer Science or related technical discipline (or equivalent).

• Experience in building highly scalable business applications, which involve implementing large complex

business flows and dealing with huge amounts of data.

• 3+ years of experience in the art of writing code and solving problems on a large scale.

• Open communicator who shares thoughts and opinions frequently, listens intently, and takes

constructive feedback.

About the Role-

Thinking big and executing beyond what is expected. The challenges cut across algorithmic problem solving, systems engineering, machine learning and infrastructure at a massive scale.

Reason to Join-

An opportunity for innovators, problem solvers & learners.  Working will be Innovative, empowering, rewarding & fun. Amazing Office, competitive pay along with excellent benefits package.

Requiremets and Responsibilities- (please read carefully before applying)

• The overall experience of 3-6 years in Java/Python Framework and Machine Learning.
• Develop Web Services, REST, XSD, XML technologies, Java, Python, AWS, API.
• Experience on Elastic Search or SOLR or Lucene -Search Engine, Text Mining, Indexing.
• Experience in highly scalable tools like Kafka, Spark, Aerospike, etc.
• Hands on experience in Design, Architecture, Implementation, Performance & Scalability, and Distributed Systems.
• Design, implement, and deploy highly scalable and reliable systems.
• Troubleshoot Solr indexing process and querying engine.
• Bachelors or Masters in Computer Science from Tier 1 Institutions

·      Core responsibilities to include analyze business requirements and designs for accuracy and completeness. Develops and maintains relevant product.

·      BlueYonder is seeking a Senior/Principal Architect in the Data Services department (under Luminate Platform ) to act as one of key technology leaders to build and manage BlueYonder’ s technology assets in the Data Platform and Services.

·      This individual will act as a trusted technical advisor and strategic thought leader to the Data Services department. The successful candidate will have the opportunity to lead, participate, guide, and mentor other people in the team on architecture and design in a hands-on manner. You are responsible for technical direction of Data Platform. This position reports to the Global Head, Data Services and will be based in Bangalore, India.

·      Core responsibilities to include Architecting and designing (along with counterparts and distinguished Architects) a ground up cloud native (we use Azure) SaaS product in Order management and micro-fulfillment

·      The team currently comprises of 60+ global associates across US, India (COE) and UK and is expected to grow rapidly. The incumbent will need to have leadership qualities to also mentor junior and mid-level software associates in our team. This person will lead the Data platform architecture – Streaming, Bulk with Snowflake/Elastic Search/other tools

Our current technical environment:

·      Software: Java, Springboot, Gradle, GIT, Hibernate, Rest API, OAuth , Snowflake

·      • Application Architecture: Scalable, Resilient, event driven, secure multi-tenant Microservices architecture

·      • Cloud Architecture: MS Azure (ARM templates, AKS, HD insight, Application gateway, Virtue Networks, Event Hub, Azure AD)

·      Frameworks/Others: Kubernetes, Kafka, Elasticsearch, Spark, NOSQL, RDBMS, Springboot, Gradle GIT, Ignite

Experience:

The candidate should have about 2+ years of experience with design and development in Java/Scala. Experience in algorithm, data-structure, database and distributed System is mandatory.

Required Skills:

Mandatory: -

1. Core Java or Scala
2. Experience in Big Data, Spark
3. Extensive experience in developing spark job. Should possess good Oops knowledge and be aware of enterprise application design patterns.
4. Should have the ability to analyze, design, develop and test complexity of spark job.
5. Working knowledge of Unix/Linux.
6. Hands on experience in Spark, creating RDD, applying operation - transformation-action

Good To have: -

1. Python
2. Spark streaming
3. Py Spark
4. Azure/AWS Cloud Knowledge of Data Storage and Compute side