Data Engineer

About the Role-

Thinking big and executing beyond what is expected. The challenges cut across algorithmic problem solving, systems engineering, machine learning and infrastructure at a massive scale.

Reason to Join-

An opportunity for innovators, problem solvers & learners.  Working will be Innovative, empowering, rewarding & fun. Amazing Office, competitive pay along with excellent benefits package.

Requiremets and Responsibilities- (please read carefully before applying)

• The overall experience of 3-6 years in Java/Python Framework and Machine Learning.
• Develop Web Services, REST, XSD, XML technologies, Java, Python, AWS, API.
• Experience on Elastic Search or SOLR or Lucene -Search Engine, Text Mining, Indexing.
• Experience in highly scalable tools like Kafka, Spark, Aerospike, etc.
• Hands on experience in Design, Architecture, Implementation, Performance & Scalability, and Distributed Systems.
• Design, implement, and deploy highly scalable and reliable systems.
• Troubleshoot Solr indexing process and querying engine.
• Bachelors or Masters in Computer Science from Tier 1 Institutions

The Sr. Analytics Engineer would provide technical expertise in needs identification, data modeling, data movement, and transformation mapping (source to target), automation and testing strategies, translating business needs into technical solutions with adherence to established data guidelines and approaches from a business unit or project perspective.

Understands and leverages best-fit technologies (e.g., traditional star schema structures, cloud, Hadoop, NoSQL, etc.) and approaches to address business and environmental challenges.

Provides data understanding and coordinates data-related activities with other data management groups such as master data management, data governance, and metadata management.

Actively participates with other consultants in problem-solving and approach development.

Responsibilities :

Provide a consultative approach with business users, asking questions to understand the business need and deriving the data flow, conceptual, logical, and physical data models based on those needs.

Perform data analysis to validate data models and to confirm the ability to meet business needs.

Assist with and support setting the data architecture direction, ensuring data architecture deliverables are developed, ensuring compliance to standards and guidelines, implementing the data architecture, and supporting technical developers at a project or business unit level.

Coordinate and consult with the Data Architect, project manager, client business staff, client technical staff and project developers in data architecture best practices and anything else that is data related at the project or business unit levels.

Work closely with Business Analysts and Solution Architects to design the data model satisfying the business needs and adhering to Enterprise Architecture.

Coordinate with Data Architects, Program Managers and participate in recurring meetings.

Help and mentor team members to understand the data model and subject areas.

Ensure that the team adheres to best practices and guidelines.

Requirements :

- Strong working knowledge of at least 3 years of Spark, Java/Scala/Pyspark, Kafka, Git, Unix / Linux, and ETL pipeline designing.

- Experience with Spark optimization/tuning/resource allocations

- Excellent understanding of IN memory distributed computing frameworks like Spark and its parameter tuning, writing optimized workflow sequences.

- Experience of relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., Redshift, Bigquery, Cassandra, etc).

- Familiarity with Docker, Kubernetes, Azure Data Lake/Blob storage, AWS S3, Google Cloud storage, etc.

- Have a deep understanding of the various stacks and components of the Big Data ecosystem.

- Hands-on experience with Python is a huge plus

4-5 years of overall experience in software development.
- Experience on Hadoop (Apache/Cloudera/Hortonworks) and/or other Map Reduce Platforms
- Experience on Hive, Pig, Sqoop, Flume and/or Mahout
- Experience on NO-SQL – HBase, Cassandra, MongoDB
- Hands on experience with Spark development,  Knowledge of Storm, Kafka, Scala
- Good knowledge of Java
- Good background of Configuration Management/Ticketing systems like Maven/Ant/JIRA etc.
- Knowledge around any Data Integration and/or EDW tools is plus
- Good to have knowledge of  using Python/Perl/Shell

Please note - Hbase hive and spark are must.

Your Opportunity

• Own and drive business features into tech requirements
• Design & develop large scale real time server side systems
• Quickly create quality prototypes
• Staying updated on emerging technologies
• Ensuring that all deliverables adhere to our world class standards
• Promote coding best practices
• Mentor and develop junior developers in the team

Required Experience:

• 4+ years of relevant experience as described below
• Excellent grasp of Core Java, Multi Threading and OO design patterns
• Experience with Scala, functional, reactive programming and Akka/Play is a plus
• Excellent understanding of data structures and algorithms
• Solid grasp of large scale distributed real time systems
• Prior experience on building a scalable and resilient micro service
• Solid understanding of relational databases, NoSQL databases and Caching systems
• Good understanding of Big Data technologies such as Spark, Hadoop is a plus
• Experience on one of AWS, Azure or GCP

Who you are :

• You have excellent and effective communication and collaborative skills
• You love problem solving
• You stay up to date with the latest technologies and then apply them in real life
• You love paying attention to detail
• You thrive in meeting tight deadlines and prioritising workloads
• Ability to collaborate across multiple functions

Education:

Bachelor’s degree in Engineering or equivalent experience within the field

Mentor To Go is an android and web based platform to help students improve their work readiness through self-learning sessions, personalised mentorship provided by mentor professionals from a range of careers, and work experience opportunities.

The technology platform implements  functionalities such as: screening surveys of mentors and mentees, training content for mentors, matching through an algorithm, and mentorship through the delivery of structured activities. 

The application technology architecture comprises of the following:

o   a Python Django web application with NGinx and Gunicorn

o   PostgreSQL database

o   A React Android and Web app

o   Rest APIs which interact between the Android app and the Django webserver

Our vision at Mentor To Go is to provide career mentorship to young people anywhere, anytime across the length and breadth of India.

We are looking for a backend software developer who is passionate about using web technologies to solve social problems and creating experiences that are elegant and effective. You will own the back end development of features of the Mentor To Go platform from conceptualisation through design and testing, working closely with other developers, the technical project manager, and the program operations team. You are user-centric, continuously demonstrate strategic & analytical abilities, and are laser focused on executing at speed. You must have the ability to succeed in a fast paced environment, where success is dependent on your ability to collaborate with cross-functional team members in a positive, productive, and transparent way.

Minimum requirements 

• Bachelor’s degree in engineering preferably  CS  or related discipline  
• 3+ years experience working in Unix/Linux environments building web and mobile applications
• Proficiency in developing and deploying cloud based Python/Postgresql applications preferably using frameworks such as Django
• Experience with design and development of moderately complex software projects
• Strong written and oral communication skills
• Familiarity with version control software such as Git

Desired  requirements 

• 5+ years of overall relevant work experience
• Experience building apps using React/React-Native
• Basic understanding of AWS cloud
• Ability to participate in technical discussions and help make technical trade-offs

About Mentor Together

https://mentortogether.org/">Mentor Together is India’s first and largest youth mentoring  non-profit organization with a mission to facilitate empowering mentoring relationships and networks that help young people break the inequalities of opportunity and actualise their potential.

http://bit.ly/mentortogo">Mentor To Go is the world’s first mobile mentoring platform created by Mentor Together with the support of Cisco India, LinkedIn Social Impact, British Telecom and Sterlite Technologies.