Engineering Manager

Location:  Pune/Nagpur,Goa,Hyderabad/Bangalore

Job Requirements:

• 9 years and above of total experience preferably in bigdata space.
• Creating spark applications using Scala to process data.
• Experience in scheduling and troubleshooting/debugging Spark jobs in steps.
• Experience in spark job performance tuning and optimizations.
• Should have experience in processing data using Kafka/Pyhton.
• Individual should have experience and understanding in configuring Kafka topics to optimize the performance.
• Should be proficient in writing SQL queries to process data in Data Warehouse.
• Hands on experience in working with Linux commands to troubleshoot/debug issues and creating shell scripts to automate tasks.
• Experience on AWS services like EMR.

Responsibilities

● Able to contribute to the gathering of functional requirements, developing technical
specifications, and test case planning
● Demonstrating technical expertise, and solving challenging programming and design
problems
● 60% hands-on coding with architecture ownership of one or more products
● Ability to articulate architectural and design options, and educate development teams and
business users
● Resolve defects/bugs during QA testing, pre-production, production, and post-release
patches
● Mentor and guide team members
● Work cross-functionally with various bidgely teams including product management, QA/QE,
various product lines, and/or business units to drive forward results

Requirements
● BS/MS in computer science or equivalent work experience
● 8-12 years’ experience designing and developing applications in Data Engineering
● Hands-on experience with Big data EcoSystems.
● Past experience with Hadoop,Hdfs,Map Reduce,YARN,AWS Cloud, EMR, S3, Spark, Cassandra,
Kafka, Zookeeper
● Expertise with any of the following Object-Oriented Languages (OOD): Java/J2EE,Scala,
Python
● Ability to lead and mentor technical team members
● Expertise with the entire Software Development Life Cycle (SDLC)
● Excellent communication skills: Demonstrated ability to explain complex technical issues to
both technical and non-technical audiences
● Expertise in the Software design/architecture process
● Expertise with unit testing & Test-Driven Development (TDD)
● Business Acumen - strategic thinking & strategy development
● Experience on Cloud or AWS is preferable
● Have a good understanding and ability to develop software, prototypes, or proofs of
concepts (POC's) for various Data Engineering requirements.
● Experience with Agile Development, SCRUM, or Extreme Programming methodologies

Carsome is a growing startup that is utilising data to improve the experience of second hand car shoppers. This involves developing, deploying & maintaining machine learning models that are used to improve our customers' experience. We are looking for candidates who are aware of the machine learning project lifecycle and can help managing ML deployments.

Responsibilities: - Write and maintain production level code in Python for deploying machine learning models - Create and maintain deployment pipelines through CI/CD tools (preferribly GitLab CI) - Implement alerts and monitoring for prediction accuracy and data drift detection - Implement automated pipelines for training and replacing models - Work closely with with the data science team to deploy new models to production Required Qualifications: - Degree in Computer Science, Data Science, IT or a related discipline. - 2+ years of experience in software engineering or data engineering. - Programming experience in Python - Experience in data profiling, ETL development, testing and implementation - Experience in deploying machine learning models

Good to have: - Experience in AWS resources for ML and data engineering (SageMaker, Glue, Athena, Redshift, S3) - Experience in deploying TensorFlow models - Experience in deploying and managing ML Flow

Responsibilities:

• Design, construct, install, test and maintain data pipeline and data management systems.
• Ensure that all systems meet the business/company requirements as well as industry practices.
• Integrate up-and-coming data management and software engineering technologies into existing data structures.
• Processes for data mining, data modeling, and data production.
• Create custom software components and analytics applications.
• Collaborate with members of your team (eg, Data Architects, the Software team, Data Scientists) on the project's goals.
• Recommend different ways to constantly improve data reliability and quality.

 

Requirements:

• Experience in a related field with real-world skills and testimonials from former employees.
• Familiar with data warehouses like Redshift, Bigquery and Athena.
• Familiar with data processing systems like flink, spark and storm. Develop set
• Proficiency in Python and SQL. Possible work experience and proof of technical expertise.
• You may also consider a Master's degree in computer engineering or science in order to fine-tune your skills while on the job. (Although a Master's isn't required, it is always appreciated).
• Intellectual curiosity to find new and unusual ways of how to solve data management issues.
• Ability to approach data organization challenges while keeping an eye on what's important.
• Minimal data science knowledge is a Must, should understand a bit of analytics.

A data engineer with AWS Cloud infrastructure experience to join our Big Data Operations team. This role will provide advanced operations support, contribute to automation and system improvements, and work directly with enterprise customers to provide excellent customer service.
The candidate,
1. Must have a very good hands-on technical experience of 3+ years with JAVA or Python
2. Working experience and good understanding of AWS Cloud; Advanced experience with IAM policy and role management
3. Infrastructure Operations: 5+ years supporting systems infrastructure operations, upgrades, deployments using Terraform, and monitoring
4. Hadoop: Experience with Hadoop (Hive, Spark, Sqoop) and / or AWS EMR
5. Knowledge on PostgreSQL/MySQL/Dynamo DB backend operations
6. DevOps: Experience with DevOps automation - Orchestration/Configuration Management and CI/CD tools (Jenkins)
7. Version Control: Working experience with one or more version control platforms like GitHub or GitLab
8. Knowledge on AWS Quick sight reporting
9. Monitoring: Hands on experience with monitoring tools such as AWS CloudWatch, AWS CloudTrail, Datadog and Elastic Search
10. Networking: Working knowledge of TCP/IP networking, SMTP, HTTP, load-balancers (ELB) and high availability architecture
11. Security: Experience implementing role-based security, including AD integration, security policies, and auditing in a Linux/Hadoop/AWS environment. Familiar with penetration testing and scan tools for remediation of security vulnerabilities.
12. Demonstrated successful experience learning new technologies quickly
WHAT WILL BE THE ROLES AND RESPONSIBILITIES?
1. Create procedures/run books for operational and security aspects of AWS platform
2. Improve AWS infrastructure by developing and enhancing automation methods
3. Provide advanced business and engineering support services to end users
4. Lead other admins and platform engineers through design and implementation decisions to achieve balance between strategic design and tactical needs
5. Research and deploy new tools and frameworks to build a sustainable big data platform
6. Assist with creating programs for training and onboarding for new end users
7. Lead Agile/Kanban workflows and team process work
8. Troubleshoot issues to resolve problems
9. Provide status updates to Operations product owner and stakeholders
10. Track all details in the issue tracking system (JIRA)
11. Provide issue review and triage problems for new service/support requests
12. Use DevOps automation tools, including Jenkins build jobs
13. Fulfil any ad-hoc data or report request queries from different functional groups

- Prior experience in Business Analytics and knowledge of related analysis or visualization tools
- Expecting a minimum of 2-4 years of relevant experience
- You will be managing a team of 3 currently
- Take up the ownership of developing and managing one of the largest and richest food (recipe, menu, and CPG) databases
- Interactions with cross-functional teams (Business, Food Science, Product, and Tech) on a regular basis to pan the future of client and internal food data management
- Should have a natural flair for playing with numbers and data and have a keen eye for detail and quality
- Will spearhead the Ops team in achieving the targets while maintaining a staunch attentiveness to Coverage, Completeness, and Quality of the data
- Shall program and manage projects while identifying opportunities to optimize costs and processes.
- Good business acumen, in creating logic & process flows, quick and smart decision-making skills are expected
- Will also be responsible for the recruitment, induction and training new members as well
- Setting competitive team targets. Guide and support the team members to go the extra mile and achieve set targets


Added Advantages :
- Experience in a Food Sector / Insights company
- Has a passion for exploring different cuisines
- Understands industry-related jargons and has a natural flair towards learning more about anything related to food

Data Architect who leads a team of 5 numbers. Required skills : Spark ,Scala , hadoop

We are a team with a mission, A mission to create and deliver great learning experiences to engineering students through various workshops and courses. If you are an industry professional and :- See great scope of improvement in higher technical education across the country and connect with our purpose of impacting it for good. Keen on sharing your technical expertise to enhance the practical learning of students. You are innovative in your ways of creating content and delivering them. You don’t mind earning few extra bucks while doing this in your free time. Buzz us at [email protected] and let us discuss how together we can take technological education in the country to new heights.

• Design secure solutions in line with the business strategy and security requirements • Contribute to the enterprise security architecture through developing Strategies, Reference Architectures, Roadmaps, Architectural Principles, Technology Standards, Security Non-Functional Requirements, Architectural Decisions and Design Patterns. • Deliver cyber security architectural artifacts such as High Level Designs and Solution Blueprints. • Ensure the enforcement of security requirements in solution architecture • Contribute to educating other architects and engineering teams in designing and implementing secure solutions Technologies The candidate should have knowledge and experience in designing and implementing the following technologies and related domains • Cloud security • Identity and Access Management • Encryption, Masking and Key Management • Data Classification, Data Privacy and Data Leakage Prevention • Infrastructure security (Network/Servers/Virtualization) • Application Security • Endpoint Security • SIEM and Log Management • Forward and Reverse Proxy • Big Data Security • IoT Security • SAP Security (Preferred) Architecture Skills • Solid experience in developing security solution architecture • Solid experience and knowledge in TOGAF and SABSA or other Enterprise Architecture frameworks. • Strong experience in developing architectural artifacts including reference architectures, roadmaps, architectural principles, technology standards, security non-functional requirements, architectural decisions and design patterns • Strong experience in documenting existing, transition and target architectures. Cyber Security Skills • Solid experience in performing security risk assessments and controls implementation • Strong experience in designing and implementing security controls by utilizing the technologies mentioned in the technologies section above • Strong knowledge in offensive and defensive aspects of cybesecurity with solid understanding of attack techniques and abuse cases. • Strong knowledge and implementation experience of cyber security standards, frameworks and regulations such as ISO27001, NIST CSF, CSA CCM, PCI-DSS, GDPR