Java Big data Developer

1. Big data developer with 8+ years of professional IT experience with expertise in Hadoop ecosystem components in ingestion, Data modeling, querying, processing, storage, analysis, Data Integration and Implementing enterprise level systems spanning Big Data.
2. A skilled developer with strong problem solving, debugging and analytical capabilities, who actively engages in understanding customer requirements.
3. Expertise in Apache Hadoop ecosystem components like Spark, Hadoop Distributed File Systems(HDFS), HiveMapReduce, Hive, Sqoop, HBase, Zookeeper, YARN, Flume, Pig, Nifi, Scala and Oozie.
4. Hands on experience in creating real - time data streaming solutions using Apache Spark core, Spark SQL & DataFrames, Kafka, Spark streaming and Apache Storm.
5. Excellent knowledge of Hadoop architecture and daemons of Hadoop clusters, which include Name node,Data node, Resource manager, Node Manager and Job history server.
6. Worked on both Cloudera and Horton works in Hadoop Distributions. Experience in managing Hadoop clustersusing Cloudera Manager tool.
7. Well versed in installation, Configuration, Managing of Big Data and underlying infrastructure of Hadoop Cluster.
8. Hands on experience in coding MapReduce/Yarn Programs using Java, Scala and Python for analyzing Big Data.
9. Exposure to Cloudera development environment and management using Cloudera Manager.
10. Extensively worked on Spark using Scala on cluster for computational (analytics), installed it on top of Hadoop performed advanced analytical application by making use of Spark with Hive and SQL/Oracle .
11. Implemented Spark using PYTHON and utilizing Data frames and Spark SQL API for faster processing of data and handled importing data from different data sources into HDFS using Sqoop and performing transformations using Hive, MapReduce and then loading data into HDFS.
12. Used Spark Data Frames API over Cloudera platform to perform analytics on Hive data.
13. Hands on experience in MLlib from Spark which are used for predictive intelligence, customer segmentation and for smooth maintenance in Spark streaming.
14. Experience in using Flume to load log files into HDFS and Oozie for workflow design and scheduling.
15. Experience in optimizing MapReduce jobs to use HDFS efficiently by using various compression mechanisms.
16. Working on creating data pipeline for different events of ingestion, aggregation, and load consumer response data into Hive external tables in HDFS location to serve as feed for tableau dashboards.
17. Hands on experience in using Sqoop to import data into HDFS from RDBMS and vice-versa.
18. In-depth Understanding of Oozie to schedule all Hive/Sqoop/HBase jobs.
19. Hands on expertise in real time analytics with Apache Spark.
20. Experience in converting Hive/SQL queries into RDD transformations using Apache Spark, Scala and Python.
21. Extensive experience in working with different ETL tool environments like SSIS, Informatica and reporting tool environments like SQL Server Reporting Services (SSRS).
22. Experience in Microsoft cloud and setting cluster in Amazon EC2 & S3 including the automation of setting & extending the clusters in AWS Amazon cloud.
23. Extensively worked on Spark using Python on cluster for computational (analytics), installed it on top of Hadoop performed advanced analytical application by making use of Spark with Hive and SQL.
24. Strong experience and knowledge of real time data analytics using Spark Streaming, Kafka and Flume.
25. Knowledge in installation, configuration, supporting and managing Hadoop Clusters using Apache, Cloudera (CDH3, CDH4) distributions and on Amazon web services (AWS).
26. Experienced in writing Ad Hoc queries using Cloudera Impala, also used Impala analytical functions.
27. Experience in creating Data frames using PySpark and performing operation on the Data frames using Python.
28. In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS and MapReduce Programming Paradigm, High Availability and YARN architecture.
29. Establishing multiple connections to different Redshift clusters (Bank Prod, Card Prod, SBBDA Cluster) and provide the access for pulling the information we need for analysis. 
30. Generated various kinds of knowledge reports using Power BI based on Business specification. 
31. Developed interactive Tableau dashboards to provide a clear understanding of industry specific KPIs using quick filters and parameters to handle them more efficiently.
32. Well Experience in projects using JIRA, Testing, Maven and Jenkins build tools.
33. Experienced in designing, built, and deploying and utilizing almost all the AWS stack (Including EC2, S3,), focusing on high-availability, fault tolerance, and auto-scaling.
34. Good experience with use-case development, with Software methodologies like Agile and Waterfall.
35. Working knowledge of Amazon's Elastic Cloud Compute( EC2 ) infrastructure for computational tasks and Simple Storage Service ( S3 ) as Storage mechanism.
36. Good working experience in importing data using Sqoop, SFTP from various sources like RDMS, Teradata, Mainframes, Oracle, Netezza to HDFS and performed transformations on it using Hive, Pig and Spark .
37. Extensive experience in Text Analytics, developing different Statistical Machine Learning solutions to various business problems and generating data visualizations using Python and R.
38. Proficient in NoSQL databases including HBase, Cassandra, MongoDB and its integration with Hadoop cluster.
39. Hands on experience in Hadoop Big data technology working on MapReduce, Pig, Hive as Analysis tool, Sqoop and Flume data import/export tools.

Responsibilities:

• Designing and implementing fine-tuned production ready data/ML pipelines in Hadoop platform.
• Driving optimization, testing and tooling to improve quality.
• Reviewing and approving high level & amp; detailed design to ensure that the solution delivers to the business needs and aligns to the data & analytics architecture principles and roadmap.
• Understanding business requirements and solution design to develop and implement solutions that adhere to big data architectural guidelines and address business requirements.
• Following proper SDLC (Code review, sprint process).
• Identifying, designing, and implementing internal process improvements: automating manual processes, optimizing data delivery, etc.
• Building robust and scalable data infrastructure (both batch processing and real-time) to support needs from internal and external users.
• Understanding various data security standards and using secure data security tools to apply and adhere to the required data controls for user access in the Hadoop platform.
• Supporting and contributing to development guidelines and standards for data ingestion.
• Working with a data scientist and business analytics team to assist in data ingestion and data related technical issues.
• Designing and documenting the development & deployment flow.

Requirements:

• Experience in developing rest API services using one of the Scala frameworks.
• Ability to troubleshoot and optimize complex queries on the Spark platform
• Expert in building and optimizing ‘big data’ data/ML pipelines, architectures and data sets.
• Knowledge in modelling unstructured to structured data design.
• Experience in Big Data access and storage techniques.
• Experience in doing cost estimation based on the design and development.
• Excellent debugging skills for the technical stack mentioned above which even includes analyzing server logs and application logs.
• Highly organized, self-motivated, proactive, and ability to propose best design solutions.
• Good time management and multitasking skills to work to deadlines by working independently and as a part of a team.

Designation: Perception Engineer (3D) 

Experience: 0 years to 8 years 

Position Type: Full Time 

Position Location: Hyderabad 

Compensation: As Per Industry standards 

About Monarch: 

At Monarch, we’re leading the digital transformation of farming. Monarch Tractor augments both muscle and mind with fully loaded hardware, software, and service machinery that will spur future generations of farming technologies. 

With our farmer-first mentality, we are building a smart tractor that will enhance (not replace) the existing farm ecosystem, alleviate labor availability, and cost issues, and provide an avenue for competitive organic and beyond farming by providing mechanical solutions to replace harmful chemical solutions. Despite all the cutting-edge technology we will incorporate, our tractor will still plow, still, and haul better than any other tractor in its class. We have all the necessary ingredients to develop, build and scale the Monarch Tractor and digitally transform farming around the world. 

Description: 

We are looking for engineers to work on applied research problems related to perception in autonomous driving of electric tractors. The team works on classical and deep learning-based techniques for computer vision. Several problems like SFM, SLAM, 3D Image processing, multiple view geometry etc. Are being solved to deploy on resource constrained hardware. 

Technical Skills: 

• Background in Linear Algebra, Probability and Statistics, graphical algorithms and optimization problems is necessary. 
• Solid theoretical background in 3D computer vision, computational geometry, SLAM and robot perception is desired. Deep learning background is optional. 
• Knowledge of some numerical algorithms or libraries among: Bayesian filters, SLAM, Eigen, Boost, g2o, PCL, Open3D, ICP. 
• Experience in two view and multi-view geometry. 
• Necessary Skills: Python, C++, Boost, Computer Vision, Robotics, OpenCV. 
• Academic experience for freshers in Vision for Robotics is preferred.  
• Experienced candidates in Robotics with no prior Deep Learning experience willing to apply their knowledge to vision problems are also encouraged to apply. 
• Software development experience on low-power embedded platforms is a plus. 

Responsibilities: 

• Understanding engineering principles and a clear understanding of data structures and algorithms. 
• Ability to understand, optimize and debug imaging algorithms. 
• Ability to drive a project from conception to completion, research papers to code with disciplined approach to software development on Linux platform. 
• Demonstrate outstanding ability to perform innovative and significant research in the form of technical papers, thesis, or patents. 
• Optimize runtime performance of designed models. 
• Deploy models to production and monitor performance and debug inaccuracies and exceptions. 
• Communicate and collaborate with team members in India and abroad for the fulfillment of your duties and organizational objectives. 
• Thrive in a fast-paced environment and can own the project end to end with minimum hand holding. 
• Learn & adapt to new technologies & skillsets. 
• Work on projects independently with timely delivery & defect free approach. 
• Thesis focusing on the above skill set may be given more preference. 

What you will get: 

At Monarch Tractor, you’ll play a key role on a capable, dedicated, high-performing team of rock stars. Our compensation package includes a competitive salary, excellent health benefits commensurate with the role you’ll play in our success.  

Technical Knowledge (Must Have)

• Strong experience in SQL / HiveQL/ AWS Athena,
• Strong expertise in the development of data pipelines (snaplogic is preferred).
• Design, Development, Deployment and administration of data processing applications.
• Good Exposure towards AWS and Azure Cloud computing environments.
• Knowledge around BigData, AWS Cloud Architecture, Best practices, Securities, Governance, Metadata Management, Data Quality etc.
• Data extraction through various firm sources (RDBMS, Unstructured Data Sources) and load to datalake with all best practices.
• Knowledge in Python
• Good knowledge in NoSQL technologies (Neo4J/ MongoDB)
• Experience/knowledge in SnapLogic (ETL Technologies)
• Working knowledge on Unix (AIX, Linux), shell scripting
• Experience/knowledge in Data Modeling. Database Development
• Experience/knowledge creation of reports and dashboards in Tableau/ PowerBI

Senior Big Data Engineer 

Note:   Notice Period : 45 days 

Banyan Data Services (BDS) is a US-based data-focused Company that specializes in comprehensive data solutions and services, headquartered in San Jose, California, USA. 

We are looking for a Senior Hadoop Bigdata Engineer who has expertise in solving complex data problems across a big data platform. You will be a part of our development team based out of Bangalore. This team focuses on the most innovative and emerging data infrastructure software and services to support highly scalable and available infrastructure. 

It's a once-in-a-lifetime opportunity to join our rocket ship startup run by a world-class executive team. We are looking for candidates that aspire to be a part of the cutting-edge solutions and services we offer that address next-gen data evolution challenges. 

Key Qualifications

·   5+ years of experience working with Java and Spring technologies

· At least 3 years of programming experience working with Spark on big data; including experience with data profiling and building transformations

· Knowledge of microservices architecture is plus 

· Experience with any NoSQL databases such as HBase, MongoDB, or Cassandra

· Experience with Kafka or any streaming tools

· Knowledge of Scala would be preferable

· Experience with agile application development 

· Exposure of any Cloud Technologies including containers and Kubernetes 

· Demonstrated experience of performing DevOps for platforms 

· Strong Skillsets in Data Structures & Algorithm in using efficient way of code complexity

· Exposure to Graph databases

· Passion for learning new technologies and the ability to do so quickly 

· A Bachelor's degree in a computer-related field or equivalent professional experience is required

Key Responsibilities

· Scope and deliver solutions with the ability to design solutions independently based on high-level architecture

· Design and develop the big data-focused micro-Services

· Involve in big data infrastructure, distributed systems, data modeling, and query processing

· Build software with cutting-edge technologies on cloud

· Willing to learn new technologies and research-orientated projects 

· Proven interpersonal skills while contributing to team effort by accomplishing related results as needed 

• We are looking for a Data Engineer to build the next-generation mobile applications for our world-class fintech product.
• The candidate will be responsible for expanding and optimising our data and data pipeline architecture, as well as optimising data flow and collection for cross-functional teams.
• The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimising data systems and building them from the ground up.
• Looking for a person with a strong ability to analyse and provide valuable insights to the product and business team to solve daily business problems.
• You should be able to work in a high-volume environment, have outstanding planning and organisational skills.

Qualifications for Data Engineer

• Working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimising ‘big data’ data pipelines, architectures, and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets. Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Looking for a candidate with 2-3 years of experience in a Data Engineer role, who is a CS graduate or has an equivalent experience.

What we're looking for?

• Experience with big data tools: Hadoop, Spark, Kafka and other alternate tools.
• Experience with relational SQL and NoSQL databases, including MySql/Postgres and Mongodb.
• Experience with data pipeline and workflow management tools: Luigi, Airflow.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift.
• Experience with stream-processing systems: Storm, Spark-Streaming.
• Experience with object-oriented/object function scripting languages: Python, Java, Scala.