Hadoop Developer

at Securonix

Posted by Ramakrishna Murthy

Responsibilities Exp 3~5 years Build up a strong and scalable crawler system for leveraging external user & content data source from Facebook, Youtube and others internet products or service. Getting top trending keywords & topic from social media. Design and build initial version of the real-time analytics product from Machine Learning Models to recommend video contents in real time to 10M+ User Profiles independently. Architect and build Big Data infrastructures using Java, Kafka, Storm, Hadoop, Spark and other related frameworks, experience with Elastic search is a plus Excellent Analytical, Research and Problem Solving skills, in-depth knowledge of Data Structure Desired Skills and Experience B.S./M.S. degree in computer science, mathematics, statistics or a similar quantitative field with good college background 3+ years of work experience in relevant field (Data Engineer, R&D engineer, etc) Experience in Machine Learning and Prediction & Recommendation techniques Experience with Hadoop/MapReduce/Elastic-Stack/ELK and Big Data querying tools, such as Pig, Hive, and Impala Proficiency in a major programming language (e.g. Java/C/Scala) and/or a scripting language (Python) Experience with one or more NoSQL databases, such as MongoDB, Cassandra, HBase, Hive, Vertica, Elastic Search Experience with cloud solutions/AWS, strong knowledge in Linux and Apache Experience with any map-reduce SPARK/EMR Experience in building reports and/or data visualization Strong communication skills and ability to discuss the product with PMs and business owners

Our client is a Gurgaon based software startup in Artificial Intelligence, Big Data and Data Science domain. Our clients, have built a data scientist, a virtual one. It's an Artificial Intelligence powered agent who can learn & work 24x7 to deliver business insights that matter the most. ● Working on a Unique concept ● Recognized by Indian Angel Network (IAN), the biggest network in India along with DIPP (Govt. Of India) and NASSCOM. ● Winner of $120K credits as part of Microsoft Bizspark Plus program ● Raised two professional rounds of funding ● Alumni of premier institutes (like IIT Bombay, IIT Delhi) on our advisory panel ● The current hiring is for the core team expansion. It will be under 10. I.e. candidate will be part of the core founding team and will get tremendous exposure. Core founding team focuses on invention and gets huge opportunities to file patents. ● Days: Monday to Saturdays ● One weekday off per month as per employee's choice. This is on top of earned/privilege leaves and bank holidays. ● Location: Gurgaon ● Line Management: Directly reporting to CxO Team Position: - Sr. Product Developer Job Description: Sr. Product Developer will be part of the client’s Lab. As a Sr. Product Developer, the candidate will be working very closely with Product Management, AI Research and Data Scientist team. Key responsibilities include, ● Design and Development of product in Big Data architecture and framework - Apache Spark, HDFS, Flume, Kafka with Scala/Java ● Development of Machine Learning Algorithms in Apache Spark framework ● Design and Development of integration connectors with external data sources like RDBMS (MySQL, Oracle etc.) and other products ● Lead, mentor and coach team members Skills ● 4+ Years of product development experience in Scala / Java ● Must possess in-depth knowledge of core design/architectural concerns like design patterns, performance, code re-usability and quality ● Should have good understanding of RDBMS and EA diagrams ● Experience on development (or upgrades) of Apache Spark libraries and contributions to Apache Spark or other open source frameworks is an added advantage  ● Understanding on data security or statistics (like probability distribution etc.) is an added advantage ● Ability to take initiatives, self-motivated and a learning attitude is must. Experience & Qualification Required - B.Tech from Tier 1.5 (NIT/IIIT/DCE) with 4+ years of Experience

Looking for a technically sound and excellent trainer on big data technologies. Get an opportunity to become popular in the industry and get visibility. Host regular sessions on Big data related technologies and get paid to learn.

Desired Skills and Experience Strong platform / infrastructure automation expertise or DevOps experience. Experience in Cloud Platforms – Amazon AWS, Cloudstack, Openstack would be a huge advantage. Working with automation technologies (Puppet, Chef, Ansible) is highly desirable. Experience with Git/Github, Gerrit/ReviewBoard/Phabricator, Jenkins/Hudson and/or other build and continuous integration systems are highly desirable. Experience with scripting languages – Bash, Python, Ruby, Powershell. Ability to build, create, version control and deploy software repositoriesExperience in proposal writing, presales, architecture, designing code is a must. Strong problem solving and debugging skills. Open source contributions in cloud domain will be a huge plus. Experience with virtualization technologies – VMware, Xenserver, Microsoft Hyper-V. Prior experience with server automation products – BMC Bladelogic, Microsoft System Configuration Manager would be a huge plus. Knowledge of ITIL is an added bonus.

Sigmoid is a fast growing Product Based BIG DATA startup. Sequoia Funded & Backed by experienced Professionals & Advisors. Sigmoid is revolutionizing business intelligence and analytics by providing unified tools for historical and real time analysis on Apache Spark. With their suite of products, Sigmoid is democratizing streaming use-cases like RTB Data Analytics, Log Analytics, Fraud Detection, Sensor Data Analytics etc. Sigmoid can enable the customers’ engineering team to set up their infrastructure on Spark and ramp up their development timelines, or enable the analytics team to derive insights from their data. Sigmoid has created a real time exploratory analytics tool using on Apache SPARK which not only vastly improves performance but also reduces the cost. A user can quickly analyse huge volumes of data, filter through multiple dimensions, compare results across time periods and carry out root cause analysis in a matter of seconds. Leading organisations across industry verticals are currently using Sigmoid’s platform in production to create success stories. ------------------------------------ What Sigmoid offers you: Work in a well-funded (Sequoia Capital) Big Data company. Deal with Terabytes of data on a regular basis. Opportunity to contribute to top big data projects. Work on complex problems faced by leading global companies in multiple areas such as fraud detection, real-time analytics, pricing modeling and so on ------------------------------------------------------ We are looking for Someone who has: 6+ years of demonstrable experience designing technological solutions to complex data problems, developing efficient and scalable code. Experience in Design, Architecture, Development of Big Data Technologies. Provides Technical leadership in Big Data space (Apache Spark, Kafka, Flink, Hadoop, MapReduce, HDFS, Hive, HBase, Flume, Sqoop, NoSQL, Cassandra, HBase) Strong understanding of databases and SQL. Defines and Drives best practices in Big Data stack. Drives operational excellence through root cause analysis and continuous improvement for Big Data technologies and processes. Operating knowledge of cloud computing platforms (AWS and/ or Azure or Google Cloud). Mentors/coaches engineers to facilitate their development and provide technical leadership to them A technologist who Loves to code and design and have great problem-solving skills, and the ability & confidence to hack their way out of tight corners. In addition, the ideal candidate would have great problem-solving skills, and the ability & confidence to hack their way out of tight corners. ------------------------------------ Preferred Qualifications: Engineering Bachelors/Masters in Computer Science/IT. Top Tier Colleges (IIT, NIT, IIIT, etc) will be preferred. Salary is not a constraint for the right talent.

Experience : Minimum of 3 years of relevant development experience Qualification : BS in Computer Science or equivalent Skills Required: • Server side developers with good server side development experience in Java AND/OR Python • Exposure to Data Platforms (Cassandra, Spark, Kafka) will be a plus • Interested in Machine Learning will be a plus • Good to great problem solving and communication skill • Ability to deliver in an extremely fast paced development environment • Ability to handle ambiguity • Should be a good team player Job Responsibilities : • Learn the technology area where you are going to work • Develop bug free, unit tested and well documented code as per requirements • Stringently adhere to delivery timelines • Provide mentoring support to Software Engineer AND/ OR Associate Software Engineers • Any other as specified by the reporting authority

Data Scientist - We are looking for a candidate to build great recommendation engines and power an intelligent m.Paani user journey Responsibilities : - Data Mining using methods like associations, correlations, inferences, clustering, graph analysis etc. - Scale machine learning algorithm that powers our platform to support our growing customer base and increasing data volume - Design and implement machine learning, information extraction, probabilistic matching algorithms and models - Care about designing the full machine learning pipeline. - Extending company's data with 3rd party sources. - Enhancing data collection procedures. - Processing, cleaning and verifying data collected. - Ad hoc analysis of the data and present clear results. - Creating advanced analytics products that provide actionable insights. The Individual : - We are looking for a candidate with the following skills, experience and attributes: Required : - Someone with 2+ years of work experience in machine learning. - Educational qualification relevant to the role. Degree in Statistics, certificate courses in Big Data, Machine Learning etc. - Knowledge of Machine Learning techniques and algorithms. - Knowledge in languages and toolkits like Python, R, Numpy. - Knowledge of data visualization tools like D3,js, ggplot2. - Knowledge of query languages like SQL, Hive, Pig . - Familiar with Big Data architecture and tools like Hadoop, Spark, Map Reduce. - Familiar with NoSQL databases like MongoDB, Cassandra, HBase. - Good applied statistics skills like distributions, statistical testing, regression etc. Compensation & Logistics : This is a full-time opportunity. Compensation will be in line with startup, and will be based on qualifications and experience. The position is based in Mumbai, India, and the candidate must live in Mumbai or be willing to relocate.

candidate will be responsible for all aspects of data acquisition, data transformation, and analytics scheduling and operationalization to drive high-visibility, cross-division outcomes. Expected deliverables will include the development of Big Data ELT jobs using a mix of technologies, stitching together complex and seemingly unrelated data sets for mass consumption, and automating and scaling analytics into the GRAND's Data Lake. Key Responsibilities : - Create a GRAND Data Lake and Warehouse which pools all the data from different regions and stores of GRAND in GCC - Ensure Source Data Quality Measurement, enrichment and reporting of Data Quality - Manage All ETL and Data Model Update Routines - Integrate new data sources into DWH - Manage DWH Cloud (AWS/AZURE/Google) and Infrastructure Skills Needed : - Very strong in SQL. Demonstrated experience with RDBMS, Unix Shell scripting preferred (e.g., SQL, Postgres, Mongo DB etc) - Experience with UNIX and comfortable working with the shell (bash or KRON preferred) - Good understanding of Data warehousing concepts. Big data systems : Hadoop, NoSQL, HBase, HDFS, MapReduce - Aligning with the systems engineering team to propose and deploy new hardware and software environments required for Hadoop and to expand existing environments. - Working with data delivery teams to set up new Hadoop users. This job includes setting up Linux users, setting up and testing HDFS, Hive, Pig and MapReduce access for the new users. - Cluster maintenance as well as creation and removal of nodes using tools like Ganglia, Nagios, Cloudera Manager Enterprise, and other tools. - Performance tuning of Hadoop clusters and Hadoop MapReduce routines. - Screen Hadoop cluster job performances and capacity planning - Monitor Hadoop cluster connectivity and security - File system management and monitoring. - HDFS support and maintenance. - Collaborating with application teams to install operating system and - Hadoop updates, patches, version upgrades when required. - Defines, develops, documents and maintains Hive based ETL mappings and scripts

It is one of the largest communication technology companies in the world. They operate America's largest 4G LTE wireless network and the nation's premiere all-fiber broadband network.

Position:-Data Scientist Location :- Gurgaon Job description Shopclues is looking for talented Data Scientist passionate about building large scale data processing systems to help manage the ever-growing information needs of our clients. Education : PhD/MS or equivalent in Applied mathematics, statistics, physics, computer science or operations research background. 2+ years experience in a relevant role. Skills · Passion for understanding business problems and trying to address them by leveraging data - characterized by high-volume, high dimensionality from multiple sources · Ability to communicate complex models and analysis in a clear and precise manner · Experience with building predictive statistical, behavioural or other models via supervised and unsupervised machine learning, statistical analysis, and other predictive modeling techniques. · Experience using R, SAS, Matlab or equivalent statistical/data analysis tools. Ability to transfer that knowledge to different tools · Experience with matrices, distributions and probability · Familiarity with at least one scripting language - Python/Ruby · Proficiency with relational databases and SQL Responsibilities · Has worked in a big data environment before alongside a big data engineering team (and data visualization team, data and business analysts) · Translate client's business requirements into a set of analytical models · Perform data analysis (with a representative sample data slice) and build/prototype the model(s) · Provide inputs to the data ingestion/engineering team on input data required by the model, size, format, associations, cleansing required