Job Description: The Data Engineering team is one of the core technology teams of Lumiq.ai and is responsible for creating all the Data related products and platforms which scale for any amount of data, users, and processing. The team also interacts with our customers to work out solutions, create technical architectures and deliver the products and solutions. If you are someone who is always pondering how to make things better, how technologies can interact, how various tools, technologies, and concepts can help a customer or how a customer can use our products, then Lumiq is the place of opportunities. Who are you? Enthusiast is your middle name. You know what’s new in Big Data technologies and how things are moving Apache is your toolbox and you have been a contributor to open source projects or have discussed the problems with the community on several occasions You use cloud for more than just provisioning a Virtual Machine Vim is friendly to you and you know how to exit Nano You check logs before screaming about an error You are a solid engineer who writes modular code and commits in GIT You are a doer who doesn’t say “no” without first understanding You understand the value of documentation of your work You are familiar with Machine Learning Ecosystem and how you can help your fellow Data Scientists to explore data and create production-ready ML pipelines Eligibility At least 2 years of Data Engineering Experience Have interacted with Customers Must Have Skills: Amazon Web Services (AWS) - EMR, Glue, S3, RDS, EC2, Lambda, SQS, SES Apache Spark Python Scala PostgreSQL Git Linux Good to have Skills: Apache NiFi Apache Kafka Apache Hive Docker Amazon Certification
RESPONSIBILITIES: 1. Full ownership of Tech right from driving product decisions to architect to deployment. 2. Develop cutting edge user experience and build cutting edge technology solutions like instant messaging in poor networks, live-discussions, live-videos optimal matching. 3. Using Billions of Data Points to Build User Personalization Engine. 4. Building Data Network Effects Engine to increase Engagement & Virality. 5. Scaling the Systems to Billions of Daily Hits. 6. Deep diving into performance, power management, memory optimization & network connectivity optimization for the next Billion Indians. 7. Orchestrating complicated workflows, asynchronous actions, and higher order components. 8. Work directly with Product and Design teams. REQUIREMENTS: 1. Should have Hacked some (computer or non-computer) system to your advantage. 2. Built and managed systems with a scale of 10Mn+ Daily Hits 3. Strong architectural experience. 4. Strong experience in memory management, performance tuning and resource optimizations. 5. PREFERENCE- If you are a woman or an ex-entrepreneur or having a CS bachelors degree from IIT/BITS/NIT. P.S. If you don't fulfill one of the requirements, you need to be exceptional in the others to be considered.
ABOUT MOOSHAK We're at a point where the urban English-speaking Indian population is almost all online.The next billion Indians online all communicate via Indian languages. Mooshak was created with the singular aim of making the Internet fun and relevant for this large, untapped segment. At Mooshak, we want to connect and engage Indians in their own language. And that presents problems in various domains, from creativity in content creation, to creating a highly scalable platform, to applying techniques in AI and NLP in Indian languages to understand what people are saying and react to what they want. Mooshak is architected to scale. Irrespective of the number of followers, the read time for a feed remains constant. We achieve this by using distributed message queues and a distributed computing engine and some nifty caching! TECHNICAL RESPONSIBILITIES Mooshak’s Tech Stack Java Node.js Mongo DB Redis Apache Kafka & Apache Storm Nginx / Jenkins Server Developer’s Roles and Responsibilities You are expected to know at least 4 of these technologies with the ability to quickly learn the others. You will play the leading role in all stages of server development Architecture Coding Final testing Shipping The APIs are written and the product works fine. You are expected to understand the architecture and enhance product functionality. Sometimes you may be required to double up as the Dev Ops guy should the servers fail or the product not be working as expected. The core APIs are written in Node.js The distributed message queue (Kafka) and compute engine (Storm) are implemented in Java. Understanding of Angular 2 is a big plus as our Web app is built on the same. NON TECHNICAL RESPONSIBILITIES We are a startup. This means that: You will be expected to be someone who comes up with solutions instead of problems. You will be expected to work non stop including weekends if the servers crash. But otherwise we are quite chill! You will be expected to talk to multiple stakeholders customers, designer, client side developer to achieve user and business needs. A high aptitude and a positive attitude are a must You should be comfortable working independently as well as in a team. We are a lean team right now, with you as the only server developer (assisted by the folks who built the platform) JOB LOCATION You would be working out of our office in Pune. You may be required to travel occasionally to Mumbai or Bangalore to interact with some other team members.
Our company is working on some really interesting projects in Big Data Domain in various fields (Utility, Retail, Finance). We are working with some big corporates and MNCs around the world. While working here as Big Data Engineer, you will be dealing with big data in structured and unstructured form and as well as streaming data from Industrial IOT infrastructure. You will be working on cutting edge technologies and exploring many others while also contributing back to the open-source community. You will get to know and work on end-to-end processing pipeline which deals with all type of work like storing, processing, machine learning, visualization etc.