(Senior) Data Engineer Job Description About us DataWeave provides Retailers and Brands with “Competitive Intelligence as a Service” that enables them to take key decisions that impact their revenue. Powered by AI, we provide easily consumable and actionable competitive intelligence by aggregating and analyzing billions of publicly available data points on the Web to help businesses develop data-driven strategies and make smarter decisions. Data Engineering and Delivery @DataWeave We the Delivery / Data engineering team at DataWeave, deliver the Intelligence with actionable data to the customer. One part of the work is to write effective crawler bots to collect data over the web, which calls for reverse engineering and writing scalable python code. Other part of the job is to crunch data with our big data stack / pipeline. Underpinnings are: Tooling, domain awareness, fast paced delivery, and pushing the envelope. How we work? It's hard to tell what we love more, problems or solutions! Every day, we choose to address some of the hardest data problems that there are. We are in the business of making sense of messy public data on the web. At serious scale! Read more on Become a DataWeaver What do we offer? ● Some of the most challenging data problems. Huge text and image datasets that you can play with! ● Ability to see the impact of your work and the value you're adding to our customers almost immediately. ● Opportunity to work on different problems and explore a wide variety of tools to figure out what really excites you. ● A culture of openness. Fun work environment. A flat hierarchy. Organization wide visibility. Flexible working hours. ● Learning opportunities with courses and tech conferences. Mentorship from seniors in the team. ● Last but not the least, competitive salary packages and fast paced growth opportunities. Relevant set of skills ● Good communication and collaboration skills with 4-7 years of experience. ● Ability to code and script with strong grasp of CS fundamentals, excellent problem solving abilities. ● Comfort with frequent, incremental code testing and deployment, Data management skills ● Good understanding of RDBMS ● Experience in building Data pipelines and processing large datasets . ● Knowledge of building crawlers and data mining is a plus. ● Working knowledge of open source tools such as mysql, Solr, ElasticSearch, Cassandra ( data stores ) would be a plus. ● Expert in Python programming Role and responsibilities ● Inclined towards working in a start-up environment. ● Comfort with frequent, incremental code testing and deployment, Data management skills ● Design and Build robust and scalable data engineering solutions for structured and unstructured data for delivering business insights, reporting and analytics. ● Expertise in troubleshooting, debugging, data completeness and quality issues and scaling overall system performance. ● Build robust API ’s that powers our delivery points (Dashboards, Visualizations and other integrations).
Organization Description Founded out of Harvard University and MIT in 2002, Dimagi is an award-winning global technology social enterprise with an impact-focused business model. We deliver open source technology to help underserved communities in over 60 countries around the world. Our partners include the World Bank, WHO, the Bill and Melinda Gates Foundation, USAID, UNICEF, Novartis, Johns Hopkins, the University of Pennsylvania, the Harvard School of Public Health and more. Dimagi’s work has been profiled in the New York Times, the Washington Post, Huffington Post, the World Economic Forum and Forbes, among others. Our team of exceptional public health and development specialists,software developers and engineers apply their experience in technology development, implementation, and global healthcare delivery to further public health and well-being. In the next few years, our India office has set aggressive goals, both for Dimagi and across the mHealth industry, to dramatically improve healthcare delivery at scale across India and beyond. We are developing and already implementing mobile applications for hundreds of thousands of health workers of the Government of India, who use our technology to provide services to hundreds of millions of beneficiaries across the country. Job Description Dimagi’s Technical Project Analysts are the main drivers in implementing our technology all over the world. They share a passion for challenging experiences and developing solutions for trying environments that enable people to live healthier and more productive lives. As a Technical Project Analyst, you will dedicate your time to one of our ground-breaking, large-scale mHealth projects, which is transforming community health systems across multiple Indian states, in partnership with central and state governments. You will work with a diverse, collaborative, and international team. You will be based out of Delhi (where our office is located). You will work on creating and deploying mobile applications, based on our CommCare platform, for strengthening service delivery of front-line workforce. You will join our global team and share lessons learned and expertise from around the world. You will receive regular mentorship and support from our office in New Delhi.In your first two years at Dimagi India, based on a combination of skills, interests, and opportunities, you will specialize in areas such as technical design and application development, quality assurance processes, and reporting functionalities on one of our large-scale projects. In the medium to long term, Dimagi is flexible in terms of growth and career progression, and strives to accommodate your interests in defining your future role. We have a skills-based performance and development system that rewards people for building their skills. Your performance will determine your future growth at Dimagi. Employment at Dimagi is at-will, however, we do expect a longer term commitment. Responsibilities Project needs assessments and technology requirements definition Design of mHealth applications, bringing together the best combination of stakeholder inputs, field feedback, technical feasibility, and robust architectural principles Preparation of detailed software specifications that convert design documents into practical instructions for CommCare application builders and for software engineers to build out requisite functionality Building mHealth and eHealth systems using Dimagi’s flagship mHealth platform, CommCare, using CommCare’s application building tools Development of software test plans, as well as load testing and quality assurance (QA) of our mobile tools Explaining project matters to the technical team and technical matters to the project team Building the capacity of the support team to address post deployment support matters Technical documentation Other project management and capacity building support Skills & Requirements Degree in computer science, engineering, information technology, or related – or equivalent work experience Experience with XML, MySQL or other relational database, Excel, and Access 1 – 4+ years of work experience Outstanding proactive oral and written communication skills Reliable, self-directed, and resourceful Able to juggle many things at once Able to get things done Flexible and able to wear many hats Well organised and detail oriented Interested and able to work in a multicultural team distributed across the globe Note that experience with software programming is NOT a requirement, since Dimagi tools allow applications to be built by system administrators through a user-friendly web interface. Desirable Fluency in Hindi or other Indian languages is an asset Quantitative data management and data analysis skills Experience with the complete software development lifecycle Experience working in India Desire to make a social impact Is this the right position for me? The ideal candidate is interested in understanding what makes technology work in practice in low-resource settings. This person is able to self-organize and self-manage, thrives on the intricacy and challenge of application design and project management and functions at a high level when interacting with partners. This person is detail-oriented and comfortable with mobile tools, and, if she does not have software development experience already, has at least been exposed to the software development process. Benefits Dimagi provides an employer-sponsored health insurance plan for all employees and their family, including an additional OPD and mental health allowance Health and Fitness allowance Air Quality reimbursement to cover air purifiers, pollution masks, etc Employee stock option plan 30 days Paid Time Off, Flex time and 10 Personal Initiative Days. Complimentary lunch provided in office Opportunities for international travel
• To own several functions for the Shiprocket Platform such as – multi channel commerce, inventory management, templates and front-end UI, back-end user interface and flows, user onboarding processes and analytics. • Work in a cross-functional team environment where you’re engaging with engineers, marketers, copyrighters, designers and senior management. • Ensure you deliver as per the product delivery plan. • Be able to think independently and own your area of the product – holistically. We believe in “full stack” product managers who can run a piece of a product entirely. • Ensuring we build for the right customer – and yet again, questioning who the right customer for us is. • Focus on solving problems by doing things that are right for the user. • Be able to re-iterate as many times as required and drive decisions based on data and analytics. • Perform primary market research, user research, competitive research and establish yourself and the company as a thought leader in the SaaS ecommerce space. • Clearly communicate product plans, enhancements, new features and launch “mini-products”. Be able to think through a feature from start to finish keeping in mind the impact on features, technology and ultimately the business. • Be able to operate within fuzziness and a state of “flux” – we truly are quite dynamic at our workplace. • Perform well under uncertainty and work with unclear communication channels intra and inter team considering that ours is an organization that’s swiftly evolving.