Job Responsibilities:
- Developing highly reliable web crawlers and parsers across various websites
- Extract structured/unstructured data and store them into SQL/No SQL database
- Work closely with Product/Research/Technology teams to provide data for analysis
- Develop frameworks for automating and maintaining constant flow of data from multiple sources
- Develop and maintain data pipelines for batch/incremental as well as real-time requirements.
- Develop a deep understanding of the data sources on the web and know exactly how, when, and which data to parse and store this data
- Create a monitoring framework to identify anomalies in web crawlers and resolve for contingencies
- Implement best practices in-house to detect / prevent crawlers on internal systems and websites
- Writing and running queries on large datasets to support analytics team or data sharing requirements.
- Dealing well with ambiguity, prioritizing needs, and delivering results in a dynamic environment
Must-Have:
- Proficient knowledge in Python language and excellent knowledge on Web Crawling in Python Scrapy / Beautifulsoup / URLlib / Selenium / WebHarvest etc.
- Experience in Data parsing and understanding of document structure in HTML – CSS/DOM/XPATH. Knowledge of JS would be a plus
- Strong experience in Data Parsing
- Experience in working with large datasets, querying terabytes of data on a regular basis – proficient in SQL
- Must be able to develop reusable code-based crawlers that are easy to modify / transform
- Proficient in GIT and better understanding of launching instances and setting up crawlers on AWS/Azure
- Understands detailed requirements and demonstrates excellent problem-solving skills
- Strong sense of ownership, drive, and ability to deliver results.
- A track record of digging in to the tough problems / challenges and bringing innovative approaches to solve for such situations. Must be highly capable of self-teaching new techniques.
B.E/B.Tech in Computer Science / IT, BCA, B.Sc in Computer Science / IT
About Impact Guru
Similar jobs
-
4+ years of data analysis experience
-
Advanced working knowledge of SQL (window functions, CTEs, etc.)
-
Experience working with a BI tool like Tableau, Chartio, or Looker
-
Knowledge of Python and/or R
-
Strong critical thinking and problem-solving skills
-
Success owning your own projects and driving these projects to
completion
-
Hands-on experience with data pipelines and/or ETL processes
-
Excellent verbal and written communication skills, with the ability to
communicate technical concepts to a non-technical audience
-
Strong business intuition and an ability to relate analyses to 6sense’s
goals and objectives
-
Ability to prioritize and execute tasks in a changing environment
Data Engineer & Sr Data Engineer
Must Have Skills:
• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skills
Job Summary :
Independently handle the delivery of analytics assignments by mentoring a team of 3 - 10 people and delivering to exceed client expectations
Responsibilities :
- Co-ordinate with onsite company consultants to ensure high quality, on-time delivery
- Take responsibility for technical skill-building within the organization (training, process definition, research of new tools and techniques etc.)
- Take part in organizational development activities to take company to the next level
Qualification, Skills & Prior Work Experience :
- Great analytical skills, detail-oriented approach
- Sound knowledge in MS Office tools like Excel, Power Point and data visualization tools like Tableau, PowerBI or such tools
- Strong experience in SQL, Python, SAS, SPSS, Statistica, R, MATLAB or such tools would be preferable
- Ability to adapt and thrive in the fast-paced environment that young companies operate in
- Priority for people with analytics work experience
- Programming skills- Java/Python/SQL/OOPS based programming knowledge
Job Location : Chennai, Work from Home will be provided until COVID situation improves
Note :
- Minimum one year experience needed
- Only 2019, 2020 and 2020 passed outs applicable
- Only above 70% aggregate throughout studies is applicable
- POST GRADUATION is must
Job Description |
Job Title: Data Engineer |
Tech Job Family: DACI |
• Bachelor's Degree in Engineering, Computer Science, CIS, or related field (or equivalent work experience in a related field) |
• 2 years of experience in Data, BI or Platform Engineering, Data Warehousing/ETL, or Software Engineering |
• 1 year of experience working on project(s) involving the implementation of solutions applying development life cycles (SDLC) |
Preferred Qualifications: |
• Master's Degree in Computer Science, CIS, or related field |
• 2 years of IT experience developing and implementing business systems within an organization |
• 4 years of experience working with defect or incident tracking software |
• 4 years of experience with technical documentation in a software development environment |
• 2 years of experience working with an IT Infrastructure Library (ITIL) framework |
• 2 years of experience leading teams, with or without direct reports |
• Experience with application and integration middleware |
• Experience with database technologies |
Data Engineering |
• 2 years of experience in Hadoop or any Cloud Bigdata components (specific to the Data Engineering role) |
• Expertise in Java/Scala/Python, SQL, Scripting, Teradata, Hadoop (Sqoop, Hive, Pig, Map Reduce), Spark (Spark Streaming, MLib), Kafka or equivalent Cloud Bigdata components (specific to the Data Engineering role) |
BI Engineering |
• Expertise in MicroStrategy/Power BI/SQL, Scripting, Teradata or equivalent RDBMS, Hadoop (OLAP on Hadoop), Dashboard development, Mobile development (specific to the BI Engineering role) |
Platform Engineering |
• 2 years of experience in Hadoop, NO-SQL, RDBMS or any Cloud Bigdata components, Teradata, MicroStrategy (specific to the Platform Engineering role) |
• Expertise in Python, SQL, Scripting, Teradata, Hadoop utilities like Sqoop, Hive, Pig, Map Reduce, Spark, Ambari, Ranger, Kafka or equivalent Cloud Bigdata components (specific to the Platform Engineering role) |
Lowe’s is an equal opportunity employer and administers all personnel practices without regard to race, color, religion, sex, age, national origin, disability, sexual orientation, gender identity or expression, marital status, veteran status, genetics or any other category protected under applicable law. |
Senior Artificial intelligence/ Machine Learning Developer
at A firm which woks with US clients. Permanent WFH.
This person MUST have:
- B.E Computer Science or equivalent
- 5 years experience with the Django framework
- Experience with building APIs (REST or GraphQL)
- Strong Troubleshooting and debugging skills
- React.js knowledge would be an added bonus
- Understanding on how to use a database like Postgres (prefered choice), SQLite, MongoDB, MySQL.
- Sound knowledge of object-oriented design and analysis.
- A strong passion for writing simple, clean and efficient code.
- Proficient understanding of code versioning tools Git.
- Strong communication skills.
Experience:
- Min 5 year experience
- Startup experience is a must.
Location:
- Remote developer
Timings:
- 40 hours a week but with 4 hours a day overlapping with client timezone. Typically clients are in California PST Timezone.
Position:
- Full time/Direct
- We have great benefits such as PF, medical insurance, 12 annual company holidays, 12 PTO leaves per year, annual increments, Diwali bonus, spot bonuses and other incentives etc.
- We dont believe in locking in people with large notice periods. You will stay here because you love the company. We have only a 15 days notice period.
Skills- Informatica with Big Data Management
1.Minimum 6 to 8 years of experience in informatica BDM development
2.Experience working on Spark/SQL
3.Develops informtica mapping/Sql
About Us
upGrad is an online education platform building the careers of tomorrow by offering the most industry-relevant programs in an immersive learning experience. Our mission is to create a new digital-first learning experience to deliver tangible career impact to individuals at scale. upGrad currently offers programs in Data Science, Machine Learning, Product Management, Digital Marketing, and Entrepreneurship, etc. upGrad is looking for people passionate about management and education to help design learning programs for working professionals to stay sharp and stay relevant and help build the careers of tomorrow.
-
upGrad was awarded the Best Tech for Education by IAMAI for 2018-19
-
upGrad was also ranked as one of the LinkedIn Top Startups 2018: The 25 most sought-
after startups in India
-
upGrad was earlier selected as one of the top ten most innovative companies in India
by FastCompany.
-
We were also covered by the Financial Times along with other disruptors in Ed-Tech
-
upGrad is the official education partner for Government of India - Startup India
program
-
Our program with IIIT B has been ranked #1 program in the country in the domain of Artificial Intelligence and Machine Learning
Role Summary
Are you excited by the challenge and the opportunity of applying data-science and data- analytics techniques to the fast developing education technology domain? Do you look forward to, the sense of ownership and achievement that comes with innovating and creating data products from scratch and pushing it live into Production systems? Do you want to work with a team of highly motivated members who are on a mission to empower individuals through education?
If this is you, come join us and become a part of the upGrad technology team. At upGrad the technology team enables all the facets of the business - whether it’s bringing efficiency to ourmarketing and sales initiatives, to enhancing our student learning experience, to empowering our content, delivery and student success teams, to aiding our student’s for their desired careeroutcomes. We play the part of bringing together data & tech to solve these business problems and opportunities at hand.
We are looking for an highly skilled, experienced and passionate data-scientist who can come on-board and help create the next generation of data-powered education tech product. The ideal candidate would be someone who has worked in a Data Science role before wherein he/she is comfortable working with unknowns, evaluating the data and the feasibility of applying scientific techniques to business problems and products, and have a track record of developing and deploying data-science models into live applications. Someone with a strong math, stats, data-science background, comfortable handling data (structured+unstructured) as well as strong engineering know-how to implement/support such data products in Production environment.
Ours is a highly iterative and fast-paced environment, hence being flexible, communicating well and attention-to-detail are very important too. The ideal candidate should be passionate about the customer impact and comfortable working with multiple stakeholders across the company.
Roles & Responsibilities-
- 3+ years of experience in analytics, data science, machine learning or comparable role
- Bachelor's degree in Computer Science, Data Science/Data Analytics, Math/Statistics or related discipline
- Experience in building and deploying Machine Learning models in Production systems
- Strong analytical skills: ability to make sense out of a variety of data and its relation/applicability to the business problem or opportunity at hand
- Strong programming skills: comfortable with Python - pandas, numpy, scipy, matplotlib; Databases - SQL and noSQL
- Strong communication skills: ability to both formulate/understand the business problem at hand as well as ability to discuss with non data-science background stakeholders
- Comfortable dealing with ambiguity and competing objectives
Skills Required
-
Experience in Text Analytics, Natural Language Processing
-
Advanced degree in Data Science/Data Analytics or Math/Statistics
-
Comfortable with data-visualization tools and techniques
-
Knowledge of AWS and Data Warehousing
-
Passion for building data-products for Production systems - a strong desire to impact
the product through data-science technique
-
About antuit.ai
Antuit.ai is the leader in AI-powered SaaS solutions for Demand Forecasting & Planning, Merchandising and Pricing. We have the industry’s first solution portfolio – powered by Artificial Intelligence and Machine Learning – that can help you digitally transform your Forecasting, Assortment, Pricing, and Personalization solutions. World-class retailers and consumer goods manufacturers leverage antuit.ai solutions, at scale, to drive outsized business results globally with higher sales, margin and sell-through.
Antuit.ai’s executives, comprised of industry leaders from McKinsey, Accenture, IBM, and SAS, and our team of Ph.Ds., data scientists, technologists, and domain experts, are passionate about delivering real value to our clients. Antuit.ai is funded by Goldman Sachs and Zodius Capital.
The Role:
Antuit is looking for a Data / Sr. Data Scientist who has the knowledge and experience in developing machine learning algorithms, particularly in supply chain and forecasting domain with data science toolkits like Python.
In this role, you will design the approach, develop and test machine learning algorithms, implement the solution. The candidate should have excellent communication skills and be results driven with a customer centric approach to problem solving. Experience working in the demand forecasting or supply chain domain is a plus. This job also requires the ability to operate in a multi-geographic delivery environment and a good understanding of cross-cultural sensitivities.
Responsibilities:
Responsibilities includes, but are not limited to the following:
- Design, build, test, and implement predictive Machine Learning models.
- Collaborate with client to align business requirements with data science systems and process solutions that ensure client’s overall objectives are met.
- Create meaningful presentations and analysis that tell a “story” focused on insights, to communicate the results/ideas to key decision makers.
- Collaborate cross-functionally with domain experts to identify gaps and structural problems.
- Contribute to standard business processes and practices as part of a community of practise.
- Be the subject matter expert across multiple work streams and clients.
- Mentor and coach team members.
- Set a clear vision for the team members and working cohesively to attain it.
Qualifications and Skills:
Requirements
- Experience / Education:
- Master’s or Ph.D. in Computer Science, Computer Engineering, Electrical Engineering, Statistics, Applied Mathematics or other related
- 5+ years’ experience working in applied machine learning or relevant research experience for recent Ph.D. graduates.
- Highly technical:
- Skilled in machine learning, problem-solving, pattern recognition and predictive modeling with expertise in PySpark and Python.
- Understanding of data structures and data modeling.
- Effective communication and presentation skills
- Able to collaborate closely and effectively with teams.
- Experience in time series forecasting is preferred.
- Experience working in start-up type environment preferred.
- Experience in CPG and/or Retail preferred.
- Effective communication and presentation skills.
- Strong management track record.
- Strong inter-personal skills and leadership qualities.
Information Security Responsibilities
- Understand and adhere to Information Security policies, guidelines and procedure, practice them for protection of organizational data and Information System.
- Take part in Information Security training and act accordingly while handling information.
- Report all suspected security and policy breach to Infosec team or appropriate authority (CISO).
EEOC
Antuit.ai is an at-will, equal opportunity employer. We consider applicants for all positions without regard to race, color, religion, national origin or ancestry, gender identity, sex, age (40+), marital status, disability, veteran status, or any other legally protected status under local, state, or federal law.