- 5+ years of industry experience in administering (including setting up, managing, monitoring) data processing pipelines (both streaming and batch) using frameworks such as Kafka Streams, Py Spark, and streaming databases like druid or equivalent like Hive
- Strong industry expertise with containerization technologies including kubernetes (EKS/AKS), Kubeflow
- Experience with cloud platform services such as AWS, Azure or GCP especially with EKS, Managed Kafka
- 5+ Industry experience in python
- Experience with popular modern web frameworks such as Spring boot, Play framework, or Django
- Experience with scripting languages. Python experience highly desirable. Experience in API development using Swagger
- Implementing automated testing platforms and unit tests
- Proficient understanding of code versioning tools, such as Git
- Familiarity with continuous integration, Jenkins
Responsibilities
- Architect, Design and Implement Large scale data processing pipelines using Kafka Streams, PySpark, Fluentd and Druid
- Create custom Operators for Kubernetes, Kubeflow
- Develop data ingestion processes and ETLs
- Assist in dev ops operations
- Design and Implement APIs
- Identify performance bottlenecks and bugs, and devise solutions to these problems
- Help maintain code quality, organization, and documentation
- Communicate with stakeholders regarding various aspects of solution.
- Mentor team members on best practices
Similar jobs
We are #hiring !!
PHP developer
Skills in HTML, CSS, Bootstrap, PHP, DHTML, XML, JavaScript, jQuery/Prototype and AJAX, WordPress
Generating WordPress themes and plugins.
Conducting website performance tests.
Full-stack developers will be most preferred.
Other CMS/framework(LARAVEL) experience will be an added advantage
Experience Preferred
Job Location: Bhubaneswar, Odisha
The Unique Culture
Title:- Senior Data Engineer
Experience: 4-6 yrs
Budget: 24-28 lpa
Location: Bangalore
Work of Mode: Work from office
Primary Skills: Data Bricks, Spark, Pyspark,Sql, Python, AWS
Qualification: Any Engineering degree
Responsibilities:
∙Design and build reusable components, frameworks and libraries at scale to support analytics products.
∙Design and implement product features in collaboration with business and Technology
stakeholders.
∙Anticipate, identify and solve issues concerning data management to improve data quality.
∙Clean, prepare and optimize data at scale for ingestion and consumption.
∙Drive the implementation of new data management projects and re-structure of the current data architecture.
∙Implement complex automated workflows and routines using workflow scheduling tools.
∙Build continuous integration, test-driven development and production deployment
frameworks.
∙Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards.
∙Analyze and profile data for the purpose of designing scalable solutions.
∙Troubleshoot complex data issues and perform root cause analysis to proactively resolve
product and operational issues.
∙Mentor and develop other data engineers in adopting best practices.
Qualifications:
Primary skillset:
∙Experience working with distributed technology tools for developing Batch and Streaming pipelines using
o SQL, Spark, Python, PySpark [4+ years],
o Airflow [3+ years],
o Scala [2+ years].
∙Able to write code which is optimized for performance.
∙Experience in Cloud platform, e.g., AWS, GCP, Azure, etc.
∙Able to quickly pick up new programming languages, technologies, and frameworks.
∙Strong skills building positive relationships across Product and Engineering.
∙Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
∙Experience with creating/ configuring Jenkins pipeline for smooth CI/CD process for Managed Spark jobs, build Docker images, etc.
∙Working knowledge of Data warehousing, Data modelling, Governance and Data Architecture
Good to have:
∙Experience working with Data platforms, including EMR, Airflow, Databricks (Data
Engineering & Delta Lake components, and Lakehouse Medallion architecture), etc.
∙Experience working in Agile and Scrum development process.
∙Experience in EMR/ EC2, Databricks etc.
∙Experience working with Data warehousing tools, including SQL database, Presto, and
Snowflake
∙Experience architecting data product in Streaming, Serverless and Microservices Architecture and platform.
What You will do
● Create beautiful software experiences for our clients using design thinking, lean and
agile methodology.
● Work on software products designed from scratch using the latest cutting edge
technologies, platforms and languages such as JAVA, python, Javascript, GoLang and
Scala.
● Work in a dynamic, collaborative, transparent, non-hierarchical culture.
● Work in collaborative, fast-paced and value-driven teams to build innovative customer
experiences for our clients.
● Help to grow the next generation of developers and have a positive impact on the
industry.
Basic Qualifications
● Experience: 4+ years.
● Hands-on development experience with a broad mix of languages such as JAVA, Python,
Javascript etc.
● Server-side development experience mainly in JAVA, (Python and nodeJS can be
considerable)
● UI development experience in ReactJS or AngularJS or PolymerJS or EmberJS or Jquery
etc. are good to have.
● Passion for software engineering and follow the best coding concepts.
● Good to great problem solving and communication skills.
Nice to have Qualifications
● Product and customer-centric mindset.
● Great OO skills, including design patterns.
● Experience with devops, continuous integration & deployment.
● Exposure to big data technologies, Machine Learning and NLP will be a plus.
Benefits
● Competitive salary.
● Work from anywhere.
● Learning and gaining experience rapidly.
● Reimbursement for basic working set up at home.
● Insurance (including a top up insurance for COVID).
Location
Remote - work from anywhere.
Responsibilities
• Build high quality systems and applications that solve key challenges for the customer
• Collaborate with product management and engineering teams to elicit and understand their requirements, challenges and develop potential solutions
• Proactively identify areas of improvements, come up with innovative solutions, and drive adoption
• Push boundaries on infrastructure and cost optimisation, reusability, simplicity and scalability • Own significant engineering projects end-to-end
• Sign-off on all significant architectures and technical designs regardless of front-end or back-end.
• Translate business requirements to tangible engineering deliverables
• Setup and streamline processes and introduce best practices to improve the overall efficiency and throughput of the engineering team
• Be an effective mentor and help in building capacity in the larger team, strategize and work towards helping junior engineers grow
• Stay current with latest tools, technologies, methodologies and ideas; spread the knowledge and conduct effective POCs to demonstrate applicability and benefits
Requirements
• 15+ years of strong experience in developing highly scalable backend and middle tier enterprise applications.
• Strong experience with large volume batch and real-time data processing applications
• Experience building enterprise SaaS applications. Strong knowledge of AWS.
• Programming Languages: Java (must have), Python (good to have)
• BS/MS in Computer Science or equivalent from premier institutes
• Strong in problem solving, data structures and algorithm design.
• Clear Object Oriented Design and Design Pattern concepts
• Significant experience working with non-functional requirements such as performance, security, compliance, automation etc.
• Experience developing AI-based solutions is a big plus.
• Comfortable working in a fast paced startup environment.
• Strong communication and collaboration skills.
• Attention to detail
-Proficiency with Javascript and Node.js + TypeScript , Express JS
-Familiar with JS tools
-Experience writing code from scratch
-Thrive in a fast-paced working environment, owning deadlines and striving for efficiency
-Familiarity with GIT, Jira, Redmine, or similar project management and issue tracking software
-Working knowledge of design principles, typography, user interfaces, Web standards and usability
-Familiar with interaction design and information architecture methods and principles
**~ Job Requirements ~**
- Handson with Node js and express.js
- working in a young, ambitious, and international team
- good to have experience building an API from scratch
- Are interested in security and its best practices
- Hands-on programming expertise in Java OR Python
- Strong production experience with Spark (Minimum of 1-2 years)
- Experience in data pipelines using Big Data technologies (Hadoop, Spark, Kafka, etc.,) on large scale unstructured data sets
- Working experience and good understanding of public cloud environments (AWS OR Azure OR Google Cloud)
- Experience with IAM policy and role management is a plus
Position: Lead - PHP Developer
About WebMD:
WebMD Health Corp., an Internet Brands Company, is the leading provider of health information services, serving patients, physicians, health care professionals, employers, and health plans through our public and private online portals, mobile platforms, and health-focused publications. The WebMD Health Network includes WebMD Health, Medscape, Jobson Healthcare Information, prIME Oncology, MediQuality, Frontline, QxMD, Vitals Consumer Services, MedicineNet, eMedicineHealth, RxList, OnHealth, Medscape Education, and other owned WebMD sites. WebMD®, Medscape®, CME Circle®, Medpulse®, eMedicine®, MedicineNet®, theheart.org®, and RxList® are among the trademarks of WebMD Health Corp. or its subsidiaries.
For company details, visit our website: http://www.webmd.com">www.webmd.com and http://www.krames.com">www.krames.com
Education: B.E. Computer Science/IT degree (or any other engineering discipline)
Experience: 7 + years
Work timings: 2 PM to 11 PM
Position Requirements:
- Technical expertise in leading a team with hands on experience with Core PHP and JavaScript
- Highly proficient with HTML, CSS, RESTful API
- Experience with SQL/NoSQL databases and their declarative query languages
- Experience in working with CI/CD tools and automation
- Experience in working on Kubernetes
- Knowledge of functional programming and object-oriented programming paradigms
- Ability to communicate with team members and stakeholders
- The candidate must work in a Collaborative style, should be results-driven and a problem solver
- Must be self-driven, detail-oriented with ability to produce high quality and timely results
Role & Responsibilities:
- Lead a technical team on building custom modules in a tool that creates static html pages
- Develop these new functionalities with the team's other frontend & backend developers while ensuring compliance with technical requirements (quality, performance, and security)
- Bring these functionalities into production and ensure their proper functioning
- Sharing knowledge with the team
➢ Agility and ability to adapt quickly to changing requirements and scope and priorities
➢ Strong proficiency with JavaScript (ECMAScript 5, 6)
➢ Knowledge of Node.js and frameworks available for it (SailsJS, Express)
➢ Understanding the nature of asynchronous programming and its quirks and workarounds
➢ Familiarity with front-end technologies
➢ User authentication and authorization between multiple systems, servers, and environments
➢ Interaction with multiple data sources
➢ Good understanding of SQL syntax
➢ Understanding fundamental design principles behind a scalable application
➢ Understanding differences between multiple delivery platforms, such as mobile vs. desktop, and optimizing output to match the specific platform.
➢ Implementing automated testing platforms and unit tests
➢ Proficient understanding of code versioning tools, such as Git .
➢ Knowledge in the field of IOT is good to have.