Loading...

{{notif_text}}

Excited to launch our new report - Understanding the mind of modern candidates. Get it here

Apache Spark Jobs

Explore top Apache Spark Job opportunities for Top Companies & Startups. All jobs are added by verified employees who can be contacted directly below.

Lead Data Engineer
Lead Data Engineer

via Lymbyc
Founded 2012
Products and services{{j_company_types[1 - 1]}}
{{j_company_sizes[3 - 1]}} employees
{{j_company_stages[3 - 1]}}
{{rendered_skills_map[skill] || skill}}
Location icon
Bengaluru (Bangalore), Chennai
Experience icon
4 - 8 years
Experience icon
Best in industry9 - 14 lacs/annum

Key skill set : Apache NiFi, Kafka Connect (Confluent), Sqoop, Kylo, Spark, Druid, Presto, RESTful services, Lambda / Kappa architectures Responsibilities : - Build a scalable, reliable, operable and performant big data platform for both streaming and batch analytics - Design and implement data aggregation, cleansing and transformation layers Skills : - Around 4+ years of hands-on experience designing and operating large data platforms - Experience in Big data Ingestion, Transformation and stream/batch processing technologies using Apache NiFi, Apache Kafka, Kafka Connect (Confluent), Sqoop, Spark, Storm, Hive etc; - Experience in designing and building streaming data platforms in Lambda, Kappa architectures - Should have working experience in one of NoSQL, OLAP data stores like Druid, Cassandra, Elasticsearch, Pinot etc; - Experience in one of data warehousing tools like RedShift, BigQuery, Azure SQL Data Warehouse - Exposure to other Data Ingestion, Data Lake and querying frameworks like Marmaray, Kylo, Drill, Presto - Experience in designing and consuming microservices - Exposure to security and governance tools like Apache Ranger, Apache Atlas - Any contributions to open source projects a plus - Experience in performance benchmarks will be a plus

Job posted by
apply for job
apply for job
Venky Thiriveedhi picture
Venky Thiriveedhi
Job posted by
Venky Thiriveedhi picture
Venky Thiriveedhi
Apply for job
apply for job

Machine Learning Engineer
Machine Learning Engineer

Founded 2018
Products and services{{j_company_types[2 - 1]}}
{{j_company_sizes[2 - 1]}} employees
{{j_company_stages[2 - 1]}}
{{rendered_skills_map[skill] || skill}}
Location icon
Bengaluru (Bangalore)
Experience icon
3 - 9 years
Experience icon
Best in industry20 - 50 lacs/annum

How often have you read job descriptions and gone ‘I have read this before’ or ‘the real job description will come out during the interviews, so why bother reading this’. In other instances when job descriptions are actually well-written, ie not just copied and pasted from somewhere and try doing justice to what you’d be doing at the job, 2-4 months of a typical interview cycle make those descriptions obsolete by the time you actually start at the job. Also not unsurprising then: just like you ignore or skim through job descriptions, most recruiters do the same with your resumes – look for specific keywords and leave all the assessment for during the interview itself. Even worse: the human recruiter in some cases is being replaced by an algorithm to automate screening. You, therefore, will try to put as many keywords in your resume to ensure you get that interview call. Nobody is being ingenuine in this process but the very process is fundamentally broken. And that is exactly what we want to solve: create an effective ‘matching of work to the worker’ that is an accurate and real-time reflection of both ends, thus increasing the actual engagement with the work itself. Responsibilities In this role, you’ll build and implement novel Machine Learning and Deep Learning systems on our platform as well as help build the infrastructure to train and deploy them. Specifically, you will: - Design and implement the infrastructure required to train models at scale. - Work with the data team’s infrastructure to build real-time and offline feature databases. - Work with the data team to create the infrastructure to build and maintain the datasets from which models are created - Build the model serving systems with which we can deploy our models to production - As we grow, scale the ML system to be able to support more use cases and ML model types. Requirements - 1+ years of experience building production-ready ML models and systems. - 3+ years of building distributed systems and/or scalable backend systems and the ability to maintain such systems in production. - Strong software engineering fundamentals - understanding of data structures and algorithms, O-notation, ability to maintain a test suite and write clear maintainable code. - Familiarity with the majority of the following tools: Tensorflow, Numpy, Scipy, SparkML, pandas, scikit-learn. - Demonstrated leadership and self-direction and willingness to both teach others and learn new techniques. - Experience with big data processing and storage systems: Hadoop, Spark, Hbase, Cassandra etc. - Strong programming skills in Python. Intermediate to Advanced knowledge of SQL and ability to wrangle data from many disparate data sources - Technologies we use: MySQL, Python, AWS, Snowflake, R, and Looker, among many others.

Job posted by
apply for job
apply for job
Maitrayee  picture
Maitrayee
Job posted by
Maitrayee  picture
Maitrayee
Apply for job
apply for job

Scala + Spark Developer
Scala + Spark Developer

Founded 2015
Products and services{{j_company_types[3 - 1]}}
{{j_company_sizes[3 - 1]}} employees
{{j_company_stages[3 - 1]}}
{{rendered_skills_map[skill] || skill}}
Location icon
Hyderabad
Experience icon
4 - 9 years
Experience icon
Best in industry12 - 30 lacs/annum

Greeting from Unify Technologies! We are looking out for some great talent with hands-on Scala (or Python or Java) Programming with Spark and BigData for one of our top projects which we are working with one of the Top Product Development company in Technology. We are sure that this experience will help scale-up your career as well! PFB the JD for your quick reference. What are we looking: We are looking for candidates who have a keen interest in security, privacy, scalability, and performance, cater to customer experience and pay attention to details. You’ll be part of the team that develops Software, builds automated tests, does release and reliability engineering for extraordinary frontend and backend systems scaling to billions of users and devices. Key Qualifications: Scala Programming or Excellent Java Programming including Web Services Hadoop Big Data Development - size 500 to 800TB Individual contributors with strong coding skills Location of work: Hyderabad Member of HackerRank, HackerEarth, Stackoverflow et al. Proficiency with Big Data processing technologies (Hadoop, Spark, Oozie). Experience in building data pipelines and analysis tools using Scala, Java, Python Experience building large-scale server-side systems with distributed processing algorithms. Aptitude to independently learn new technologies. Strong problem solving skills Excellent oral and written English communication skills Our Company: Unify Technologies Our Website: http://unifytech.com/ Linked In: https://www.linkedin.com/company/9206998 Offices in: Gurgaon, Pune, Hyderabad - India, and Seattle-USA Industry/Domain: Cloud/Product - Cloud Automation, Data Engineering, Mobile Few words about Unify Technologies: Unify is a pioneer in developing technology solutions towards imparting greater value and creating collaboration amongst global businesses. Unify leads the way in changing the conventional wisdom to assure greater returns on investments made. Unify helps customers focus on their business while taking care of your software needs with a global strategy to transform their company. Employment Type: Full-Time Joining time: Immediate to 30 days Work Location: Hyderabad - India Education: Bachelor's degree or equivalent in Computer Science and others related fields from reputed colleges Job Summary: A Job at UNIFY is Inspired and Innovative. If you enjoy working on unique and challenging problems? Our Project - Apple’s Enterprise Technology Services (ETS) needed engineers to be part of internet-scale systems and platforms that power all of Apple’s enterprise applications and customer-facing products including iCloud, iTunes, Retail and Online stores. Kindly check below detailed JD and company details. Let us know your interest in pursuing this position. Thanks, SudhEe Sudheendra Srinivasan Lead Recruiter | 91 333 73693 unifytech.com

Job posted by
apply for job
apply for job
Prakash G picture
Prakash G
Job posted by
Prakash G picture
Prakash G
Apply for job
apply for job

Backend Engineer - Java/Scala/Distributed System/NoSQL
Backend Engineer - Java/Scala/Distributed System/NoSQL

Founded 2015
Products and services{{j_company_types[3 - 1]}}
{{j_company_sizes[2 - 1]}} employees
{{j_company_stages[3 - 1]}}
{{rendered_skills_map[skill] || skill}}
Location icon
Bengaluru (Bangalore)
Experience icon
3 - 7 years
Experience icon
Best in industry8 - 9 lacs/annum

Systems EngineerAbout Intellicar Telematics Pvt LtdIntellicar Telematics Private Limited is a vehicular telematics organization founded in 2015 with the vision of connecting businesses and customers to their vehicles in a meaningful way. We provide vehicle owners with the ability to connect and diagnose vehicles remotely in real-time. Our team consists of individuals with an in-depth knowledge and understanding in automotive engineering, driver analytics and information technology. By leveraging our expertise in the automotive domain, we have created solutions to reduce operational and maintenance costs of large fleets, and ensure safety at all times.Solutions :- Enterprise Fleet Management, GPS Tracking- Remote engine diagnostics, Driver behavior & training- Technology Integration : GIS, GPS, GPRS, OBD, WEB, Accelerometer, RFID, On-board Storage.Intellicar's team of accomplished automotive Engineers, hardware manufacturers, Software Developers and Data Scientists have developed the best solutions to track vehicles and drivers, and ensure optimum performance, utilization and safety at all times.We cater to the needs of our clients across various industries such as: Self drive cars, Taxi cab rentals, Taxi cab aggregators, Logistics, Driver training, Bike Rentals, Construction, ecommerce, armored trucks, Manufacturing, dealership and more. Desired skills as a developer :- Education: BE/B.Tech in Computer Science or related field.- 4+ years of experience with scalable distributed systems applications and building scalable multi-threaded server applications.- Strong programming skills in Java or Scala on Linux or a Unix based OS.- Understanding of distributed systems like Hadoop, Spark, Cassandra, Kafka.- Good understanding of HTTP, SQL, Database internals.- Good understanding of Internet and how it works- Create new features from scratch, enhance existing features and optimize existing functionality, from conception and design through testing and deployment.- Work on projects that make our network more stable, faster, and secure.- Work with our development QA and system QA teams to come up with regression tests that cover new changes to our software

Job posted by
apply for job
apply for job
Lata Patil picture
Lata Patil
Job posted by
Lata Patil picture
Lata Patil
Apply for job
apply for job
Why apply via CutShort?
Connect with actual hiring teams and get their fast response. No 3rd party recruiters. No spam.