BIg Data/ Pyspark AWS Engineer

Siri InfoSolutions Inc

San Jose, CA (United States)

Category: Programming

Posted on: 22 Feb 2021

Hadoop Java Linux Python

Job DescriptionThe Big Data Engineer is responsible for engaging in the design, development and maintenance of the big data platform and solutions at Quicken Loans. This includes the platform host data sets that support various business operations and enable data-driven decisions as well as the analytical solutions that provide visibility and decision support using big data technologies. The Big Data Engineer is responsible for administering a Hadoop cluster, developing data integration solutions, resolving technical issues, and working with Data Scientists, Business Analysts, System Administrators and Data Architects to ensure the platform meets business demands. This team member also ensures that solutions are scalable, include necessary monitoring, and adhere to best practices and guidelines. The Big Data Engineer helps mentor new team members and continues to grow their knowledge of new technologies.
Responsibilities
Develop ELT processes from various data repositories and APIs across the enterprise, ensuring data quality and process efficiency
Develop data processing scripts using Spark
Develop relational and NoSQL data models to help conform data to meet users' needs using Hive and HBase
Integrate platform into the existing enterprise data warehouse and various operational systems
Develop administration processes to monitor cluster performance, resource usage, backup and mirroring to ensure a highly available platform
Address performance and scalability issues in a large-scale data lake environment
Provide big data platform support and issue resolutions to Data Scientists and fellow engineers
Requirements
Master's degree in computer science, software engineering or a closely related field
2 years of experience with Hadoop distribution and ecosystem tools such as Hive, Spark, NiFi and Oozie
2 years of experience developing batch and streaming ETL processes
2 years of experience with relational and NoSQL databases, including modeling and writing complex queries
Proficiency in at least one programming language, such as Python or Java
Experience with Linux system administration, scripting and basic network skills
Excellent communication, analytical and problem-solving skills

  Siri InfoSolutions Inc

  <p></p>

  Address
  San Jose,
     CA

     USA

    View all jobs at Siri InfoSolutions Inc

  Report Job

Want more jobs like this?

Apply Now

By clicking "Apply Now" or "Subscribe and Apply" you will be redirected to an external website. By clicking "Subscribe & Apply" you give TechieJobs consent to process and store your data, and send you emails to the provided email address.

Job Source: Ziprecruiter (Will expire by: 2021-04-09 00:00:00)

Apply Now

New

Techies Talent Network

Upload your resumes and connect with IT recruiters, setup personalized job alerts and receive professional tips to achieve your career goals, and even more. Join today!

Join talent network
Quick poll
In your opinion which programming language has a brighter future?
  • GoLang
  • Java
  • Python
  • Scala
  • Ruby
  • PHP
  • JavaScript
  • C++
  • Other

* This poll is anonymous.