Sr. Data Engineer : 24-01959 Jobs in New York NY Data Engineering, Big Data & Hadoop Job Recruiters

Posted: 08/13/2024 Job Category: Data Engineering, Big Data & Hadoop Job Number: 22582544

Job Description

Primary Skills: SQL, ETL, Python, Scala, Airflow, Snowflake
Contract Type: W2
Duration: 6 Months
Location: New York, NY, Burbank CA, or Seattle WA, Santa Monica CA (Hybrid)
Pay Range: $70.00 to 80.00 Per hour

JOB SUMMARY:
A leading fortune 500 Media and Entertainment company is looking for a Sr. Data Engineer with experience in Scala and Python that abstract complex business logic to allow consistent functionality across all data pipelines across the Data organization who can Collaborate with product managers, architects, and other engineers to drive the success of the Core Data platform and contribute to developing and documenting both internal and external standards and best practices for pipeline configurations, naming conventions, partitioning strategies, and more. Ensure high operational efficiency and quality of the Core Data platform datasets to ensure our solutions meet SLAs and project reliability and accuracy to all our stakeholders (Engineering, Data Science, Operations, and Analytics teams.

JOB RESPONSIBILITIES:

Contribute to maintaining, updating, and expanding existing Core Data platform data pipelines in Scala and Python / Spark while maintaining strict uptime SLAs.
Extend functionality of current Core Data platform offerings, including metadata parsing, extending the Metastore API, and building new integrations with APIs both internal and external to the Data organization
Implement Ingestion of new batch and streaming data pipelines using Scala, Databricks, and Airflow
Implementation shared libraries in Scala and Python that abstract complex business logic to allow consistent functionality across all data pipelines across the Data organization
Tech stack includes Airflow, Spark, Databricks, Delta Lake, Snowflake, Scala, Python
Implement the Lakehouse architecture, working with customers, partners, and stakeholders to shift towards a Lakehouse centric data platform.

JOB REQUIREMENTS:

5+ years of data engineering experience developing large data pipelines
Deep Understanding of AWS or other cloud providers as well as infrastructure as code
Experience in Data Modeling techniques and Data Warehousing standard methodologies and practices
Strong fundamental Scala and Python programming skills.
Strong SQL skills and ability to create queries to analyze complex datasets
Hands-on production environment experience with distributed processing systems such as Spark and Airflow for creating and maintaining data pipelines
Some scripting language experience-Preferred.

ABOUT AKRAYA
Akraya is an award-winning IT staffing firm consistently recognized for our commitment to excellence and a thriving work environment. Most recently, we were recognized Inc's Best Workplaces 2024 and Silicon Valley's Best Places to Work by the San Francisco Business Journal (2024) and Glassdoor's Best Places to Work (2023 & 2022)!

Industry Leaders in IT Staffing
As staffing solutions providers for Fortune 100 companies, Akraya's industry recognitions solidify our leadership position in the IT staffing space. We don't just connect you with great jobs, we connect you with a workplace that inspires!

Join Akraya Today!
Let us lead you to your dream career and experience the Akraya difference. Browse our open positions and join our team!

Apply Now:

Apply Later

Share This Job:

Apply Online Apply with LinkedIn Apply with Facebook Apply with X

Share This Job:

Related Jobs:

About New York, NY

Discover exciting job opportunities in the vibrant area around New York, New York! With its iconic skyline, diverse cultural scene, and endless career growth prospects, the Big Apple and its surroundings promise an unparalleled work-life experience. Nestled between Broadway theaters, world-class art galleries like The Metropolitan Museum of Art, and globally renowned cuisine spots in Little Italy and Chinatown, finding your dream job in New York is your gateway to thriving in a city that never sleeps. Embrace the energy of Times Square, catch a game with the Yankees at Yankee Stadium, or unwind in the serenity of Central Park. Explore our job listings today and unlock your potential in the heart of opportunity.