Lead Data Engineer Job at WorkHQ, Los Angeles, CA

QkhOY0YxanpSMS9PZmVzSzBxSHpvUXI2NVE9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Remote work, Shift work,

Similar Jobs

IDR, Inc.

Workday Integration Analyst (Grand Prairie) Job at IDR, Inc.

 ...IDR is seeking a Workday Integration Developer to join one of our top clients for an opportunity in Grand Prairie, TX. This role is part of a critical transition project where the organization is migrating from Lawson to Workday, supporting large-scale system integration... 

Trucker Job USA

Dedicated Regional Trucker-Great Benefits-Short Haul Pay-Fort Collins, CO Job at Trucker Job USA

 ...family. This opportunity is truly exceptional, and we would like you to consider exploring this job opening. Benefits: Great Benefits Pay Salary: .60 .70 DOE cpm solo / .70 .80 cpm team Average earnings per week: $1,500 + solo / $1,800 per driver for... 

Cargill

Licensed Electrician Job at Cargill

 ...of the National Electric Code Mechanical aptitude to work from all types of blueprints, maintenance manuals, and drawings Apprentice, Journeyman or above license with the state of Nebraska or equivalent A 2-year degree in electromechanical, electrical system,... 

Confidential

Chief Executive Officer (CEO), Fundraising & Social Impact Job at Confidential

 ...Chief Executive Officer (CEO), Fundraising & Social Impact About the Company Non-profit organization fighting for girls' rights Industry Non-Profit Organization Management Type Non Profit Founded 2018 Employees 11-50 About the Role The Company... 

The Planet Group

Sr. Workday Benefits Analyst (Houston) Job at The Planet Group

 ...Title: Sr. Workday Benefits Analyst Location: Onsite in Houston, TX Type: Direct Hire Were seeking an experienced Senior Workday Benefits Analyst to support an upcoming Workday HCM implementation for a global organization. This role will focus on the configuration...