Senior Data Engineer

Remote - North America - LATAMRemotesenior

Data EngineerData

0 views0 saves0 applied

Apply Now

Quick Summary

Key Responsibilities

Design, build, and evolve the core data platform infrastructure e.g., distributed query engines, orchestration, warehousing, cataloging, and more. Own our lakehouse infrastructure as code,

Requirements Summary

5+ years of experience in Data Engineering, including 2+ years building and operating scalable, low-latency data platforms handling > 100M events/day.

Technical Tools

Data EngineerData

We're a dynamic team of 400+ globally distributed members who thrive working from our favorite places around the world, with teammates spanning the USA, Canada, Japan, Hungary, Nigeria, Brazil, the UK, and beyond!

We're searching for passionate individuals eager to contribute to Alpaca's rapid growth. If you align with our core values—Stay Curious, Have Empathy, and Be Accountable—and are ready to make a significant impact, we encourage you to apply.

Responsibilities

~1 min read

→Design, build, and evolve the core data platform infrastructure e.g., distributed query engines, orchestration, warehousing, cataloging, and more.
→Own our lakehouse infrastructure as code, managing deployments through Terraform and Ansible on Kubernetes.
→Build and maintain low-latency streaming and CDC ingestion pipelines, as well as batch ingestion paths landing in Iceberg.
→Develop and scale our BI landscape so downstream teams and agents get performant, self-serve access to lakehouse data.
→Enforce platform reliability best practices, including monitoring and alerting, on-call rotations, incident response, maintenance windows, runbooks, and SLAs.
→Partner with DevOps, Analytics Engineering, and other stakeholders to close infrastructure gaps and support new data requirements.

5+ years of experience in Data Engineering, including 2+ years building and operating scalable, low-latency data platforms handling > 100M events/day.
Strong hands-on experience running data infrastructure on Kubernetes, with cloud-native tooling like Docker and Helm.
Production experience with IaC: Terraform, Ansible, and ArgoCD (or equivalents).
Deep knowledge of distributed systems (storage, transactions, and query processing) with hands-on experience operating open-source query engines like Trino or Presto.
Strong experience with object storage and open table formats, specifically Apache Iceberg.
Experience with streaming and CDC systems: Kafka, Redpanda, and Debezium.
Hands-on experience with orchestration frameworks (Airflow) and ELT tools (Airbyte).
Strong working knowledge of Python and SQL for building pipelines and platform tooling.
Experience with Google Cloud Platform and its data services (GCS, Cloud Build, Cloud SQL, Dataproc, etc); or related experience with other cloud services.
Ability to thrive in a fast-paced startup environment and adapt infrastructure to rapidly changing needs.