Senior Data Pipeline Engineer

💰Salary: $200-240k + Bonus

• →You've onboarded to Clever's AWS environment, Terraform repos, Airflow/MWAA setup, and Databricks workspace, and can independently triage and resolve pipeline alerts.
• →You've established a working relationship with the data engineering team and understand how each data product depends on the infrastructure you now own.
• →You've completed a documented audit of the current infrastructure state, identifying any immediate risks or technical debt.

• Pipeline reliability has measurably improved: fewer unresolved alerts, faster mean-time-to-resolution, and no extended outages caused by infrastructure gaps.
• You've delivered at least one new data ingestion pipeline for an operational data source supporting speed-to-match and/or algorithmic content.
• You've proposed and begun executing on an infrastructure roadmap that addresses cost optimization, security posture, and reduced single-points-of-failure.

• Clever's data platform is more resilient, better documented, and less dependent on any single person than when you started.
• Operational data ingestion is running reliably for multiple new sources, enabling the data team to support Bench Score, Market Score, and speed-to-match with fresh, trustworthy data.
• You've made meaningful progress on at least one strategic infrastructure initiative (e.g., orchestration consolidation, Databricks optimization, or compliance automation).

• You've spent 8+ years in data engineering or data platform roles, with at least 2-3 years focused on infrastructure ownership (not just writing transformations).
• You're comfortable being the primary infrastructure owner on a small team. You see this as an opportunity to shape strategy, not a burden.
• You’ve administered Databricks on AWS and have familiarity with the suite of Databricks tools, including Unity Catalog.
• You have deep, hands-on AWS experience — not just using services, but configuring VPCs, managing security groups, setting up peering, and debugging networking issues.
• You've managed Terraform in production and understand state management, module structure, and the discipline required to keep IaC reliable.
• You've administered Airflow (ideally MWAA) and have opinions about DAG design, monitoring, and failure handling.
• You're pragmatic about tooling: you'll maintain what works, improve what's fragile, and propose changes only when the ROI is clear.
• You communicate clearly with non-infrastructure stakeholders and can explain tradeoffs without jargon.

• 8+ years of experience in data engineering or data platform roles
• Databricks administration and lakehouse architecture
• Apache Airflow administration and DAG development (MWAA preferred)
• AWS (EC2, RDS, VPC, S3, MWAA): our AWS environment is central to all data operations and requires deep, hands-on expertise
• Terraform (infrastructure as code): managing multiple repos across RDS, CDC, and orchestration infrastructure
• Python (PySpark, general scripting)
• PostgreSQL / RDS management, including replica promotion, security group configuration, VPC peering
• VPN setup and management (Pritunl on EC2)
• SSO configuration
• Excellent communication skills, able to influence across teams and levels
• Experience with SOC-2 compliance infrastructure requirements is a plus
• Real estate, fintech, or marketplace industry experience a plus

Clever Real Estate provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.