Senior Data Engineer

United StatesRemoteFull-timesenior

Data EngineerData

1 views0 saves0 applied

Apply Now

Quick Summary

Key Responsibilities

Design, develop, and maintain scalable, high-throughput data pipelines and platform systems supporting blockchain data across more than 30 networks.

Requirements Summary

Design, develop, and maintain scalable, high-throughput data pipe

Technical Tools

Data EngineerData

The Senior Data Engineer will design, build, and operate the core data infrastructure powering advanced blockchain intelligence and analytics products.
You will take ownership of end-to-end data systems, from architecture and development through production deployment, monitoring, and continuous optimization.
Working across more than 30 blockchain networks, you will build scalable, high-throughput pipelines capable of processing billions of records and tens of terabytes of data.
The role combines backend engineering, distributed data processing, and analytical database optimization, with a strong focus on performance, reliability, and cost efficiency.
You will collaborate closely with product, analytics, and investigations teams to translate complex data requirements into robust technical solutions.
Joining a small, senior engineering team, you will have significant influence over architecture decisions, technical standards, and the evolution of a sophisticated data platform.

You will own the design, development, and operation of production-grade data infrastructure, ensuring that data systems remain scalable, reliable, efficient, and performant. Your responsibilities will include:

Design, develop, and maintain scalable, high-throughput data pipelines and platform systems supporting blockchain data across more than 30 networks.
Build backend data services and data loaders using Go and Python.
Design and evolve analytical data models and workloads in ClickHouse, optimizing schemas and queries to achieve sub-second performance.
Develop incremental, idempotent data pipelines using Databricks to process billions of records and tens of terabytes of information.
Continuously improve platform scalability, reliability, data freshness, processing performance, and infrastructure cost efficiency.
Investigate complex performance bottlenecks and reliability issues, implementing effective solutions and following them through to resolution.
Collaborate with product, analytics, and investigations teams to understand data requirements and translate them into scalable platform architectures.
Contribute to technical direction through architecture discussions, design documentation, technical reviews, and code reviews.
Mentor fellow engineers and promote strong development, operational, and engineering practices.
Implement monitoring and instrumentation for production systems, maintain operational runbooks, and conduct blameless post-incident reviews.
Manage data backfills, pipeline failures, and processing consistency across distributed environments.
Apply production-scale testing, performance profiling, and continuous optimization to maintain high engineering standards.
Leverage AI-assisted development tools while applying sound technical judgment when reviewing and integrating generated code.

Requirements

~2 min read

The ideal candidate is an experienced data engineer with strong backend development capabilities, deep expertise in large-scale data processing, and a hands-on approach to production operations. You should bring:

At least 5 years of experience building and operating production data systems, including ETL and ELT pipelines.
At least 2 years of experience owning systems that other engineers or teams depend on.
Expert-level Python skills and strong SQL proficiency, including experience writing typed, tested code and working with asyncio and concurrency.
Demonstrated experience building incremental data models and processing datasets containing billions of records.
Production experience with Go, particularly developing and operating high-throughput backend services and data loaders.
Strong hands-on experience with ClickHouse, including MergeTree schema design, sort-key optimization, materialized views, bulk loading, and query performance tuning.
Production experience with Databricks, including PySpark, Delta Lake, and Unity Catalog, as well as designing incremental and idempotent tables and optimizing compute costs.
Practical experience managing large-scale data pipelines involving billions of records and tens of terabytes.
Strong understanding of partitioning, clustering, file sizing, and their impact on infrastructure costs and query latency.
Production experience with Kafka, Airflow, and Kubernetes, including delivery semantics, idempotent writes, backfills, and failure recovery.
Experience running containerized data workloads using Docker and Helm.
Familiarity with Google Cloud Platform, which is preferred, or AWS.
Strong operational and communication skills, including system instrumentation, technical documentation, runbook creation, and post-incident reviews.
Ability to produce clear architecture and design documents that support effective collaboration across distributed teams and time zones.

Additional experience that would be advantageous includes:

Familiarity with blockchain data structures and models, including EVM, UTXO, and non-EVM networks.
Experience retrieving blockchain data from nodes through JSON-RPC or gRPC.
Knowledge of graph databases, distributed key-value stores, entity resolution, or large-scale clustering.
Experience with infrastructure-as-code, GitOps, and observability tools such as Terraform, Helm, ArgoCD, OpenTelemetry, Prometheus, and Grafana.
Experience designing load tests and building platforms supporting multiple data sources, datasets, or product teams.
Familiarity with AI-assisted software development.
Previous experience in fintech, regulatory compliance, or Anti-Money Laundering (AML).