Staff Software Engineer

Design, develop, and maintain scalable batch and streaming data pipelines on Google Cloud Platform. Build enterprise data processing solutions using services such as BigQuery, Dataflow, Dataproc, Cloud Storage, Pub/Sub, and Cloud Composer. Develop robust ETL/ELT frameworks for ingesting data from databases, APIs, files, messaging platforms, and other enterprise data sources. Design and optimize data models and datasets in BigQuery for analytics and downstream consumption. Develop data transformation and processing components using Python, SQL, Apache Beam, and/or Spark. Build workflow orchestration and scheduling solutions using Cloud Composer / Apache Airflow. Implement real-time and event-driven data processing using Pub/Sub and Dataflow. Ensure pipelines meet requirements for performance, scalability, reliability, data quality, security, and cost optimization. Implement appropriate logging, monitoring, alerting, retry, reconciliation, and error-handling mechanisms for production pipelines. Troubleshoot production issues, perform root-cause analysis, and implement permanent fixes. Participate in production deployments, maintenance activities, and operational support of data platforms. Work with DevOps and Cloud Engineering teams to implement CI/CD and Infrastructure-as-Code practices for data engineering workloads. Implement GCP security best practices including IAM, service accounts, secrets management, encryption, and least-privilege access. Collaborate with data architects, analysts, application teams, ML engineers, and business stakeholders to translate requirements into scalable technical solutions. Participate in technical design reviews, code reviews, performance optimization, and engineering best practices. Mentor junior engineers and contribute to reusable data engineering frameworks, standards, and patterns. GCP Data Engineering Google BigQuery Google Cloud Storage (GCS) Google Cloud Dataflow Google Cloud Dataproc Google Cloud Pub/Sub Cloud Composer / Apache Airflow Experience designing and implementing both batch and streaming data pipelines is expected. Experience with Apache Beam and/or Apache Spark. Experience working with structured, semi-structured, and large-scale datasets. IAM and Service Accounts Secret Manager Cloud Logging and Cloud Monitoring Compute Engine / Cloud Run / GKE exposure VPC and basic networking concepts Cloud Scheduler / Cloud Functions where applicable GCP security and access-control best practices Cost monitoring and optimization Experience implementing CI/CD for data engineering applications. Experience with containerization using Docker is desirable. The candidate should have a strong understanding of: Data lakes and cloud data warehouses ETL vs. ELT architectures Batch vs. streaming architectures Data ingestion and integration patterns Data modeling and dimensional modeling Data quality and reconciliation Metadata and schema management Data security and governance Pipeline observability and monitoring Performance and cost optimization High availability and fault-tolerant pipeline design