DevOps Engineer II - DataBricks

CommerceIQ is building the AI platform that runs commerce for the world's largest brands. We are not selling AI demos. We are shipping AI agents for content, media, and sales into the workflows of the Fortune 100 every week.

Customers include Coca-Cola, Nestlé, Colgate-Palmolive, Mondelez, Samsung, and Kellogg's. Backed by SoftBank, Insight Partners, and Madrona. Headquartered in Mountain View with teams across the US, India, Canada, and the UK. Pre-IPO.

We are looking for a DevOps Engineer II with strong hands-on experience in Databricks, databases, Kubernetes, and AWS.

Key Responsibilities

• Manage and maintain Databricks infrastructure and production workloads.
• Manage clusters, jobs, compute resources, configurations, permissions, and integrations.
• Monitor Databricks workloads and troubleshoot performance and reliability issues.
• Drive Databricks cost optimization through cluster sizing, autoscaling, job optimization, scheduling, and resource utilization.
• Establish best practices around Databricks governance, security, access management, and operational efficiency.

• Strong hands-on experience managing databases in production environments.
• Perform database maintenance, monitoring, troubleshooting, performance tuning, backup/recovery, and capacity management.
• Work with databases such as PostgreSQL, MySQL, Aurora, Redis, or similar technologies.
• Troubleshoot database performance, connectivity, storage, replication, and availability issues.
• Identify opportunities for database cost and performance optimization.

• Hands-on experience with AWS infrastructure and services, including EKS, EC2, RDS/Aurora, S3, IAM, CloudWatch, VPC, ECR, and related services.
• Manage and maintain production AWS infrastructure with a strong focus on reliability, security, scalability, and cost optimization.
• Troubleshoot complex infrastructure and networking issues.

• Strong hands-on experience with Kubernetes, preferably AWS EKS.
• Experience with deployments, troubleshooting, scaling, networking, workloads, ingress, storage, and cluster maintenance.
• Experience with Helm, ArgoCD, Terraform, GitHub Actions/Bitbucket Pipelines, or similar DevOps tools.
• Build and maintain CI/CD pipelines and automate repetitive operational tasks.
• Implement Infrastructure as Code and follow best practices for infrastructure provisioning and management.

• Experience with monitoring and observability tools such as Prometheus, Grafana, CloudWatch, OpenTelemetry, or similar.
• Build dashboards, alerts, and operational metrics for production systems.
• Participate in incident management, root-cause analysis, and reliability improvements.
• Ensure production systems meet availability, performance, and operational requirements.

• 4–7+ years of DevOps / Cloud / Infrastructure experience
• Strong hands-on experience with Databricks
• Strong experience with AWS
• Strong hands-on experience with Kubernetes / EKS
• Strong database administration and troubleshooting experience
• Terraform / Infrastructure as Code
• CI/CD and Git-based workflows
• Helm and GitOps tools such as ArgoCD
• Monitoring and observability
• Strong Linux and networking fundamentals
• Strong troubleshooting and problem-solving skills
• Experience working with production environments at scale