Sr DataOps Engineer

As a Senior DataOps Engineer, you will help operate, scale, and optimize the data platforms that power critical analytics, engineering, and business operations. You will work extensively with Databricks, AWS, data pipelines, databases, and infrastructure automation to ensure platform reliability, security, performance, and cost efficiency. Collaborating with cross-functional teams, you will strengthen observability, streamline operational workflows, and lead the resolution of complex production incidents. You will also drive infrastructure-as-code practices, improve access governance, and build automation that reduces repetitive work. This role is ideal for a hands-on engineer who combines strong technical expertise with a proactive approach to troubleshooting and continuous improvement. You will also have opportunities to explore AI-powered tools that make operations smarter, faster, and more effective.

• Administer and support Databricks workspaces, compute configurations, cluster policies, jobs, workflows, libraries, service principals, permissions, and platform integrations.

• Support Unity Catalog and implement secure data access governance, catalog and schema permissions, and data-sharing controls.

• Monitor and troubleshoot Spark jobs, Delta Lake workloads, Delta Live Tables, and data pipelines, addressing failures, performance bottlenecks, recovery needs, and data-quality issues.

• Maintain secure, reliable integrations between Databricks and AWS services, including Amazon S3, IAM, VPC connectivity, networking, and external data sources.

• Support AWS infrastructure for data platforms, including EC2, VPC, IAM, security groups, patching, networking, and operational access.

• Maintain production database environments across MariaDB/MySQL and MongoDB, including availability, monitoring, connectivity, backups, performance troubleshooting, and recovery.

• Build and maintain observability solutions, including dashboards, logs, metrics, and alerts across data platforms, pipelines, databases, infrastructure, and integrations.

• Participate in on-call rotations and lead production incident investigation, mitigation, recovery, stakeholder communication, and root-cause analysis.

• Develop and maintain infrastructure-as-code and automation solutions using Terraform, Terraform Enterprise or equivalent tools, Git, GitHub Actions, CI/CD pipelines, and scripting languages.

• Automate recurring operational tasks such as infrastructure provisioning, access configuration, monitoring deployment, recovery procedures, and platform changes.

• Manage security controls, secrets, certificates, TLS configurations, service accounts, and IAM roles across platform services.

• Improve operational readiness through runbooks, recovery procedures, architecture diagrams, technical standards, escalation paths, and documentation.

• Collaborate with Data Engineering, Analytics, Security, Cloud Infrastructure, Database, and application teams to improve capacity planning, platform performance, cost visibility, and resource utilization.

• Identify practical applications for AI-assisted tools and automation to enhance troubleshooting, incident response, documentation, and operational efficiency.

• Strong experience supporting production data platforms, cloud infrastructure, or DataOps environments.

• Hands-on experience administering Databricks, including workspaces, jobs, workflows, compute resources, cluster policies, and access controls.

• Experience supporting Apache Spark, Delta Lake, and Delta Live Tables workloads.

• Strong knowledge of AWS services, including EC2, VPC, IAM, S3, security groups, CloudWatch, and cloud networking.

• Experience with infrastructure as code using Terraform, Terraform Enterprise, or comparable tools.

• Production experience supporting MariaDB/MySQL and MongoDB environments.

• Strong understanding of monitoring, observability, alerting, incident management, and production support practices.

• Experience with CI/CD, Git, GitHub Actions, and automation using Bash, Python, PowerShell, or similar scripting languages.

• Proven troubleshooting skills across platform performance, connectivity, permissions, certificates, TLS, and service integrations.

• Experience with root-cause analysis, post-incident reviews, recovery procedures, and on-call operations.

• Strong understanding of security and access management, including IAM, role-based access control (RBAC), secrets management, certificates, and secure connectivity.

• Excellent communication and collaboration skills, with the ability to coordinate effectively across engineering, analytics, security, infrastructure, database, and application teams.

• A proactive, hands-on mindset focused on identifying root causes, resolving complex technical challenges, improving reliability, and reducing operational overhead.

• Curiosity about AI and a willingness to experiment with AI-assisted tools, learn new capabilities, and apply sound technical judgment to practical use cases.

• Experience with Unity Catalog, Databricks SQL, Databricks CLI, REST APIs, and Databricks asset or deployment bundles.

• Kubernetes administration and production troubleshooting experience.

• Familiarity with observability platforms such as Prometheus, Grafana, Datadog, or similar tools.

• Experience with Rootly or comparable incident-management platforms.

• Familiarity with MongoDB Atlas, Ops Manager, or equivalent MongoDB administration tools.

• Experience with MariaDB/MySQL replication, schema migrations, indexing, backup validation, and restore procedures.

• Knowledge of capacity planning, cloud cost optimization, and workload performance tuning.

• Experience implementing or improving data governance, compliance controls, and audit processes.

• Experience using AI-assisted tools to improve operational workflows, troubleshooting, documentation, or automation.