Senior Data Engineer, Databricks

United States·Denversenior

Data EngineerData

3 views0 saves0 applied

Apply Now

Quick Summary

Key Responsibilities

medallion layering, Delta table design, and the gold layer that feeds Power BI. Set the platform roadmap and defend pattern-level decisions — batch vs. streaming, Delta Live Tables vs. Workflows,

Requirements Summary

Full-time salaried exempt employees are eligible for Colas Group benefits, as outlined in the plan documents. Available benefits include medical, dental, and vision insurance, term life insurance,

Technical Tools

Data EngineerData

Job Description

Senior Data Engineer, Databricks

Company: COLAS USA

Job Location: DENVER, COLORADO, USA

Type of Contract: Regular

Posting Start Date: 9/2/26

Colas IS Support is the information technology arm of the North America business for Colas and is based in Denver, Colorado. Colas ISS strives to make Colas an industry leader through developing custom mobile applications and business intelligence tools, supporting core applications, maintaining our infrastructure, and much more. Through a partnership with the Colas Companies in the USA & Canada Colas IS Support helps empower our employees for success while also providing direct contact with users and other innovators across the organization. To learn more about Colas IS Support visit www.colasiss.com/.

What We Offer

~1 min read

*Estimated target starting compensation; final compensation will be determined based on an assessment of an individual’s qualifications, education, and experience relevant to the role.

Benefits: Full-time salaried exempt employees are eligible for Colas Group benefits, as outlined in the plan documents. Available benefits include medical, dental, and vision insurance, term life insurance, supplemental life insurance, 401(K) with company match, short- and long-term disability. Other benefits include paid holidays, paid sick leave, accrued vacation time off, and paid parental leave.

Other Compensation: This position may be considered for annual bonus compensation.

Responsibilities

~1 min read

Own the technical design of the ERDW.Next lakehouse: medallion layering, Delta table design, and the gold layer that feeds Power BI.
Set the platform roadmap and defend pattern-level decisions — batch vs. streaming, Delta Live Tables vs. Workflows, where transformation logic belongs.
Define enterprise data modeling standards: conformed dimensions, gold-layer star schemas, slowly changing dimensions, grain, and naming.
Partner with the Enterprise Architect on the data governance framework, JD Edwards data strategy, and enterprise integration patterns.

Stay hands-on in the codebase, writing and reviewing production pipelines as the primary way you set technical standards.
Build and optimize pipelines in PySpark, Spark SQL, and Delta Live Tables, orchestrated through Databricks Workflows.
Engineer ingestion from JD Edwards, BMS, CMS, Cority, HCSS, Intelex, and Anaplan using CDC, Auto Loader, and MERGE.
Drive migration of legacy workloads (SQL Server procedures, SSIS, Synapse, ADF, Qlik Replicate) with a defensible parallel-run and cutover approach.

Own technical delivery of the data platform workstream: sequence the build, surface dependencies and risk early, and land your half to plan.
Tune performance and cost: liquid clustering, OPTIMIZE / Z-ORDER, Photon, and right-sized clusters, with clear accountability for consumption.
Engineer reliability: data quality expectations, restartable pipelines, monitoring, and dependable SLAs through month-end close.
Establish and maintain CI/CD: Git-based development, Databricks Asset Bundles, promotion across environments, and automated testing.

Design and implement the Unity Catalog governance model: catalog structure, entitlements, lineage, classification, and audit.
Build semantic models with rich metadata, business definitions, and synonyms, so data is trustworthy for reporting and AI consumption alike.
Capture design rationale in architecture decision records and runbooks, so the platform never depends on one person or partner.

Curate Databricks Genie Agents over the gold layer: datasets, example queries, business semantics, and verified answers.
Work directly with business users and citizen developers across finance, construction, equipment, and safety.
Make certified data the easiest path to use and raise their capability, rather than gatekeeping behind a request queue.
Partner with Power BI developers on the serving layer beneath their semantic models, deciding what belongs where.

Use agentic coding tools as a regular part of your workflow to accelerate pipeline development, legacy code migration, and testing.
Own the correctness of AI-generated output: you review it, verify it, and remain accountable for what reaches production.

Provide senior-level support and change-request delivery on the legacy ERDW estate (T-SQL, SSRS, Synapse, ADF) while it remains in service.
Hold the engineering quality bar across offshore partner work (Datavail, Birlasoft) through design and code reviews and a clear definition of done.
Communicate architecture, risk, and cost trade-offs to IT leadership and business stakeholders in plain language.

Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field; or equivalent practical experience.
5+ years of hands-on experience building production data platforms on Databricks / Apache Spark, backed by 8+ years of overall experience in data engineering, data warehousing, and business intelligence.
Demonstrated ownership of the technical design of at least one enterprise data warehouse or lakehouse platform end to end—from target-state design through production delivery—rather than solely executing a design set by others.
Currently hands-on in code and intending to remain so. This is not a design-only, advisory, or oversight position; candidates seeking to move away from day-to-day engineering will not be a fit.
Proven track record migrating legacy on-premises or first-generation cloud warehouse workloads (SQL Server, SSIS, Synapse, ADF) to a modern lakehouse architecture.
Experience sourcing data from ERP and operational systems (JD Edwards strongly preferred) and delivering finance and operational reporting at enterprise scale.
Experience enabling a self-service analytics community—business users, analysts, or citizen developers—through curated data and guardrails rather than a ticket queue.
Agentic coding tools are part of your regular workflow, and you verify and take ownership of what they produce.
Experience reviewing the work of offshore or partner development teams and holding a consistent quality bar across them.
Preferred certifications: Databricks Certified Data Engineer Professional and / or Databricks Certified Data Architect; Azure DP-203 or AZ-305 a plus.

Platform Design & Data Strategy: Able to define a target-state design and a sequenced migration roadmap, and to defend the trade-offs to both engineers and executives (must-have).
Databricks Platform: Expert across Databricks SQL, Workflows / Jobs, Delta Live Tables, Unity Catalog, Auto Loader, Photon, and cluster / SQL warehouse configuration (must-have).
Spark & Python: Advanced PySpark and Spark SQL, writing modular, tested, reusable code rather than one-off notebooks (must-have).
Delta Lake: Deep proficiency—MERGE / upserts, time travel, OPTIMIZE, Z-ORDER, liquid clustering, partitioning, VACUUM, and schema evolution.
Enterprise & Dimensional Data Modeling: Star schema, conformed dimensions, slowly changing dimensions, and grain design—including a gold layer purpose-built for a Power BI semantic layer (must-have).
SQL: Expert SQL / T-SQL, including the ability to read and re-engineer large, complex legacy stored procedures.
Data Governance: Unity Catalog, access control and entitlements, lineage, data classification, data quality frameworks, and master data / metric definition.
Conversational & Agentic Analytics: Curating natural-language query layers over governed data—dataset selection, business semantics, synonyms, verified answers, and evaluation. Databricks Genie experience a plus.
Self-Service Enablement: Making certified data usable by business users and citizen developers through documentation, guardrails, and coaching rather than gatekeeping.
Azure Data Platform: ADLS Gen2, Azure Data Factory, Azure Synapse, Key Vault, Entra ID, and the networking fundamentals behind secure data services.
Ingestion & Replication: CDC and replication tooling (Qlik Replicate or equivalent), API and file-based ingestion, and Structured Streaming concepts.
DevOps & CI/CD: Git, Azure DevOps or GitHub Actions, Databricks Asset Bundles / Repos, automated testing, and infrastructure as code (Terraform a plus).
Performance & Cost Optimization: Job and query tuning, cluster right-sizing, workload isolation, and demonstrable cloud cost management.
BI Integration: Solid understanding of Power BI semantic models and of Direct Lake / DirectQuery / import trade-offs, plus Microsoft Fabric and OneLake.
Legacy Estate: Working capability across SQL Server, SSIS, SSRS / paginated reporting, Synapse, and ADF to support the existing ERDW platform during transition.
AI-Assisted Delivery: Fluent use of agentic coding assistants in day-to-day engineering, paired with the judgment to know when the output is wrong.
Business Domain: Finance / P&L reporting logic; construction, equipment, or safety domains a plus. JD Edwards source-system knowledge is highly valued.

Communication Skills: Ability to convey complex technical concepts and architectural trade-offs to technical and non-technical stakeholders, including leadership.
Collaboration: Working effectively with cross-functional teams including developers, project managers, business analysts, citizen developers, and offshore delivery partners.
Problem-Solving: Analyzing issues and developing effective solutions while considering constraints and trade-offs.
Technical Influence: Influencing technical decision-making through demonstrated expertise rather than formal authority, and raising the capability of the engineers around you.
Ownership & Bias for Action: Comfortable acting as the go-to technical authority and driving decisions to closure while the platform is still being defined.
Adaptability: Openness to change and emerging technologies.
Critical Thinking: Evaluating options and making informed decisions.
Time Management: Prioritizing effectively across new platform build, legacy support, and governance work.
Empathy: Understanding team and stakeholder perspectives to foster collaboration.

This job description is not intended to be an exhaustive list of duties, responsibilities, or qualifications and may be amended to meet business needs.

Colas Inc. and its subsidiaries are an Equal Opportunity Employer. For details on the specifics of this commitment, please click here to view the EEO document. If you are an individual with a disability and require a reasonable accommodation to:

to meet the requirements of the role in which you are applying
complete any part of the application process
access or use the online application process and need an alternative method for applying

Please contact Colas Inc. at 973-290-9082 or send an email to ColasRecruiter@colasinc.com.