2d ago
New

Databricks Architect

United StatesUnited States·Chicagomid
ArchitectConstruction & Real Estate
3 views0 saves0 applied

Quick Summary

Overview

Confiz is seeking a Databricks Architect in Chicago, IL to lead the data-layer design and delivery for enterprise AI use cases, serving as the technical authority on the Databricks platform.

Technical Tools
ArchitectConstruction & Real Estate

Confiz is seeking a Databricks Architect in Chicago, IL to lead the data-layer design and delivery for enterprise AI use cases, serving as the technical authority on the Databricks platform. This role owns the end-to-end architecture of the data foundation that powers AI/BI solutions from data discovery and readiness through governed, secure delivery, ensuring every solution aligns with enterprise standards. The Architect explicitly owns security design for the platform, including the Unity Catalog governance model, fine-grained access controls, and PII handling for natural-language query experiences.

Work Model: Hybrid, 2-3 days onsite in Chicago, IL

Responsibilities

~1 min read
  • Lead the architecture, design, and delivery of the data layer supporting prioritized AI use cases, from source ingestion through curated, consumption-ready data products.
  • Translate AI/BI use case requirements into concrete data models, pipelines, and platform patterns on Databricks (Delta Lake, medallion architecture, streaming and batch ingestion).
  • Define and enforce data quality, lineage, and observability standards so downstream AI models and natural-language query experiences operate on trusted data.
  • Establish reusable reference architectures and design patterns that accelerate delivery across successive use cases.
  • Co-facilitate live data discovery sessions with the Lead AI/BI Engineer, working directly with business stakeholders and data owners to identify, profile, and validate candidate data sources.
  • Assess data readiness for each use case — completeness, quality, granularity, latency, and access — and produce clear readiness findings with remediation plans.
  • Drive resolution of data readiness issues, coordinating with source-system owners, data engineering teams, and governance stakeholders to close gaps on schedule.
  • Maintain a data discovery playbook and artifacts (source inventories, profiling results, gap logs) that make each engagement faster than the last.
  • Architect the Databricks platform solution — workspace topology, compute strategy, storage layout, networking, and CI/CD — aligned to enterprise standards.
  • Define environment strategy (dev/test/prod), promotion paths, and infrastructure-as-code practices for repeatable, auditable deployments.
  • Advise on cost optimization, performance tuning, and capacity planning across clusters, SQL warehouses, and serverless compute.
  • Stay current on the Databricks roadmap (Unity Catalog, Genie/AI-BI, Delta Sharing, serverless) and guide adoption decisions.
  • Own the platform security design end to end, including identity integration, workspace access, secrets management, and network isolation.
  • Design and implement the Unity Catalog governance model: catalog/schema structure, ownership model, access policies, tagging, and lineage.
  • Define and implement row-level and column-level security, dynamic data masking, and attribute-based access controls to enforce least-privilege data access.
  • Own PII handling for natural-language query experiences — classification, masking/tokenization strategies, and guardrails that prevent sensitive data exposure through conversational and generative interfaces.
  • Partner with enterprise security, privacy, and compliance teams to ensure designs satisfy regulatory and audit requirements, and document controls for review.
  • Serve as the primary data-architecture counterpart to the Lead AI/BI Engineer, aligning the data layer with semantic models and AI/BI experiences.
  • Provide technical direction and design review for data engineers delivering pipelines and data products.
  • Communicate architecture decisions, trade-offs, and risks clearly to both technical teams and business stakeholders.

Requirements

~1 min read
  • 8+ years in data architecture or data engineering, with 3+ years architecting solutions on Databricks in production environments.
  • Deep expertise with the Databricks Lakehouse platform: Delta Lake, Unity Catalog, Databricks SQL, workflows/jobs, and medallion architectures.
  • Demonstrated ownership of data security and governance design, including Unity Catalog governance models, row/column-level security, and data masking.
  • Hands-on experience with PII classification and protection strategies, ideally in the context of AI, natural-language query, or conversational analytics workloads.
  • Strong data modeling skills (dimensional, data vault, or domain-driven data product design) and proficiency in SQL and Python (PySpark).
  • Experience with at least one major cloud platform (Azure, AWS, or GCP), including networking, identity (e.g., Entra ID/IAM), and storage services.
  • Proven ability to facilitate discovery workshops and translate ambiguous business needs into actionable data designs.
  • Experience delivering within enterprise architecture and governance frameworks and standards.
  • Databricks certifications (e.g., Data Engineer Professional, Platform Architect accreditation).
  • Experience supporting GenAI/LLM or AI-BI (e.g., Databricks Genie) use cases, including retrieval patterns and semantic layer design.
  • Familiarity with data privacy regulations (GDPR, CCPA, HIPAA as applicable) and audit/compliance processes.
  • Experience with infrastructure-as-code (Terraform) and CI/CD for Databricks (Asset Bundles, GitHub Actions/Azure DevOps).
  • Background in consulting or multi-stakeholder delivery environments.
  • AI use case data layers delivered on schedule with documented, standards-aligned architectures.
  • Data readiness issues identified early and resolved without derailing delivery timelines.
  • Zero PII exposure incidents through natural-language query or AI interfaces; security designs passing enterprise security and audit review.
  • Unity Catalog governance model adopted as the enterprise pattern, with measurable reuse across use cases.

We have a global team of amazing individuals working on highly innovative enterprise projects & products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast growth fintech, and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM) and ISO 14001:2015 (EMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.

People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves. 

To know more about Confiz Limited, visit https://www.linkedin.com/company/confiz/

Location & Eligibility

Where is the job
Chicago, United States
On-site at the office
Who can apply
US

About this job

Who can apply
United States
Posted
2 days ago
Employer
has 46 open jobs on Jobera in 3 countries.

Listing Details

Posted
November 17, 2026
First seen
November 17, 2026
Last seen
November 19, 2026

Posting Health

Days active
1
Repost count
0
Trust Level
56%
Scored at
November 19, 2026

Signal breakdown

freshnesssource trustcontent trustemployer trust
Newsletter

Stay ahead of the market

Get the latest job openings, salary trends, and hiring insights delivered to your inbox every week.

A
B
C
D
Join 12,000+ marketers

No spam. Unsubscribe at any time.

Databricks Architect