Sr Platform Engineer

Datavations is a leading New York-based data and AI software specializing in the $2.3 trillion dollar building materials industry. Datavations gives building materials and home improvement manufacturers real-time, store-level visibility into pricing, assortment, and inventory across major retailers, including Home Depot, Lowe’s, and Menards. Manufacturers use this data to make sharper decisions about pricing, distribution, and how they show up on the shelf. 

Product taxonomy and attributes form the foundation of every insight Datavations delivers to our customers. We are looking for a senior engineer to own this domain end-to-end—driving both the data systems that organize the market and the applications that enable our teams and customers to interact with them. This is a hands-on role with significant architectural latitude for someone who wants to take full ownership of a system that already powers the business.

• The attribute extraction engine — LLM-based extraction at production scale, its configuration model, its quality gates, and its cost profile.
• The transformation layer for this domain — the dbt models that turn extracted values into published attributes: standardization, and the override model that lets human judgment reliably beat the machine. Built on our ClickHouse warehouse alongside the data platform team.
• Orchestration and reliability for these pipelines — the Dagster jobs, sensors, and schedules that run taxonomy and attribute processing, including run monitoring, retries, alerting, and recovery tooling. Reliability is not a separate team here; for this domain, it is this seat.
• The applications — internal and customer-facing. You own the screens people actually use, not only the services behind them: the internal platform our teams run taxonomy and attributes from, and the customer-facing views of this data as they move onto it. React/Next.js front end, backend services and APIs, application architecture, deployment and CI/CD.
• The write path — every change validated, logged, and visible before it ships, with approval where it matters.
• Data quality and observability — automated testing on the data itself, plus statistical detection of what goes wrong quietly: outliers, drift, a retailer that stopped updating, a value that flips between runs.
• Engineering standards for the domain — documentation, tests, runbooks, and review culture as the system and the team around it scale.

• A meaningful part of this platform is AI, and not bolted on the side: an LLM extraction engine already running at scale, a rules engine that produces a confidence score per item, and an in-app assistant that answers questions in natural language and proposes changes for review. We are looking for someone who has built this kind of system properly, not someone who has called a completion endpoint.
• Evaluation before assertion — golden sets, regression suites that run when a prompt changes, and a defensible answer to “did that make it better?” Quality you can measure, not quality you claim.
• Prompts and rules as versioned data — stored, diffable, testable, auditable, with a clear record of what changed and what it did. Not constants in a file that move on deploy.
•  Tool-calling and MCP — exposing internal systems to agents safely: scoped, read-first, audited. It is how our teams will query and operate the platform, and how we already work internally.
• Cost and latency as design constraints — model tiering, caching, batching, circuit breakers. Knowing what a run costs before it runs, and why a small model first is usually the right answer.

• A modernization already in motion, not a greenfield and not a rescue: a production system that has grown fast with the business, a detailed technical map of it, an engaged leadership team, a modern monorepo, and a new internal platform in its first release — with full air cover to do it properly. You own the taxonomy and attribute systems end to end; the warehouse, ingestion, and shared infrastructure are owned alongside you by the data platform team.

• At least 5 years of experience in Cloud Infrastructure, Site Reliability Engineering (SRE), or Platform Engineering.
• Strong expertise with Terraform and using infrastructure as code (IaC).
• Experience with CI/CD pipelines (GitHub Actions, Jenkins, GitLab CI/CD, ArgoCD).
• Strong expertise in architecting solutions using AWS services.
• Experience in Python, SQL, dbt, and modern orchestration tools (e.g., Dagster, Airflow, Prefect).
• Proficiency with a columnar warehouse (e.g., ClickHouse, Snowflake, BigQuery).
• Experience with full-stack development (TypeScript, Next.js or similar).
• Experience building production data pipelines and maintaining observability/reliability.
• Experience with AI/LLM-based systems, including evaluation and versioning of prompts.
• Experience with data visualization tools (e.g., Tableau, Power BI) and project management tools (e.g., Jira).

• You think like a product person as well as an engineer — you form opinions about what the people using this should be able to do, and would rather understand the problem behind a request than build the ticket as written.
• You build with AI daily — both in what you ship and in how you work. AI-assisted development is a habit, not a novelty.
• You write things down by default, and can take ownership of an existing codebase without needing its original author in the room.
• Bonus: retail, e-commerce, or point-of-sale data.

• You want a greenfield with no history, or “it ran green” is your definition of done.

Preferred Location: NY, Chicago, Dallas, Cincinnati, Minneapolis

• Customer Obsession: We are integrated in the industry with a customer-first obsession.
• Proactive Ownership: We have agency for our actions and embody an entrepreneurial mindset.
• Bias for Action: We maintain momentum with a can-do attitude; we value progress over perfection.
• Eager to Grow: We stay curious and hungry to learn, viewing every failure with humility as an opportunity to grow.
• Foster Collaboration: We work across boundaries with quiet competence, welcoming diverse perspectives and offering help unselfishly.