Clarityinnovates7d ago

Principal Data Engineer

Columbia · Columbialead

EngineeringData EngineeringData Engineer

3 views0 saves0 applied

Apply Now

Quick Summary

Overview

Clarity Innovations is a trusted national security partner,

Technical Tools

EngineeringData EngineeringData Engineer

Clarity Innovations is a trusted national security partner, dedicated to safeguarding our nation’s interests and delivering innovative solutions that empower the Intelligence Community (IC) and Department of Defense (DoD) to transform data into actionable intelligence, ensuring mission success in an evolving world.

Our mission-first software and data engineering platform modernizes data operations, utilizing advanced workflows, CI/CD, and secure DevSecOps practices. We focus on challenges in Information Warfare, Cyber Operations, Operational Security, and Data Structuring, enabling end-to-end solutions that drive operational impact.

We are committed to delivering cutting-edge tools and capabilities that address the most complex national security challenges, empowering our partners to stay ahead of emerging threats and ensuring the success of their critical missions. At Clarity, we are people-focused and set on being a destination employer for top talent, offering an environment where innovation thrives, careers grow, and individuals are valued. Join us as we continue to lead innovation and tackle the most pressing challenges in national security.

The Data Transport Engineer II is responsible for the design, implementation, and validation of secure data movement within the Unified Data Model (UDM) team. This role focuses on integration between UDM, Apache NiFi, and external stakeholder systems in constrained and classified environments where data flow correctness, schema integrity, and transport reliability are mission-critical. The position is delivery and compliance-oriented rather than availability or on-call focused.

Responsibilities

~1 min read

→Active TS/SCI w/ CI Poly security clearance (required prior to start).
→Ability to operate in classified environments and comply with all applicable government and program security protocols.

Hands-on experience with Apache NiFi for data flow design, pipeline configuration, and processor development.
Working knowledge of data format conversion and schema validation (JSON, XML, Avro, Protobuf, or similar).
Familiarity with data modeling concepts and structured data interchange patterns.
Experience with REST APIs and integration patterns for data transport between distributed systems.
Proficiency in at least one scripting or programming language (Python, Groovy, Java) for pipeline customization.
Understanding of data transport security requirements including encryption in transit and access control.

5-10 years of experience in data engineering, integration engineering, or a directly related role.
Experience working in government, defense, or other regulated/classified environments preferred.
Demonstrated ability to troubleshoot integration failures and resolve data consistency issues in production pipelines.

Requirements

~1 min read

Experience with Kubernetes-based deployment environments and containerized data pipelines.
Familiarity with document/data ingestion workflows.
Experience with CI/CD pipelines and DevSecOps practices in a government contracting context.
Familiarity with Agile/Scrum delivery and working in cross-functional product teams.

We are an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status.