Senior Data Engineer

Ad Hoc is a technology company that empowers organizations to deliver scalable, impactful digital services. Using modern, agile methods, our team creates products that meet people’s needs and transform their experience of government.

Our collaborations have shaped some of the defining moments in public-sector service delivery. We’ve helped build products that connect Veterans to tailored services, help millions access affordable health care, and support important programs like Head Start. As we work with agencies to deliver critical services, we’re also changing how the government approaches technology.

Our culture, communications, and tools are built for remote work, enabling us to bring together top talent nationwide. At Ad Hoc, remote life empowers our teams to design work environments that fit their lives and that foster flexibility and collaboration to achieve positive outcomes for our customers.

Ad Hoc values acceptance, accountability, and humility. We aren’t heroes. We learn from our mistakes and improve the process for the next time. We build small, inclusive teams to collaborate closely with our partners to solve the right problems and deliver software that works.

The Federal Civilian business unit supports many customers spanning the federal, commercial, and nonprofit space. Our customers include NASA, the General Services Administration, Office of Personnel Management, the Library of Congress, Health & Human Services, and the FDIC. We partner with these agencies to build new capabilities, deliver products, establish data as a strategic asset for informed decision-making, modernize legacy systems, and build the digital service infrastructure necessary to scale their mission impact.

• Evaluate and recommend multiple technical approaches to solve complex data engineering and architecture challenges.
• Design, develop, maintain, and optimize scalable data pipelines supporting both batch and real-time processing.
• Design and implement robust ETL/ELT workflows that transform raw data into reliable, consumable datasets.
• Build and maintain scalable data models, data warehouses, and cloud-native data architectures.
• Develop solutions for structured, semi-structured, and unstructured data sources.
• Generate data architecture recommendations and successfully implement approved solutions.
• Ensure data quality, integrity, governance, lineage, security, and observability across data platforms.
• Optimize database performance, query execution, storage strategies, and overall system scalability.
• Diagnose and resolve production issues while implementing long-term improvements to increase system reliability and performance.
• Collaborate with cross-functional teams to translate business requirements into scalable technical solutions.
• Present technical designs, architecture diagrams, and implementation strategies to clients, stakeholders, partners, and engineering teams.
• Champion data engineering best practices, coding standards, automation, and operational excellence.
• Mentor junior engineers through technical guidance, code reviews, design discussions, and knowledge sharing.
• Lead small projects or serve as the technical lead for data engineering initiatives when needed.
• Effectively communicate technical challenges, risks, and progress with engineering teams, leadership, clients, and stakeholders.
• Participate in technical interviews and contribute to hiring decisions.

• Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, or a related technical discipline with 7+ years of professional experience. Relevant years of experience may be substituted for formal education.
• 7+ years of experience designing, building, and maintaining enterprise-scale data platforms and data pipelines.
• Strong proficiency in SQL with experience developing, optimizing, and troubleshooting complex queries and large datasets.
• Professional experience developing software and data solutions using Python, Java, Scala, or a comparable programming language.
• Experience designing, developing, and maintaining ETL/ELT pipelines for batch and real-time data processing.
• Experience working with cloud-based data platforms and services within AWS, Microsoft Azure, or Google Cloud Platform.
• Experience with modern cloud data platforms such as Snowflake, Databricks, Amazon Redshift, Google BigQuery, or Azure Synapse Analytics.
• Experience with relational database technologies such as PostgreSQL, SQL Server, Oracle, or MySQL, along with familiarity with NoSQL database solutions.
• Experience with distributed data processing frameworks such as Apache Spark or equivalent big data technologies.
• Strong understanding of data modeling, dimensional modeling, schema design, data warehousing, and database optimization.
• Experience implementing data quality, validation, governance, metadata management, and data lineage best practices.
• Experience working with structured, semi-structured, and unstructured data from multiple sources.
• Experience with version control systems such as Git and CI/CD practices supporting data engineering workflows.
• Understanding of data security, privacy, encryption, and access control principles.
• Experience monitoring, troubleshooting, and optimizing production data systems for scalability, availability, and performance.
• Experience working within Agile software development environments and collaborating across cross-functional engineering teams.
• Strong analytical, troubleshooting, and problem-solving skills with the ability to make sound technical decisions.
• Excellent written and verbal communication skills with the ability to explain technical concepts to both technical and non-technical audiences.
• Demonstrated ability to mentor junior engineers through code reviews, technical guidance, and knowledge sharing.
• Ability to obtain and maintain a U.S. Public Trust clearance.

• Experience supporting U.S. Federal Government programs or other highly regulated environments.
• Experience with orchestration platforms such as Apache Airflow, Prefect, Azure Data Factory, or AWS Step Functions.
• Experience with streaming technologies such as Apache Kafka, Amazon Kinesis, or Azure Event Hubs.
• Experience using dbt or other modern data transformation frameworks.
• Experience with Infrastructure as Code tools such as Terraform or AWS CloudFormation.
• Experience working with Docker and Kubernetes in cloud-native environments.
• Experience building analytics and business intelligence solutions using Tableau, Power BI, Looker, or similar visualization tools.
• Existing Public Trust or higher security clearance.
• Experience serving as a technical lead or mentoring engineers on enterprise-scale data initiatives.