Senior Data Engineer

M Science is a data-driven research and analytics firm, uncovering new insights for leading financial institutions and corporations. We come to work every day to create actionable insights from alternative data. M Science is revolutionizing research, discovering new data sets and pioneering methodologies to provide actionable intelligence. Our research teams have decades of experience working with massive amounts of unstructured data in near real-time to discern critical insights that help clients make smarter, more informed decisions. We combine the best of finance, data and technology to create a truly unique value proposition for both financial services firms and major corporations.

We're looking for a Senior Data Engineer who is passionate about building robust, scalable data infrastructure and pipelines. You'll work with petabytes of data daily using cutting-edge technologies including Databricks, SQL, Python, Airflow, and modern AI development tools. This role requires someone who can architect solutions, mentor junior engineers, and drive technical excellence across our data platform.

• →Design, build, and maintain scalable ETL/ELT pipelines processing petabytes of data;
• →Architect data solutions using modern cloud technologies (AWS, Databricks, Snowflake);
• →Implement real-time and batch processing workflows using Apache Airflow;
• →Optimize data storage and retrieval patterns for analytical workloads;
• →Implement comprehensive data quality frameworks and monitoring systems;
• →Establish data governance standards and best practices;
• →Build automated testing and validation for data pipelines;
• →Develop data lineage and metadata management solutions;
• →Lead technical design reviews and architecture decisions;
• →Mentor junior data engineers and promote best practices;
• →Drive adoption of modern development practices including CI/CD for data pipelines;
• →Leverage AI coding assistants (Cursor, Claude, GitHub Copilot) to accelerate development;
• →Implement AI-driven data quality checks and anomaly detection;
• →Stay current with emerging AI tools and their applications in data engineering.

• 3-8 years of data engineering experience;
• Expert-level SQL - Complex queries, performance optimization, window functions;
• Advanced Python - Object-oriented programming, async/await, data processing libraries;
• Apache Airflow - DAG development, custom operators, workflow orchestration;
• Databricks - Spark optimization, Delta Lake, Unity Catalog governance;
• Cloud Platforms - AWS (S3, EMR, Lambda, Glue) or equivalent;
• Data Modeling - Dimensional modeling, data vault, modern data stack patterns;
• Experience mentoring junior engineers and leading technical initiatives;
• Strong communication skills for cross-functional collaboration;
• Experience with Agile methodologies and project management;
• Ability to translate business requirements into technical solutions.

• Experience in financial services, fintech, or data intelligence companies;
• Background in consumer retail, digital media, or technology verticals;
• Advanced degree in Computer Science, Statistics, Mathematics, or related field.