Data Engineer - ETL

 

We are seeking a Data Engineer to simplify and modernize our on-premises SQL Server data environment and support the transition to a corporate-managed Databricks platform. This role will redesign legacy ETL processes, build reliable ODS and DWH pipelines, and embed operational controls, data quality and governance into the engineering lifecycle.

 

 

Key Functions/Duties of Position:

 

·         Assess, document and rationalize existing SQL Server ETL pipelines, databases and data silos.

·         Refactor redundant or fragile SQL processes into standardized, maintainable ETL/ELT pipelines.

·         Design and develop ingestion, transformation and reconciliation pipelines for governed ODS and DWH environments.

·         Implement logging, monitoring, error handling, auditability, lineage and repeatable deployment practices.

·         Develop Databricks notebooks and pipelines using SQL and, as appropriate, Python or PySpark.

·         Partner with corporate platform teams to align solutions with enterprise architecture, security and engineering standards.

·         Support source-to-target mapping, testing, production support and migration validation.

• Perform other duties and/or projects as assigned

 

Bachelor’s degree in Computer Science, Information Systems, Data Analytics, Data Engineering, or a related field

 

 

·                     Strong SQL and Microsoft SQL Server development experience.

·                     Hands-on experience designing, building and supporting ETL/ELT pipelines.

·                     Knowledge of relational databases, data warehousing and incremental data processing.

·                     Experience troubleshooting complex data issues and reconciling source-to-target results.

·                     Experience with source control and disciplined software development practices.

·                     Strong analytical, documentation and communication skills.

 

Preferred Qualifications

·                     Databricks, Python, PySpark or Apache Spark experience.

·                     Experience migrating legacy ETL workloads to a cloud data platform.

·                     Dimensional modeling and data quality automation experience.

·                     Experience in a SOX-controlled or regulated environment.

·                     Property and casualty insurance industry knowledge, including policy, billing, claims or underwriting data, is a strong plus.

·                     Experience working with Guidewire applications or data, including Guidewire ClaimCenter (GWCC), Guidewire    BillingCenter (GWBilling) or related policy systems.

·                     Experience engineering feature-ready datasets or pipelines for predictive analytics and machine learning.

·                     Familiarity with model lifecycle concepts, including experimentation, deployment, monitoring and reproducibility.

·                     Exposure to generative AI, large language models (LLMs), retrieval-augmented generation (RAG), vector search or AI-assisted data engineering.

Only solicited resumes/candidate profiles that are submitted by a Berkley Preferred Agency Partner that have been requested by Berkley Internal Recruiting to engage in a particular search will be subject to the payment of Agency fees as defined in the formal agency agreement. Any unsolicited resumes/candidate profiles are not subject to payment of the agency fees. The actual salary for this position will be determined by a number of factors, including the scope, complexity and location of the role; the skills, education, training, credentials and experience of the candidate; and other conditions of employment.

Sponsorship not Offered for this Role