Databricks Data Engineer

Horizon Industries Limited (Horizon) is a dynamic IT and Management Consulting firm based in the Washington, DC area. Founded in 1996, Horizon Industries Limited (Horizon) has evolved into a company of highly qualified professionals that provides full-cycle IT Consultant and Management support in both the private and public sectors.

Horizon’s culture is based on our core values, and we PRIDE ourselves on a diverse, employee, and family-centric approach.

We are seeking a Databricks Data Engineer for a full-time position supporting our various clients

We are committed to your growth and success in the IT software development industry. We are looking for a Databricks Data Engineer ready to assume leadership responsibilities in a supportive team environment focused on your success as a function of client success.

• →Build end-to-end implementation of multiple ETL/ELT pipelines, demonstrating efficient data transformation and ingest patterns to move raw data from data producers to an enterprise data ecosystem, with a focus on performance and reliability
• →Assess and understand the ETL jobs, workflows, BI tools, and reports
• →Address technical inquiries concerning customization, integration, enterprise architecture and general feature / functionality of data products
• →Experience in crafting database / data warehouse solutions in the cloud (Preferably AWS, Azure, Alternatively GCP)
• →Key must have skill sets – Python, SQL, Databricks, AWS Data Services
• →Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores
• →Experience manipulating, processing, and extracting value from large, disconnected datasets
• →Experience manipulating structured and unstructured data for analysis
• →Experience with data modeling tools and processes
• →Experience aggregating and transforming data from multiple datasets to create data products
• →Support an Agile software development lifecycle

• Ability to hold a position of public trust with the US government.
• S. in Computer Science or equivalent
• 4+ years' experience in data engineering and big data. At least 2 years' of professional services experience interacting directly with clients.
  • Big data tools: Hadoop, Spark, Kafka, etc.
  • Relational SQL and NoSQL databases and experience working with relational databases
  • AWS cloud services: EC2, S3, RDS, Glue, Step Functions, Lamda, EMR, DynamoDB, DocumentDB, Redshift, Aurora, Athena
  • Data Platforms: Databricks
  • Data streaming systems: Batch,Kafka, Storm, Spark-Streaming, etc.
  • Languages: Python, R, Scala, Go
• Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.
• Extensive knowledge of data warehousing concepts and hands-on experience deploying pipelines using Databricks **a must
• Data modeling and database design skills and knowledge of version control
• Excellent verbal and written communication skills
• Experience architecting scalable and fault-tolerant data solutions across Azure, AWS, and Databricks
• Databricks Data Engineer Professional certification a plus.
• Preference for candidates with Databricks Professional certifications