Elastic Data Engineer

Keystone Solutions is seeking an experienced Data Engineer for a consultancy mission at a client site. The client is undergoing an ambitious digital and organizational transformation, focusing on modern processes, stronger operational support, and more data-driven services. You will play a crucial role in establishing and managing an event-driven data architecture utilizing Apache Kafka as a buffering and processing layer, and Elasticsearch as a data lake solution for both structured and unstructured data.

• →Design, implement, and document necessary components for the data architecture.
• →Set up performant Kafka and Elastic stacks, ensuring schema management, topic management, and indexing strategies.
• →Ensure data consistency, scalability, and observability throughout the entire data ingestion to consumption process.
• →Make data available via a data service layer (REST APIs, microservices) and handle error management and optimization of the entire pipeline.
• →Work with Docker, Kubernetes, and Openshift for the deployment of components.
• →Maintain observability, error handling, and optimization through CI/CD pipelines (GitLab CI) and Python scripting.

• Proven experience with event-driven systems and real-time data integration.
• In-depth knowledge of Apache Kafka (Streams, Connect, Schema Registry, Avro, ksqlDB) and Elasticsearch (including Kibana, ingest pipelines, index lifecycle management).
• Methodical and precise work ethic to ensure data consistency and scalability.
• Ability to work autonomously, propose solutions, and adapt quickly to changes.
• Strong documentation skills for analyses, pipelines, schemas, configurations, and technical choices.
• Excellent written and verbal communication skills for effective collaboration with specialists and user representatives.
• Ability to prioritize within a complex environment with multiple data pipelines and processes.
• Client-oriented and solution-focused attitude, capable of working under stress and creatively addressing challenges.

Dutch
Level Active knowledge
English
Level Active knowledge
French
Level Active knowledge

• Apache Kafka (Streams, Connect, Schema Registry, Avro, ksqlDB) - Level: Confirmed - Most recent: Any time
• CI/CD tools (GitLab CI) - Level: Confirmed - Most recent: Any time
• Data Service Layer (REST API’s, microservices) - Level: Confirmed - Most recent: Any time
• Docker, Kubernetes en Openshift - Level: Confirmed - Most recent: Any time
• Elasticsearch (Kibana, ingest pipelines, index lifecycle management) - Level: Confirmed - Most recent: Any time
• Indexingstrategieën en data-consistentie (Elasticsearch) - Level: Confirmed - Most recent: Any time
• Observability, foutafhandeling en optimalisatie van data-pipelines - Level: Confirmed - Most recent: Any time
• PostgreSQL - Level: Confirmed - Most recent: Any time
• Python - Level: Confirmed - Most recent: Any time
• Schema management en topicbeheer (Kafka) - Level: Confirmed - Most recent: Any time