Data Engineer

We are seeking a data professional to use data to solve business problems and build the infrastructure needed to improve processes. In this role, you will streamline data science workflows to enhance our products, lifecycle operations, and retention models. You will collaborate closely with data science and business intelligence teams to design data models and pipelines for research, reporting, and machine learning. Additionally, you will champion best practices and promote continuous learning across the organization.

A Data Engineer is responsible for designing, building, and maintaining large-scale data systems, architectures, and pipelines. Key responsibilities include:

• →

  Data Architecture: Designing and implementing data warehouses, lakes, and pipelines to store and process large datasets.

• →

  Data Ingestion: Developing data ingestion pipelines to collect data from various sources, such as APIs, files, and databases.

• →

  Data Processing: Building data processing workflows using tools like Apache Beam, Spark, or Flink to transform, aggregate, and analyze data.

• →

  Data Storage: Managing data storage solutions like relational databases, NoSQL databases, or cloud-based storage systems.

• →

  Data Quality: Ensuring data quality, integrity, and security by implementing data validation, data cleansing, and data governance processes.

• →

  Collation: Working with data scientists, analysts, and other stakeholders to understand data requirements and deliver data products.

• →

  Troubleshooting: Identifying and resolving data pipeline issues, optimizing data processing workflows, and ensuring data system reliability.

• Bachelor's degree: In Computer Science, Information Technology, Industrial Engineering, or a related field.
• Academic achievement more than 3.3 CGPA

• Programming languages: Java, Python, Scala
• Data processing frameworks: Apache Spark, Apache Beam, Apache Flink
• Data storage solutions: Relational databases, NoSQL databases, cloud-based storage systems
• Data ingestion tools: Apache Kafka, Apache NiFi, AWS Kinesis

• Communication: Collaborating with stakeholders to understand data requirements
• Problem-solving: Identifying and resolving data pipeline issues
• Time management: Prioritizing tasks and managing multiple projects
• Continuous learning: Staying up-to-date with new technologies and trends in data engineering.