IC3 - DataOps Engineer

Purpose of this position
As a DataOps Engineer, you will be responsible for designing, implementing, and optimizing data processes and workflows for our fintech multi-product B2C and B2B platform on AWS cloud. Your primary goal is to ensure efficient, secure, and scalable data management, facilitating collaboration between development, operations, and data analysis teams. You will work closely with other Data and Platform Engineering teams to ensure data solutions meet business requirements and adhere to the highest performance and security standards.

Main Responsibilities
1. Design and Implementation of Data Pipelines: Design, develop, and implement robust and scalable data pipelines to facilitate the ingestion, processing, and analysis of large volumes of data.
2. Automation and Orchestration: Develop and maintain scripts and tools to automate data processes, ensuring efficiency and reducing errors in data operations.
3. Monitoring and Optimization of Data Processes: Continuously monitor the performance of data pipelines, proactively identifying and resolving issues to ensure data availability and reliability.
4. Incident and Problem Management: Manage and resolve incidents related to data processes, minimizing the impact on business operations.
5. Security and Compliance: Implement and maintain security measures to protect data, ensuring compliance with industry best practices and regulatory requirements.
6. Cross-Functional Collaboration: Work closely with development, operations, and data analysis teams to understand their needs and provide effective technical solutions.
7. Technical Documentation: Create and maintain detailed documentation on data architectures, system configurations, operational procedures, and best practices.
8. Resource Optimization: Monitor and optimize the use of resources in data operations to identify opportunities for improvement and cost reduction.
9. Continuous Improvement: Propose and lead continuous improvement initiatives to optimize data processes and operational workflows.
10. Training and Mentorship: Provide guidance and training to team members on new tools, technologies, and best practices in DataOps.
11. Evaluation of New Technologies: Evaluate and recommend new technologies and tools that can enhance data operations and internal services.
12. Development of Policies and Procedures: Develop policies and procedures for data management, ensuring consistency and reproducibility of data processes.

13. Promote Autonomous Teams: Encourage and support the development of autonomous teams by fostering a culture of self-management, accountability, and proactive problem-solving.
14. Serve as a Spin Culture Ambassador: Foster and maintain a positive, inclusive, and dynamic work environment that aligns with the company's values and culture.

Required Knowledge and Experience
1. Bachelor's degree in computer science, Systems Engineering, Information Technology, Mathematics, or a related field.
2. 2-4 years of experience in DataOps, data engineering, or a similar role, with experience managing large volumes of data and implementing data pipelines.
3. Experience with DataOps tools and technologies such as Apache Airflow, Luigi, Jenkins, Kafka.
4. Advanced knowledge of SQL and NoSQL database administration.
5. Proficiency in scripting and automation (Python, Bash, SQL).
6. Experience with cloud platforms (AWS, Azure, Google Cloud), specifically AWS.
7. Familiarity with infrastructure such as code tools (Terraform, CloudFormation).
8. Experience using version control systems (Git).
9. Knowledge of Agile methodologies and ITIL principles.
10. Excellent communication skills and ability to work in a team.
11. Strong analytical and problem-solving skills.
12. Ability to manage multiple tasks and priorities effectively.
13. High organizational skills and attention to detail.
14. Mentorship and professional development skills.