Data Engineer

Data Engineer - Detalhes da Vaga. ● The primary responsibility of this role will be to design, develop, and maintain data pipelines for threat intelligence ingestion, validation, and export automation flows. ● Responsibilities:Design, develop, and maintain data pipelines for ingesting threat intelligence data from various sources into our data ecosystem. ●Implement data validation processes to ensure data accuracy, completeness, and consistency. ●Collaborate with threat analysts to understand data requirements and design appropriate solutions. ●Develop automation scripts and workflows for data export processes to external systems or partners. ●Optimize and enhance existing data pipelines for improved performance and scalability. ●Monitor data pipelines and troubleshoot issues as they arise, ensuring continuous data availability and integrity. ●Document technical specifications, data flows, and procedures for data pipeline maintenance and support. ●Stay updated on emerging technologies and best practices in data engineering and incorporate them into our data ecosystem. ●Provide technical guidance and support to other team members on data engineering best practices and methodologies. ●Requirements:Proven experience as a Data Engineer or similar role, with a focus on data ingest, validation, and export automation. ●Strong proficiency in Python. ●Experience with data pipeline orchestration tools such as Apache Airflow, Apache NiFi, or similar. ●Familiarity with cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform. ● Experience with data validation techniques and tools for ensuring data quality. ●Experience building and deploying images using containerization technologies such as Docker and Kubernetes. ●Excellent problem-solving skills and attention to detail. ●Strong communication and collaboration skills, with the ability to work effectively in a team environment. ●. Requisitos para participar do processo. ● . O que oferecemos.