Engenheiro de Dados SR

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Engenheiro de Dados SR based in Brazil. Join a technology-driven environment focused on innovation, digital transformation, and high-impact data solutions. As a Senior Data Engineer, you will design, develop, and optimize scalable data architectures that support business intelligence, analytics, and data-driven decision-making. You will work with modern platforms, including Cloudera, Databricks, and AWS, to process and integrate large volumes of information. Your work will help strengthen data quality, security, reliability, and performance across the organization. Collaborating with architecture, analytics, data science, and business teams, you will contribute to the evolution and modernization of the data platform. This role is ideal for an experienced professional who enjoys solving complex technical challenges and building robust, efficient data solutions. Accountabilities - Design, develop, and maintain scalable, efficient, and reliable data pipelines to support large-scale data processing. - Implement data ingestion, transformation, and integration workflows using both batch and streaming architectures. - Build and optimize data engineering solutions with Cloudera, Databricks, and Amazon Web Services (AWS). - Develop data processing and transformation routines using Python, SQL, PySpark, and Apache Spark. - Create, maintain, and improve ETL/ELT processes to ensure efficient and consistent data movement. - Design and work with Data Lake, Lakehouse, and distributed processing architectures. - Integrate data from multiple sources while ensuring data quality, consistency, integrity, and reliability. - Monitor pipeline execution, processing jobs, and queries, identifying bottlenecks and implementing performance improvements. - Apply best practices for data security, governance, cataloging, lineage, and observability. - Contribute to the evolution of data architecture, platform modernization, and continuous process improvement.

- Collaborate with architecture, analytics, data science, and business teams to align data solutions with technical and business requirements. - Support troubleshooting and maintain reliable, maintainable, and production-ready data workflows. Requirements - Proven experience working as a Senior Data Engineer, with strong expertise in data solution development, architecture, and optimization. - Practical experience with Cloudera and Databricks, particularly in processing and transforming large volumes of data. - Knowledge of AWS and its core services for data engineering and cloud-based data processing. - Strong command of SQL and hands-on experience with Python and PySpark. - Experience with Apache Spark and distributed data processing technologies. - Solid understanding of ETL/ELT processes and relevant tools. - Experience designing and working with Data Lake and Lakehouse architectures. - Knowledge of data modeling and integrating information from multiple data sources. - Experience with Git and code version control practices. - Ability to monitor data pipelines, troubleshoot issues, and optimize processing performance. Additional qualifications that would be an advantage: - Experience with AWS data services such as Amazon S3, AWS Glue, Amazon Redshift, AWS Lambda, and Amazon Athena. - Familiarity with cloud-based data processing, workflow orchestration, and pipeline management services. - Experience with Delta Lake and Hadoop ecosystem technologies, including Hive, HDFS, and Impala. - Knowledge of Apache Kafka and real-time data processing architectures. - Experience with Apache Airflow or similar orchestration tools. - Familiarity with CI/CD practices, infrastructure as code, and automation.