Senior Data Engineer ID86297
AgileEngine
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for an experienced Senior Data Engineer to help modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers. This role also uses AI tools like Claude and GitHub Copilot to speed up development.
WHAT YOU WILL DO
- Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
- Model and maintain a medallion (bronze/silver/gold) architecture serving analytics, reporting, and machine learning consumers.
- Migrate legacy ETL and data warehouse workloads onto the Lakehouse with validated data parity and minimal business disruption.
- Use Claude or Github Copilot as a development accelerator, generating code scaffolding, writing and reviewing tests, creating documentation and prototyping solutions.
- Write clean, well-tested Python and SQL; maintain high standards through code review and documentation.
- Optimize Spark jobs and Delta tables for performance and cost, including partitioning, clustering, caching, and cluster sizing.
- Implement data quality, lineage, and governance controls using Unity Catalog and automated validation checks.
- Debug, troubleshoot, and resolve pipeline failures, data defects, and production incidents.
- Participate in Agile or product-centric delivery practices including sprint planning and retrospectives.
- Collaborate with DevOps, platform, and analytics engineers on observability, security, and compliance best practices.
MUST HAVES
- 4+ years of professional experience in data engineering , featuring direct expertise with Apache Spark and cloud-based data architectures.
- Strong hands-on experience building data pipelines with Databricks, Apache Spark (PySpark), and Delta Lake .
- Advanced SQL and Python , with strong data modeling skills across dimensional and Lakehouse patterns.
- Experience with streaming ingestion using Structured Streaming, Auto Loader, Kafka, or Event Hubs .
- Experience with workflow orchestration ( Databricks Workflows, Airflow, or Azure Data Factory ).
- Experience with legacy platform migrations, ETL modernization, or managing data hygiene when porting old systems.
- Strong problem-solving, collaboration, and communication skills, including mentoring junior engineers and explaining data concepts to non-technical stakeholders.
- Familiarity with Unity Catalog , data governance, access control, and PII handling.
- Experience with dbt or an equivalent transformation framework.
- Familiarity with secure coding standards and industry security best practices.
- Experience delivering production data platforms at scale.
- Upper-intermediate English level.
NICE TO HAVES
- Experience with Infrastructure as Code (IaC) using Terraform and CI/CD using Azure Devops.
- Experience working with relational databases (specifically PostgreSQL) and data persistence concepts.
- Familiarity with logging and monitoring tools (e.g., Dynatrace, CloudWatch, Databricks system tables).
- Experience working in Agile or team-based development environments preferred.
PERKS AND BENEFITS
- Growth without limits : build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
- Competitive compensation : get recognition that reflects your skills and impact, with regular performance and compensation reviews
- Flexibility : work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
- Meaningful, modern projects : build impactful products using modern technologies alongside global teams and leading brands
- Collaborative culture : join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
- Well-being & support : access local well-being programs and people-focused support tailored to your location
- Sap Abap Consultant Senior - Detalhes da Vaga. ● Proficient in technical documentation and adherence to best practices while supporting modernization initiatives. ●#J-18808-Ljbffr. Requisitos para participar do processo. ● . O que oferecemos.
Caderno Nacional
Sorocaba, SPhá 5 horas atrás R$ 14,2 por hora
...Excel e nas ferramentas do pacote Office. conhecimento prévio em rotinas e processos de estoque. Diferencial:Vivência com o sistema Senior X. especialização ou curso técnico em Logística, Gestão de Estoque ou áreas correlatas. Como enviar meu currículo? Whats app (15)...SANTHA ASSESSORIA EM RH LTDA
Sorocaba, SPhá 2 dias atrás- A pessoa receberá treinamento e amplo conhecimento e desenvolvimento de habilidades em vários departamentos:Logística, Financeiro, Fiscal, Vendas e Compras, tudo que envolve na gestão de um estoque desde sua aquisição, entrada e saída. Atividades:Dar suporte ao atendimento...
Fastenal brasil import exportação e distribuição
Sorocaba, SPhá 2 dias atrás - Controle de produtos e especificações. Atendimento ao cliente on-site. Abastecimento de Suprimentos na linha fabril do cliente via vending machines. Cotações, revisões de preços e negociações com os fornecedores/clientes. Curso Técnico ou Superior nas áreas de Administração...
Fastenal brasil import exportação e distribuição
Sorocaba, SPhá 2 dias atrás - VISÃO GERAL Trabalhando como Almoxarife e reportando ao Gerente Geral no período integral, você desempenhará uma variedade de atividades que apoiarão e ajudarão no processamento da Fastenal na região de Curitiba. RESPONSABILIDADES Recebimento de Materiais, Conferência ...
Fastenal brasil import exportação e distribuição
Sorocaba, SPhá 2 dias atrás - ...atento, comunicativo, proativo e dinâmico; Facilidade para trabalhar com alto volume de demandas; Desejável conhecimento nos sistemas Senior e HK; Residir em Sorocaba/SP; Disponibilidade para início imediato. Salário R$ 2.518,47 Benefícios Vale-refeição de R$ 26,03 por...
Grupo souza lima
Sorocaba, SPhá 2 dias atrás - ...A UniFacens está em busca de um(a) Analista Sênior de TI (ERP RM) para atuar em um momento importante de evolução tecnológica e fortalecimento dos processos internos.Essa posição tem papel estratégico na sustentação e evolução do ERP TOTVS RM, sendo responsável por garantir...
Facens
Sorocaba, SPhá 2 dias atrás - ...e à individualidade é a base das nossas relações. VENHA SER CEDRENSE! Estamos em busca de um(a) Desenvolvedor(a) BackEnd .Net e C# SÊNIOR para atuar no desenvolvimento de um projeto de cadastro para corretora de investimentos, com esteira completa da experiência do cliente...
Cedro Finances
Votorantim, SPhá 2 dias atrás - Desafie o mercado com a IHARA ao seu lado!Somos IHARA porque assim escolhemos ser! Sonhamos grande. Acreditamos que a capacidade humana é ilimitada e nos desafiamos sempre.Venha com a IHARA, contribuir para que o progresso e a competitividade brasileira, continuem a alimentar...
Iharabras
Sorocaba, SPhá 2 dias atrás - ...grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE We are looking for a Middle Data Engineer to help modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming...
- ...ABOUT THE ROLE We are looking for a Full Stack Software Engineer to build and maintain web applications for a tax research and workpapers... ...in React and back-end services in Node.js, C#, and Python, with data managed across PostgreSQL and DynamoDB. This is a short-term...
- ...grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE We are looking for a Senior Quality Engineer to drive quality throughout the software development lifecycle by partnering with Product Managers, Software Engineers, and...
- ...love to meet you! ABOUT THE ROLE We are looking for a Senior Data Scientist to drive rigorous statistical execution, multi-... ...flaws before deployment. - Collaborate daily with internal engineering cross-functions via Slack and Google Meet to rapidly resolve...
R$ 3.000
...Experiência na área de saúde Conhecimento sobre audiometria Experiência com CRM; Conhecimento de técnicas de vendas consultivas. Condições da vaga Jornada: segunda a sexta das 8h30 às 17h30 / sábado das 8h às 12h Carga horária: 44 Senioridade: Analista...- ...Qualificações Ensino superior completo em Administração, Ciências Contábeis, Economia, Finanças ou áreas correlatas. Experiência sólida/sênior em contas a receber, crédito e cobrança, faturamento ou tesouraria. Inglês fluente obrigatório, escrito e verbal. Excel...
- ...Qualificações Ensino superior completo em Administração, Ciências Contábeis, Economia, Finanças ou áreas correlatas. Experiência sólida/sênior em contas a pagar, tesouraria ou rotinas financeiras. Inglês fluente obrigatório, com capacidade de comunicação oral e escrita...
- ...ou stakeholders internacionais em inglês. Qualificações Ensino superior completo em Ciências Contábeis. Experiência sólida/sênior em contabilidade corporativa, fechamento contábil e conciliações. Estar cursando ou possuir pós-graduação concluída em...
- ...ABOUT THE ROLE We are looking for a ServiceNow Discovery Engineer to join an enterprise-scale Discovery Engineering squad that executes... ...including MID Servers, Patterns/Probes, and CMDB/CSDM data quality. WHAT YOU WILL DO - Continuously assess the health...
- ...of accuracy and consistency. MUST HAVES - Strong attention to detail and accuracy. - Comfortable performing repetitive data-entry and back-office tasks. - Ability to work efficiently across multiple browser-based applications and websites. - Good general...
- ...program, or industry experience strongly preferred; familiarity with accounting ledger concepts, real-time decisioning systems, or rules engines is a plus. - Upper-intermediate English level. NICE TO HAVES - Familiarity with UX research and best practices. -...
- ...ABOUT THE ROLE We are looking for a Senior BI Product Manager to act as the subject... ...-to-Cash processes, connecting financial data to real business decisions. This person... ...with Technical Delivery Managers (TDMs) and engineering teams who architect the technical...
- ...ABOUT THE ROLE We are looking for a Senior Business Analyst to connect business... ...techniques such as BPMN, swimlane diagrams, and data flow diagrams. - Facilitate... ...ML initiatives. - Exposure to prompt engineering concepts or evaluating generative AI tools...
- ...behind them. WHAT YOU WILL DO - Act as the Squad Lead, driving the team to execute on deliverables while maintaining high engineering standards. - Spend a significant portion of your time designing and implementing robust backend services using Java and...
- ...de liderança da empresa normalmente define a direção estratégica e os resultados que deseja alcançar. Com base nisso, o(a) Gerente Sênior de Projetos monitora o progresso de vários projetos sob sua responsabilidade para garantir que sigam essa direção e superem as metas...
Emerson
Sorocaba, SPhá 2 dias atrás - ...world sustainably . The Opportunity We are looking for an engineer or a professional with a related background to join our... ...Inventor, Fusion 360, SolidWorks, etc.) – Not mandatory . Data management and analysis (Excel, Power BI, Visual Basic) – Not...
- ...working experience. ~ Technical or Bachelor degree in Renewable Energies, Architecture, Industrial Design, Electrical or Civil Engineering or related field. ~ You interest in learning and evolving to other technical roles. ~ You will use our ERP SAGE software for...
- ...pipelines, and ensure a positive recruitment experience. This position offers the opportunity to influence hiring outcomes through data-driven decision-making, strong stakeholder partnerships, and a commitment to operational excellence, innovation, integrity, collaboration...
Emerson
Sorocaba, SPhá 2 dias atrás - ...grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE We are looking for a Senior Full Stack Engineer to build robust server-side solutions for a SaaS platform operating at scale, working across Java, React, SQL, Redis,...
- ...Rails . - Strong command of API design patterns and best practices in Rails . - Solid understanding of protocols and JSON data structures . - Experience with relational databases, e.g. PostgreSQL . - Familiarity with Git and collaborative...
- ...to grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE We are looking for a Senior Data Engineer to own ingestion, pipeline orchestration, and the Single Source of Truth architecture for new product modules. You will...
Deseja receber mais vagas?
Assine e receba vagas semelhantes a Senior Data Engineer ID86297. Seja o primeiro a se candidatar!
