Data Engineer – Databricks, PySpark, Delta Lake

Job Title: Senior Data Engineer – Databricks, PySpark, Delta Lake Location: USA – Remote Duration: 3 Months (Contract) – Expected extension possible but not guaranteed Years of Experience Required: 10+ Years Visa Status: US Citizen / Green Card / Any Independent Visa. Education Background: Bachelor's or master's degree in computer science, Engineering, Data Science, or a related field Domain / Industry: IT (Healthcare) Interview Details: Virtual – Technical + Panel discussion NOTE (Mandatory for Submission) Attached Driver's License, Visa / H1B copy, Travel history, and I-140 / I-940 approval (if applicable). Candidate Disqualifiers No candidates from JNTU University Extension / Full-Time Possibility Contract extension beyond the initial 3 months is possible based on performance and business need Potential for long-term engagement; full-time conversion is not guaranteed Top Required Skills Databricks (Workspace, Jobs, Clusters, Performance Tuning) PySpark (Advanced DataFrame & Spark SQL development) Delta Lake (ACID transactions, MERGE, Schema Evolution) Data Engineering & ETL Pipeline Design Azure Cloud (ADLS Gen2, ADF, Synapse, Key Vault) Data Modeling (Fact/Dimension, SCD Type 1 & 2) Agile / Scrum methodology Production-grade coding, CI/CD, Git version control Job Description (JD) We are seeking a Senior Data Engineer with strong hands-on expertise in Databricks, PySpark, and Delta Lake to design, build, and maintain scalable, enterprise-level data pipelines in an Azure cloud environment. The ideal candidate will have deep experience delivering production-quality data solutions and working in fast-paced agile teams. Key Responsibilities: Design, develop, and optimize end-to-end data pipelines using Databricks and PySpark Implement robust ETL/ELT frameworks for batch and streaming data processing Create and manage Delta Lake tables, ensuring ACID compliance, schema enforcement, and schema evolution Apply strong data modeling techniques to support analytics, reporting, and downstream consumption Optimize Spark jobs for performance, scalability, and cost efficiency Handle data quality, validation, error handling, and pipeline monitoring Collaborate with cross-functional teams including product owners, architects, QA, and DevOps Participate in agile ceremonies (stand-ups, sprint planning, retrospectives) Deliver production-ready code with proper logging, testing, documentation, and version control Troubleshoot and resolve data pipeline, performance, and scalability issues Preferred Qualifications: Databricks Certification (Associate or Professional) Experience with Structured Streaming Prior experience supporting healthcare or retail data platforms Remote Skills: Agile Programming Methodologies, Application Programming Interface (API), Automation, Automation Engineering, Compensation and Benefits, Computer Science, Continuous Deployment/Delivery, Continuous Integration, Data Quality, DevOps, GitHub, Healthcare, Integration Testing, Microsoft C# (C Sharp), Microsoft Windows Azure, Object Oriented Programming (OOP), Pharmacy, Quality Assurance, SQL (Structured Query Language), Selenium, Software Design for Test (SDET), Test Automation, Test Data, Test Plan/Schedule, Test Suite, Testing, United States Citizen, User Interface/Experience (UI/UX) About the Company: FL Tech Solutions LLC Apply tot his job Apply tot his job Apply To this Job