Databricks Data Engineer – Sales Incentive Compensation | New York, NY

Databricks Data Engineer – Sales Incentive Compensation

📍 Location: New York, New York
🏢 Work Arrangement: Hybrid
📄 Contract Length: Six Months
💰 Hourly Rate: Approximately $75 per hour
💼 Employment Type: Contract
🚫 Visa Sponsorship: Not Available
🚚 Relocation Assistance: Not Available
🏦 Industry: Insurance and Financial Services

Build the Data Infrastructure Behind Enterprise Sales Compensation

A leading global organization is seeking an experienced Databricks Data Engineer to design, develop and optimize enterprise data solutions supporting analytics, performance reporting and Sales Incentive Compensation.

This is not purely a pipeline-development position. The successful candidate will combine strong Databricks engineering expertise with an understanding of incentive compensation processes, including quota allocation, attainment calculations, commission processing and payout reporting.

You will work closely with Sales Operations, Finance, Business Intelligence and technology teams to translate complex business rules into scalable, accurate and auditable data solutions.

Why This Opportunity Stands Out

This opportunity will allow you to:

• Own the design and development of enterprise Databricks solutions

• Influence the wider Databricks architecture and technical roadmap

• Build data pipelines supporting business-critical compensation processes

• Work with Delta Lake, Delta Live Tables, Unity Catalog and Databricks Workflows

• Translate complex incentive-plan rules into reliable transformation logic

• Improve automation, integration, data quality and platform performance

• Partner directly with senior Finance, Sales Operations and technology stakeholders

• Deliver solutions that directly influence commission and compensation accuracy

What You’ll Be Doing

Databricks Architecture and Engineering

• Design, develop and implement scalable Databricks-based data solutions

• Build and maintain data pipelines supporting ingestion, transformation, reporting and analytics

• Develop data models, Delta Lake tables, notebooks and reporting datasets

• Take ownership of the overall Databricks architecture

• Ensure new pipelines and enhancements integrate effectively with existing workflows

• Maintain and optimize Databricks workspaces, pipelines and orchestration processes

• Identify opportunities to improve automation, performance and data integration

• Design reliable ETL and ELT processes across multiple enterprise data sources

Sales Incentive Compensation

• Build data pipelines supporting Sales Incentive Compensation processes

• Translate incentive-plan rules into accurate and auditable data transformations

• Support quota allocation and quota-management processes

• Develop logic for attainment tracking and performance calculations

• Support commission calculations and payout processing

• Produce reliable datasets for compensation reporting and reconciliation

• Work closely with Sales Operations and Finance to understand plan structures and business rules

• Ensure calculation logic is transparent, documented and capable of being audited

• Integrate incentive-compensation platforms with enterprise data and analytics solutions

Performance and Data Quality

• Test, debug and optimize Databricks pipelines and notebooks

• Investigate and resolve data-quality, performance and integration issues

• Improve the reliability and efficiency of existing workflows

• Implement appropriate controls and validation processes

• Monitor data pipelines and troubleshoot failures or inconsistencies

• Deliver a high-quality experience for analysts and other data consumers

Stakeholder and Technical Collaboration

• Work with business stakeholders to understand complex data requirements

• Translate functional requirements into scalable technical architectures

• Challenge requirements constructively when a more effective solution is available

• Partner with Finance, Sales Operations, IT and Business Intelligence teams

• Provide technical support to analysts and end users

• Communicate technical concepts clearly to both technical and non-technical audiences

• Manage multiple priorities within a fast-paced enterprise environment

Documentation and Platform Development

• Document data architectures, models, pipelines and configuration decisions

• Produce clear technical specifications and development documentation

• Maintain documentation to support knowledge transfer and long-term platform ownership

• Stay current with Databricks capabilities and platform enhancements

• Identify opportunities to use features such as Unity Catalog, Delta Live Tables and Photon

What You’ll Bring

Essential Databricks and Data Engineering Experience

• At least seven years of professional data-engineering experience

• A minimum of three years of hands-on Databricks experience

• Extensive experience building and managing production Databricks solutions

• Strong knowledge of:

• Delta Lake

• Delta Live Tables

• Unity Catalog

• Databricks Workflows

• Databricks notebooks and workspaces

• Advanced proficiency in Python and/or Scala

• Strong SQL development expertise

• Experience with data ingestion, transformation, ETL/ELT and data modelling

• Experience designing and orchestrating enterprise data pipelines

• Strong understanding of data quality, scalability and performance optimization

Sales Incentive Compensation Experience

• Functional and technical experience with Sales Incentive Compensation processes

• Understanding of incentive-plan design and compensation rules

• Experience supporting quota allocation and management

• Knowledge of sales-attainment calculations

• Experience with commission calculation and payout processing

• Ability to model complex incentive rules within data pipelines and transformation logic

• Experience partnering with Sales Operations and Finance stakeholders

Cloud and Integration Experience

• Experience integrating Databricks with at least one major cloud platform:

• Microsoft Azure

• Amazon Web Services

• Google Cloud Platform

• Experience working with integration and ingestion technologies such as:

• Azure Data Factory

• Azure Event Hubs

• Apache Kafka

• Comparable cloud-based data-integration tools

• Ability to create holistic, end-to-end data solutions rather than isolated pipelines

Professional Capabilities

• Strong stakeholder-management and consulting skills

• Ability to translate complex business requirements into scalable data architectures

• Excellent analytical and problem-solving capabilities

• Strong written and verbal communication skills

• Ability to work effectively with cross-functional teams

• Resourceful and comfortable working with minimal direction

• Strong technical-documentation skills

• Ability to manage competing priorities and deadlines

• Bachelor’s degree in Computer Science, Data Engineering, Information Technology or a related discipline

Preferred Experience

• Experience integrating Databricks with platforms such as:

• Anaplan

• Oracle Incentive Compensation

• Varicent

• Comparable Sales Performance Management platforms

• Databricks Certified Data Engineer Associate certification

• Databricks Certified Data Engineer Professional certification

• Experience using Photon to improve query and workload performance

• Previous experience within insurance, financial services or another large regulated enterprise

• Experience delivering auditable financial or compensation-related data solutions

The Ideal Candidate

The ideal candidate will be an experienced Databricks engineer who understands that compensation data requires exceptional accuracy, traceability and control.

You will be comfortable moving between detailed technical development and strategic conversations with Finance, Sales Operations and technology leaders. You should be willing to challenge requirements constructively, propose stronger architectural solutions and take ownership of delivery from initial design through testing, deployment and production support.

Most importantly, you will be able to combine enterprise data-engineering expertise with a practical understanding of how incentive plans, quotas, attainment calculations, commissions and payouts operate.

Contract Details

• Initial Contract: Six months

• Hourly Rate: Approximately $75 per hour

• Location: New York, New York

• Working Arrangement: Hybrid

• Visa Sponsorship: Not available

• Relocation Assistance: Not available

Ready to Build Business-Critical Data Solutions?

If you have deep Databricks expertise and experience translating complex Sales Incentive Compensation processes into scalable, accurate and auditable data solutions, this position offers the opportunity to make an immediate impact within a major enterprise environment.

Data Engineer | Azure, Databricks, Python, SQL, Spark | Hybrid – Netherlands (€3,500–€5,000/month)

Data Engineer

📍 Location: Eindhoven area or Randstad, Netherlands (Hybrid – 3 office days / 2 home days)
💼 Employment Type: Full-time
💵 Salary: €3,500 – €5,000 per month (€45,360 – €64,800 annually)
🎯 Experience Level: Mid-level | 2–3 years’ experience

About the Role

Do you love working with data — from digging into sources and writing clean ingestion scripts to ensuring a seamless flow into a data lake? As a Data Engineer, you’ll design and optimize data pipelines that transform raw information into reliable, high-quality datasets for enterprise clients.

You’ll work with state-of-the-art technologies in the cloud (Azure, Databricks, Fabric) to build solutions that deliver business-critical value. In this role, data quality, stability, and monitoring are key — because the pipelines you create will be used in production environments.

Key Responsibilities

• Develop data connectors and processing solutions using Python, SQL, and Spark.

• Define validation tests within pipelines to guarantee data integrity.

• Implement monitoring and alerting systems for early issue detection.

• Take the lead in troubleshooting incidents to minimize user impact.

• Collaborate with end users to validate and continuously improve solutions.

• Work within an agile DevOps team to build, deploy, and optimize pipelines.

Requirements

• 🎓 Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.

• 2–3 years of relevant experience in data ingestion and processing.

• Strong knowledge of SQL, Python, and Spark.

• Familiarity with container environments (e.g., Kubernetes).

• Experience with Azure Data Factory, Databricks, or Fabric is a strong plus.

• Experience with data model management and dashboarding (e.g., PowerBI) preferred.

• Team player with strong communication skills in Dutch and English.

• Familiarity with enterprise data platforms and data lakes is ideal.

What We Offer

• 💶 Salary: €3,500 – €5,000 per month

• 🌴 26 vacation days

• 🚗 Lease car or mobility budget (€600)

• 💻 Laptop & mobile phone

• 💸 €115 monthly cost allowance

• 🏦 50% employer contribution for health insurance

• 📈 60% employer contribution for pension scheme

• 🎯 Performance-based bonus

• 📚 Training via in-house Academy (hard & soft skills)

• 🏋️ Free use of on-site gym

• 🌍 Hybrid work model (3 days in office, 2 days at home)

• 🤝 Start with a 12-month contract, with option to move to indefinite after evaluation

Ideal Candidate

You are a hands-on data engineer who enjoys data wrangling and building robust pipelines. You take pride in seeing your code run smoothly in production and know how to troubleshoot quickly when issues arise. With strong technical skills in SQL, Python, and Spark, plus familiarity with cloud platforms like Azure, you’re ready to contribute to impactful enterprise projects.

👉 Ready to make data flow seamlessly and create business value? Apply now to join a passionate, innovation-driven team.

Azure Cloud Developer | Insurance | Princeton, NJ | $85/hr

Job Title: Azure Cloud Developer

Location: Princeton, NJ (On-site)
Experience Level: Mid-senior
Experience Required: 5 Years
Education Level: Bachelor’s Degree
Job Function: Information Technology
Industry: Insurance
Pay Rate: $85 per hour
Total Positions: 1
Relocation Assistance: Not available
Visa Sponsorship: Not available

Role Overview

We are seeking a skilled Azure Cloud Developer to support cloud development projects and create innovative data solutions for various business use cases. This role involves collaborating on cloud-based architecture designs, providing technical expertise throughout development phases, and delivering cutting-edge solutions to meet business requirements.

Key Responsibilities

• Design and develop solutions using Azure cloud-native technologies.

• Collaborate on solution architecture discussions and implementation strategies.

• Support all project phases, including analysis, design, development, and deployment.

• Develop and maintain CI/CD pipelines using tools like Azure DevOps and GitHub.

• Write and execute Unit, Integration, and System Testing using tools such as Postman and Fiddler.

• Work with backend databases and queuing technologies, including relational and NoSQL databases.

Technical Skills Required

• Proficient with Azure Data Factory, Azure Logic Apps, Azure Functions, Azure Service Bus, Azure SQL Database, SQL, and Databricks.

• Knowledge of CI/CD tools (e.g., Azure DevOps, GitHub).

• Experience with backend databases (PostgreSQL, MongoDB) and Azure technologies such as Data Lake, Storage, API Management, and Active Directory.

Qualifications

• 3+ years of programming experience with JavaScript.

• 2+ years of experience in development using Azure services (e.g., Azure Data Factory, Logic Apps, Functions, and Service Bus).

• Hands-on experience with Agile/SCRUM methodologies.

• Strong problem-solving skills and the ability to work independently.

• Exceptional communication skills for collaborating with stakeholders.

Preferred Certifications

• Azure Fundamentals

• Azure Developer Associate

This version is streamlined and tailored to the key qualifications and requirements of the role.

Azure Cloud Architect - USA, Remote $140,000 to $180,000

Azure Cloud Architect

USA, Remote

$140,000 to $180,000

Responsibilities 

• Architect and design solutions to meet functional and non-functional requirements  

• Create and review architecture and solution design artifacts 

• Evangelize re-use through the implementation of shared assets  

• Enforce adherence to architectural standards/principles, global product-specific guidelines, usability design standards, etc.  

• Proactively guide engineering methodologies, standards, and leading practices  

• Guidance of engineering staff and reviews of as-built configurations during the construction phase  

• Provide insight and direction on roles and responsibilities required for solution operations  

• Identify, communicate and mitigate Risks, Assumptions, Issues, and Decisions throughout the full lifecycle  

• Considers the art of the possible, compares various architectural options based on feasibility and impact, and proposes actionable plans  

• Demonstrate strong analytical and technical problem-solving skills  

• Ability to analyze and operate at various levels of abstraction  

• Ability to balance what is strategically right with what is practically realistic 

Minimum qualifications 

• Excellent technical architecture skills, enabling the creation of future-proof, complex global solutions  

• Excellent interpersonal communication and organizational skills are required to operate as a leading member of global, distributed teams that deliver quality services and solutions.  

• Ability to rapidly gain knowledge of the organizational structure of the firm to facilitate work with groups outside of the immediate technical team.  

• Knowledge and experience in IT methodologies and life cycles that will be used.  

• Familiar with solution implementation/management, service/operations management, etc.  

• Leadership skills can inspire others and persuade  

• Maintains close awareness of new and emerging technologies and their potential application for service offerings and products.  

• Bachelor’s Degree or equivalency (CS, CE, CIS, IS, MIS, or engineering discipline) or equivalent work experience  

• Experience in a solution architecture role using service and hosting solutions such as private/public cloud IaaS, PaaS, and SaaS platforms.  

• Experience in architecting and designing technical solutions for Microsoft-centric solutions based on industry standards using Azure IaaS, PaaS, and SaaS capabilities.  

• Experience with any of the following: Azure, Azure Stack, Azure AD  

• Must have hands-on experience on various services like ADF, ADLS, Security, Monitoring, Governance

• Must have experience to design platform on Azure.

• 3+ years of hands-on Experience to design and build Databricks based solution on Azure platform

• 1+ year of hands-on experience to design and build solution powered by DBT models and integrate with databricks.

• Must be very good designing End-to-End solution on Azure platform.

• Must have good knowledge of Data Engineering concept and related services of Azure

• Must have good experience in Python and Spark

• Must have good experience in setting up development best practices.

• Intermediate level knowledge is required for Data Modelling

• Good to have knowledge of docker and Kubernetes

• Experience with claims-based authentication (SAML/OAuth/OIDC), MFA, and RBAC  

• Knowledge of cloud security controls including tenant isolation, encryption at rest, encryption in transit, key management, vulnerability assessments, application firewalls, SIEM, etc.  

• Experience building and supporting mission-critical technology components with DR capabilities  

• Experience with multi-tier system and service design and development for large enterprises  

• Extensive, real-world experience designing technology components for enterprise solutions and defining solution architectures and reference architectures with a focus on cloud technologies.  

• Exposure to infrastructure and application security technologies and approaches  

• Familiarity with requirements gathering techniques.  

• Microsoft, TOGAF, Zachman, ITIL, CCNA, CISSP, PMP, etc. 

Preferred qualifications 

• Experience with configuration management and automation tools such as Ansible, DSC, Puppet, Chef, Salt, etc.  

• Experience with relational, graph and/or unstructured data technologies such as SQL Server, Azure SQL, Azure Data Lake, HD Insights, Hadoop, Cloudera, MongoDB, MySQL, Neo4j, Cassandra, Couchbase  

• Software development full lifecycle methodologies, patterns, frameworks, libraries, and tools  

• Knowledge of programming and scripting languages such as JavaScript, PowerShell, Bash, SQL, Java, Python, etc.  

• Experience with data ingestion technologies such as Azure Data Factory, SSIS, Pentaho, Alteryx  

• Experience with visualization tools such as Tableau, Power BI

• Experience with analytics tools including SPSS.  

• Experience with machine learning tools such as Azure ML, IBM, AWS, etc.  

• Experience in distilling complex technical challenges to actionable decisions for stakeholders and guiding project teams by building consensus and mediating compromises when necessary.  

• Experience coordinating the intersection of complex system dependencies and interactions. 

• Experience in solution delivery using common methodologies especially SAFe Agile but also Waterfall, Iterative, etc.  

• Demonstrated knowledge of relevant industry trends and standards 

Why we're a great company
We are an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. We are committed to creating a dynamic work environment that values diversity and inclusion, respect and integrity, customer focus, and innovation.