Query Optimization

Senior Software Engineer, Backend - Internal Tools | Los Angeles, CA

The opportunity

Build a new internal tools platform from the ground up, bringing legacy systems together into a modern advertiser management platform. You will shape APIs, middleware, backend services and the data model, working across product and engineering teams to deliver reliable, high-performance software.

Role details

Location: Los Angeles, California. Candidates must be based in the Pacific Time zone; Los Angeles is preferred.

Work arrangement: Hybrid preferred, with three days per week in Santa Monica. Fully remote or hybrid flexibility is available depending on location, with some travel to Santa Monica.

Employment: Full-time

Base salary: $175,000-$195,000 per year, depending on location, knowledge, skills and experience.

Relocation assistance: Not available. Visa sponsorship: Not available now or in the future.

Security clearance: Not required. Travel: Occasional.

What you will do

Design, build and maintain scalable Java and Spring Boot backend services, RESTful APIs and microservices.

Help define system architecture and technology choices, with a focus on reliability, performance and maintainability.

Design and optimise relational schemas and queries, protecting data integrity and consistency.

Consolidate internal tools and integrate systems across Publisher, Merchant and other business teams.

Develop APIs and services that can support AI use cases.

Troubleshoot complex issues, improve performance and refactor the codebase.

Maintain strong testing, code review and documentation practices.

Evaluate useful tools and frameworks, including AI-assisted development tooling.

What you will bring

6+ years of professional backend development experience, including delivery of high-performance systems.

Strong Java and hands-on Spring Boot experience, plus solid RESTful API design and implementation skills.

Expertise in relational databases such as MySQL or PostgreSQL, including schema design and query optimisation.

Strong software architecture, design patterns, distributed systems and microservices knowledge.

Practical daily use of AI coding tools such as Claude Code, GitHub Copilot or Cursor for generation, refactoring, testing and debugging. You must critically review generated code and take ownership of what you ship.

Excellent communication, cross-functional collaboration and analytical problem-solving skills; experience working in Agile teams.

A professional LinkedIn profile. A Computer Science, Engineering or related degree is preferred, not required.

Useful additional experience

Python, Node.js or Go; NoSQL databases or analytical warehouses such as BigQuery.

Integrating LLM APIs or building internal AI-powered tools.

Building new internal platforms and integrating multiple tools or legacy systems.

Benefits

Medical, dental and vision cover; 401(k) matching; life and AD&D insurance; short- and long-term disability; maternity, paternity and medical leave; flexible time off and paid holidays; learning and development; wellness resources; equity; team events and office food and snacks.

Databricks Data Engineer โ€“ Sales Incentive Compensation | New York, NY

Databricks Data Engineer โ€“ Sales Incentive Compensation

๐Ÿ“ Location: New York, New York
๐Ÿข Work Arrangement: Hybrid
๐Ÿ“„ Contract Length: Six Months
๐Ÿ’ฐ Hourly Rate: Approximately $75 per hour
๐Ÿ’ผ Employment Type: Contract
๐Ÿšซ Visa Sponsorship: Not Available
๐Ÿšš Relocation Assistance: Not Available
๐Ÿฆ Industry: Insurance and Financial Services

Build the Data Infrastructure Behind Enterprise Sales Compensation

A leading global organization is seeking an experienced Databricks Data Engineer to design, develop and optimize enterprise data solutions supporting analytics, performance reporting and Sales Incentive Compensation.

This is not purely a pipeline-development position. The successful candidate will combine strong Databricks engineering expertise with an understanding of incentive compensation processes, including quota allocation, attainment calculations, commission processing and payout reporting.

You will work closely with Sales Operations, Finance, Business Intelligence and technology teams to translate complex business rules into scalable, accurate and auditable data solutions.

Why This Opportunity Stands Out

This opportunity will allow you to:

  • Own the design and development of enterprise Databricks solutions

  • Influence the wider Databricks architecture and technical roadmap

  • Build data pipelines supporting business-critical compensation processes

  • Work with Delta Lake, Delta Live Tables, Unity Catalog and Databricks Workflows

  • Translate complex incentive-plan rules into reliable transformation logic

  • Improve automation, integration, data quality and platform performance

  • Partner directly with senior Finance, Sales Operations and technology stakeholders

  • Deliver solutions that directly influence commission and compensation accuracy

What Youโ€™ll Be Doing

Databricks Architecture and Engineering

  • Design, develop and implement scalable Databricks-based data solutions

  • Build and maintain data pipelines supporting ingestion, transformation, reporting and analytics

  • Develop data models, Delta Lake tables, notebooks and reporting datasets

  • Take ownership of the overall Databricks architecture

  • Ensure new pipelines and enhancements integrate effectively with existing workflows

  • Maintain and optimize Databricks workspaces, pipelines and orchestration processes

  • Identify opportunities to improve automation, performance and data integration

  • Design reliable ETL and ELT processes across multiple enterprise data sources

Sales Incentive Compensation

  • Build data pipelines supporting Sales Incentive Compensation processes

  • Translate incentive-plan rules into accurate and auditable data transformations

  • Support quota allocation and quota-management processes

  • Develop logic for attainment tracking and performance calculations

  • Support commission calculations and payout processing

  • Produce reliable datasets for compensation reporting and reconciliation

  • Work closely with Sales Operations and Finance to understand plan structures and business rules

  • Ensure calculation logic is transparent, documented and capable of being audited

  • Integrate incentive-compensation platforms with enterprise data and analytics solutions

Performance and Data Quality

  • Test, debug and optimize Databricks pipelines and notebooks

  • Investigate and resolve data-quality, performance and integration issues

  • Improve the reliability and efficiency of existing workflows

  • Implement appropriate controls and validation processes

  • Monitor data pipelines and troubleshoot failures or inconsistencies

  • Deliver a high-quality experience for analysts and other data consumers

Stakeholder and Technical Collaboration

  • Work with business stakeholders to understand complex data requirements

  • Translate functional requirements into scalable technical architectures

  • Challenge requirements constructively when a more effective solution is available

  • Partner with Finance, Sales Operations, IT and Business Intelligence teams

  • Provide technical support to analysts and end users

  • Communicate technical concepts clearly to both technical and non-technical audiences

  • Manage multiple priorities within a fast-paced enterprise environment

Documentation and Platform Development

  • Document data architectures, models, pipelines and configuration decisions

  • Produce clear technical specifications and development documentation

  • Maintain documentation to support knowledge transfer and long-term platform ownership

  • Stay current with Databricks capabilities and platform enhancements

  • Identify opportunities to use features such as Unity Catalog, Delta Live Tables and Photon

What Youโ€™ll Bring

Essential Databricks and Data Engineering Experience

  • At least seven years of professional data-engineering experience

  • A minimum of three years of hands-on Databricks experience

  • Extensive experience building and managing production Databricks solutions

  • Strong knowledge of:

    • Delta Lake

    • Delta Live Tables

    • Unity Catalog

    • Databricks Workflows

    • Databricks notebooks and workspaces

  • Advanced proficiency in Python and/or Scala

  • Strong SQL development expertise

  • Experience with data ingestion, transformation, ETL/ELT and data modelling

  • Experience designing and orchestrating enterprise data pipelines

  • Strong understanding of data quality, scalability and performance optimization

Sales Incentive Compensation Experience

  • Functional and technical experience with Sales Incentive Compensation processes

  • Understanding of incentive-plan design and compensation rules

  • Experience supporting quota allocation and management

  • Knowledge of sales-attainment calculations

  • Experience with commission calculation and payout processing

  • Ability to model complex incentive rules within data pipelines and transformation logic

  • Experience partnering with Sales Operations and Finance stakeholders

Cloud and Integration Experience

  • Experience integrating Databricks with at least one major cloud platform:

    • Microsoft Azure

    • Amazon Web Services

    • Google Cloud Platform

  • Experience working with integration and ingestion technologies such as:

    • Azure Data Factory

    • Azure Event Hubs

    • Apache Kafka

    • Comparable cloud-based data-integration tools

  • Ability to create holistic, end-to-end data solutions rather than isolated pipelines

Professional Capabilities

  • Strong stakeholder-management and consulting skills

  • Ability to translate complex business requirements into scalable data architectures

  • Excellent analytical and problem-solving capabilities

  • Strong written and verbal communication skills

  • Ability to work effectively with cross-functional teams

  • Resourceful and comfortable working with minimal direction

  • Strong technical-documentation skills

  • Ability to manage competing priorities and deadlines

  • Bachelorโ€™s degree in Computer Science, Data Engineering, Information Technology or a related discipline

Preferred Experience

  • Experience integrating Databricks with platforms such as:

    • Anaplan

    • Oracle Incentive Compensation

    • Varicent

    • Comparable Sales Performance Management platforms

  • Databricks Certified Data Engineer Associate certification

  • Databricks Certified Data Engineer Professional certification

  • Experience using Photon to improve query and workload performance

  • Previous experience within insurance, financial services or another large regulated enterprise

  • Experience delivering auditable financial or compensation-related data solutions

The Ideal Candidate

The ideal candidate will be an experienced Databricks engineer who understands that compensation data requires exceptional accuracy, traceability and control.

You will be comfortable moving between detailed technical development and strategic conversations with Finance, Sales Operations and technology leaders. You should be willing to challenge requirements constructively, propose stronger architectural solutions and take ownership of delivery from initial design through testing, deployment and production support.

Most importantly, you will be able to combine enterprise data-engineering expertise with a practical understanding of how incentive plans, quotas, attainment calculations, commissions and payouts operate.

Contract Details

  • Initial Contract: Six months

  • Hourly Rate: Approximately $75 per hour

  • Location: New York, New York

  • Working Arrangement: Hybrid

  • Visa Sponsorship: Not available

  • Relocation Assistance: Not available

Ready to Build Business-Critical Data Solutions?

If you have deep Databricks expertise and experience translating complex Sales Incentive Compensation processes into scalable, accurate and auditable data solutions, this position offers the opportunity to make an immediate impact within a major enterprise environment.