Software Engineer Java + Data (PySpark)

Lineate is a US-based international software development company with over two decades of experience.

From Intelligent Document Processing(IDP) and Agentic RAG systems to scalable cloud architectures, we turn complex ideas into real, measurable results.

We deliver AI-driven custom solutions for FinTech, HealthTech, AdTech, and beyond, empowering businesses to grow smarter, faster, and more efficiently.

Our expertise falls into three main categories:

• Building Custom AI Solutions: Deploying high-impact, AI-enabled technology utilizing IDP, Agentic RAG.
• Cloud and Data Infrastructure: Optimizing business operations with our data management and cloud computing solutions.
• Team Augmentation: Providing specialized experts in FinTech, AdTech, and HealthTech to integrate seamlessly and accelerate project timelines.
• Our goal is not just to build technology, but to build the future operating model for our clients.

• Design, develop, and maintain scalable backend services using Java and Python
• Build and optimize data pipelines for large-scale data processing (primarily with PySpark)
• Collaborate closely with ML engineers to prepare and transform data for ML models
• Work with large datasets to support search relevance and optimization systems
• Improve system performance, scalability, and reliability
• Contribute to architecture decisions and technical design
• Write clean, maintainable, and well-documented code

• 6+ years of commercial software development experience
• Strong hands-on experience with both Java and Python (required)
• Solid experience with Spark / PySpark or similar data processing frameworks
• Proven experience building data pipelines and working with large datasets
• Strong understanding of backend development, system design, and distributed systems
• Experience with APIs and microservices
• Good knowledge of data structures, algorithms, and performance optimization
• Ability to work in hours close to the New York timezone (at least 6 hours overlap with NY team)

• Experience with search and relevance optimization, including:
  Elasticsearch / Solr (lexical search)
  Semantic search, vector search, or RAG-based systems
  Search relevance tuning
• Experience supporting ML workflows, such as:
  Data preparation or feature engineering
  Recommendation systems or user behavior prediction
• Experience with Databricks (preferred) or similar environments:
  AWS EMR, Hadoop, Spark ecosystem