Staff Software Development Test Engineer

Positively disrupting an industry that has not seen any innovation in over 50 years, Tekion has challenged the paradigm with the first and fastest cloud-native automotive platform that includes the revolutionary Automotive Retail Cloud (ARC) for retailers, Automotive Enterprise Cloud (AEC) for manufacturers and other large automotive enterprises and Automotive Partner Cloud (APC) for technology and industry partners. Tekion connects the entire spectrum of the automotive retail ecosystem through one seamless platform. The transformative platform uses cutting-edge technology, big data, machine learning, and AI to seamlessly bring together OEMs, retailers/dealers and consumers. With its highly configurable integration and greater customer engagement capabilities, Tekion is enabling the best automotive retail experiences ever. Tekion employs close to 3,000 people across North America, Asia and Europe.

Technical Skills & Frameworks 

Core Programming 

• →Python — expert level; test automation, eval pipelines, data analysis (Pandas, NumPy, Pytest) 

• →SQL — data output validation, ground truth querying, pipeline data quality checks 

GenAI & Evaluation 

• →RAGAS / TruLens / DeepEval / Promptflow etc — LLM evaluation frameworks for measuring faithfulness, hallucination rate, and task success 

• →LangChain / LangSmith / LlamaIndex — agent workflow testing, prompt tracing, and LLM response debugging 

• →OpenAI / Anthropic / Hugging Face APIs — direct LLM endpoint testing and output consistency validation 

• →Vector DBs — retrieval quality testing, embedding validation, and latency benchmarking 

• →Pandas / NumPy etc. — statistical analysis for output validation and error pattern investigation, data profiling, schema validation, and pipeline integrity checks 

API & Automation 

• →Pytest — modular, reusable test framework for AI eval and automation suites 

• →Postman / REST Assured / Requests — API contract validation and service-level integration testing 

MLOps & CI/CD 

• →MLflow — tracking model versions and eval runs to detect regressions across updates 

• →Docker / GitHub Actions / Jenkins — containerised test environments and deployment pipeline automation 

Observability 

• →Grafana / Kibana / OpenTelemetry — monitoring AI system health, output drift, and distributed tracing across agent pipelines 

Good to Have 

• →Cloud AI Services — AWS Bedrock, Azure OpenAI, or GCP Vertex AI for testing managed model endpoints and cloud-deployed agents 

• →MLOps Platforms — MLflow, Kubeflow, Weights & Biases, or Feast feature stores for experiment tracking and model governance 

• →ML Frameworks — Scikit-learn, TensorFlow, or PyTorch familiarity for understanding model internals and validating training pipelines 

• →Infrastructure as Code — Docker, Kubernetes, Terraform for managing containerised test environments at scale 

• →UI Automation — Playwright or Cypress for end-to-end conversational AI application testing 

• →Performance Engineering — Locust or JMeter for load testing heavy AI inference endpoints under peak traffic 

• →Statistical Hypothesis Testing — t-tests, confidence intervals, significance testing to distinguish real quality signal from noise 

• →Synthetic Data Generation — using LLMs to generate diverse test cases and evaluation datasets at scale