Data Scientist – AI & Quality Assurance

Job Title: Data Scientist – AI & Quality Assurance Company: Ariveguru Technology Solutions Job Category: Data Science Roles & Responsibilities: AriveGuru is seeking a Data Scientist with combined expertise in Artificial Intelligence and Quality Assurance. The selected candidate will be responsible for developing data science and machine learning solutions and for ensuring that every AI\-driven capability we release meets defined standards of accuracy, reliability and consistency. Key Responsibilities * Develop, test and refine machine learning and LLM\-based solutions, including retrieval\-augmented generation (RAG), classification and information extraction * Perform data analysis, cleaning and preparation to support model development and evaluation * Design and iterate on prompts using structured, repeatable testing methods * Design evaluation datasets, reference (“golden”) answers and test cases to measure the accuracy, relevance and consistency of AI outputs * Define quality metrics, acceptance criteria and release thresholds for AI\-driven features * Build automated evaluation and regression\-testing pipelines in Python, integrated into the development and release workflow * Identify, document and report failure modes, including hallucinations, incomplete or incorrect responses, bias and formatting errors, and coordinate with engineering teams on resolution * Apply LLM\-as\-a\-judge methods and calibrate automated scoring against human review * Monitor production performance, detect model drift and quality degradation, and escalate issues promptly * Validate the completeness, consistency and correctness of datasets used by our systems * Opportunity to work on production\-grade AI systems where quality standards directly influence outcomes * Exposure to the complete lifecycle of AI development, evaluation and deployment * Defined career growth toward senior data science and AI quality leadership roles * Close collaboration with the leadership and engineering teams Work Location: In person Work Location: Bengaluru, India Availability: Full time

This is a hybrid position. Candidates must demonstrate working proficiency in both AI/data science and software quality assurance practices. Required Qualifications The position requires proficiency in both AI/data science and quality assurance. Applicants with experience in only one area will not be considered. * Bachelor’s degree in Computer Science, Data Science, Statistics, Mathematics or a related discipline * Approximately 2 years of hands\-on experience in data science, machine learning or AI projects * Strong proficiency in Python and its data libraries (Pandas, NumPy, Scikit\-learn) * Good working knowledge of SQL and relational databases such as PostgreSQL * Sound understanding of machine learning fundamentals, statistics and evaluation metrics (precision, recall, F1 score, etc.) * Practical experience with large language models, including prompt engineering, RAG, and an understanding of hallucination and grounding * Working knowledge of QA fundamentals: test planning, test case design, edge\-case analysis, defect reporting and regression testing * Experience testing AI or ML outputs for accuracy, consistency and bias * Ability to build automated tests and evaluation pipelines in Python * Experience in data quality validation * Ability to define pass/fail criteria and release standards * Strong attention to detail and a disciplined, evidence\-based approach to verifying results Preferred Qualifications * Experience with evaluation frameworks such as Ragas, DeepEval or LangSmith * Familiarity with LangChain or LlamaIndex * Knowledge of test automation and API testing tools (Pytest, Postman, Selenium or equivalent) * Experience with Git, CI/CD pipelines and AWS * Data visualization skills (Matplotlib, Plotly, Power BI or equivalent) * Exposure to NLP, embeddings and vector databases Benefits: Key Skills: Data Science, Python, SQL, Pandas, NumPy, Statistics, Data Visualization, Machine Learning, Power BI Salary: INR 500000 - 800000 per yearly