Pathi Manikanta
AI Evaluation Specialist, ML Engineer and RLHF/RLAIF contributor. I audit agentic tool-use trajectories, design pass/fail rubrics and pytest verifiers, and build end-to-end ML systems. I care about finding where a model breaks - and proving it with evidence.
Evaluation, engineering, and taxation
AI Evaluation Specialist, ML Engineer and Full Stack Developer from Andhra Pradesh, India.
B.E. Computer Science and Engineering (Data Science), Anna University (2025). At Scale AI (Outlier) I reviewed frontier AI and was promoted from Attempter to Reviewer. I designed binary pass/fail rubrics backed by pytest verifiers and audited agentic tool-use trajectories. That engagement has concluded and I am now open to new roles.
I work across AI evaluation, LLM engineering, RLHF/RLAIF and full-stack ML. My focus is finding the exact turn where a model breaks and building the deterministic verifier that proves it.
Languages: Telugu, English, Hindi, Tamil
Finance and Taxation
The full stack of building and proving
From rubric design and trajectory review to ML systems, web engineering, and taxation.
Pipelines, rubrics, and verifiers
Golden/silver trajectory pipelines, safety gates, and QC tooling for frontier models.
OpenClaw Atlas
PrivateGolden/silver memory-trap pipeline for authoring and QC-ing agentic RL trajectories: story -> discrepancy -> failure grade -> silver fix -> pytest verifiers -> QC.
Blue Shell
PrivateTwo-turn golden/silver capabilities-RL pipeline across six tools where Turn 1 must fail a rubric trap and a Turn-2 nudge must pass 100 percent.
Lobster Safety
PrivateNine-worker safety-RL pipeline with a hard PASS/STOP gate, grading trajectories against an SSOT taxonomy (F1-F10 failure types, S0-S3 severity).
TrajLens
PublicInteractive agentic-trajectory auditor: grades 14 recorded tool-use traces against a 9-criterion rubric of deterministic verifiers (incl. safety checks for buried errors and unauthorized actions), and measures an LLM-judge against human labels with Cohen's kappa. Paste your own trajectory JSON and grade it live.
JudgeLab
PublicLLM-as-a-judge reliability lab: measures GPT-4-as-a-judge against 3,355 real human expert votes from the MT-Bench study. Cohen's kappa 0.46 on decisive verdicts, a 15.8% order-swap inconsistency rate (position bias), a quantified verbosity preference, and per-category reliability showing the judge collapses on writing tasks. Real public data, no API key.
EvalGate
PublicCI/CD pipeline that blocks bad models (GitHub Actions): every push runs 11 tests, retrains the model, enforces a hard quality gate (ROC-AUC threshold plus a CV-vs-test drift check), promotes the gated artifact between workflows, and auto-deploys a live model-quality dashboard to GitHub Pages. Weekly scheduled retrains, deployment environments, zero manual steps.
Project Seal - Authoring
PrivateRunbook for writing hard single-answer multi-hop search questions that defeat frontier models; 8 quality rules, 18 techniques.
Shipped, end to end
HandPilot
Computer Vision, 2026Webcam gesture control for Windows: MediaPipe and OpenCV hand tracking drive the pointer, pinch clicks, scrolling, and an on-screen keyboard, with optional object and pen following. PySide6 overlay, Windows CI tests.
Chatbox
Full-stack, 2026Streaming chat app with a pluggable model backend: FastAPI and Server-Sent Events on the server, React and TypeScript in the browser. Ships an offline replay provider so it runs with no API key; also supports Gemini and Claude. CI included.
Customer Churn Predictor
ML & Data, 2026End-to-end ML app predicting telecom churn (ROC-AUC 0.843) on 7,000+ customers; Gradient Boosting; Streamlit; live risk scoring.
Diabetes Foot Pressure Detection
ML & Data, 2025Full ML pipeline from Arduino sensors to classifiers at 92.5% accuracy; caught and fixed session-level data leakage with GroupKFold.
Spam Email Classifier
NLP, 2024NLTK + TF-IDF, 97.8% accuracy with Linear SVM on 1000+ emails.
Face Recognition (OpenCV)
Computer Vision, 2024Real-time detection and recognition at 15+ FPS, 70%+ confidence, Haar Cascades + LBPH.
IoT Environment Monitor
Hardware, 2023Arduino DHT11/BMP180 streaming temperature, humidity, pressure every 2 seconds to a remote dashboard.
Google Doc Grid Parser
ML & DataParses coordinate triples from a published Google Doc and prints the hidden block-letter message.
Where I have worked
AI Evaluation Specialist, Reviewer - Scale AI (Outlier)
Promoted from Attempter to Reviewer. Designed binary pass/fail rubrics backed by pytest verifiers and audited agentic tool-use trajectories on frontier AI.
Software and ML Engineer - Tensium
Author CPU-based long-horizon ML tasks requiring multistep reasoning and code generation; develop, debug, and refactor Python pipelines for maintainability and performance.
AI Evaluation Contributor - Handshake AI
Authored hard single-answer multi-hop search questions and built automated QC tooling against rejection rules.
MCX Advisor - Zebu Share and Wealth Management
4 years of trading experience across crypto and Indian equities. Supported client trading operations: order processing, position monitoring, and profit-and-loss updates.
Selected highlights
Promoted Attempter to Reviewer in 3 months
Recognized for evaluation accuracy, guideline compliance, and consistent task quality.
Shipped 3 live AI-evaluation tools
Public deployed apps for agentic-trajectory auditing, LLM-judge reliability, and CI/CD model gating.
CI/CD pipeline green for months
Automated weekly retraining and quality-gated deployments running unattended on GitHub Actions.
Caught a data-leakage bug others missed
Traced inflated 99% accuracy to session leakage and corrected it to a truthful 92.5% with GroupKFold.
HackerRank certified
Foundations and credentials
B.E. CSE (Data Science)
Higher Secondary (XII) MPC
Secondary (X)
Let us build and prove the next model
Open to AI Evaluation, LLM Engineering, RLHF/RLAIF and ML Engineer roles. Remote or relocation. Fastest reply by email.