cognisos AI Systems Evaluation & Reliability Internship Remote · Intern

Apply for AI Systems Evaluation & Reliability Internship

Cognisos is a research lab that is actively building and shipping products. Our first product is Fabric, an evolving memory layer for agents. Through Fabric, agents reduce hallucinations, improve context windows and deliver higher quality. The final result is we unlock quality, trustworthy workflows to build at scale faster than any alternative. We are a small, yet powerful team today, blending expertise in different domains to drive everything we do. Come join us!

Role

We are looking for an engineering intern to help us over the summer in bringing our product to market. The primary role needed is around validating the core processing pipeline throug: benchmarks, regression tests, PR QA, failure analysis, and reproducible reports.

This is primarily an engineering role. Product/backend work is Rust-first, while benchmark and evaluation workflows are more Python-heavy. The ideal candidate is systems-minded, comfortable learning Rust, and effective with Python for scripting and analysis.

What You Will Work On

• Run and analyze memory/recall benchmarks.
• Add benchmark controls and ablations, such as no-context baselines, retrieval-vs-reader failure analysis, and regression checks.
• QA agent-generated PRs by running tests, reading logs, checking edge cases, and validating behavior.
• Write small Python, Rust, and SQL tools for benchmark analysis, smoke tests, and CI validation.
• Maintain concise runbooks so benchmark and development workflows are repeatable.
• Help summarize results clearly for the team, including what passed, what failed, and what the result actually means.

Our Ideal Candidate

• Strong CS fundamentals and fast learning ability.
• Comfortable with terminal workflows, Git/GitHub, and CI logs.
• Python for benchmark/eval scripting.
• Rust interest or experience, since the product codebase is Rust-first.
• Basic SQL/Postgres familiarity.
• Good testing and debugging instincts.
• Familiarity with or strong interest in LLM/RAG concepts: embeddings, vector search, evals, model-as-judge, context windows.
• Clear written communication and high attention to detail.
• High agency: able to take an ambiguous validation task and make steady progress.

Nice To Have

• Rust project experience.
• Docker or CI experience.
• Prior ML/AI project.
• Basic statistics: accuracy, confidence intervals, sampling, ablations.
• Ideal Project For July-August
• Help build the validation layer around Fabric: benchmark runbooks, no-context controls, regression smoke tests, PR QA checklists, failure analysis reports, and lightweight tools that let us ship agent-generated code faster without losing correctness.

Ideal Profile

Someone careful, technical, and high-agency who can help turn fast agent-driven development into trusted, benchmarked, shippable work.