# Rhesis AI > Open-source collaboration layer that gets domain-expert knowledge into AI agent development. Domain experts annotate agent behaviour, engineers get that feedback as annotated test sets, and the same expectations drive adversarial test generation, multi-turn conversation simulation, and traces linked back to test failures. ## Platform - [Collaboration layer for teams developing AI agents](https://rhesis.ai/): Open-source collaboration layer that gets domain-expert knowledge into AI agent development. Shared workspace where experts annotate agent behaviour and engineers get the feedback they need, via UI, SDK, or MCP. - [Conversation simulation](https://rhesis.ai/conversation-simulation): Penelope simulates realistic multi-turn conversations with your AI, testing goal completion, context retention, and behavioral consistency. - [Observability](https://rhesis.ai/observability): OpenTelemetry-based tracing that links production traces back to test failures. Close the gap between observability and testing for AI applications. - [Editions](https://rhesis.ai/editions): Community and Enterprise editions compared. The full collaboration workflow between domain experts and engineers is open source and self-hostable; the Enterprise Edition adds SSO (OIDC), RBAC, and API Clients, plus support and an agreement. ## Company - [About us](https://rhesis.ai/about): Rhesis is the collaboration layer that gets domain-expert knowledge into AI agent development. Open source, built for experts, engineers, and product managers. - [Watch demo](https://rhesis.ai/watch-demo): Short videos showcasing test-case generation, conversation simulation, traces, and collaborative reviews. - [Talk to us](https://rhesis.ai/talk-to-us): Schedule a demo, discuss your testing needs, or ask questions about the Rhesis platform. - [Blog](https://rhesis.ai/blog): Guides, insights, and best practices for testing LLM and agentic applications. ## Optional - [Privacy policy](https://rhesis.ai/privacy-policy): Privacy policy and data protection information for Rhesis AI GmbH. ## Blog - [We tested our own healthcare agent. It missed five emergencies out of twenty.](https://rhesis.ai/post/testing-our-own-healthcare-agent): A field report: 151 single-turn tests against our own healthcare agent. Same emergency, three wordings, three different behaviours, and what the traces showed was causing it. - [Clinical Annotation for Conversational Healthcare AI](https://rhesis.ai/post/clinical-annotation-healthcare-ai): Clinical annotation for healthcare AI: inter-rater agreement, rubric design, adjudication, and where automated judges hold or collapse. - [Building a Test Harness for Healthcare Conversational AI](https://rhesis.ai/post/healthcare-ai-test-harness): Build a healthcare AI test harness: real clinical seeds, persona design, adversarial mutation operators, multi-turn escalation, and directional gates. - [Testing conversational AI for healthcare: why it's different](https://rhesis.ai/post/testing-conversational-ai-healthcare): Healthcare conversational AI fails in ways generic LLM testing cannot see. What the 2026 research shows, and what medical benchmarks can and cannot tell you. - [Top 6 AI quality platforms for human annotation (2026)](https://rhesis.ai/post/ai-quality-platforms-human-annotation): Rhesis, Langfuse, Opik, Arize Phoenix, Orq.ai, and DeepEval compared for human annotation in 2026: reviewer access, what experts can review, pricing, and open source. - [Rhesis vs Opik: head-to-head comparison (2026)](https://rhesis.ai/post/rhesis-vs-opik): Rhesis vs Opik in 2026: two open-source platforms compared on evaluation, simulation, annotation, tracing, CI, pricing, and when to pick each. - [Rhesis vs LangWatch: head-to-head comparison (2026)](https://rhesis.ai/post/rhesis-vs-langwatch): Rhesis vs LangWatch in 2026: two open-source platforms compared on evaluation, simulation, annotation, tracing, CI, pricing, and when to pick each. - [Rhesis vs Confident AI: head-to-head comparison (2026)](https://rhesis.ai/post/rhesis-vs-confident-ai): Rhesis vs Confident AI in 2026: near feature parity, but Rhesis open-sources the whole workspace, UI included. Features, pricing, and when to pick each. - [Scoped access, managed secrets, self-healing deploys: our move to Kubernetes](https://rhesis.ai/post/migrating-to-kubernetes): How we migrated from Cloud Run to private Kubernetes clusters: WireGuard access, centralized secrets management, and self-healing GitOps deploys, hardened for scale. Costs, trade-offs, and lessons from a four-month migration. - [Watching Isn't Testing: The Case for Two-Way Connections](https://rhesis.ai/post/two-way-connections-llm-testing): Why testing an LLM application requires a two-way connection, not just observability, and how to set one up with Rhesis REST Endpoints or the Python SDK connector. - [Deploying a Custom LLM in Production: Four Architectures, Only One Works](https://rhesis.ai/post/deploying-custom-llm-in-production): How we deployed an 8B parameter model on GCP from oversized Docker images to a low-latency Vertex AI endpoint with vLLM. Real data, real tradeoffs. - [What EvalOps is and why AI teams can't ship without it](https://rhesis.ai/post/what-is-evalops): EvalOps explained: the operational discipline behind reliable AI applications. Covers the lifecycle, real-world failures, and what a complete EvalOps stack contains. - [Why conversational AI needs adversarial testing](https://rhesis.ai/post/llm-application-adversarial-testing): How adversarial testing catches jailbreaks, prompt injection, hallucination, and other failures that standard testing misses. A practical strategy for technical teams. - [We asked 22 LLMs to generate adversarial test cases - most refused to help](https://rhesis.ai/post/adversarial-llm-benchmark): We benchmarked 22 language models for adversarial prompt generation. Commercial models refused up to 65% of critical test cases. Here's what actually works. - [How we automated multi-component releases with GitHub Actions](https://rhesis.ai/post/ai-testing-platform-release-automation): How we automated multi-component releases with GitHub Actions: release branches, cron-triggered PRs, LLM-generated changelogs, and production deployment. - [Offline vs. online evaluation at the application layer: a practical guide](https://rhesis.ai/post/offline-vs-online-evaluation-llm-applications): How offline and online evaluation work at the LLM application layer: requirements testing, production monitoring, and the feedback loop. Read the full guide. - [Why we moved from Auth0 to native authentication](https://rhesis.ai/post/auth0-to-native-authentication-migration): How our LLM testing platform replaced Auth0 with a pluggable auth stack that works on-prem and air-gapped. Architecture, migration path, and lessons. - [9 LLM evaluation & testing tools compared (2026)](https://rhesis.ai/post/best-llm-evaluation-testing-tools): Compare 9 LLM evaluation tools for 2026: Rhesis, LangWatch, Confident AI, LangSmith, Braintrust, DeepEval, RAGAS, Arize Phoenix, and Opik. - [Multi-turn agent simulation: frameworks compared](https://rhesis.ai/post/agent-simulation-frameworks): Compare GOAT, ADaPT, DeepEval, promptfoo, and Penelope for multi-turn LLM testing. Covers design challenges, research foundations, and open evaluation problems. - [How to pick an agentic framework in 2026](https://rhesis.ai/post/picking-agentic-framework-2026): Side-by-side comparison of 8 agent frameworks—LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, Google ADK, Semantic Kernel. Architecture, multi-agent support, and real tradeoffs. - [10 challenges building an OTel LLM tracer](https://rhesis.ai/post/10-challenges-otel-llm-tracer): Ten challenges building an OpenTelemetry tracer for LLM apps: trace formats, naming, token extraction, auto-instrumentation, cost calculation, and more. - [Tracing for agentic applications: a developer's guide](https://rhesis.ai/post/tracing-agentic-applications-developers-guide): Where traditional APM breaks down for LLM apps, how OpenTelemetry fills the gap, and which tracing tools work for agentic workflows. Context propagation, multi-agent spans, and real tradeoffs. - [AI agent test coverage: what actually matters](https://rhesis.ai/post/test-coverage-llm-agentic-apps): Code coverage means nothing for LLM agents. Learn how to build a multi-dimensional coverage map that reveals where your test suite is blind. - [Beyond evals: what enterprise AI agent testing requires](https://rhesis.ai/post/ai-agent-testing-beyond-evals): Evaluation frameworks check outputs but miss the full picture. What production AI agent testing looks like when engineers, QA, and product all need to stay in the loop. - [A PM's guide to testing AI agents](https://rhesis.ai/post/guide-testing-ai-agents-as-pm): A practical playbook for AI PMs: move from spreadsheets and vibe-testing to behaviors, metrics, tests, and operational confidence. With Rhesis as reference. - [Our first community hour: Building together](https://rhesis.ai/post/our-first-community-hour-building-together): Join Rhesis AI's Community Hour - a virtual meetup for Gen AI testing. Learn about our first session on evaluation backend design. - [Self-hosting Rhesis with Docker Compose](https://rhesis.ai/post/self-hosting-llm-evaluation-framework): Learn how Rhesis built a one-command Docker Compose setup. Discover solutions for cross-container communication and multi-service orchestration. - [Why we rebranded Rhesis AI](https://rhesis.ai/post/from-enterprise-saas-to-open-source-why-we-rebranded-rhesis-ai): Learn why Rhesis AI pivoted from enterprise SaaS to open source. Discover how user feedback shaped our rebrand and collaborative vision. - [Why quality assurance matters for trustworthy AI](https://rhesis.ai/post/ensuring-trustworthy-ai): Explore why Quality Assurance matters for trustworthy AI. Learn about QA challenges in AI and best practices for the AI lifecycle. - [Lessons from 10+ AI conferences](https://rhesis.ai/post/lessons-from-10-ai-conferences-on-gen-ai-application-development): Insights from 10+ AI conferences on moving Gen AI to production. Learn about the Production Trifecta: governance, regulation, and evaluation. - [Are insurance chatbots trustworthy?](https://rhesis.ai/post/gen-ai-chatbots-in-the-insurance-industry-are-they-trustworthy): Explore LLM chatbot benchmarks in insurance. Learn about reliability, compliance challenges, and trustworthy AI in regulated industries. - [Building Penelope: A multi-turn testing agent](https://rhesis.ai/post/penelope-multi-turn-testing-agent): Learn how Rhesis built Penelope, an autonomous testing agent for conversational AI. Discover the architecture behind complex agentic workflows. - [RPC testing setup with Rhesis connector](https://rhesis.ai/post/llm-testing-setup-rhesis-connector): Learn how Rhesis built a WebSocket-based RPC connector for LLM testing. Discover automatic endpoint registration and distributed execution. - [Conversational AI testing glossary](https://rhesis.ai/post/conversational-ai-glossary): Learn essential vocabulary for testing conversational AI. Comprehensive glossary covering metrics, behaviors, endpoints, and test runs. - [Observability vs. testing: Dependency binding](https://rhesis.ai/post/observability-vs-testing): Learn how Rhesis's bind parameter enables remote LLM testing with dependency injection, beyond Langfuse, TruLens, and OpenTelemetry. - [How to test LLM-applications: A six-phase cycle](https://rhesis.ai/post/how-to-test-llm-applications): Learn the six-phase testing cycle for LLM applications: configure, define, select metrics, generate tests, evaluate, and collaborate. - [Using AI to test AI: The only approach that scales](https://rhesis.ai/post/ai-testing-ai): Why using AI to test AI is the only approach that scales. Learn how Rhesis uses LLM-as-judge methodology to evaluate AI applications. - [Optimizing Docker images: Lessons from Rhesis](https://rhesis.ai/post/docker-optimization-lessons-learned): Learn how Rhesis reduced Docker images from 16GB to 4.8GB. Discover multi-stage builds, layer optimization, and cache mounting techniques. - [Building custom LLM evaluation metrics](https://rhesis.ai/post/llm-judge-metrics): Learn to build custom LLM-as-a-Judge metrics. Discover best practices for prompt design, scoring calibration, and bias mitigation. - [Engineer's guide to testing conversational AI](https://rhesis.ai/post/testing-conversational-ai): Comprehensive guide for testing conversational AI. Learn strategies, metrics, and best practices for LangChain, LangGraph, and custom agents. - [Why domain experts matter more than AI skills](https://rhesis.ai/post/collaborative-llm-evaluation): Discover why domain experts are essential to AI development. Learn how collaborative testing bridges technical teams and subject matter experts. - [Building MCP connections: PRDs vs simple MVPs](https://rhesis.ai/post/building-mcp-connections-for-llm-applications): Learn lessons from building MCP integrations for LLMs. Discover why simpler MVPs outperform over-designed PRDs and earn complexity through feedback.