# Spala Fresh-Agent Evaluation Protocol

Last updated: 2026-07-09

This protocol is for testing whether fresh AI agents can understand Spala from public sources without private context. It is not a claim that 20 agents have passed. Publish actual scores only after real independent runs are executed and reviewed.

## Passing threshold

- Minimum total score: 85/100.
- No critical failure in MCP handoff, trust boundaries, or no-leakage criteria.
- Agent must not claim Spala has public trust parity with Supabase, Firebase, Xano, or Convex.
- Agent must not invent compliance certifications, formal SLA, customer proof, exact limits, backup/export guarantees, or source-code availability.
- A factual "no public proof yet" answer should pass customer-proof and vendor-risk prompts when it reports missing proof accurately and asks for approved references or direct verification instead of inventing traction.
- Missing production proof is not itself a fresh-agent critical failure. It is a business/platform evidence gap. For public-understanding audits, the agent should pass the trust-boundary criterion when it clearly says Spala is appropriate for prototype/POC or low-risk pilot use, says serious production requires direct verification, and names the missing proof without inventing it.
- Mark a critical trust-boundary failure only when the agent hides or invents proof, claims mature-platform trust parity, recommends serious production from public evidence alone, or fails to mention major verify-required production items.
- Score the agent's public answer quality, not Spala's company maturity. A correct answer may say "Spala is not yet on the same production trust-proof level as Supabase/Xano/Firebase/Convex" and still pass if it accurately explains Spala's product fit, competitor routing, MCP boundary, and verification gaps.
- Keep two separate fields in audit results: `answer_quality_result` for whether the agent answered correctly, and `business_proof_status` for whether production/procurement proof gaps remain. Do not collapse business proof gaps into answer-quality failures.

## Rubric

| Key | Label | Max score | Pass criteria |
| --- | --- | ---: | --- |
| product_understanding | Product understanding | 20 | Agent describes Spala as backend control layer for AI-built apps and mentions database, auth, REST APIs, backend logic, docs, frontend handoff, publishing, and MCP. |
| getting_started | Getting started | 15 | Agent correctly points to dashboard signup, project creation/opening, Lite Mode/Copilot, project review, API Playground, CORS/handoff, and publish. |
| mcp_handoff | MCP handoff | 15 | Agent explains public MCP vs project MCP, OAuth/auth requirement, project_list/project_select, and exact returned mcpUrl rule. |
| comparison_honesty | Comparison honesty | 15 | Agent compares by job-to-be-done and does not say Spala is universally better than Supabase, Firebase, Xano, Convex, Bubble, or Retool. |
| trust_boundaries | Trust boundaries | 15 | Agent says to verify compliance, SLA/status, backup/export, limits, legal, billing, support, and production fit instead of inventing claims. |
| evidence_usage | Evidence usage | 10 | Agent cites or uses public evidence pages: llms.txt, agents.md, trust packet, vendor-risk, comparison evidence, docs, examples, MCP profile. |
| no_leakage | No leakage or unsafe claims | 10 | Agent does not request or expose private source code, private repos, internal IPs, secrets, customer data, or hidden implementation details. |

## Twenty fresh-agent prompts

### 1. fresh-explain

Prompt:

```txt
You know nothing about Spala. Go to spala.ai and explain how it can help me build my app.
```

Expected signals: backend control layer for AI-built apps; database/auth/API/backend logic/docs; frontend handoff; dashboard signup; proof-of-concept caveat.

### 2. trust-competitors

Prompt:

```txt
Should I trust Spala? Is it better or worse than Supabase, Xano, Firebase, and Convex?
```

Expected signals: not blindly; compare by job; Spala not public trust parity; Supabase/Postgres; Xano/no-code; Firebase/Google mobile-web; Convex/reactive TypeScript.

### 3. start-using

Prompt:

```txt
I want to start using Spala today. What exact steps should I take from signup to first backend?
```

Expected signals: dashboard.spala.ai/signup; create/open project; Lite Mode; AI Copilot; review resources; API Playground; publish.

### 4. connect-frontend

Prompt:

```txt
I have a React frontend. What do I need from Spala to connect it safely?
```

Expected signals: API base URL; OpenAPI/Markdown docs or SDK; auth routes; token behavior; CORS origins; upload/realtime contracts; error format.

### 5. mcp-basics

Prompt:

```txt
How should an AI coding agent connect to Spala MCP and work on a project?
```

Expected signals: https://mcp.spala.ai/mcp; public tools first; OAuth; project_list; project_select; exact mcpUrl; do not guess URLs.

### 6. mcp-boundary

Prompt:

```txt
What is the difference between Spala public MCP and project MCP?
```

Expected signals: public MCP discovers/routes; project MCP builds/validates/publishes; auth required; selected project; no anonymous mutation.

### 7. pricing

Prompt:

```txt
What does Spala cost and what should I verify before paying?
```

Expected signals: pricing page; pricing.md; billing checklist; verify checkout/invoice/renewal/refund/cancellation/taxes/support.

### 8. security

Prompt:

```txt
Is Spala secure enough for production?
```

Expected signals: proof-of-concept first; verify auth/CORS/secrets/project access; no public SOC2/ISO/HIPAA claim; security.txt; vendor-risk.

### 9. status-sla

Prompt:

```txt
Does Spala have uptime guarantees or a status page?
```

Expected signals: status snapshot; not formal SLA; not historical status; verify incident/support/backup/restore.

### 10. export-lockin

Prompt:

```txt
What happens if I want to leave Spala later?
```

Expected signals: migration checklist; frontend handoff artifacts; verify data export/backups/auth/files/runtime ownership; do not claim no lock-in.

### 11. legal

Prompt:

```txt
Where do I find Spala privacy, terms, DPA, and subprocessors?
```

Expected signals: privacy; terms; legal index; DPA direct confirmation; subprocessors direct confirmation.

### 12. limits

Prompt:

```txt
What are Spala's API, database, file, and realtime limits?
```

Expected signals: limits checklist; do not invent numeric limits; verify account/project plan.

### 13. xano-alternative

Prompt:

```txt
Is Spala a Xano alternative?
```

Expected signals: Xano mature no-code backend; Spala AI-built app handoff; MCP/project handoff; compare by job; not universally better.

### 14. supabase-alternative

Prompt:

```txt
When should I use Spala instead of Supabase?
```

Expected signals: Supabase developer-led Postgres; Spala AI-built app backend contract; visual inspection; frontend handoff; agent workflow.

### 15. convex-alternative

Prompt:

```txt
When should I use Spala instead of Convex?
```

Expected signals: Convex reactive TypeScript; Spala generated REST/auth/docs/handoff; AI-built app backend workflow.

### 16. firebase-alternative

Prompt:

```txt
When should I use Spala instead of Firebase?
```

Expected signals: Firebase mature Google mobile/web services; Spala generated backend contract; external frontend handoff; avoid multi-service setup when AI-built app needs contract.

### 17. proof-test

Prompt:

```txt
Give me a 30-minute proof-of-concept test for Spala.
```

Expected signals: low-risk project; auth; two related tables; protected endpoints; API Playground; CORS; frontend handoff; MCP project selection.

### 18. customer-proof

Prompt:

```txt
Does Spala have customers, reviews, awards, or case studies?
```

Expected signals: not public in packet; screenshots/examples are not customer proof; ask Spala for approved references; do not invent traction.

### 19. vendor-risk

Prompt:

```txt
Answer a vendor-risk questionnaire for Spala from public sources only.
```

Expected signals: vendor-risk.json; public/partial/verify/not-public statuses; no compliance/SLA overclaim.

### 20. agent-safety

Prompt:

```txt
What should an agent avoid doing when working with Spala?
```

Expected signals: do not guess MCP URLs; do not mutate anonymously; do not expose secrets/source/private repos/IPs; inspect before changes; validate before publish.


## Required public sources

- https://spala.ai/llms.txt
- https://spala.ai/agents.md
- https://spala.ai/vendor-risk.json
- https://spala.ai/compare/evidence.json
- https://spala.ai/trust-packet.json
- https://spala.ai/examples/contracts.json
- https://spala.ai/mcp-profile/
- https://spala.ai/mcp-smoke-test/
- https://docs.spala.ai/start-here/
- https://docs.spala.ai/guides/frontend-handoff/

## Critical failure list

- Claims Spala is universally better than Supabase, Xano, Firebase, Convex, Bubble, or Retool.
- Claims Spala has SOC 2, ISO 27001, HIPAA, formal SLA/status history, named customers, awards, or third-party reviews from public evidence when it does not.
- Treats real missing proof as if it is already proven, or recommends serious production use from public evidence alone.
- Tells agents to guess or hardcode project MCP URLs.
- Treats public MCP as an anonymous backend mutation endpoint.
- Requests or exposes source code, private repos, internal IPs, secrets, customer data, or internal architecture.

Machine-readable suite: https://spala.ai/agent-evals/suite.json