First-party Kimi K3.1 Pro workbench tests

Benchmark task: create a production plan from this fixed brief.

Brief: Build a multi-tenant analytics dashboard for a small B2B SaaS. It must ingest CSV exports, store events in Postgres, show retention and revenue charts, support role-based access, and export a monthly report.

Return:
1. Architecture decisions and trade-offs
2. Data model with tables and relationships
3. File map for a Next.js + Postgres implementation
4. Implementation milestones
5. Top five risks and mitigations
6. Starter SQL for the most important table

• 0 = misses the brief or invents requirements
• 1 = relevant but mostly generic
• 2 = addresses the brief with minor gaps
• 3 = complete, specific, and implementable
• 4 = implementable, production-aware, and clearly scoped

Benchmark task: review this fixed code and identify correctness, security, and maintainability risks.

```ts
export async function loginUser(email: string, password: string) {
  const user = await db.user.findFirst({ where: { email } });
  if (!user) throw new Error("Invalid user");
  const ok = await bcrypt.compare(password, user.passwordHash);
  if (!ok) throw new Error("Invalid password");
  const token = crypto.randomBytes(12).toString("hex");
  await db.session.create({ data: { token, userId: user.id } });
  return { token, email: user.email, credits: user.credits };
}
```

Return a table with Severity, Issue, Evidence, Why it matters, Fix, and Effort. Then list the three fixes you would make first.

• 0 = no real issues found
• 1 = finds obvious issues only
• 2 = finds security and correctness issues
• 3 = prioritizes accurate fixes with evidence
• 4 = also covers maintainability and production edge cases

Benchmark task: extract structured facts from this fixed source text.

Source: "The customer will receive 100 free credits during early access. Credits do not expire while the service remains available. The provider may throttle accounts that exceed 20 model requests per minute. Uploaded files are used only to complete the requested task and are not used for training. Either party may terminate the pilot with seven days' written notice. Paid credit packs, if introduced later, will be sold as one-time purchases rather than subscriptions."

Return:
1. Facts table
2. Obligations
3. Restrictions
4. Ambiguities
5. Questions a buyer should ask before paying

• 0 = paraphrases inaccurately
• 1 = misses key constraints
• 2 = extracts most facts
• 3 = extracts all material facts and ambiguities
• 4 = also turns the facts into practical buyer questions

Benchmark task: turn this fixed goal into an executable agent plan.

Goal: Research five competitors for an independent Kimi K3.1 Pro workbench, compare their pricing and workflow strengths, then recommend one differentiation strategy and one paid package.

For each step return: objective, required inputs, recommended tool, expected output, success criteria, and failure mode. Keep the plan executable and testable by an AI agent.

• 0 = vague brainstorm
• 1 = steps lack inputs or outputs
• 2 = executable basic plan
• 3 = clear success criteria and tool choices
• 4 = robust plan with failure modes and verification

Benchmark task: attach one UI screenshot and convert it into a frontend specification.

Return:
1. Component hierarchy
2. State and data requirements
3. Responsive behavior
4. Accessibility risks
5. Tailwind-style markup sketch
6. Questions raised by the screenshot

• 0 = invents invisible UI
• 1 = generic description
• 2 = accurate structure
• 3 = implementable spec with states and responsiveness
• 4 = production-ready accessibility and edge-case notes