HyperAgent Field Notes #1: Setup, First Skill, and the Lesson from Day One

Field Notes series — We are part of the closed beta of HyperAgent. In this series we share what we learn, what works and what doesn't. Today: day one.

In 30 Seconds

• Onboarding is fast: workspace, first model routing, Slack integration ≈ 15 minutes.
• First skill ≠ dream project. Start with something small, repetitive, with a clear definition of done.
• Build the eval rubric BEFORE the first run. Otherwise you can't measure improvement later.
• Biggest mental shift: you're not building a bot, you're building a role with a job description.

What we did

We had two hours for a first session. Plan: set up the workspace, identify a productive use case, build the first skill, ship the first run.

1. Setup (≈ 15 min)

• Create workspace, invite team
• Pick default model (we start with Claude for reasoning, a GPT-4-class model for tool calls)
• Connect Slack workspace – we want triggers from the start
• A test run against a browse task: works, returns a clean trace

Nothing exciting yet. The platform feels like a very polished studio IDE for agents.

2. Use-case selection: this was the important decision

Our first reflex: "Let's build a sales research agent!" – the ambitious use case with the wow effect.

We dropped it. Instead we asked:

Which task does someone on our team do multiple times per week manually, with a clear output format and little context dependency?

Answer: competitor watchlist update. Every Monday we look at 8 competitor domains, check blog/pricing/changelog for changes since last week, summarize that into a short Slack message.

Small. Clear. Repeated. Perfect for a first skill.

3. The first skill

We defined a skill competitor-watchlist-scan. Roughly:

• Input: list of URLs + last scan timestamp
• Steps: fetch each URL, diff against snapshot, distinguish "real" changes from dynamic content
• Output: structured list { domain, change_type, summary, evidence_url }
• Definition of done: Slack post with at most 5 bullet points, each linked

Writing the skill took longer than expected (≈ 30 min) – not because of the platform, but because we noticed: we had never cleanly documented this process in our own heads. That's exactly the value. HyperAgent forces you to codify your implicit process knowledge.

4. Eval rubric before the first run

We almost made a mistake here: just run it, then look.

Instead we defined a simple rubric:

LLM-as-Judge scores every run against this rubric. Result: from run to run we can objectively measure whether a skill update is better or worse – instead of guessing.

5. The first real run

First run: 78% score. Three false positives (CSS changes were flagged as "pricing update"), one domain was blocked by a cookie banner.

Both are clear signals for what to improve in the skill. Exactly the loop we wanted.

Three takeaways from day one

1. Skills aren't fancy prompts – they're process documents

We underestimated how much explicit process description a good skill needs. The good news: what you write down once runs 1,000 times after that. Skills are onboarding documents for machines.

2. Eval first – or you optimize blind

Without a rubric, after three runs you have no idea whether you got better or worse. With a rubric you see trends. Sounds obvious, biggest gamechanger.

3. Starting small was right

A "small" skill that runs reliably every week is worth more than a "big" skill that impresses once and then breaks. Consistency first, complexity later.

What's next

In Field Notes #2 we'll cover how this skill becomes a deployable role – with a Slack trigger, budget limit, and permission scope. In Field Notes #3 we then orchestrate multiple roles as a fleet with hand-off contracts and concurrency limits.

→ Field Notes #2: From skill to deployable role → Field Notes #3: From role to fleet → HyperAgent tool overview → HyperAgent full review → The 5 building blocks of an AI agent → Agentic Engineering – how we partner with teams