I don't prompt agents anymore...

What is Graph Engineer00:00

The term "graph engineer" has become confusing because people are mixing three different concepts under the same term. Peter Wei's tweet about shifting from loops to graphs was the catalyst, but the discussion became muddled as people conflated control graphs, knowledge graphs, and graph of loops.

Three Types of Graphs Being Discussed00:30

Control Graph

Control graph refers to dynamic workflow systems like LangGraph or dynamic workflow features. These are essentially SOPs (standard operating procedures) that ensure agents produce more reliable results. This is the graph Peter Wei was actually referring to, as his use case involved shipping code changes with defined control flow.

Knowledge Graph

Knowledge graphs represent relationships between different entities for more effective data retrieval. This concept is completely different from control graphs but got conflated when Andrew Ng released a one-hour course on building knowledge graphs. Companies like Graphiti have been shipping with graph memory for years.

Graph of Loops

This concept, brought forward by Colors, addresses how to manage multiple loops running simultaneously in future companies. The challenge is ensuring all loops execute well and improve continuously, since errors and improvements compound quickly across multiple loops. This remains an unexplored space with no established solutions yet.

The Shift from Human Prompting to Agent Prompting03:00

The fundamental shift occurring is teams moving from humans prompting agents to having other agents prompt agents on behalf of humans. Two patterns are emerging:

Loops Pattern: Instead of human prompting for every task, triggers can be:

• Time-based (daily GitHub issue fixes)
• Goal-based (continuous frontend optimization)
• Event-based (email-triggered agent responses)

Humans only become involved when agents determine it's necessary.

Octra Trader Pattern: Humans communicate with an orchestrator agent that has full context and can spin up teams of agents for execution and monitoring. Boris (creator of Claude Code) noted he now talks to Claude who talks to other Claudes rather than prompting agents directly.

The Need for Reliability06:30

As humans move one layer above actual work, reliability becomes critical. Setting up verifiers in codebases provides confidence about what agents are shipping. However, verifiers are just one component of the guardrails needed for different business processes.

Control Graph Components07:00

Control graph-based workflow automation has existed for decades with three key components:

• Nodes: Actions the system will take
• Edges: What happens next after each node
• State: Data carried over between nodes/steps

The innovation is what nodes can now represent - from simple scripts to agents capable of complex tasks.

Existing Graph Examples08:00

Claude Code's Go Feature

This is implemented as a graph using agent loops plus stop hook system. The graph receives user prompts, runs through agent loops, then uses an LLM to classify if the goal is satisfied.

Andrew Copsy's Auto Research

This project puts agents in a for-loop to continuously improve language model systems. The process involves:

• Receiving program.md file with process instructions
• Agent updates train.py with experimental ideas
• Runs experiments and evaluates results
• Keeps changes if results improve, otherwise discards and tries new ideas

This achieves a graph through simple prompting without fancy mechanisms.

Deep Research Pattern

Research agents typically follow this graph:

• Receive prompt
• Orchestrator plans topics to cover
• Fan out sub-agents for research
• Review if information is sufficient
• Generate report if complete, otherwise return to research step

Two Implementation Methods11:00

Large Model as Graph

Modern models can understand SOP workflows described in text, JSON, or Mermaid charts. SOPs can be documented as skills that agents invoke as context.

Code as Graph

Traditional code-based approaches using dynamic workflow from Claude Code/CodeX or LangGraph for building graph workflows.

Design Patterns for Large Model as Graph12:30

When implementing SOPs as skills, several design patterns are crucial:

Separate Agent Nodes: While models are improving, it's important to separate nodes because models aren't great at self-verifying their own work. Verifier agents should be separate nodes. For complex tasks, dedicated planner agents focused on reasoning are necessary.

Use Code When Appropriate: Scripts should handle complex data fetching, dev server setup, result evaluation, and end-to-end tests.

Define Input/Output Boundaries: Clear expectations for each agent node ensure proper workflow execution.

State Management: A markdown file documenting latest status and a pen-only log from each agent provides quick state understanding across different nodes.

Super Design Implementation Examples15:00

Large Model as Graph: Daily Bad Design Triage

This loop reviews user-generated designs daily to identify bad designs for an evaluation dataset. The process:

• Daily agent filters designs using heuristic indicators
• Pulls raw data from database for potential candidates
• Runs deterministic scripts for obvious issues (errors, broken designs, non-responsive)
• Spins up sub-agents for vision logic model judging on remaining designs
• Main agent ranks worst designs and adds to daily report

Implementation uses a skill documenting the procedure alongside scripts for deterministic steps, with output contracts and guardrails.

Code as Graph: Ship Change Workflow

Dynamic workflow breaks this into three phases:

• Setup and implement
• Verify the work
• Simplify and PR if verification passes

Dynamic workflow works by defining schemas for each agent session, allowing easy construction of prompts for subsequent nodes using output from previous sessions.

Key Recommendations21:00

• Breaking down tasks alone won't work without giving agents proper tools to test things
• Verifier setup skills are available in the public AI Builder repo
• Large model as graph is preferred for most cases due to conversation resumption limitations in current dynamic workflow implementations
• Code as graph works better for extremely large tasks