Orient phase · steps 01–02
Step 01
Orient
Define your use case
Name the job before choosing the technology.
Build output
Leave this step with:
- Task and success criteria
- Required capabilities
- Privacy, latency, and budget limits
Build the system.
Keep it replaceable.
A practical roadmap from first use case to production. Start with one model, then add the layers that make your AI useful, safe, and replaceable.
OrientConnectExtendGroundShip
Architecture blueprint
Choose the first deployment
Keep routine or sensitive work local and escalate selectively.
Local + cloud
Route by task
Unified API
Same harness contract
Context + tools
RAG · memory · checks
Request entersVerified output leaves
Build state · 0/10 complete
Assemble one reliable layer at a time.
The roadmap
A system, in order.
Everything is open below. Scroll from step 01 to 10; mark a step complete only if you want a lightweight checklist.
01
01
02
02
Step 02
Orient
Choose a model
Compare local and cloud models against the work they must do.
Build output
Leave this step with:
- Local and cloud shortlist
- Task prompts for comparison
- Scorecard for quality, speed, cost, and privacy
03
03
Connect phase · steps 03–04
Step 03
Connect
Build the provider and inference layer
Give the harness one API for local, private GPU, and cloud inference.
Build output
Leave this step with:
- chat, generate, vision, and embedding methods
- Adapters for each provider
- Consistent errors, streaming, and model metadata
04
04
Step 04
Connect
Route work to the right model
Use a small local model for simple work and stronger models only when needed.
Build output
Leave this step with:
- Task categories and routing rules
- Fallback behavior
- A reason for every model choice
05
05
Extend phase · steps 05–06
Step 05
Extend
Choose a harness and build the agent loop
Start with an existing harness, then make plan, act, observe, verify, and finish explicit.
Build output
Leave this step with:
- Harness choice: use, modify, fork, or build
- Bounded retries and stop conditions
- One workflow tested end to end
06
06
Step 06
Extend
Add tools, MCP, plugins, and skills
Let the model reason while typed tools and portable integrations perform real work.
Build output
Leave this step with:
- Small schemas with validated arguments
- MCP or plugin boundary
- Approval rules for risky actions
07
07
Ground phase · steps 07–08
Step 07
Ground
Add RAG, memory, and vision
Ground the model with current knowledge, durable state, and visual input only where useful.
Build output
Leave this step with:
- Chunk → embed → retrieve pipeline
- Separate short-term and long-term memory
- Vision path with image limits
08
08
Step 08
Ground
Build context and permissions
Select relevant context and make tool access proportional to risk.
Build output
Leave this step with:
- System prompt + task + memory + tool results
- Token and freshness budgets
- Approval gates for delete, push, deploy, and payment
09
09
Ship phase · steps 09–10
Step 09
Ship
Verify, evaluate, then fine-tune
Use external checkers and a repeatable eval set before changing model weights.
Build output
Leave this step with:
- Compiler, tests, Python, geometry, or citations
- Quality, latency, cost, and tool-error metrics
- A diagnosed problem and rollback plan
10
10
Step 10
Ship
Observe, deploy, and optimize
Make production behavior visible, choose a deployment shape, and optimize only after it works.
Build output
Leave this step with:
- Request, provider, tool, cost, latency, and verification traces
- Local, cloud, or private GPU release path
- One measured optimization with a quality guard
Examples to study
Start with a harness that already works.
Use an existing harness, inspect its loop, then modify or fork it when your workflow needs something different.
Personal AI system
OpenClaw
A personal assistant layer that can connect to external coding harnesses and tools.
Self-improving agent
Hermes Agent
A tool-using agent with memory, skills, and support for multiple model endpoints.
Coding agent toolkit
Pi coding agent
A compact agent loop, unified model API, TUI, and coding-agent CLI you can extend.
Docs guide
Schema-first tool calling
Define a tool contract, validate arguments, authorize the action, and return a structured result.
Troubleshooting map
Fix the layer that is failing.
Before fine-tuning, identify the failure. The simplest correct layer is usually the most maintainable one.
Missing knowledge
Use RAG
Retrieve the right documents at request time.
Wrong calculation
Use a tool
Let Python or a domain engine do the math.
Wrong workflow
Fix the harness
Make state, tools, retries, and checks explicit.
Not capable enough
Change the model
Select a better model before fine-tuning.
System recipe
Model + provider + harness + context + tools + verification.