Cheva Labs Logo
Cheva Labsweb

Run your first AI model

Learn how to run AI on a laptop, a headless server, or through a hosted API.

From a blank terminal to your first answer.

You do not need to train a model. You only need a runner, a model, and a prompt. Pick the path that matches your computer and follow the steps in order.

Run on my computer Call an API
You

A prompt

Runner

Local or cloud

Result

An answer

Local:your files stay on your machine.
API:a provider runs the model for you.

Choose your first path

Both paths use the same basic idea. The difference is where the model runs.

Path A · local

Run AI on your computer

Best when you want privacy, offline use, and no per-message bill. You will install Ollama and run a small model from your terminal.

PrivateNeeds RAMNo API key

Path B · API

Call a model over the internet

Best when your computer is not powerful enough or you want the fastest start. You will create a key and send one request to OpenRouter.

Easy startInternet neededWatch cost

Run guides

Short routes for the most common ways to run an AI model.

01

Your first local chat

Install Ollama and ask a model a question on your own computer.

02

Pick a local runner

Compare easy apps, portable engines, GPU servers, and low-memory tools.

03

Run AI headlessly

Keep the model on a Linux server and reach it safely over SSH.

04

Your first API call

Create one key, call a hosted model, and read the answer in Python.

Before you start

You only need these three ideas.

Runner

The app that loads a model and sends it your prompt. Ollama is one runner.

Model

The trained file that reads your words and predicts a useful answer.

API

A web address your program can call to ask another computer for an answer.

Path A · local

Run AI on your computer

We will use Ollama because it has a short install path, works on macOS, Windows, and Linux, and gives you a local API after it is installed.

About 10 min
  1. 1

    Install Ollama

    Download it for your operating system, install it, and open a new terminal window.

    Download Ollama

  2. 2

    Run a model

    This command downloads the model the first time. Then it opens a chat in your terminal.

    terminalbash
    1ollama run gemma4
    When you see a prompt, type: Explain what a black hole is in two sentences. You are now running AI locally.
  3. 3

    Use the local model from Python

    Ollama serves a local API at localhost:11434. Install the small requests library, then run this file.

    terminalbash
    1python -m pip install requests
    local_request.pypython
    1import requests
    2 
    3response = requests.post(
    4 "http://localhost:11434/api/generate",
    5 json={
    6 "model": "gemma4",
    7 "prompt": "Explain gravity in two short sentences.",
    8 "stream": False,
    9 },
    10)
    11 
    12print(response.json()["response"])
What just happened?

Ollama loaded the model into your computer's memory, gave it your prompt, and returned the answer. The Python code talks to the same local service instead of opening a website.

Popular local runners

A runner is the program that loads a model. Pick one by the job you want to do.

Ollama

Best first step

A friendly command-line app for downloading and running models locally.

Good for

First local chat

ollama run gemma4

Open docs

LM Studio

Best visual app

A desktop app for downloading models, chatting, and starting a local server.

Good for

No-terminal beginners

Developer → Start server

Open docs

llama.cpp

Most portable

A small C/C++ engine that runs GGUF models across CPUs, GPUs, and edge devices.

Good for

Control and portability

llama-server -hf ggml-org/gemma-3-1b-it-GGUF

Open docs

vLLM

GPU server

A high-throughput server for serving models to many users through an OpenAI-style API.

Good for

NVIDIA GPU servers

vllm serve Qwen/Qwen2.5-1.5B-Instruct

Open docs

SGLang

High throughput

A production-focused serving framework for low-latency and large concurrent workloads.

Good for

Advanced GPU serving

python3 -m sglang.launch_server --model-path MODEL

Open docs

LocalAI

Docker + API

A self-hosted API that can run text, image, audio, and other models on hardware you control.

Good for

One private API

docker run -p 8080:8080 localai/localai:latest

Open docs

Jan

Desktop + API

A local-first desktop app that can also expose a model through an OpenAI-compatible API.

Good for

Learning with a GUI

jan serve MODEL_ID

Open docs

AirLLM

Low VRAM

A specialist Python library that loads model layers one at a time to reduce GPU memory use.

Good for

Memory-limited experiments

pip install airllm

Open docs

Run AI on a headless server

A headless server has no monitor or keyboard. You connect with SSH, start the runner there, and call it from your laptop or app.

  1. 1

    Connect to the server

    Use a Linux machine, home server, or rented GPU server. Replace the username and address with yours.

    your-laptopbash
    1ssh user@SERVER_IP
  2. 2

    Start one runner on the server

    Use Ollama for a simple server, or vLLM when you have a compatible GPU and want an OpenAI-style API.

    serverbash
    1# Choose one runner
    2 
    3# Simple local server
    4curl -fsSL https://ollama.com/install.sh | sh
    5ollama pull gemma4
    6OLLAMA_HOST=127.0.0.1:11434 ollama serve
    7 
    8# GPU server
    9python -m pip install vllm
    10vllm serve Qwen/Qwen2.5-1.5B-Instruct --host 127.0.0.1 --port 8000
  3. 3

    Open a safe tunnel from your laptop

    Keep the model listening on 127.0.0.1 on the server. This SSH tunnel lets only your laptop reach it.

    your-laptopbash
    1# Ollama tunnel
    2ssh -N -L 11434:127.0.0.1:11434 user@SERVER_IP
    3 
    4# vLLM tunnel
    5ssh -N -L 8000:127.0.0.1:8000 user@SERVER_IP
Do not expose an open model port

Avoid opening port 8000 or 11434 to the public internet without authentication and a firewall. The SSH tunnel is the simple safe starting point.

Path B · API

Call an AI model over the internet

We will use OpenRouter because one API gives you access to many models. Start with its free model router for learning, then choose a specific model when you know what you need.

About 10 min
  1. 1

    Create an API key

    An API key is like a password for your program. Create one in OpenRouter, and set a small spending limit if you add paid credits.

    Create an OpenRouter key

  2. 2

    Keep the key outside your code

    Put it in your terminal as an environment variable. Do not paste a real key into a GitHub repository or a browser app.

    terminalbash
    1export OPENROUTER_API_KEY="paste-your-key-here"
    If your key ever appears in a screenshot, chat, or code repository, revoke it and create a new one.
  3. 3

    Send your first request

    Install requests, run this file, and read the answer printed in your terminal.

    terminalbash
    1python -m pip install requests
    api_request.pypython
    1import os
    2import requests
    3 
    4response = requests.post(
    5 "https://openrouter.ai/api/v1/chat/completions",
    6 headers={
    7 "Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}",
    8 "Content-Type": "application/json",
    9 },
    10 json={
    11 "model": "openrouter/free",
    12 "messages": [
    13 {
    14 "role": "user",
    15 "content": "Explain gravity in two short sentences.",
    16 }
    17 ],
    18 },
    19)
    20 
    21print(response.json()["choices"][0]["message"]["content"])
What just happened?

Your Python program sent a request to OpenRouter. OpenRouter picked a free model, waited for its answer, and sent the answer back as JSON.

Popular API providers

These services host models for you. Most use an API key and charge by usage, while some offer limited free access.

OpenRouter

Best first API

Model API

One endpoint for many models. Change the model name instead of rewriting your app.

Good for

Comparing models

Open docs

Hugging Face

Model catalog

Model API

Try open models on the Hub and call supported providers from one client.

Good for

Open model discovery

Open docs

Together AI

Open models

Model API

Hosted open models with official Python and TypeScript SDKs and a quick first request.

Good for

Open-weight apps

Open docs

Fireworks AI

Hosted inference

Model API

An OpenAI-compatible API for serving open models with extra controls for production use.

Good for

Structured output

Open docs

Groq

Speed focused

Model API

A fast hosted API with familiar chat-completion calls and a simple Python SDK.

Good for

Fast responses

Open docs

Replicate

Many model types

Model API

Run models from a web playground or Python API, including image, video, audio, and text.

Good for

Trying different tasks

Open docs

Cerebras

Very fast

Model API

A hosted inference API with a short Python quickstart and a developer playground.

Good for

Low-latency chat

Open docs

RunPod

Bring your GPU

GPU platform

Rent GPU machines or deploy serverless workers when you want more control than a model API.

Good for

Custom servers

Open docs

Modal

Code-first GPU

GPU platform

Write a Python function and let Modal provide the containers and GPUs behind it.

Good for

Python experiments

Open docs

Local or API?

There is no universally correct choice. Start with the row that sounds like your project.

QuestionLocalAPI
Where does it run?Your computerA provider's computer
What do I pay?Your hardware and electricityUsually per request or token
What is easiest?Ollama + one terminal commandOpenRouter + one API key
Best first projectPrivate notes or an offline helperA web app or a quick prototype

When your first run fails

Most beginner problems have a simple cause. Check these before changing code.

“Command not found”

Close and reopen your terminal after installing Ollama. Your terminal may not know about the new program yet.

It is very slow

The model may be too large for your memory. Stop it and try a smaller model from the provider's model list.

401 or missing key

Start a new terminal after setting the environment variable, and check that the key name matches your code exactly.

Good next steps

One working request is enough for today.

Once the example works, change one thing at a time: use your own prompt, choose another model, then move the request into a small app.

Run it again

Sources checked

Provider docs used for this guide

Commands and endpoints can change. Use the official docs when you move from a first experiment to a real project.

OllamaLocal quickstart and APIllama.cppPortable local enginevLLMGPU serverSGLangHigh-throughput servingAirLLMLow-memory inferenceLocalAISelf-hosted APIOpenRouterMulti-model APIHugging FaceOpen model providersTogether AIHosted open modelsReplicateMany model typesRunPodServerless GPUModalCode-first GPU