Your dedicated GPU machine

It works like a GPU instance on EC2. You SSH into your own container and run what you want, and your API is served at this address over HTTPS.

What you have

The machine is reserved for Risika only. It is not shared with other customers.

Get started

1. Log in

   Once we have added your SSH public key:

   ssh -p 2601 root@193.181.215.114

   Host key fingerprint (ED25519): SHA256:o3qDRo9yFJoirUhFjN7p/LUEXYVTnm11qK2G1Vu2tfA. To use just ssh herd, add this to ~/.ssh/config:

   Host herd
     HostName 193.181.215.114
     Port 2601
     User root

2. Start a model

   The Mistral Small 3.2 weights are already downloaded. To serve it in FP8 across both GPUs:

   /workspace/examples/mistral-small.sh

   The first start takes about 3 minutes while vLLM compiles. When it's ready, the status pill at the top of this page turns green. The very first request after a start is slow (up to a minute). After that, expect around 40 tokens/s. Run it inside tmux to keep it running after you log out, or see "Keep it running" below.

3. Call your API from Risika

   Every request needs the API key we sent you separately, as a bearer token. The endpoint is OpenAI-compatible:

   Python curl

   import os
   from openai import OpenAI
   
   client = OpenAI(
       base_url="https://risika.herdinfrastructure.com/v1",
       api_key=os.environ["HERD_API_KEY"],
   )
   r = client.chat.completions.create(
       model="mistralai/Mistral-Small-3.2-24B-Instruct-2506",
       messages=[{"role": "user", "content": "Hello!"}],
   )
   print(r.choices[0].message.content)

   Anything you serve on port 8000 in your container is available here, not only vLLM. Authentication is handled by our gateway, so don't add --api-key to vLLM.

Keep it running

If /workspace/start.sh exists and is executable, it runs every time the container starts, so your API comes back on its own after a restart. Its output goes to /workspace/start.log.

cp /workspace/examples/mistral-small.sh /workspace/start.sh

What persists

Install your own packages into a venv on /workspace, so they survive updates:

python3 -m venv --system-site-packages /workspace/venv
source /workspace/venv/bin/activate

Good to know