Using It From Anywhere

At the end of lesson five you had a model answering questions, and it stops the moment you close that terminal window.

This lesson fixes four things, in the order they matter.

Link to Make it survive a rebootMake it survive a reboot

Right now the server is running because you typed a command. Close the window and it dies. Restart the machine and it stays dead.

The fix on Linux is a service, which is a job the operating system starts on its own, keeps an eye on, and restarts if it crashes. Everything running in the background on your machine is a service.

You describe one in a small text file. Make the folder it lives in:

mkdir -p ~/.config/systemd/user

mkdir makes a folder and -p creates any missing folders along the way. The ~ is shorthand for your home folder.

Then create the file:

nano ~/.config/systemd/user/llama-server.service

nano is a simple text editor that runs in the terminal. Paste in the following, then press Ctrl-O and enter to save, and Ctrl-X to leave.

[Unit]
Description=llama.cpp server
 
[Service]
Type=simple
ExecStart=/home/<your username>/llama.cpp/build/bin/llama-server \
  --model /home/<your username>/models/<your model>.gguf \
  --host 0.0.0.0 --port 8080 \
  --ctx-size 32768 --spec-type draft-mtp
Restart=on-failure
RestartSec=5
 
[Install]
WantedBy=default.target

Replace both <your username> placeholders with your actual username. Run whoami if you are unsure.

What the parts mean:

• [Unit], [Service], [Install] are section headings. The file is read in three parts: what this is, how to run it, and when to start it.
• ExecStart is the command to run, the same one from lesson five with full paths instead of relative ones. A service has no idea which folder you were standing in.
• Restart=on-failure restarts it if it crashes.
• RestartSec=5 waits five seconds first, so a broken configuration does not spin in a tight loop.
• WantedBy=default.target means start this when the user session normally starts things.

Now switch it on:

systemctl --user daemon-reload
systemctl --user enable --now llama-server
loginctl enable-linger $USER

• daemon-reload tells the system to re-read its service files, since you just added one.
• enable --now does two jobs: start it immediately, and start it again on future boots.
• enable-linger matters more than it looks. Without it your services only run while you are logged in, so after a reboot with nobody signed in the model does not come back and nothing tells you why. $USER fills in your own username automatically.

You might expect a line telling it to wait for the network. There is not one, because the per-user part of the system has no network signal to wait on. Restart=on-failure covers it: if the network is not ready, the server exits and gets started again five seconds later.

From now on:

systemctl --user status llama-server    # is it alive
systemctl --user restart llama-server   # after changing settings
systemctl --user stop llama-server      # free the memory

And when it does not start, this shows you why:

journalctl --user -u llama-server -n 50

journalctl reads system logs and -n 50 shows the last fifty lines. Run it first whenever something misbehaves.

Link to Reach it from anywhereReach it from anywhere

Your Spark sits on your home network. Your phone, once you leave the house, does not.

The old answer was opening a hole in your router, which is fiddly and exposes the thing to the entire internet. The modern answer is a mesh VPN.

A VPN, virtual private network, makes distant machines behave as though they share one small private network. A mesh one connects your own devices to each other rather than routing everything through one central server. The easiest is Tailscale.

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

The first line downloads Tailscale's installer and runs it immediately. If you have been told not to run scripts off the internet, that is a good instinct. What makes it acceptable here is that the address is Tailscale's own domain over an encrypted connection, and you can paste that URL into a browser and read the script first if you would rather see it.

The second line prints a link. Open it, sign in, and the machine joins your private network. Install the Tailscale app on your phone, sign in with the same account, and you can reach the model from anywhere.

Most of the time your devices connect straight to each other. When a network blocks that, Tailscale passes the traffic through its own relay servers instead. Either way it works, and either way it is encrypted end to end, so the relay cannot read it.

One benefit worth calling out. Home networks hand out addresses that change, and local names sometimes stop resolving. A Tailscale address stays with the machine for as long as it stays registered. Remove it from your account, reinstall Tailscale, or wipe the disk and it gets a new one, but nothing your router does will move it. That removes a whole category of "it worked yesterday" problems.

The catch, and it is a real one. Tailscale connects your devices. It does not keep your machine alive. The Spark still has to be powered on and have working internet at home. If your router dies while you are away, your phone has nothing to connect to and no way to fix it.

If you plan to rely on this while travelling, a battery backup covering the Spark and the router is what makes it trustworthy. Backing up only the Spark gives you a running machine you cannot reach.

Link to Put a password on itPut a password on it

Until now your server has no authentication at all. Anyone who can reach that port can use your model.

On a home network that is contained. Once it is reachable from anywhere, add a key.

echo "pick-a-long-random-string" > ~/.llama-api-key
chmod 600 ~/.llama-api-key

echo prints text, and > sends that text into a file instead of onto your screen. chmod 600 sets the file's permissions so only you can read it. The 600 is a compact way of saying "owner can read and write, nobody else can do anything."

Then add this to your service file's command and restart:

--api-key-file /home/<your username>/.llama-api-key

Use the file version, not the plain --api-key flag. Anything you type on a command line is visible to every user on the machine through the list of running processes. A file with locked-down permissions is not.

systemctl --user daemon-reload
systemctl --user restart llama-server

One thing that will catch you. The moment the key is live, your browser tab stops working and starts returning errors, because it does not know the key yet. That is expected. Open Settings in the chat interface, paste the key in, and it works again. Do the same on your phone.

Link to See what it is doingSee what it is doing

Add one more flag to the command and the server starts publishing statistics about itself:

--metrics

Then you can read them:

curl -H "Authorization: Bearer <your key>" http://localhost:8080/metrics

curl fetches a web address from the terminal instead of a browser. The -H part sends your key along, because the metrics sit behind the same password as everything else.

What comes back:

The llamacpp: prefix is part of the name. Searching for the bare name finds nothing.

For most people that is enough. If you want graphs over time rather than a snapshot, the format those numbers are in is a standard one called Prometheus, and pairing it with a graphing tool called Grafana is the usual route. Both are open source and both run on the Spark itself. That is a project in its own right rather than a step in this lesson.

The simpler daily check is the log, which prints the speed of every request:

journalctl --user -u llama-server -f

-f follows the log live, so you watch requests as they happen. Press Ctrl-C to stop.

Link to Where to go nextWhere to go next

You now have a model that runs privately, survives reboots, is reachable from anywhere, and is behind a password.

Reasonable next steps, roughly in order of value:

Point something at it. The server speaks the same request format as the big commercial APIs, so most tools built for those can be aimed at your machine by changing one address. Do that and this stops being a toy.

Run more than one model. A tool called llama-swap sits in front of several and loads them on demand, so only what you are using takes up memory.

Find the limits. Try a model much larger than seems sensible and watch where it slows down. The arithmetic from lesson three tells you what to expect, and checking a prediction against reality is the fastest way to make it stick.

The mental models here transfer. Capacity against bandwidth, reading fast and writing slow, weights plus conversation cache. Those hold on any machine you ever run a model on, rented or owned.