# Save Your Work Before You Destroy a Cloud GPU: Checkpoints, rsync and Hub Uploads (2026)

**On CloudGPU, destroying an instance deletes its disk, and there is no persistent storage across instances yet, so anything you want to keep — checkpoints, LoRA adapters, generated images, logs — must be copied off the machine before you destroy it.** An instance is also destroyed automatically if your balance runs out. This post is the checklist and the commands we use.

## What happens to your files, and when

| Event | What happens to the disk |
| --- | --- |
| You click Destroy | The instance stops, billing settles, the disk is deleted |
| Your balance cannot cover the next hour | The instance is destroyed and its disk deleted |
| You want to “pause” and resume tomorrow | Not available yet: stopping releases the GPU; you start a new instance next time |
| The JupyterLab tab or SSH session closes | Nothing — the instance keeps running (and billing) until destroyed |

Because billing is per minute, the economical pattern is: work, copy outputs off, destroy. The minutes spent copying are billed like any other minute, so plan the copy rather than leaving the instance up “just in case”.

## Pick a destination

| Destination | Good for | Command family |
| --- | --- | --- |
| Your own computer | Small to medium outputs, anything private | `rsync` / `scp` over SSH |
| Hugging Face Hub (private repo) | LoRA adapters, merged models, datasets you will reuse | `huggingface-cli upload` |
| S3-compatible object storage (S3, R2, B2…) | Large checkpoints, image batches, logs | `rclone copy` |
| JupyterLab download button | A handful of small files | Browser |

Do **not** push large files through the instance’s public API endpoint: the tunnel is capped at 3 MB/s and is meant for API responses, not file transfer. Use SSH or an upload from the machine.

Outbound speed from the supplier data centre to a given destination varies. Test with a small file first and add the copy time to your budget.

## Commands

### 1. Pull with rsync from your laptop (resumable)

```bash
# run on your laptop; user, host and port are on the instance page
rsync -avP -e "ssh -p <ssh-port>" <ssh-user>@<ssh-host>:/workspace/out/ ./run-2026-10-23/
```

`-P` shows progress and resumes partial files if the connection drops. Re-run the same command at the end; rsync only copies what changed.

### 2. Keep pulling during a long run

```bash
# laptop: pull new checkpoints every 10 minutes while training runs
while true; do
  rsync -aP -e "ssh -p <ssh-port>" <ssh-user>@<ssh-host>:/workspace/out/ ./run/ && date
  sleep 600
done
```

Pair it with frequent saves in your trainer (for LLaMA-Factory, `save_steps` in the YAML; for Hugging Face `Trainer`, `save_steps` and `save_total_limit`).

### 3. Push a LoRA adapter to a private Hugging Face repo

```bash
pip install -U "huggingface_hub[cli]"
huggingface-cli login            # paste a fine-grained write token
huggingface-cli upload <your-user>/qwen3-8b-qlora-run1 /workspace/out/qwen3-8b-qlora . --private
```

Use a fine-grained token scoped to the one repo and revoke it after the run.

### 4. Copy a large output folder to object storage

```bash
curl https://rclone.org/install.sh | bash          # you are root on the instance; or: apt-get install -y rclone
rclone config                                       # add your S3 / R2 / B2 remote
rclone copy /workspace/out remote:my-bucket/run1 --progress --transfers 8
```

### 5. Tar first when there are many small files

```bash
tar -C /workspace -czf /workspace/outputs.tgz out/
```

Thousands of small PNGs or log files copy much faster as one archive.

## The pre-destroy checklist

| # | Check | Why |
| --- | --- | --- |
| 1 | Training or batch has finished (or you saved a checkpoint) | Partial runs are lost on destroy |
| 2 | Outputs copied and **verified** (file count, sizes, or `sha256sum`) | A failed copy looks like a successful one until you need it |
| 3 | Configs, scripts and the exact command line saved | So you can reproduce the run on a fresh instance |
| 4 | Environment captured (`pip freeze > requirements.txt`) | Next instance starts clean |
| 5 | Tokens you pasted on the machine revoked | Good hygiene, even though the disk is deleted |
| 6 | Click Destroy | Unused minutes of the pre-authorised hour return to your balance |

## Budget for the copy, and for the balance

Two numbers keep long runs safe:

* **Balance buffer.** An instance is destroyed when the balance runs out. For an 8-hour run on a card listed at \$0.65/h (RTX 4090 48G, 2026-10-23 23:58 Beijing time), the run itself is \$5.20; keep that plus a margin for setup and copying.
* **Copy time.** Estimate `size ÷ measured upload speed`, then add it to the job. On a card listed at \$0.59/h, 10 extra minutes of copying cost about \$0.10.

Stock and prices change during the day; check the [Deploy page](https://cloudgpu.app/deploy). Billing rules are on [per-minute billing](https://cloudgpu.app/gpu-rental-per-minute-billing).

## A note on where your data goes

If your data must not leave a particular jurisdiction, do not train on it here, and choose your copy destination with the same care.

## FAQ

**What happens to my files when I destroy a cloud GPU instance on CloudGPU?**

The disk is deleted. Copy anything you need off the machine before you destroy it.

**Is there persistent storage between instances?**

Not yet. Stopping releases the GPU and the next session starts on a new instance, so keep checkpoints and outputs elsewhere.

**What if my balance runs out during training?**

The instance is destroyed and its disk deleted. Keep enough balance for the whole run plus a margin, and copy checkpoints off as you go.

**How do I download checkpoints from a rented GPU?**

Use rsync or scp over SSH, or upload from the machine to a model hub or object storage. Do not use the public API endpoint, which is capped at 3 MB/s.

**Am I billed while I copy files off?**

Yes. Billing runs per minute until you destroy the instance, so plan the copy and destroy promptly afterwards.

---

If the checkpoints you are saving come from a fine-tune, the [LoRA and QLoRA fine-tuning page](https://cloudgpu.app/rent-gpu-for-lora-fine-tuning) covers card choice and billing for those runs. *Originally published at [cloudgpu.app](https://cloudgpu.app/blog/save-work-before-destroying-cloud-gpu-2026).*

> Prices are list prices read on 2026-10-23 23:58 Beijing time (UTC+8); current prices on [/deploy](https://cloudgpu.app/deploy) take precedence.