Auto-scaling - San Francisco Compute Docs

Use a spot deployment to keep a target number of instances running. SF Compute buys the compute time to run them automatically. To scale, update the deployment’s target instance count. The deployment buys compute time to cover new instances and offers unneeded compute time for sale when you scale down. This pattern fits inference workers, batch workers, continuous integration (CI) runners, or any other fleet whose instance count changes over time.

Spot deployments require preview access. Contact us to enable them for your organization.

Prerequisites

SF Compute CLI installed and authenticated (sf login)
Credits on your organization (sf billing balance)
Which hardware you want (sf skus list)

Create a pool

The fleet needs a pool, the container that holds your compute allocation over time.

sf pools create --name workers

Create an instance template

Define the image and startup script for the instances in the fleet.

startup.sh

#!/bin/bash

mkdir -p /root/.ssh
cat >>/root/.ssh/authorized_keys <<"EOF"
ssh-ed25519 AAAA... you@example.com
EOF

# Start your worker process here

Create the template from the image and the startup script.

sf instance-templates create \
  --name worker \
  --image sfc:image:sfcompute:public:ubuntu-22.04.5-cuda-12.7 \
  --cloud-init ./startup.sh

See Instance templates for details on cloud-init and image configuration.

Create a spot deployment

Pass --instance-sku to pin the fleet to specific hardware; see SKUs for the catalog.

sf deployments create \
  --type spot \
  --name workers \
  --pool workers \
  --instance-template worker \
  --instance-sku sku_4UpxzQw7A8N \
  --target-instance-count 4 \
  --max-rate 20.00 \
  --min-runtime 2h

The deployment places buy orders for 4 nodes; each node runs 1 instance. As orders fill, allocation lands on the pool and instances move to running. --max-rate caps what the deployment pays, in dollars per node-hour. --min-runtime sets the minimum continuous window of compute time the deployment buys before launching each instance.

Scale up

Raise the target instance count.

sf deployments set workers --target-instance-count 8

The deployment queues 4 new instances and buys compute time to cover them. Availability depends on matching sell orders within your maximum rate.

Scale down

Lower the target instance count.

sf deployments set workers --target-instance-count 2

The deployment stops excess instances and offers their remaining compute time for sale. When a pool’s allocation drops below the number of running instances, SF Compute terminates the lowest-priority instances first. Within a priority, it terminates the newest first. To protect specific instances, transfer their allocation to a separate pool; see Protect instances from termination.

Handling interruptions

Spot compute is not guaranteed. Instances may shut down if the market price exceeds your maximum rate or other buyers place reservations that consume the capacity. Design workloads to handle instances being replaced.

Make workers stateless. Download model weights on boot; local disk does not persist between instances.
Add load balancer health checks so traffic reaches only instances that are ready.
A longer --min-runtime (for example, 6h) buys a longer guaranteed window per instance but commits more spend per purchase.

Monitoring

Check the deployment, its instances, and the orders it has placed.

sf deployments get workers            # Deployment status, target, and reconciliation
sf instances list                     # Individual instance status
sf orders list --deployment workers   # Buy and sell orders the deployment placed

Plain sf orders list omits deployment-managed orders; the --deployment filter includes them.

Next steps

To shield part of the fleet from interruption, buy a reserved block of compute time on a separate pool.

sf orders create \
  --pool workers-reserved \
  --side buy \
  --count 4 \
  --instance-sku sku_4UpxzQw7A8N \
  --start "in 5h" \
  --duration 24h \
  --max-rate 20.00

Then run a standard deployment on that pool to manage its instances.

​Prerequisites

​Create a pool

​Create an instance template

​Create a spot deployment

​Scale up

​Scale down

​Handling interruptions

​Monitoring

​Next steps

Prerequisites

Create a pool

Create an instance template

Create a spot deployment

Scale up

Scale down

Handling interruptions

Monitoring

Next steps