Automatic scaling with Auto-Scale and a Load Balancer

Traffic in SaaS products is rarely flat. Campaigns, month-end reports or onboarding a new customer create sudden spikes. This guide walks through moving your application tier on Bulutlan Edge Cloud to a setup that scales automatically with Auto-Scale and a load balancer, at a conceptual level.

How it works

The setup has two parts:

• Load balancer: Distributes incoming traffic across the servers behind it. On Edge Cloud, the load balancer is built into the platform (elastic), with no extra hardware, licenses or complex configuration. L3 and L7 options are available.
• Auto-Scale: Adjusts the size of a server group to match the load. You define minimum, desired and maximum capacity, and the group grows and shrinks within those limits.

When a new server comes online, the load balancer starts sending traffic to it. When load drops, extra servers shut down, and with hourly billing you pay only for the time they ran.

Before you start

Automatic scaling requires an application that is ready to scale out:

• Stateless application: Session and user data should live in a database or a shared layer, not on the server's local disk.
• Shared files: Files uploaded by users should be stored in a shared layer such as Object Storage.
• Automatic startup: When a new server boots, the application should start running without manual steps.
• Health check: There should be a simple endpoint that shows the application is healthy.

1. Prepare a template server

Set up a server running your application, configure it and test it. Then create an image from that server. Since every new server in the Auto-Scale group will boot from this image, it matters that the image is current and clean.

2. Create the load balancer

Open the load balancer wizard in the panel and select the network your application runs in. Choose the layer based on the type of traffic:

• L7: Suited to HTTP and HTTPS traffic; routing decisions can be based on the content of the request.
• L3: Suited to cases that need distribution at the network layer, independent of the application protocol.

Define the health check; whether unhealthy servers stop receiving traffic depends on this setting.

3. Define the scaling group

In the Auto-Scale wizard, choose the template image, the server profile and the network. Then set the capacity limits:

• Minimum capacity: The number of servers that run no matter how low the load is. For high availability, at least two servers are recommended.
• Desired capacity: The target number of servers under normal conditions.
• Maximum capacity: The upper limit that is never exceeded, however high the load. It keeps costs from running away in an unexpected situation.

4. Set the scaling rules

Define the rules that decide when the group grows and shrinks. You will see the available metrics and thresholds in the wizard. When setting thresholds, make scale-out decisions fast and scale-in decisions more cautious; otherwise the group may keep growing and shrinking at short intervals.

5. Attach the group to the load balancer

Connect the scaling group to the load balancer you created. Every new server added to the group then starts receiving traffic automatically, and servers that shut down are removed from rotation.

6. Test it

Run a load test before going live:

• Confirm that new servers start and begin receiving traffic as load increases.
• Check that the group returns to minimum capacity as load decreases.
• Stop one server and watch the load balancer shift traffic to the others.

Common mistakes

• Setting minimum capacity to one: in a group running a single server, any problem with that server stops the service.
• Leaving maximum capacity open-ended: a faulty rule or unexpected traffic can start more servers than you need. Always set the upper limit deliberately.
• Scaling with an old image: when you update the application, update the template image too; otherwise new servers come up on the old version.
• Writing to the local disk: data left on the local disk of a server that shuts down is lost. Keep persistent data in the database or in Object Storage.

Cost

Servers added by Auto-Scale are billed only for the hours they run. Compared with a fixed fleet sized for your busiest moment, a group that runs at minimum capacity and grows when needed usually delivers the same service quality at a lower cost. You can model your scenario in the Pricing Calculator.

With Terraform

You can define the same setup as code with Terraform. According to RNG Technology, the load balancer can be applied automatically with Auto-Scale or Terraform. See the Terraform guide for details.

Tips for SaaS

• In private SaaS environments for enterprise customers, use a separate scaling group and load balancer per customer.
• For higher availability, consider an active-active architecture across the İstanbul and Konya regions.

Related pages: Auto-Scale, Load Balancer.