Serverless Architecture

This is the third of a five-part series on Branchstack, the approach behind Platter's latest Branching Postgres product. Read the series introduction here and read the previous post about Environmentless development here

As applications have moved away from self-hosted, on-premise machines to cloud vendors, software developers have responded by adapting their application architecture around a cloud-hosted execution model called "serverless" computing. Instead of managing the hardware requirements behind their code, developers can now build modular codebases that scale up and down with use automatically.

In this post, we'll learn about Serverless architecture as a pattern, then we'll explore how this architectural pattern is integral to Branchstack methodology.

Serverless Computing

"Serverless" is a term that has gotten a lot of traction recently from JAMstack providers like Netlify and Vercel. The "A" in JAMstack refers to APIs that are either provided by a third-party (like Auth0 or Stripe) or included as a part of the application itself (usually written in "J"-for-JavaScript functions that respond to HTTP requests). These modular functions are then deployed to providers like AWS (as Lambdas), GCP (as Cloud Functions), or Cloudflare (as Workers), waiting to run in sandboxed environments whenever a request is recieved at a particular endpoint.

This pattern of modular development and HTTP-based request handling has a few important consequences for developers:

1. Because of the modular, independent nature of these functions, they scale automatically with usage. Each function is invoked once per request, so there's no need to worry about any request going unhandled because of under-provisioned server hardware.
2. Since function invocation scales with use, so too can function cost. Depending on the provider, each invocation will usually be charged for the few milliseconds of time that it takes for the function to handle the incoming request. This means that costs can be lower for rarely-used endpoints than would be possible with dedicated compute resources. And for applications that see a sudden spike in usage, downtime can be avoided (for a price).
3. Dividing an API into a set of discrete functions means that it's easier to change individual pieces of a backend in-place without requiring complex rollover logic (like you might see in a Kubernetes Deployment).

While these seem to be generally positive features, there are some cons to consider as well:

1. Rarely-used endpoints are usually subject to a "cold start" that delays initial responses to requests by a few seconds. This behavior can be unacceptable in some cases, requiring some clever hacks to keep functions "warm".
2. Serverless functions are usually built for a particular cloud provider's serverless runtime (whatever that might be). That means that local development requires either careful emulation (a la netlify dev) or an Environmentless approach that leverages the production serverless environment (a la wrangler dev).
3. Since each function is stateless, it's difficult to interact with resources that assume a stateful connection (like databases).

These cons are not insurmountable, though! Serverless function providers will keep improving cold start times, more providers will adopt an Environmentless approach to feature development as Branchstack methodology becomes more widely-known, and stateful resources will be built to handle the unique needs of Serverless backends. The trends are clear: Serverless architecture is gaining traction, and new cloud products need to treat Serverless as a first-class citizen.

So what does this have to do with Branchstack?

Serverless Branchstack

In the context of Branchstack-ready cloud resources, "Serverless" means two things:

1. resources scale automatically with usage, just like a Serverless function
2. resources handle the high request volumes of Serverless applications without the assumption of shared state

It does not mean that there is no "server". It means instead that it's not up to developers to configure or manage that server. It's up to a third party (usually a provider like Platter) to handle allocation of those resources on behalf of developers.

Serverless also doesn't mean that the resource in question is built out of some of the functions above. But Branchstack resources should behave like those functions in certain respects, and should be ready for use in Serverless backends. With that in mind, let's look at how to apply these principles to Branchstack methodology.

Automatic Scaling

Branchstack is all about building features with more confidence and speed. The first part of that equation means adopting an Environmentless approach, where there is a single cloud-based "environment" that handles every branch of your stack on the same hardware. If this is to work, your production environment needs to be able to scale quickly with both user-generated traffic and development-induced traffic. In the case of many resources, this is not trivial! Databases, for example, are difficult to configure ahead-of-time to handle in-place copying of large data sets quickly. Instead, your team needs to be confident that the resource in question scales with usage, just like serverless functions.

In addition to providing confidence for users and devs alike, it frees developers from even thinking about things like CPU or RAM usage, disk size provisioning, or hard request limits. This brings the experience of working with those traditionally-precious resources closer to that of working with disposable code, without the ceremony of a separate resource-provisioning step.

High Request Volumes

In addition, the statelessness of HTTP-handling functions poses a problem for resources like databases. Most databases are built on the assumption that a consumer will use stateful connections that persist between requests. This means that connections usually have a high initiation cost (in time and compute) and a high persistence cost (usually in RAM). The way that stateful applications handle the former is by creating a "pool" of connections when the application starts, paying that connection price once before the service starts handling traffic. The way they handle the latter is by sharing those pooled connections for individual queries within a long-running process, returning those connections to the pool when a query is finished.

Serverless functions cannot leverage this pooling strategy for database connections, as there is no state to share between invocations. Creating new connections for each invocation is both too slow and potentially disastrous for databases that are not built to handle thousands of connections at once. The same applies for any resource that relies on persistent, stateful connections, like message queues, pub/sub brokers, and databases of every flavor.

Branchstack resources need to be able to handle those high request volumes if they are to be useful in the Serverless era. Most of the time, that means offloading pooling and scaling strategies onto a provider instead of a consumer. As an example of how this works, let's take a look at what makes Platter Postgres Serverless-ready.

Serverless Postgres

Platter's Branching Postgres product was built from the ground up to handle the high request volumes of Serverless applications. There are two strategies at work, depending on the way you connect to the database itself:

1. Users of the Node.js Client use gRPC requests instead of direct connections to query databases. These requests are backed by an application-level set of managed connection pools so that requests are handled quickly.
2. Direct connections using the platter postgres branch url subcommand are made against a pgbouncer-based connection pool, which means that users of e.g. pg or an ORM can still leverage direct connections in Serverless contexts without needing to worry about overloading the database itself with connections.

And while Platter Postgres uses tiered pricing instead of strict pay-as-you-go pricing, we're currently working on features to enable more seamless automatic scaling features in the spirit of Serverless. As we add more usage-based tiers, though, we'll be making sure that transitions between those tiers happens automatically, and without an explicit configuration step on your part!

If you have any questions about how Platter Postgres gives your Serverless applications the power of relational data, please send us an email at [email protected] or join us on the #kitchen Slack channel.