Managing Cloud Infrastructure State Files

Cloud Infrastructure State Files

In order to bring your project into existence, you need to deploy your application code, along with the necessary cloud infrastructure, into a cloud provider of your choice (for example AWS ).

By default, for everything deployment-related, Webiny relies on Pulumi, a modern infrastructure as code (IaC) framework. Essentially, the framework enables us to define needed cloud infrastructure via code, and ultimately, deploy it into one or more separate environments.

One of the fundamental concepts of any IaC framework are cloud infrastructure state files. Every framework handles them differently, but in all cases, state files represent the state of all deployed cloud infrastructure resources. Things like which resources are currently deployed, which configuration params were used upon deploying them, and more, are just some of the information these state files contain.

It’s important to know that state files must not be lost or deleted, because otherwise, the connection between our IaC framework and the actual cloud infrastructure resources would also be lost. Meaning, we would lose the ability to update existing cloud infrastructure resources or even delete them (using the IaC framework).

Managing Cloud Infrastructure State Files

Because of the fact that our cloud infrastructure state files must be preserved, we are often talking about cloud infrastructure state management, or, in context of IaC frameworks and cloud infrastructure deployments, just state management.

Commonly Used Backends

The following are three backends that are commonly used with Webiny projects.

Local File System

This is the default backend, with which, all of the state files are stored within your Webiny project, inside of a single .pulumi folder, located in your project root. Note that this folder contains cloud infrastructure state files for all project applications you might have in your Webiny project.

The following directory tree shows the .pulumi folder which is being utilized by all three default project applications (api/.pulumi, apps/admin/.pulumi, apps/website/.pulumi):

.
├── apps
│   ├── core
│   │   ├── .pulumi
│   │   │   ├── backups
│   │   │   ├── history
│   │   │   └── stacks
│   ├── api
│   │   ├── .pulumi
│   │   │   ├── backups
│   │   │   ├── history
│   │   │   └── stacks
│   ├── admin
│   │   ├── .pulumi
│   │   │   ├── backups
│   │   │   ├── history
│   │   │   └── stacks
│   └── website
│       ├── .pulumi
│       │   ├── backups
│       │   ├── history
│       │   └── stacks
│  
└── (...)

This backend can be a solid choice for local development purposes, where you typically don’t need to share the state files with other team members and are only relevant to you. Because of this, in every new Webiny project, by default, the .pulumi folder isn’t checked into your version control.

Amazon S3

With this backend option, we can store our cloud infrastructure state files in a remote Amazon S3 bucket.

This is useful when deploying our project into long-lived shared environments, like staging or even prod (production). Environments like these can certainly be considered as sensitive, so, it is critical that we have a single source of truth of our cloud infrastructure state and also, that the state files are stored in a reliable and secure way. We definitely want to avoid having multiple versions of our production cloud infrastructure state, or have our state files publicly exposed.

Pulumi Service

You can certainly use the Amazon S3 for storing cloud infrastructure state files, but if you need more, you can also explore the Pulumi Service .

It is a service that not only gives you the ability to store your cloud infrastructure state files in an easy, secure, and reliable way, but also offers a couple of other interesting features, like:

• concurrent state locking to prevent corrupting your infrastructure state in a team environment
• full deployment history for auditing and rollback purposes
• encrypted state in transit and at rest
• and more

Of course, it does come with a price, so make sure to check their official pricing page before making a decision on which backend you’re about to use.

Using Different Backends

Amazon S3

If we wanted to use Amazon S3 as our backend of choice, we could assign the following environment variable:

WEBINY_PULUMI_BACKEND=s3://my-s3-bucket/some-folder-maybe

Pulumi Service

On the other hand, if we wanted to use the Pulumi Service, we could assign the following environment variables:

WEBINY_PULUMI_BACKEND=https://api.pulumi.com
PULUMI_ACCESS_TOKEN=pul-xyzabc123

Note that, when using the Pulumi Service, it’s recommended we remove the PULUMI_SECRETS_PROVIDER and PULUMI_CONFIG_PASSPHRASE environment variables. Essentially, these variable instruct Pulumi CLI which secrets provider we want to use, and since Pulumi Service has its own built-in one, we don’t need these anymore.

What to Use

Development / Short-Lived Environments

That being said, there might still be cases in which you’ll want to share the cloud infrastructure state files, for example with your team members or maybe you just want to continue developing on a different machine.

And while both can be achieved, note that sharing cloud infrastructure state files implies the same cloud (AWS) account is used on every machine on which the files end up. Which means that sharing them with multiple team members requires every member to use the same cloud account and this is something we usually wouldn’t recommend. Ideally, developers should have their own cloud account for development purposes.

In case of the latter, where developers just wants to continue working from one of their other machines (using the same cloud account), sharing the cloud infrastructure state files can be achieved in a relatively straightforward way.

Simply copy and paste your .pulumi folder to the other machine. Once you’ve done that, also note that all of the extra configuration arguments, that were passed to Pulumi via environment variables, should also be copied. Usually, here we’re referring to values that were specified in your root .env file, where by default, the PULUMI_SECRETS_PROVIDER and PULUMI_CONFIG_PASSPHRASE values are set.

CI/CD / Long-Lived Environments

For CI/CD purposes, we recommend using one of the remote backend options - Amazon S3 or Pulumi Service.

Note that in case of the former, since we recommend a separate cloud (AWS) account to be used with every long-lived environment, you will end up with a separate Amazon S3 bucket for each environment.

FAQ

Should I Use Different Amazon S3 Buckets for Different Shared Long-Lived Environments?

If you’re using a separate cloud (AWS) account for every long-lived environment, then the answer is yes and this is the approach we recommend you adapt.

For Storing Cloud Infrastructure State of My Long-Lived Environments in Amazon S3 Buckets, Must I Create These on My Own, or Does This Happen Automatically in Some Way?

If you decide to use Amazon S3 as a way to store your cloud infrastructure state files for your long-lived environments, then be aware that the actual buckets will need to be created manually. At the moment, this is not something Webiny does automatically for you (but we are thinking about it).

The most straightforward way to do it would be to open the AWS Management Console for every separate AWS account you might have, and simply create the bucket from there.

For Development Purposes, Can I Check in the .pulumi Folder Into VCS?

We don’t recommend doing it for of a couple reasons, mainly because this can quickly get messy, especially if multiple developers are working on the project.

Still, if you have a strong opinion on this, let us know , we’d like to hear your thoughts!

What Happens if Multiple Pulumi Deployments Are Triggered at the Same Time?

No matter which Pulumi backend you end up choosing, if two or more Pulumi deployments are triggered simultaneously, only the first one will be performed. The rest will be rejected, with an appropriate error being returned in your terminal. The error should look similar to the following:

error: the current deployment has 1 resource(s) with pending operations:
  * urn:pulumi:dev::website::aws:cloudfront/distribution:Distribution::delivery, interrupted while updating

How Do I Pass the WEBINY_PULUMI_BACKEND in My CI/CD?

Different CI/CD providers offer different options when it comes to setting environment variables. For example, GitHub Actions enable this via repository secrets . On the other hand, if using AWS CodeBuild , you can use the AWS CodePipeline service.

Make sure to read the relevant documentation for best practices around how to properly set environment variables in your CI/CD provider.

Is It Possible to Move My Locally Stored State Files Into an Amazon S3 Bucket and Continue Using It as My Backend?

Yes, it can be done by taking the following steps.

First, create a new Amazon S3 bucket. You can do this via the AWS console or any other way you prefer. Do note that the bucket has to be created in the same region where your Webiny project is deployed.

Once the bucket has been created, assign its URI to the WEBINY_PULUMI_BACKEND environment variable, in your .env file. For example:

WEBINY_PULUMI_BACKEND=s3://my-project-state-files

Finally, copy all of the folders and files located in your project root .pulumi folder into the newly created bucket. Once copied, in your Amazon S3 bucket, you should end up with the following paths:

/apps/core/.pulumi/...
/apps/api/.pulumi/...
/apps/admin/.pulumi/...
/apps/website/.pulumi/...

Can State Files Contain Secrets or Passwords? How Are These Stored?

The easiest way to do this would be to use the pulumi.secret method, for example:

pulumi.secret(process.env.MY_API_TOKEN);

With this approach, values will still end up in your state files, but they will be encoded, which means nobody will be able to actually read them.