
I was working with [Pulumi](https://www.pulumi.com) to migrate AWS cloud
resources of a company to IaC. In this article, we shall explore why I chose
Pulumi, how the migration was, and the standards and processes defined along
the way.

## Looking for an IaC solution

Terraform is the first IaC solution that comes to anyone's mind. Although it is
widely used in the industry, I found it a little intimidating to use HCL, as it
reinvented some of the basic programming constructs such as loops. I will take
an example straightly from the [Terraform
Docs](https://www.terraform.io/language/expressions/for#filtering-elements) to
illustrate my point.

```hcl
variable "users" {
  type = map(object({
    is_admin = bool
  }))
}

locals {
  admin_users = {
    for name, user in var.users : name => user
    if user.is_admin
  }
  regular_users = {
    for name, user in var.users : name => user
    if !user.is_admin
  }
}
```

In a general-purpose programming language, say TypeScript, we can write this
code as:

```typescript
type User = {
  is_admin: boolean,
}

let users: Map<string, User> = {} // The value is coming from somewhere, say a
config or input

let admin_users = new Map(
  [...users].filter(
    ([name, user]) => user.is_admin
  )
)

let regular_users = new Map(
  [...users].filter(
    ([name, user]) => !user.is_admin
  )
)

```

While anyone comfortable with JavaScript or TypeScript can understand the below
code, it takes some time to understand HCL code and get used to its syntax. I
preferred general-purpose programming languages for IaC for a few reasons:

1. Developers can write IaC in the language that they are used to write
applications, this eliminates the need to learn a new language
2. A developer with sufficient knowledge can maintain the IaC codebase; no need
to have dedicated personnel for this
3. One can use the _best practices_ of the programming language to organize the
code, such as refactoring into modules, etc.
4. In some cases, debugging IaC is also possible by using common debuggers by
stepping into the code and inspecting variables!

When I saw a [Fireship
video](https://www.youtube.com/watch?v=tomUWcQ0P3k&lc=UgxQFl_FF5yB0bKhXF14AaABAg&ab_channel=Fireship)
about Terraform, a comment mentioned that he should make a video of Pulumi as
well; that's how I got to know about Pulumi! Going through their
[website](https://www.pulumi.com/) and [docs](https://www.pulumi.com/docs/), it
seemed they offered the exact needs that I wanted. After I read the article
[_"Imperatively
Declarative"_](https://chris.nunciato.org/blog/how-pulumi-is-declarative/) by
Christian Nunciato, I had a clear idea of what Pulumi does, and why it is
flexible over other solutions.

While Pulumi offers support for many languages (**latest**: [Pulumi now
supports JVM languages and YAML as
well!](https://www.pulumi.com/blog/pulumi-universal-iac)), I chose to go ahead
with TypeScript + NodeJS for Pulumi for a few reasons:

- Project management with `npm` is easy, it just takes doing `npm install` to
set up the project on a new machine
- At that time, Pulumi's [`awsx`](https://www.npmjs.com/package/@pulumi/awsx)
and [`eks`](https://www.npmjs.com/package/@pulumi/eks) packages were only
available on `npm`; more about them later!
- Going with TypeScript over JavaScript is natural, due to the benefits offered
by static type checking. Plus, Pulumi's packages are also in TypeScript,
providing excellent typing support.

## The Highlights

In the following sections, I will pick some interesting points about how Pulumi
helped write a clean IaC project. Let's use the following AWS architecture for
illustration purposes.

- VPC with public and private subnets, replicated for production and staging
- An EC2 jumpbox VM in the public subnet of staging VPC, which is in its own
security group
- VPC peering connection between production and staging VPCs
- An RDS instance placed in the private subnet of the VPC, with connections
allowed from the jumpbox machine, replicated for production and staging

### Stacks, Outputs, and References

In Pulumi, a _stack_ consists of a group of cloud resources. From one codebase,
we can create as many stacks as we wish. One typical use case of the stacks
would be to have `staging` and `production` environments, with nearly identical
resources. Apart from the environments, the stacks also help to break down the
infrastructure into several units using [Stack Reference
Architecture](https://www.pulumi.com/registry/packages/aws/how-to-guides/aws-ts-stackreference-architecture/).

We can break down our architecture into `network` and `database` units, with
`stage` and `prod` environments. We shall use `<unit>.<env>` syntax to name the
stacks. Each unit sits in its own repository and has a separate stack file. 

Let's focus on the jumpbox in the `network.stage` stack. Since we want the RDS
to be accessible from the jumobox, we export its security group ID as the stack
output.

```typescript
// network/index.ts
import * as aws from "@pulumi/aws"

function createJumpbox(name: string) {
  const securityGroup = new aws.ec2.SecurityGroup(
    `${name}-sg`,
    {
      ingress: [
        {
          cidrBlocks: ["0.0.0.0/0"],
          fromPort: 22,
          toPort: 22,
          protocol: "tcp",
          description: "Allow SSH connection",
        },
      ],
      egress: [
        {
          cidrBlocks: ["0.0.0.0/0"],
          fromPort: 0,
          toPort: 0,
          protocol: "all",
          description: "Allow all outgoing connections",
        },
      ],
    }
  )
  const vm = new aws.ec2.Instance(name, {
    ami: "<add-some-ami-here>",
    instanceType: "t3.medium",
    vpcSecurityGroupIds: [securityGroup.id],
    // Add other options here, such as SSH Key, subnets
  })
  return { securityGroup, vm }
}

const { securityGroup, vm } = createJumpbox("jumpbox")

// Export the security group ID as the stack output
export const jumpboxSecurityGroupId = securityGroup.id

```

On the `database` unit, to allow _only_ the connections from jumpbox at port
`5432` (which is the port at which PostgreSQL listens), we can do it this way:

```typescript
// database/index.ts

import * as pulumi from "@pulumi/pulumi"
import * as aws from "@pulumi/aws"

// If we are in the stack `db.stage`, environment is `stage`
const getEnvironment = () => pulumi
  .getStack()
  .split(".")
  .slice(-1)[0]

const environment = getEnvironment()
// Create a Stack Reference to the `network` stack in the same environment
const networkStack = new pulumi.StackReference(
  `network.${environment}`
)
// Get the jumpbox security group ID
const jumpboxSecurityGroupId = networkStack.requireOutput(
  "jumpboxSecurityGroupId"
) as pulumi.Output<string>
// Create Security group and allow incoming connection from jumpbox security
group
const securityGroup = new aws.ec2.SecurityGroup("db-sg", {
  ingress: [
    {
      securityGroups: [jumpboxSecurityGroupId],
      fromPort: 5432,
      toPort: 5432,
      protocol: "tcp",
      description:
        "Allow incoming connection from jumpbox at port 5432",
    },
  ],
})
// Create RDS Instance
const rdsInstance = new aws.rds.Instance("db", {
  instanceClass: "db.m3.medium",
  vpcSecurityGroupIds: [securityGroup.id],
  // Associate subnet group etc
})

```

The `requireOutput()` method of the `StackReference` class is the key here.
Using the stack reference pattern, one can easily split a large infrastructure
into smaller _units_ and link them.

> **Note**: When using the stack reference, if the output of a stack changes,
we have to update other stacks manually by selecting the stack `pulumi stack
select <stack name>` and `pulumi update.
>
> Sometimes, not doing so might break the things elsewhere.

The stack reference can also be used to refer to the stacks in the same project
as well. As an example, in `network.prod`, we create [VPC
Peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html)
connection to `network.stage` and export `jumpboxSecurityGroupId` reusing it
from `network.stage`

```typescript
// network/index.ts

import * as pulumi from "@pulumi/pulumi"
import * as aws from "@pulumi/aws"

// Get the current environment, getEnvironment() defined earlier
const environment = getEnvironment()

const isStaging = environment === "stage"

let securityGroupId = null

if (isStaging) {
  // Function createJumpbox() defined in earlier example
  const { securityGroup: jumpboxSecurityGroupId, vm } =
    createJumpbox("jumpbox")
  securityGroupId = jumpboxSecurityGroupId.id
} else {
  const stagingEnvironment = new pulumi.StackReference(
    "network.stage"
  )
  // Create VPC peering
  securityGroupId = stagingEnvironment.requireOutput(
    "jumpboxSecurityGroupId"
  )
}

export const jumpboxSecurityGroupId = securityGroupId

```



### Config File, Secrets

In the above examples, you may have seen that we have hardcoded the EC2
instnace type in the code. Whenever we want to change the instance type,
editing the code does not seem to be a good idea. Instead of that, Pulumi
offers us configs, which can be different for different stacks in the same
project. In the `network` project in the above example, we can refactor the
instance type to `Pulumi.network.<env>.yaml`, using `pulumi` CLI.

```sh
pulumi config set jumpboxInstanceType t3.medium
```

We can also check the config file to verify that the value has been added.

```yaml
# Pulumi.test.stage.yaml
config:
  test:jumpboxInstanceType: t3.medium
```

Then, in the code, we can access config values using the `Config` class.

```typescript
import * as pulumi from "@pulumi/pulumi"
import * as aws from "@pulumi/aws"

const config = new pulumi.Config()

const instanceType = config.require("jumpboxInstanceType")

function createJumpbox(
  name: string,
  args: { instanceType: string }
) {
  const { instanceType } = args
  // same as above
  // const securityGroup =
  const vm = new aws.ec2.Instance(name, {
    ami: "<add-some-ami-here>",
    instanceType,
    vpcSecurityGroupIds: [securityGroup.id],
    // Add other options here, such as SSH Key, subnets
  })
  return { vm, securityGroup }
}
// Create jumpbox instance
const { securityGroup, vm } = createJumpbox("jumpbox", { instanceType })
```

Pulumi can also store sensitive values such as API keys and passwords in the
config via secrets. We can pass the `--secret` flag to `pulumi config set`
command to store a secret.

```sh
pulumi config set --secret secretTest someValue
```

It will add the encrypted `"someValue"` to the YAML file. Note that the
encrypted value will be different based on your key.

```yaml
# Pulumi.test.stage.yaml
config:
  stack-test:secretTest:
    secure: v1:ZweLtoIrBL/w8QjD:EGbnlgUGg1RcOb+lrQK7o7yXsE0oAh2Hqw==
```

To access the decrypted value, we use `Config.requireSecret()` function in the
code.

```typescript
import * as pulumi from "@pulumi/pulumi"

const config = new pulumi.Config()

const secretValue = config.requireSecret("secretTest")

// secretValue is pulumi.Output<any>, so we log it once we have a value for it
secretValue.apply(value => {
    console.log(value)
    pulumi.log.debug(value)
})

export const secretOutput = secretValue
```

While the `secretValue` in the above code is accessible by other resources as
the plain string, it is not printable. The log statements above just print
`[secret]`, instead of giving out the value. This value is stored encrypted
even in the state file!

> **Note**: Although Pulumi ensures that the secret value is not stored in
plaintext from its side, it is still your responsibility to store them securely
on the cloud side.
>
> As an example, let's say we are using an API key as an environment variable
to AWS Lambda. While we can store the API key as a secret in Pulumi config, it
is still viewable plaintext at Lambda-side. 

### Crosswalk Packages

The [Pulumi CrossWalk](https://www.pulumi.com/docs/guides/crosswalk/aws/)
packages encapsulate commonly used infrastructure patterns as a
`ComponentResource`. While we can create VPC using `@pulumi/aws`, by using
`Vpc` and `Subnet` classes along with `RouteTable` class to connect the
subnets, and `NatGateway` and `InternetGateway` to configure the network
access, the creation of the same using `@pulumi/awsx` package is as simple as,

```typescript
// network/index.ts
import * as awsx from "@pulumi/awsx";

type SubnetArgs = awsx.ec2.VpcSubnetArgs;

const publicSubnetCount = 2;
const privateSubnetCount = 2;

// Returns array with n elements
const range = (n: number) => [...Array(n).keys()];

const publicSubnets: SubnetArgs[] = range(
  publicSubnetCount
).map(() => ({ type: "public" }));
const privateSubnets: SubnetArgs[] = range(
  privateSubnetCount
).map(() => ({ type: "private" }));

const vpc = new awsx.ec2.Vpc("vpc", {
  cidrBlock: "10.0.0.0/16",
  subnets: [...publicSubnets, ...privateSubnets],
  numberOfNatGateways: 1,
});

```

### Lambda Magic

When we write the IaC in TypeScript, we are in the realm of code. Pulumi offers
excellent ways to utilize this realm and provides some helper functions with
callbacks, which are then [magically converted into AWS
Lambdas](https://www.pulumi.com/docs/guides/crosswalk/aws/lambda/). As a
concrete example, we can listen to an event on an S3 bucket by writing a
callback function in the IaC.

```typescript
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("media-bucket");

bucket.onObjectCreated("bucketEvent", (args) => {
  console.log("Got ObjectCreated event");
  // You can do something useful with args
  // This will be executed in the Lamdba
});

```

Such functionality is available for a variety of event sources. I implemented
this functionality for MQ by overriding the source code in the project. I plan
to test this functionality thoroughly using LocalStack and open a PR near soon.

## Defining the Standards

Pulumi does not enforce any restrictions on project organization. While nothing
is wrong with creating every resource of the stack in the `index.ts` file,
keeping the code organized helps to maintain it. Here are some of the standards
I defined and followed while writing the IaC:

- Stack names should follow the `projectName.environment` syntax.
  - `orgName/projectName` is not supported in the S3 backend, thus this naming
convention.
  - Thanks [@briggsl](https://twitter.com/briggsl) for pointing that out!

- Every source file should be under the `src/` directory. By default, Pulumi's
`aws-typescript` template creates `index.ts` in the project root.
- No resource should be directly created in `index.ts`, which is not directly
relevant to the main resource that the stack creates. They should be rather
refactored into `resource.ts` exporting `createResource()` function.
  - If the resource creation takes only one call, it can still be in the
`index.ts`, as there is no point in wrapping it with other function.
  - Mostly, these were related to IAM; Creating roles and policies were
refactored into a separate file.
- No function should decide the resource name on its own, rather it should be
passed top-down from `index.ts`.
  - This makes it easy to follow the path from which a resource is coming.
  - The functions can prepend some identifier to the name while passing it to
sub-functions
  - The name should be the first argument of the function, for example,
`createResource(name, args)`. This is in line with Pulumi's convention of
creating resources.
- No hard-coded values in the code, they should be rather in the config.
  - The config values are stored under an object, whose type is declared in the
`index.ts` as `ProjectConfig`.
  - Then, one can use `config.requireObject<ProjectConfig>("projectConfig")` to
retrieve the config.
  - No other file than `index.ts` should read the config! This would make it
easy to track the flow of the config values.
- No call to `Config.getX()` and `StackReference.getOutput()` methods!
  - These methods will silently return an empty value if the key is not present
  - The `require` methods are rather strict and will throw an exception if the
key is not present, thus, eliminating a few errors

## CI/CD for the Infra

While [Pulumi Service](https://www.pulumi.com/product/pulumi-service/) provides
a nice way to preview PR changes and look at the current state of the stack, I
went with S3 to store the state file, and used GitLab CI/CD pipelines for
deploying infrastructure changes. The rules of the pipeline looked as:

- Pipeline contains two stages:
  - `preview`, to preview infrastructure changes
  - `deploy`, to deploy the changes
- `deploy` is only available for the `main` branch, and contains manually
triggered jobs
- After `deploy` is complete, a Slack notification is triggered
- There can be multiple jobs in the `preview` and `deploy` stages, depending on
the environments

Then, the developer workflow would be:

- Create a branch for proposed infrastructure changes
- Commit and push the code, verify if changes look OK
- Start a PR, the reviewer looks at the code _along with_ the `preview` job
output to ensure nothing is breaking
- Once the PR is accepted and merged, a maintainer can trigger `deploy`
- Team is notified on Slack about new changes :tada:

## Overall Experience

Pulumi was my first experience with IaC, and it helped me to get started with
AWS more easily, thanks to the nice APIs in the code. Whenever I was struck
with something, I would go to the [Pulumi Slack
Community](https://slack.pulumi.com/), and people are very helpful there.

Pulumi brought IaC to General Purpose Languages, which also can benefit from
existing good practices regarding reusability, as I experienced it with
`TypeScript`. I used Pulumi at a beginner level, and there is still a lot to
learn regarding `ComponentResource`, stack functions, and Policy as a Code.


Experience and learnings during project that migrated the infra of a company to IaC using Pulumi

Pulumi

Pulumification