Using Projen for CDK Constructs

, , ,

Up until recently at my $DAYJOB I have been using the AWS CDK cli to create Python based CDK Constructs and then use twine to deploy them to a private pypi repo. This works fairly well but the built in Jest testing with TypeScript is a lot more feature complete. You can then use jsii to compile the TypeScript into multiple language outputs. This is the same thing the AWS CDK uses to publish their own packages.

So I decided to explore setting up a pipeline to do so and that lead me down a long rabbit-hole to the tool called projen.

What is projen?

projen‘s project description calls it a

A new generation of project generators

This would put it on a similar level as [cookiecutter] or [yeoman]. Both of which can be used as cli tools to bootstrap new projects, generating boilerplate config and code files to get you up and running quickly.

That doesn’t give a whole lot of details, the README further describes it as a way to

Define and maintain complex project configuration through code.

and

JOIN THE #TemplatesAreEvil MOVEMENT!

While I am not sure if templates are actually evil, it is a pretty cool tool. When using the aforementioned project generator tools, you can only run them once usually. That means any additional setup must be done manually and commited. projen has a different approach of doing all configuration via it and enforcing updating those changes. That way it can be run multiple times, ensuring the same output and not needing to manually tweak files.

Using projen

Initially a CDK projen project can be bootstrapped using the cli:

1
npx projen new awscdk-construct

Code can then be added into the src/ directory to create your CDK construct.

So far this is pretty typical CDK development, but let’s say you want to bump the CDK version. Typically you would have to go in and edit your package.json and bump the version of quite a few CDK packages. But projen makes it easier. To do it simply edit a single line in the .projenrc.js config file.

cdkVersion

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
const { AwsCdkConstructLibrary } = require('projen');
const project = new AwsCdkConstructLibrary({
  author: 'David Michael',
  authorAddress: '1.david.michael@gmail.com',
  cdkVersion: '1.113.0',
  defaultReleaseBranch: 'main',
  name: '@1davidmichael/cloudwatch-alarms-to-teams',
  repositoryUrl: 'https://github.com/1davidmichael/Cloudwatch-Alarms-to-Chat-Platforms',
  cdkDependencies: [
    '@aws-cdk/core',
    '@aws-cdk/aws-lambda',
    '@aws-cdk/aws-lambda-event-sources',
    '@aws-cdk/aws-sns',
    '@aws-cdk/aws-cloudwatch',
    '@aws-cdk/aws-cloudwatch-actions',
  ],
  python: {
    distName: 'cloudwatch-alarms-to-teams',
    module: 'cloudwatch_alarms_to_teams',
  },
});
project.gitignore.addPatterns('.venv/');
project.synth();

Or add a directory to be ignored in .gitignore. In this case I use a python development environment to work on my lambda. The virtual env directory is within the repo at .venv.

1
project.gitignore.addPatterns('.venv/');

Then run npx projen and watch the config files in the repo get updated. Many files managed by projen will let you know via a comment in them. That way it is known not to directly edit it but to instead use projen.

Example in .gitignore:

1
2
3
4
# ~~ Generated by projen. To modify, edit .projenrc.js and run "npx projen".
*.lcov
*.log
*.pid

GitHub Actions

The nice thing about the CDK project project is it sets up a build and release pipeline within GitHub Actions with minimal effort. All of this is done via the jsii-release package. To get this working a few things are needed. In my case I am releasing to npm for TypeScript and pypi for Python languages. So I need the following tokens added as secrets to my GitHub Project:

Once those are added any PR to the project will result in a build and test be completed via actions. Then once merged into main a release workflow will run and push those packages to the artifact repositories.

Another added benefit is a API.md file is generated with references for the Constructs created. Here is an example.

An interesting problem I ran into when using the PythonFunction with GitHub Actions is the construct uses Docker under the hood to install dependencies. This caused issues because Docker was unable to be called within the Action. The solution is to use the L2 Construct SingletonFunction and the local bundle option. This is well described in this AWS blog post.

Here is an example of how it was done in TypeScript CDK with a Python Lambda:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
this.lambdaFunction = new lambda.SingletonFunction(this, 'TransformFunction', {
  code: lambda.Code.fromAsset(path.join(__dirname, '..', 'src', 'functions', 'teamsLambda'), {
    bundling: {
      image: lambda.Runtime.PYTHON_3_8.bundlingImage,
      local: {
        tryBundle(outputDir: string) {
          try {
            execSync('pip3 --version', execOptions);
          } catch {
            return false;
          }
          execSync(`pip3 install -r ${path.join(__dirname, '..', 'src', 'functions', 'teamsLambda', 'requirements.txt')} -t ${outputDir}`);
          execSync(`cp -au ${path.join(__dirname, '..', 'src', 'functions', 'teamsLambda')} ${outputDir}`);

          return true;
        },
      },
      command: [
        'bash', '-c',
        'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output',
      ],
    },
  }),
  uuid: 'b1475680-a6b6-4c58-9fb8-19ffd4325f45',
  handler: 'index.handler',
  runtime: Runtime.PYTHON_3_8,
  environment: { WEBHOOK: props.webhookUrl },
});

One other thing to call out here, the use of SingletonFunction, this way the lambda will be created only once, only if the Construct is actually used multiple times in a given stack.

My first multi-language construct

All of this was discovered when writing and creating my first open source CDK construct: https://github.com/1davidmichael/Cloudwatch-Alarms-to-Chat-Platforms

Does it work? I think so. Is it good? Maybe.

I’ve learned a lot through this about projen and have appreciated what it sets up, its replayability in generating configs, and the automated workflows it sets up.