Local Serverless Development Part 2: Running API Gateway Locally

PUBLISHED ON MAR 28, 2018 — AWS, SERVERLESS

This is part 2 in my Serverless mini-series. You can find part 1 here.

As I mentioned in the previous post, I’ve been working on a local development environment for a serverless architecture. The previous post covered how to invoke Lambdas locally whether they were written in Java, NodeJs or Python.
In this part of the series, I’m going to expand on the previous work in order to give us a way of running those Lambdas locally as an API Gateway.
As always I’ve added the code for this post on Github, you can find the code for the series here.

Part 2

We already have three Lambdas which we can invoke locally using Serverless Framework.
The next step was to use the API Gateway in AWS for invoking Lambdas, and as such I wanted a good local environment for developing and testing this, without needing to deploy anything.
Unfortunately, at the time of writing this post, it’s not possible to use the Serverless Framework to run Lambdas through API Gateway locally, so we’re going to need another tool.

Enter AWS SAM Local.
AWS SAM (Serverless Application Model) Local is a CLI developed by Amazon for local development and testing of serverless applications.
SAM does this by pulling down a container specific to the runtime for the function being invoked whenever the Lambda is invoked. You can invoke individual functions much as we have with Serverless. But what we want to use SAM for is to run them through API Gateway locally, which it does by exposing the functions on the endpoints specified in the template file, and hosts them on your machine at http://localhost:3000 (by default).

Now as we’ve already mentioned the word container, you may have guessed we’re going to need to have Docker installed and running. You can find that here if you don’t already have it.

In the last post I created three lambdas in varying runtimes. Unfortunately, there’s a bug in one of the plugins I’ve used which means things don’t work so smoothly with multiple runtimes in the same API.
There’s a PR to fix this, I’m just waiting for it to be merged.
Rest assured that you can have multiple runtimes in API Gateway, but until that PR gets merged it would involve manually editing a template file - and I’m not down for that. As such, I’ve used Python for both Lambdas in my API. I’ll update this post at a later date to cover using multiple different runtimes, once the PR has been merged.

If you take a look at the serverless.yml file you can see that I’ve modified the Lambdas so that they include information about the HTTP events that they will get invoked by.
Our handler paths have been updated to include the folder for each lambda:

service: HelloGoodbyeApi

provider:
  name: aws
  stage: local
  environment: ${file(env.${self:provider.stage}.yml)}
  runtime: python3.6

functions:
  HelloFunction:
    handler: HelloPython/handler.hello
    events:
      - http:
          path: hello
          method: get
  GoodbyeFunction:
    handler: GoodbyePython/handler.goodbye
    events:
      - http:
          path: goodbye
          method: get

As you can see, the structure is essentially the same as when we defined our standalone Lambdas, but this time we’re specifying that they will be invoked by a HTTP GET event - and since we have the Serverless template a level above the Lambdas, the folder path has been included in the handler.

So first things first, we need to have a template.yml file that follows the SAM specification.
Now, given we have a serverless.yml file that already contains the information we want, and in my opinion is easier to read, I don’t think you want to be maintaining two separate files that essentially will do the same thing.
Instead, we’ll add a plugin to Serverless that we can use to generate the SAM template for us.
To do this, you want to install the serverless-sam plugin by running the following command:
npm install serverless-sam --save-dev
Now that’s installed, we need to modify our serverless.yml file to add that plugin, so at the bottom of the file I’ve added this:

plugins:
- serverless-sam

This just lets Serverless know that we’re expecting that plugin to be available.

Now you can generate the template by running the following command:
serverless sam export -o template.yml
This will give us a SAM compliant template which includes each function, the permissions for the function, and the API Gateway definition. I’ve trimmed down the output to show just one of the Lambdas, but the template looks something like this:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: 'SAM template for Serverless framework service: '
Resources:
  HelloFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: HelloPython/handler.hello
      Runtime: python3.6
      CodeUri: >-
        /Users/ianrufus/Personal/BlogPosts/ServerlessMiniSeries/2/.serverless/HelloGoodbyeApi.zip
      MemorySize: 128
      Timeout: 3
      Environment:
        Variables:
          RUNNING_ENVIRONMENT: local
      Events:
        Event1:
          Type: Api
          Properties:
            Path: /hello
            Method: get
            RestApiId:
              Ref: HelloGoodbyeApi
  HelloGoodbyeApi:
    Type: 'AWS::Serverless::Api'
    Properties:
      StageName: local
      DefinitionBody:
        swagger: '2.0'
        info:
          title:
            Ref: 'AWS::StackName'
        paths:
          /hello:
            get:
              x-amazon-apigateway-integration:
                httpMethod: POST
                type: aws_proxy
                uri:
                  'Fn::Sub': >-
                    arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${HelloFunction.Arn}/invocations
              responses: {}
          /goodbye:
            get:
              x-amazon-apigateway-integration:
                httpMethod: POST
                type: aws_proxy
                uri:
                  'Fn::Sub': >-
                    arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GoodbyeFunction.Arn}/invocations
              responses: {}
  HelloFunctionLambdaPermission:
    Type: 'AWS::Lambda::Permission'
    DependsOn:
      - HelloFunction
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName:
        Ref: HelloFunction
      Principal: apigateway.amazonaws.com
  GoodbyeFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: GoodbyePython/handler.goodbye
      Runtime: python3.6
      CodeUri: >-
        /Users/ianrufus/Personal/BlogPosts/ServerlessMiniSeries/2/.serverless/HelloGoodbyeApi.zip
      MemorySize: 128
      Timeout: 3
      Environment:
        Variables:
          RUNNING_ENVIRONMENT: local
      Events:
        Event1:
          Type: Api
          Properties:
            Path: /goodbye
            Method: get
            RestApiId:
              Ref: HelloGoodbyeApi
  GoodbyeFunctionLambdaPermission:
    Type: 'AWS::Lambda::Permission'
    DependsOn:
      - GoodbyeFunction
    Properties:
      Action: 'lambda:InvokeFunction'
      FunctionName:
        Ref: GoodbyeFunction
      Principal: apigateway.amazonaws.com

As you can see, the Serverless plugin handles our environment variable configuration, and specifies the ARNs for our Lambdas.
Here you can see the individual Lambda functions being defined so they can be deployed, and the API Gateway is defined making use of those Lambdas on the routes we specified.
If you want to see the full template, run the command yourself to generate the template. I’ve purposefully not committed the SAM template to Git - I only want to maintain one template, so my plan is to generate this from the Serverless template whenever it’s required.
It’s also worth noting that you don’t need to remove that template when you regenerate, as it will be overwritten with the new version supplied from the plugin.

Now you have an AWS SAM template for the Lambdas that can be used by SAM.
We need to install AWS SAM Local before we can use it, which can be done through NPM:
npm install aws-sam-local --save-dev
To run the API Gateway, we can now run the following command:
sam local start-api
In the terminal you’ll see that it’s pulling down a runtime container for each runtime specified - in our case that’s just Python because of the aforementioned bug. As such this might take a while on the first run, but it will re-use those images for subsequent runs.

As you can see from the terminal output, this hosts our Lambdas on the localhost, and prints out the location, path and method for each of the Lambdas. It also notes that you don’t need to re-start the API for any code changes to be used.
Now you can either CURL the address, or browse to the location in your web browser. When you do so, you’ll see SAM output information to the terminal about the Lambda being invoked, alongside any logging you’ve added.

And that’s it, we now have a locally running API Gateway for our Lambdas!

Now there’s the problem that if we want to interact with any other AWS services then, as it stands, they’ll need to be deployed.
As such, in the next post I’ll cover how to mimic AWS services locally using LocalStack, instead of relying on an internet connection and deployed services.

As always, you can find the code for this post here. And feel free to get in touch with any questions or criticism :)

comments powered by Disqus