In AWS re:Invent 2018, AWS have launched a great feature called Layers in the Lambda service.  Layers let you deploy custom runtime along with additional code/package, which can be further used/imported in the lambda function. It let you keep the deployment package small. A layer can be used within multiple lambda function, hence you can re-use the packages/runtime by deploying it once.

In this tutorial, I’m going to take you through, how to create custom layer and use it in the lambda function. Also, there exists the pre-published layers by AWS. Here will use following services.

  1. IAM
  2. S3
  3. EC2
  4. Lambda Function
  5. Lambda Layers

IAM

We will get started with AWS IAM service. Here, we will create two roles

  • IAM role for Lambda Function (For publishing the CloudWatch Logs)
    • Name: lambda_basic_execution
    • Permission: AWSLambdaBasicExecutionRole
  • IAM Role for EC2 Instance (For uploading the package to S3 from EC2)
    • Name: s3-admin-access
    • Permission: AmazonS3FullAccess]

Looking for step by step tutorial on creating the IAM role? Refer how to create IAM role

S3

Here, we will create the bucket in which we will upload the package for Lambda layers. Go ahead and create the bucket. In my case, I have created layer-release-test.

EC2

Now let’s go ahead and launch the EC2 instance (Micro instance with ubuntu is sufficient for the purpose of this tutorial). In step 3 (Configure Instance), make sure to attach the IAM role, which we created for the EC2 instance. Once the instance is launched, SSH into the instance via terminal and execute below commands.

  1. sudo apt-get update
  2. python3 -V (Checks the Python version. Here, in this tutorial we will deploy the package/runtime for Python3.6)
  3. sudo apt install python3-pip (Installing pip3)
  4. mkdir -p build/python/lib/python3.6/site-packages (create the directory)
  5. pip3 install pandas -t build/python/lib/python3.6/site-packages/ —system (Here, we will build layers for pandas package)
  6. cd build
  7. zip -r package.zip . (It will create package.zip)
  8. aws s3 cp package.zip s3://layer-release-test (Uploading the package.zip to s3 bucket, which we created in the previous step)

Note: Path (build/python/lib/python3.6/site-packages) plays the key role in Layers, make sure the path is correct.

Lambda Layers

Here, we will create the layer. Navigate to Lambda Management Console -> Layers -> Create Layer. Fill the asked configuration details as shown below and Create the Layer.

lambda layers

Lambda Function

Here, we will create the Lambda function and use/import the pandas package using layers. Go ahead and create the lambda function and make sure to assign the IAM role that we created in the IAM role section (lambda_basic_execution). Refer for step by step tutorial.

Once the function is created, click on the Layers from Designer pane. Scroll down to Referenced layers pane -> Add a layer. In the Layer dropdown, you will see the pandaDependency layer (one we created) under My layers -> pandasDependency -> Version-> 1 -> Add -> Save. Now, lets use pandas in our code.

import pandas as pd
import json

def lambda_handler(event, context):
    print(pd.__version__)
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

This function should run successfully and this is how you can use AWS Lambda Layers.



Keep sharing. Stay tuned for more. Follow me on Twitter