Creating an AWS Lambda Python Layer by Logging into an AWS EC2 Linux Instance

by

In this tutorial, we’ll walk through the steps to create an AWS Lambda Python layer using an AWS EC2 Linux instance. If you’re looking to package dependencies for your Lambda function and find that doing so on your local machine isn’t feasible (due to OS or environment mismatches), then using an EC2 instance is a great alternative.

Step 1: Try Locally First

Before jumping into creating an EC2 instance, try creating your Lambda layer locally. The process is straightforward if you have a compatible environment, but if you run into issues, only then proceed with the EC2 method.

Step 2: Set Up Your AWS Environment

  1. Create an S3 Bucket: To store your zipped dependencies, you’ll need an S3 bucket. Here’s how to create one:
    • Log in to your AWS Management Console.
    • Navigate to the S3 service.
    • Click on “Create bucket.”
    • Enter a unique bucket name (e.g., ranjith-demo-yt), choose a region, and adjust the settings as needed.
    • Click on “Create bucket” to finalize.
  2. Create an IAM User: You’ll need an IAM user with Programmatic access to get an Access Key ID and Secret Access Key, which are essential for configuring AWS CLI.
    • Go to the IAM service in AWS.
    • Click on “Users” and then “Add user.”
    • Give your user a name (e.g., lambda-layer-user) and select “Programmatic access.”
    • Attach necessary policies, like AmazonS3FullAccess.
    • Review and create the user, then save the Access Key ID and Secret Access Key.
  3. Configure AWS CLI: On your local machine or EC2 instance, configure AWS CLI using the following command:bashCopy codeaws configure
    • Enter the Access Key ID, Secret Access Key, region, and output format.

Step 3: Launch an AWS EC2 Instance

If you couldn’t create the layer locally, proceed to create an EC2 instance:

  1. Create an EC2 Instance:
    • Go to the EC2 service in AWS.
    • Click on “Launch Instance.”
    • Choose an Amazon Linux 2 AMI.
    • Select an instance type (e.g., t2.micro if you’re on the free tier).
    • Configure security groups and key pairs as needed.
    • Launch the instance.
  2. Connect to Your EC2 Instance Using EC2 Connect:
    • In the EC2 dashboard, select your instance and click on “Connect.”
    • Choose “EC2 Instance Connect” and click on “Connect.”
    • This method is more straightforward than using SSH and doesn’t require key pairs.

Step 4: Install Python and Dependencies on EC2

Once logged into your EC2 instance, follow these steps:

  • Install Python 3 and Pip: 
sudo yum install python3-pip
  • Install the Required Python Package: Install psycopg2-binary and package it for Lambda:
pip3 install psycopg2-binary -t .
  • Verify the Installation: List the contents to ensure everything is installed correctly:
ls -lrt
  • Package the Dependencies: Zip the contents into a file that will be uploaded to S3:
zip -r psycopg2-aws.zip python/

Step 5: Upload to S3

Once the package is zipped, upload it to your S3 bucket:

aws s3 cp psycopg2-aws.zip s3://ranjith-demo-yt/psycopg2-aws.zip

Step 6: Create the Lambda Layer

Now that your package is in S3, you can create the Lambda layer:

  1. Go to the AWS Lambda Console:
    • Navigate to the Lambda service in the AWS Management Console.
    • Click on “Layers” in the sidebar and then “Create layer.”
  2. Upload Your Zip File from S3:
    • Choose “Upload a file from Amazon S3.”
    • Enter the S3 path to your zip file (s3://ranjith-demo-yt/psycopg2-aws.zip).
    • Name your layer and specify the runtime (Python 3.x).
  3. Publish the Layer:
    • Click on “Create,” and your layer will be ready for use in your Lambda functions.

Conclusion

By following these steps, you can successfully create a Python layer for AWS Lambda using an EC2 instance. Remember to try creating the layer locally first, and only resort to the EC2 method if necessary. This approach ensures your Lambda function has all the dependencies it needs, packaged in a way that’s compatible with AWS Lambda’s environment.

Leave a Comment