Using Amazon EFS with Lambda

Lambda functions are quite convenient when creating web servers. Their serverless architecture allows us to stop worrying about managing servers and infrastructure and dealing with scaling. Also, in some cases, they could save us some money by paying only for the compute time we use. However, despite its benefits, Lambda has some limitations, particularly when it comes to accessing and persisting data.

In such cases, AWS Elastic File System (EFS) comes to our rescue. EFS is the AWS service that enables us to share access to files residing on the cloud. It provides scalable file storage and a standard file system interface. This service is commonly used along with EC2, but in this case, we will configure our Lambda function to use it.

Throught this guide we are going to:

• Create a security group to allow access to the file system.
• Create the file system with the security group attached to it and an access point that will server as a gateway.
• Create a role for the Lambda function.
• Create the Lambda function.
• Testing everything we created works correctly.

Creating a Security Group

To start, we need to create a Security group that will allow other services to access the EFS Otherwise, our Lambda function will not be able to access it.

To create a Security Group:

Within the EC2 console, navigate to Security groups and select Create security group. Provide a name and a description and then add a new inbound rule as shown in the image.

Note that every service created in this tutorial must be located on the same VPC. Make sure that the following services you create in the next steps are within the same VPC as this security group.

Hit the Create security group button to finish this step.

Create the Elastic File System (EFS)

In this, step we are going to create the file system and its access point. To do that:

Naviagte to the EFS console and choose Create file system. Optionally, provide a name for the file system and make sure to place it in the same VPC as the recently created security group.

Don’t hit create yet. We need to change some configurations before that. First, select Customize

• Leave Step 1 as it is.
• In Step 2, modify the mount targets to use the security group created earlier. Make this change in each availability zone.

• Lastly, leave Step 3 as it is and create the file system.

Once the EFS is created, select it and navigate to the Access point tab. Here we will create a new access point, serving as the gateway for our Lambda function to interact with the file system.

Choose Create access point and configure it as follows:

In my case I prefer to specify a root directory path with /access. Feel free to leave it empty if you prefer.

After specifying these configurations, complete the step by hitting Create access point.

Create a Role for the Lambda

Before creating the Lambda function, first we want to create a role for it. This role will provide the necessary permissions for the function to use the EFS service.

Under the IAM console, select Create role and choose Lambda as the use case.

In Step 2, we will provide the role with the following permissions

• AmazonElasticFileSystemClientFullAccess
• AWSLambdaExecute
• AWSLambdaVPCAccessExecutionRole

To finish this step hit the Create role button.

Create the Lambda function

Now that we have our EFS and permissions setted up, the final step is to create the Lambda function and set it to interact with all the created services.

Go to the Lambda console and create a new function. Give it a name and choose which runtime will you use.

In the permissions section, select Use an existing role and select the role you created earlier.

Next, open the Advanced settings menu and select Enable VPC. Make sure you select the VPC where you created the EFS, select all of its subnets and finally select the created security group in the first step.

Create the function and once it finishes loading, inside the Configuration tab, select File systems and then choose Add file system.

Select the created EFS and its access point and provide the local mount path. In my case, this is /mnt/access, but /mnt if you left it empty before.

Finally, hit save. The EFS is now mounted in our Lambda.

Test our solution

With our resources connected, we can now test them. To do this, we can attempt writing a file in the EFS from the Lambda function.

Modify the Lambda function code to access the File system and write a .txt file in it.

import json
import os
 
def lambda_handler(event, context):
 
    print("ls before", os.listdir("/mnt/access"))
 
    with open("/mnt/access/somefile.txt", "w") as f:
        f.write("Writing a file from Lambda")
 
    print("ls after", os.listdir("/mnt/access"))
 
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

This code simply executes the ls command inside the file system, then creates a .txt file with some text in it and finally executes the ls command again to verify that the file was succesfully created.

In my case, I configured the Lambda function to use Python. If you are using Node.js, the same code can be easily written using the Node.js file system module.

After modifying the code, don’t forget to hit the Deploy button, as this action isn’t performed automatically. This ensures that the changes you made are deployed and available for execution by the Lambda function.

To execute the Lambda function and test if everything is working correctly, we can use the integrated Test functionality provided by the Lambda console.

Select “Test” and create a new Test event with the configurations shown in the image and then hit Save.

Now, hit Test again and the function should execute with the created test event. It should give us an output like this one:

By the output, we can verify that initially the directory was empty. Then the .txt file was created and finally checked that with the second ls.

Next steps

Our file system is now mounted on a Lambda function. Additionally, we have the flexibility to mount other Lambda functions or even access it from an EC2 instance. This would be a great case if we needed to install a Python or Node.js library, for example, since it is a better and more interactive way to access the file system.