AWS lambda is the great service to upload and run the code within the AWS infrastructure. I have been using Lambda since long. Let’s go through the quick overview on AWS Lambda.

“AWS Lambda is a compute service that lets you run code without provisioning or managing servers. AWS Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second.” Learn more about lambdas on AWS Lambda docs

Well, It comes with certain limitations. Some of the limitations of the AWS services can be overcome by contacting AWS but some are default and it can’t be overcome even by contacting Amazon. The great AWS Lambda micro-service comes with limitation of 5 minutes execution and this limit is by design. (5 minutes time limitation)

Task/code that can execute within 5 minutes is good to go but what if the task execution goes beyond 5 minutes? There are various ways like SNS, SQS that can come to rescue to overcome this limitation. Apart from that i have implemented two approaches.

 

Write multiple lambdas

I had a scenario that process data and impose two logical script on that processed data (that goes beyond 5 minutes). To overcome this scenario, i wrote three lambda function that calls each after the execution. But, again there are two issues that needs to take care of

  • AWS Lambda keeps invoking other lambda after one minute
  • Managing three lambda function is a daunting task

To overcome constant invoking of one lambda function from another lambda function botocore comes handy. Set, socket read timeout to 300 seconds, so that it cannot keeps the connection open till 5 minutes. And set retry-config-lambda to zero. Below is the quick code.

#defining config
config = botocore.config.Config(connect_timeout=300, read_timeout=300)

#passing config
invokeLam = boto3.client('lambda', region_name='us-east-1', config=config)

#setting max retry to zero
invokeLam.meta.events._unique_id_handlers['retry-config-lambda']['handler']._checker.__dict__['_max_attempts'] = 0

To overcome daunting task of managing three lambdas Invoking recurrent lambdas comes handy.

 

Invoke recurrent lambdas

Instead of managing multiple lambdas, i created 3 class for each and hosted it on single lambda. So, all i have to manage the desired class. Here, i am invoking the same lambda back based on the time remaining using boto3 in python. And the good part is that you can manage all the class by passing the payload. Below is the quick code for self lambda invocation.

#this will return 10 seconds
max_ms = 10 * 1000

#getting the current remaining time
remain_ms = context.get_remaining_time_in_millis()

#Checking if time is less than 10 seconds
if remain_ms < max_ms:
    try:
        #defining payload
        payload = {"count": count,
                   "again": True}
        #invoking lambda with payload
        resp = invokeLam.invoke(FunctionName = 'functionname', InvocationType = 'Event', Payload = json.dumps(payload))
        return "Called"
    except Exception as ex:
        print(ex)

This is how we can overcome the AWS Lambda 5 minutes execution limit. Well, EC2 also comes at rescue. Keep sharing. Stay tuned to Tech Tunes.

 

Follow me on Twitter