What is AWS Lambda?

What is AWS Lambda? Its approach to serverless computing has transformed DevOps. But how do you get the benefits without sacrificing observability?

The 2014 launch of AWS Lambda marked a milestone in how organizations use cloud services to deliver their applications more efficiently, by running functions at the edge of the cloud without the cost and operational overhead of on-premises servers.

What is AWS Lambda?

AWS Lambda is a serverless compute service that can run code in response to predetermined events or conditions and automatically manage all the computing resources required for those processes. It also enables DevOps teams to connect to any number of AWS services or run their own functions.

Organizations are realizing the cost savings and management benefits of serverless automation. But with the benefits also come concerns about observability, and how to monitor and manage ever-expanding cloud software stacks.

The Amazon Web Services ecosystem

AWS has worked for years to build a comprehensive suite of web services for customers. These include website hosting, database management, backup and restore, IoT capabilities, e-commerce solutions, app development tools and more, with new services released regularly. Lambda, the company’s serverless computing platform, integrates with these AWS services to provide customers with a rich ecosystem for creating and running large, complex applications.

Organizations can offload much of the burden of managing app infrastructure and transition many functions to the cloud by going serverless with the help of Lambda.

AWS Lambda enables organizations to access many types of functions from AWS’ cloud-based services, such as:

• Data processing, to execute code based on triggers, system states, or user actions
• Real-time file processing, for quickly indexing files, processing logs, and validating content
• Real-time stream processing to perform live activity tracking, data cleansing, metrics generation, and more
• Machine learning, allowing companies to preprocess data in the cloud before feeding it to ML models

There is a vast and growing range of use cases for these on-the-fly, front- and back-end processing capabilities — from manipulating data for modifying genomes to identifying trends in sensor data and setting triggers for automated responses.

Where does Lambda fit in the AWS ecosystem?

Customizing and connecting these services requires code. You will likely need to write code to integrate systems and handle complex tasks or incoming network requests. This is where Lambda comes in: Developers can deploy programs with no concern for the underlying hardware, connecting to services in the broader ecosystem, creating APIs, preparing data, or sending push notifications directly in the cloud, to list just a few examples.

How does AWS Lambda work?

AWS generates containers when you package your code with the Serverless Application Model (SAM) or directly in the Lambda console. A trigger causes your container to work until it reaches a period of inactivity — the end of the job. The container goes inactive until triggered again. Many events can trigger a lambda function.

Some common examples include:

• A request through API Gateway or Amplify
• A new record entering a database table
• The completion of a Step Function
• Data entering a stream
• An email pushed to AWS Simple Email Service

The function itself performs a small unit of work and Lambda charges subscribers by the millisecond.

The benefits of serverless Lambda functions

The greatest benefit of Lambda is the ability to rapidly scale up an organization’s computing capabilities and capacity. Lambda’s highly efficient, on-demand computing environment aligns with today’s microservices-centric architectures, and readily integrates with other popular AWS offerings that an organization may already be using. Lambda’s toolbox of automated processes helps developers streamline to build fast, robust, and scalable applications on accelerated timelines.

As The New Stack reports, developers spend only 32% of their time at work actually coding. Using AWS Lambda lets developers offload the administrative portion of their work to focus on their other core function — writing great code. As a bonus, operations staff never needs to update operating systems or hardware, because AWS manages servers with no stoppage of application functionality.

Finally, SAM templates enable you to configure your stack programmatically, and continue to use your preferred code repositories, such as GitHub and GitLab. Creating and managing a complicated codebase might negate the benefits of serverless technologies, but Lambda eliminates this concern.

When not to use AWS Lambda

Despite these cost-cutting, time-saving advantages, using AWS Lambda is not ideal for every use case.

Lambda works well in cases where functions deal with a steady stream of requests, handling them in less than a few seconds. Tasks like API requests, database calls, and file system management are perfect candidates for this service. On the other hand, handling CPU-intensive tasks and constantly high request volumes will run up a hefty bill.

For such functions, organizations will be better off using an EC2 instance or their own hardware to interact with data. Before you commit to leveraging AWS services, architect your solution and perform a cost analysis to make sure you can gain the most benefit.

Optimizing Lambda for performance

One factor that dissuades many from using Lambda is the need to restart containers. You can eliminate the latency issues caused by cold starts — an increase in normal response time when a new instance receives its first request — by using edge-optimized functions that run code closer to users and other projects. AWS continues to improve how it handles latency issues. For example, the new HTTP API reduces response times while also promising a reduction in cost by as much as 80%.

Still, demanding applications may run into issues related to spinning up new containers. You can use CloudWatch to keep the container “warm” by targeting each endpoint periodically for a minimal cost. This is also a great way to check on the health of your functions and optimize the amount of memory and processing power available.

How do AWS Lambda functions impact monitoring?

Relying on microservices or serverless functions will add complexity to an application, making it all the more important that organizations are able to keep tabs on their environments.

When you set up AWS to push logs to CloudWatch, each function receives its own log group under which logs aggregate. Unfortunately, the Lambda console does not let you push to a new group, making it difficult to track many functions in their full context. An application could rely on dozens or even hundreds of Lambdas and other infrastructure. Trying to discern a problem’s root cause through manual log analysis in this scenario is daunting.

Default logging in Lambda is useful, but it makes end-to-end monitoring of your functions difficult. An even bigger challenge is gaining meaningful visibility into traces, start-to-finish records of all the events that occur along the path of a given request. Although distributed tracing is critical for evaluating performance and mapping dependencies throughout the cloud stack, handling this data without automated help is impossible as the number and complexity of services grows, which creates blind spots for your staff.

How to get the most out of Lambda without sacrificing observability

AWS Lambda’s wide array of data processing and computing capabilities helps organizations accelerate innovation, but often at the cost of visibility and end-user context. When developers and DevOps teams must spend more time instrumenting Lambda apps and services, they can’t focus on building and shipping new services.

To get the most out of it and the systems it interacts with, teams need end-to-end observability that uses automation and AI-assistance to extend beyond metrics, logs, and traces and includes data from open-source initiatives and additional context from the end-user perspective.

Dynatrace and its deterministic AI engine, Davis, provides automatic and intelligent observability across the entire application ecosystem, offering precise insights with root cause detailing how Functions as a Service (FaaS) impact user experience and business outcomes. With full visibility into AWS serverless workloads in full context of the user experience and business outcome metrics, DevOps and SRE teams can optimize AWS Lambda functions, enabling them to innovate faster with greater confidence and less risk.

Automatic, cold-start detection for every Lambda invocation is another crucial capability teams need to avoid significant lags and poor customer experience. Dynatrace automatically detects and tracks cold starts of serverless resources so teams can design strategies for handling the effects of cold starts, like warming up functions or configuring provisioned concurrency for your functions.

But beyond these IT and APM benefits, Dynatrace assists its customers with workflow management via DevOps optimizations. It helps SRE teams automate responses. It can prioritize issues identified from RUM data. It shows teams in real-time how every process affects the end-user experience and calculates the impact of downtime, latency, and inefficiency on business outcomes.