Operations   Logging   Connectors & Integrations  

Announcing Honeycomb’s extension for the AWS Lambda Runtime Logs API

By George Miranda  |   Last modified on January 31, 2021

Today, AWS announced the AWS Lambda Runtime Logs API, a new way to easily send logs from AWS Lambda functions directly to your destination of choice. 

AWS Lambda Extensions, announced in October, provide the ability to run code in parallel that is independent of your function’s lifecycle. We’ve created an extension that utilizes the Lambda Runtime Logs API to send your logs directly to Honeycomb.

Using our new extension, you can more reliably send Lambda events to Honeycomb while also decreasing any added overhead, latency, and cost.

AWS Lambda Runtime Logs API

AWS Lambda now automatically captures logs and allows you to stream them to custom locations using extensions. The stream contains the logs generated by your functions, the Lambda service, and Lamba extensions. Extensions can run logging tools to receive log streams directly from within the Lambda execution environment, and then send them directly to any destination. Logging extensions can also process, filter, and send those streams to any preferred destination.

Thanks to the new AWS Lambda Runtime Logs API, it’s now possible to send events from your Lambda functions directly to Honeycomb.

CloudWatch Workarounds, No Longer!

Serverless applications using AWS Lambda faced inherent challenges when sending data to Honeycomb. Lambda functions typically have very short runtimes. Any child processes are stopped when the function terminates. Honeycomb users sending Lambda event logs faced two options: ensure data integrity (but add latency) by blocking the termination of your function until outgoing queues are flushed, or optimize for performance but potentially lose some events.

Historically, the workaround for Honeycomb customers has been to send their Lambda log messages to CloudWatch. Forwarding those logs to CloudWatch Logs makes logging a non-blocking operation in Lambda. Our CloudWatch Logs integration listens to your CloudWatch Log Groups, parses them, and sends those events to Honeycomb.

This workaround strikes a balance between minimal overhead and data integrity. But it also has some drawbacks: notably, cost and little realized benefit from using the CloudWatch Logs service itself. The new Honeycomb extension now removes the need for you to implement this workaround.

How it works

Add the Honeycomb Lambda Logs extension to your Lambda function as a Layer. Then add your Honeycomb API Key and Dataset as environment variables in your Lambda function runtime environment.

That’s it.

As long as your Lambda functions emit structured data to STDOUT, you don’t need to worry about flushing queues or connecting to CloudWatch. If you were previously using CloudWatch, having completed those steps, you can then revoke your function’s permission to write to CloudWatch.

The new Honeycomb Lambda Logs extension decreases the overhead, latency, and cost of sending events to Honeycomb while increasing the reliability of that transmission. We’re excited to make this available because it should be a big win for anyone using AWS Lambda with Honeycomb.

Give it a shot and let us know what you think.

 

Related Posts

Tracing   Logging   Customer Stories  

Confidently Shifting from Logs-Centric to a Unified Trace-First Approach: Ritchie Bros. Journey to Modern Observability

Transitioning from a monolithic system to a cloud-native microservices environment, Ritchie Bros. sought to modernize their observability infrastructure to support the transition and fuel future...

Logging   Customer Stories  

Empowering Engineering Excellence: Achieving a 26% Reduction in On-call Pages at Amperity with Modern Observability for Logs

Amperity required an observability partner to facilitate their transition into the modern engineering era as their previous tooling struggled to support their growth strategy....

Logging   Customer Stories  

Rocking the Logs: Fender's Journey to Modern Observability

Fender faced challenges with log analysis, finding it slow and complex to navigate, leading to inefficient troubleshooting and a need for a more user-friendly and...