How log management can give Asian businesses a competitive edge

By 2025, global data creation is projected to grow to more than 180 zettabytes (ZB). For organisations across Asia working on digital transformation initiatives, it's no surprise that the data generated as part of these projects can make the creation of actionable insights incredibly challenging. How can this data be managed? The answer is log management. An integral part of DevOps and IT practices, logs are a type of telemetry data; recording everything that happens in an application, network, or server. 

Thanks to effective log management, India's largest omni-commerce jeweller, CaratLane, has ensured uptime, performance, and reliability, resulting in developers having a comprehensive view of what's happening anytime. The result? The business has scaled to handle an unprecedented increase in demand, with online sales up 90% year-on-year.

With log management having such a positive impact, New Relic commissioned the 2022 State of Logs report, which analysed data taken from millions of applications and identified three key insights that technology teams should explore to take full advantage of this vital data management approach.   

1. View logs in context

Despite its apparent benefits, many organisations still adopt a decentralised approach to log management. This outdated method of using separate tools to accomplish tasks is confusing, costly, and time-consuming and can significantly impact the ability to detect and respond to security incidents.

To ensure that collected application logs are useful, engineers and developers must be given the opportunity to view logs in context. They should have access to log data alongside application performance monitoring (APM) and infrastructure monitoring, making it easier for them to find the problematic needle in the haystack.

The 2022 State of Logs report found that 56% of customers use infrastructure monitoring together with logs, and approximately 14% use logs alongside APM monitoring. The number of customers using APM alongside logs has grown by a whopping 68% year-over-year, with this percentage expected to rise in the future.

2. Leverage open-source tools to prevent information silos

Tech teams have a long history of using open-source tools, saving them a tremendous amount of time examining thousands of log files from multiple sources to pinpoint and resolve problems.

The 2022 State of Logs report identified Fluent Bit as the most used open-source processor and forwarder tool amongst New Relic users. Thanks to its ability to manage observability data at scale across cloud-native, IoT, and bare metal environments, it is now viewed as the industry standard.

3. All eyes on Amazon Firehose 

The use and growth of Amazon Firehose has been significant in recent years, even while Lambda has been the primary Amazon Web Services (AWS) serverless tool for engineers seeking to transfer logs and metrics into CloudWatch automatically.

Currently, 32% of New Relic accounts using AWS have adopted Amazon Firehose, but its usage has increased 62% year-over-year compared with 23% year-over-year growth for Lambda. The report also shows that engineers continue forwarding logs from Lambda but that new deployments increasingly favour forwarding from Firehose.

With the 2022 State of Logs report finding that there has been a 35% year-on-year increase in logging data, it's clear that this information is a crucial resource for technology teams. Log management improves operational efficiency and supports teams to find the right information quickly, which is why its adoption is only set to grow in the future. If organisations leverage their telemetry data, they are likely to become proactive rather than reactive, which can be a huge competitive advantage in a crowded market.