Google recently announced the preview of a new feature called Log Analytics in its Cloud Logging service, allowing businesses to analyze data collected from their cloud environments.
Cloud Logging is a fully managed, real-time log management service that provides storage, search, analysis, and alerting. With the Logs Explorer feature, for example, users can search, sort, and query logs. The addition of Log Analytics powered by BigQuery allows them to perform advanced analytics using SQL to query the logs. For example, they can run a query to determine the average latency of requests sent to a cloud application. Additionally, there is a new user interface optimized for log data analysis (private preview).
When using BigQuery, users must analyze their logs with data stored outside of Logging. This can be accomplished by upgrading a log bucket to use Log Analytics and creating a linked dataset. Subsequently, with the linked dataset, users can associate their log data with other data, such as a dataset that stores known malicious URLs or data generated from hacking tools. business intelligence such as Looker and Data Studio.
Google Cloud introduced log analytics; use #bigquery to analyze your data, note that the data is not duplicated as it uses a linked dataset, querying the data in log analysis itself is free, still using standard SQL!! ! Google Cloud Analytics is another league!!!
Additionally, Charles Baer, Product Manager at Google Cloud, explains in a Google blog post that Log Analytics is powerful for its:
- Centralized logging: there is no need for duplicate copies since log data is collected and stored centrally in a dedicated log bucket
- Reduced cost and complexitybecause data can be reused across the organization
- Ad hoc analysis: users can perform ad-hoc log analysis at query time
- Scalable platformas it can scale for observability using the BiqQuery serverless platform.
Google’s competitor in the public cloud space, Microsoft, offers a similar service in Azure, also called Log Analytics (part of Azure Monitor). This is an Azure portal tool that allows users to modify and run log queries against data in the Azure Monitor Logs store. Queries are performed with a Kusto proprietary query language (KQL).
Log Analytics pricing is included in standard Cloud Logging pricing. According to the company, queries made through the Log Analytics user interface do not incur additional costs. Also, enabling analytics in BigQuery is optional. If enabled, queries submitted against the BigQuery linked dataset, including Data Studio, Looker, and via the BigQuery API, incur the standard BigQuery query cost.
Finally, users can start using the sample queries available and sign up for the charting feature in Log Analytics, which is still in private preview.