Azure Monitor: Collect Logs and Metrics from On-Premises

Azure Monitor: Collect Logs and Metrics from On-Premises

Reading Time: 5 minutes

In this blog we are going to discuss how we can collect logs and metrics from the Azure resource and on-prem infrastructure to the azure monitor.

data flow between different components

Suppose we have an application running on on-premises so we need to collect the logs and metrics from it and send it to Azure log analytics for analysis of the logs and metrics and create dashboards for the same.

Azure Monitor can collect data directly from your physical or virtual Linux computers in your environment into a Log Analytics workspace for detailed analysis and correlation using the azure log analytics agents.

Installing the Log Analytics agent allows Azure Monitor to collect data from a data center.

Before analyzing and acting on collected data, you first need to install log analytics agents on all of the machines that you want monitor.

Log analytics agent sends

• System logs,
• Performance metrics
• Custom logs from any location

Prerequisite:

• Log analytics workspace 
• Log analytics workspace ID and Primary key

The agent communicates outbound to the Azure Monitor service over TCP port 443

Supported Linux operating systems:

• Amazon Linux 2017.09 (x64)
• CentOS Linux 6 (x86/x64) and 7 (x64)
• Oracle Linux 6 and 7 (x86/x64)
• Red Hat Enterprise Linux Server 6 (x86/x64) and 7 (x64)
• Debian GNU/Linux 8 and 9 (x86/x64)
• Ubuntu 14.04 LTS (x86/x64), 16.04 LTS (x86/x64), and 18.04 LTS (x64)
• SUSE Linux Enterprise Server 12 (x64) and 15 (x64)

Network firewall requirements

The information below lists the proxy and firewall configuration information required for the Linux and Windows agents to communicate with Azure Monitor logs.

Agent ResourcePortsDirectionBypass HTTPS inspection.ods.opinsights.azure.comPort 443OutboundYes.oms.opinsights.azure.comPort 443OutboundYes.blob.core.windows.netPort 443OutboundYesFirewall Rules

Install the log analytics agent for Linux

To configure the Linux computer to connect to a Log Analytics workspace, run the following command providing the workspace ID and primary key copied earlier. 

Step-1: The following command downloads the agent validates its checksum and installs it.

Enter the loganalytics workspace id and primary key.

wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -w <YOUR WORKSPACE ID> -s <YOUR WORKSPACE PRIMARY KEY>

Step-2: Restart the agent by running the following command:

Enter the loganalytics workspace id.

sudo /opt/microsoft/omsagent/bin/service_control restart [<workspace id>]

Collect event and performance data

Azure Monitor can collect events from the Linux Syslog and performance counters that you specify for longer-term analysis and reporting. It can also take action when it detects a particular condition.

Follow these steps to configure the collection of events from the Linux Syslog, and performance counters.

1. Go to Azure Portel
2. Search log analytics
3. Select your log analytics workspace
4. Click on advance setting
5. Select Data, and then select Syslog.
6. You add Syslog by typing in the name of the log. Enter Syslog and then select the plus sign +.
7. In the table, uncheck the severities Info, Notice and Debug.
8. Select Save at the top of the page to save the configuration.
9. Select Linux Performance Data to enable collection of performance counters on a Linux computer.
10. When you first configure Linux Performance counters for a new Log Analytics workspace, you are given the option to quickly create several common counters. They are listed with a checkbox next to each.
11. Select Apply below configuration to my machines and then select Add the selected performance counters. They are added and preset with a ten-second collection sample interval.
12. Select Save at the top of the page to save the configuration.

View collected data

Now that you have enabled data collection, let’s run a simple log search example to see some data from the target computer.

1. Go to your log analytics workspace
2. In the selected workspace, from the left-hand pane, select Logs.
3. On the Logs query page, type Perf in the query editor and select Run.For example, the query in the following image returned 10,000 Performance records. Your results will be significantly less.

Log Analytics Agent data sources:

These are the data source of the agent:

Performance Metrics to Monitor the Instance:

These are the performance metrics which we are getting by the azure log analytics agent.

Object NameCounter NameLogical Disk% Free InodesLogical Disk% Free SpaceLogical Disk% Used InodesLogical Disk% Used SpaceLogical DiskDisk Read Bytes/secLogical DiskDisk Reads/secLogical DiskDisk Transfers/secLogical DiskDisk Write Bytes/secLogical DiskDisk Writes/secLogical DiskFree MegabytesLogical DiskLogical Disk Bytes/secMemory% Available MemoryMemory% Available Swap SpaceMemory% Used MemoryMemory% Used Swap SpaceMemoryAvailable MBytes MemoryMemoryAvailable MBytes SwapMemoryPage Reads/secMemoryPage Writes/secMemoryPages/secMemoryUsed MBytes Swap SpaceMemoryUsed Memory MBytesNetworkTotal Bytes TransmittedNetworkTotal Bytes ReceivedNetworkTotal BytesNetworkTotal Packets TransmittedNetworkTotal Packets ReceivedNetworkTotal Rx ErrorsNetworkTotal Tx ErrorsNetworkTotal CollisionsPhysical DiskAvg. Disk sec/ReadPhysical DiskAvg. Disk sec/TransferPhysical DiskAvg. Disk sec/WritePhysical DiskPhysical Disk Bytes/secProcessPct Privileged TimeProcessPct User TimeProcessUsed Memory kBytesProcessVirtual Shared MemoryProcessor% DPC TimeProcessor% Idle TimeProcessor% Interrupt TimeProcessor% IO Wait TimeProcessor% Nice TimeProcessor% Privileged TimeProcessor% Processor TimeProcessor% User TimeSystemFree Physical MemorySystemFree Space in Paging FilesSystemFree Virtual MemorySystemProcessesSystemSize Stored In Paging FilesSystemUptimeSystemUsers

Data Retention in azure log analytics workspace:

The retention period of the collected data stored in the database depends on the selected pricing plan. Collected data is available for 31 days by default but can be extended to 730 days. Data is stored encrypted at rest in Azure storage, to ensure data confidentiality, and the data is replicated within the local region using locally redundant storage (LRS). The last two weeks of data are also stored in SSD-based cache and this cache is encrypted.

Conclusion:

We can collect logs(System or application logs) and performance metrics from the log analytics agent and It is totally secure. But this agent can’t send the application level metrics like java application. For the application-level metrics, we should go for the azure application insight which is part of the azure monitor.

Thank you for sticking to the end. If you like this blog, please do show your appreciation by giving thumbs ups and share this blog and give me suggestions on how I can improve my future posts to suit your needs. Follow me to get updates on different technologies

References: