Example blog

How to configure automated log collection with PowerShell

Computer science is one of the few jobs where you actively seek trouble.

Administrators should get into the routine of checking the logs of on-premises Windows Server systems and the Office 365 environment to avoid being taken by surprise. Part of the problem is the amount of work required to gather logs from disparate locations. But an automated log collection script written in PowerShell can run regularly to keep administrators informed of any concerning developments, regardless of their origin.

Why reviewing security logs should be routine

A key part of a security assessment is whether IT or the security team regularly reviews the logs. In many cases, these logs are only reviewed when trying to resolve an ongoing issue or find out how an incident started. This makes the log review process for the IT team and the end user slow, time-consuming, and often tedious.

Over the past few years, cybersecurity events have increased and become more sophisticated and will often overwhelm the security tools designed to detect them. The benefit of standard log checks is that evidence of identifiable entries can help find the source of an attack. Many organizations that store events in security information and event management (SIEM) platforms still require proactive interaction and execution to resolve specific events. For other organizations that don’t use a SIEM platform, there needs to be a mechanism to efficiently retrieve entries. Many applications use different formats and getting the logs can be tedious.

Why is on-premises and cloud log collection difficult?

Many organizations have difficulty retrieving log entries from on-premises and cloud services for several reasons. The most common is format and authentication. For example, the format of the Windows Server event log differs from Office 365 service audit logs or even direct logs from a specific service such as Exchange Online. The formatting process to format the data can be complicated and time-consuming.

Authentication for on-premises log collection tends to be much easier, whereas the same administrative work for a cloud service requires specific PowerShell modules, credentials, and commands. For example, retrieving all entries from the Security event log on a Windows server, you can use the Event Viewer interface and export as needed. To retrieve journal entries for Exchange Online, you must use PowerShell. First, you need to install and import the required module, login to the service with the necessary account permissions, and then run the appropriate PowerShell command. It becomes much more complicated to collect log entries from multiple on-premises servers or cloud services.

How does PowerShell make log retrieval easier?

On-premises servers and cloud services such as Microsoft 365 support PowerShell to make automated log collection less difficult. For example, you can create one script to query multiple Windows Server logs and another to query Microsoft 365 service logs, then store them in a particular location for further inspection. You can then convert these scripts into scheduled tasks for a fully automated method. With Microsoft cloud services, you can schedule PowerShell tasks in Azure to perform these tasks as needed.

How can you retrieve event logs from data center servers?

Windows Server provides the Event Viewer interface for manually managing event logs and entries. This is the easiest way to work with journal entries. The Event Viewer displays the different locations such as Application, Security, System, as well as the specifics of the application; for example, if you use Active Directory Federated Services (ADFS) on a server, a corresponding log segments entries for the application.

Windows Event Viewer provides a graphical interface used to view and export log entries.

In Event Viewer, you can export entries to a flat file and then import the log into other applications for further querying and inspection.

Another option to export the log is to use PowerShell. A command called Get-WinEvent is designed to retrieve event log entries. For example, the following command retrieves all events from the Security Log:

Get-WinEvent -LogName "Security"

To limit the entries returned, set the maximum number to return.

Get-WinEvent -LogName "Security" -MaxEvents 10

To filter the same log entries on a specific event ID, you use a Hash table filtered.

$id = "4798"
Get-WinEvent -FilterHashtable @{ LogName="Security"; Id=$id }

A standard PowerShell export command generates the selected entries.

$path = "C:ExportResults.csv"
$id = "4798"
$events = Get-WinEvent -FilterHashtable @{ LogName="Security"; Id=$id }
$events | Select ID, Message | Export-Csv -Path $path -NoTypeInformation

To run the same export command for multiple servers, you can use the following command.

$path = "C:Export"
$id = "4798"
$computers = @("DC","ADFS","SQL","WEB")
foreach ($computer in $computers)
$events = Get-WinEvent -ComputerName $server -FilterHashtable @{ LogName="Security"; Id=$id }
$events | Select ID, Message | Export-Csv -Path "$path$server.csv" -NoTypeInformation

PowerShell automation makes hard but necessary work easier. This review process should be part of the routine of an IT department.

How do I retrieve Microsoft 365 Services event logs?

For Office 365/Microsoft 365 customers, Microsoft offers PowerShell modules to connect, perform configuration tasks, and collect things like log entries. The various admin consoles also run queries and provide the ability to download results, but the best approach is to use PowerShell, especially when merging results with on-premises server logs.

The complicated part of using PowerShell with Office 365 is understanding which PowerShell module to use and the different commands. Each service has its own module and commands, with services such as Exchange Online and security and compliance features grouped together in the same module. The two primary ways to sign in to Microsoft 365, regardless of module, are either with pass-through credentials or with application registration in Azure Active Directory (Azure AD). The most common approach is to use credentials.

You can also retrieve log entries from the security and compliance components, which is the central access for all audit logs. To retrieve audit log entries, use the command Search-UnifiedAuditLog Where Search-AdminAuditLog, depending on whether you need end-user or admin entries. Office 365 also provides a dedicated command to search for Exchange Online mailbox entries called Search-MailboxAuditLog.

Using the standard audit log requires installing and importing the appropriate PowerShell module.

Install-Module ExchangeOnlineManagement
Import-Module ExchangeOnlineManagement
Connect-ExchangeOnline -UserPrincipalName [email protected]

Once the PowerShell modules are loaded and you sign in to Microsoft 365, you can run the required commands. A basic search for end-user log entries uses the following command.

Search-UnifiedAuditLog -StartDate 1/1/2022 -EndDate 2/1/2022

If you know the record types or input types you need, you can filter them using the following commands.

Search-UnifiedAuditLog -StartDate 1/1/2022 -EndDate 2/1/2022 -RecordType SharePointFileOperation
Search-UnifiedAuditLog -StartDate 1/1/2022 -EndDate 2/1/2022 -RecordType MicrosoftTeams
Search-UnifiedAuditLog -StartDate 1/1/2022 -EndDate 2/1/2022 -RecordType ExchangeAdmin

You can use the same commands described for on-premises servers to export entries.

$path = "C:ExportResults.csv"
$events = Search-UnifiedAuditLog -StartDate 1/1/2022 -EndDate 2/1/2022 -RecordType MicrosoftTeams
$events | Select UserIds, Operations, AuditData | Export-Csv -Path $path -NoTypeInformation

As you can see, the PowerShell syntax is relatively straightforward for on-premises Windows Server systems and cloud services. By combining the two areas of concern and scheduling them as tasks, IT teams will develop a proactive way to review and monitor logs to detect potential security incidents or to control issues before they cause harm. disruptions that affect the entire organization.

Source link