Parallel processing using multiple proxies

For on-prem sites, customers may wish to parallel process using multiple proxies. For example:

An organization has six major data centers.
Each data center has dozens of servers.
A frequent  scan of all server EventLogs is required to monitor for errors and warnings.

By deploying a proxy to each location, parallel processing can be performed.

This results in a challenge when using a single script to have all data merged into a single view:

How can each proxy know for which servers it's responsible? 

This article describes an approach to solve this problem.

Instructions

These instructions provide general guidance and should be modified to address specific organizational needs.

Prerequisite: Each server will belong to an OU that indicates the data center in which it is located.

1. Create a script (we should provide a template script) called "ServerOUs" that, for all servers in Active Directory, collects:
   1. The server name; and,
   2. Its respective data center OU.
2. Install a proxy in each data center.
3. Create a ConnectID for each proxy and for each ConnectId
   1. Create a variable called DataCenterOU
      1. Instead of being dependent on a custom OU that needs to be created up front, how about just using AD sites and focusing a proxy on the servers within that site?  This could be the default behaviour so it just works without any configuration, but if the custom OU is there, that is the trigger to override the default behaviour.
   2. For the DataCenterOU value, list the OUs, comma separated, for which this proxy will be responsible
4. Create a script to query the EventLogs that:
   1. Uses the data query feature to query the data from the "ServerOUs" script
      1. restrict the query to only the servers in the target OUs listed in the ConnectId variable "ServerOUs"
      2. This will retrieve the target list of servers
   2. Loop through the target list of servers and for each
      1. Collect the needed EventLog data
      2. Consider using PowerShell parallel processing
         1. To perform well in medium- to large-sized environments, we will absolutely need this.
5. Data will be returned and merged automatically by Cloudbridge

Related articles