Telecommunication Company’s Use of AutoMate BPA Server Tackles their Big Data Problem

by Dallas Li, in Ask an ASE, posted 5/24/12
image for article

The Challenge

We’ve recently completed an Automation Audit with a multi-billion dollar cable telecommunications company to evaluate the feasibility of automating a mission-critical, daily process. My client’s IT team is responsible for processing and finding error codes for more than 500,000 verification files each day—that’s 21,000 files an hour, 24x7! Not only do they need to find error codes from these files, they also need to log each error count and calculate the percentage of each error occurrence. After processing these error statistics, the IT team is required to upload and update the statistical report, then compress all the verification files by regions and archive to a network location for later viewing by corporate executives.

This process was originally handled by a few batch jobs (DOS .BAT files) scheduled on Windows Task Scheduler (WTS), as well as many hours of manual work from the IT team. But the IT team soon found these batch jobs were not handling the work effectively. It took many hours, sometimes days, to track down issues if statistics were not calculated correctly or if the files were not archived properly. The business needed a better, more reliable process. Scouring possible replacement approaches from Microsoft (on MSDN and Technet) required (at a minimum) recoding all the .BAT scripts to .PS1 (Powershell scripts), and manually deploying the PS1 files and WTS controls to each Windows machine used in the process.

The Solution

The Workflow Designer and agent deployment wizard were major features in choosing AutoMate BPA Server. This would allow the cable operator to reuse existing .BAT scripts where possible, design new automation and control it all centrally using BPA’s SMC (System Management Console). After the completion of the Automation Audit, we further discovered that all the batch processes could be fully automated—in fact, reading and calculating of error statistics could also be automated to reduce latency and improve efficiency.

The entire process was converted into an AutoMate workflow by using the Workflow Designer which allows the IT team to get a big picture and to easily access the entire process. With half a million verification files running daily, the visual Workflow Designer was ideal to organize and coordinate all the files across multiple machines. And because of the unlimited agent deployment, our IT manager could spread and monitor the workload to various machines.

A note: While this AutoMate project was all implemented using the customer’s on-premise storage and machines, my client does have alternative options if his company continues to grow. For example, if my client’s business increases their workload from processing 500,000 verification files to 1,000,000, AutoMate BPA Server offers expansion options without spending heavily on efforts for redeployment and recreation of the AutoMate workflow. With the scalability of AutoMate BPA Server, additional agents can easily be deployed with just a few clicks using BPA Server’s Agent Deployment Wizard.  Moreover, with AutoMate’s built-in cloud-computing actions, such as Microsoft Azure Storage and Amazon Web Services, my client can easily migrate to cloud storage (e.g. Azure Blobs) to handle storing millions of files without the extra cost of storage hardware.

The Design

The AutoMate setup and workflows are streamlined to boost efficiency. The AutoMate workflow is responsible for monitoring different folder locations for complete creation of verification files from each region throughout the nation. Then, it utilizes 10 AutoMate agents to spread the workload to process and read more than 500,000 verification files. After all the error codes are logged and counted, the workflow calculates the error statistics for each region to create a report, which is uploaded to their network share. If my customer would like to eventually move to the cloud, the changes to the workflow would be very simple, and would only involve adding a few new actions into their existing AutoMate workflow. 

The work that used to take a full working day with a team of 10 IT professionals is now fully automated, and the entire process is completed in the background in just about two hours, without any human intervention.

To learn about this project, stay tuned for next month’s blog, I will show you how the workflow is designed and how each task is built.