White Papers and AutoMate Literature

Solving Scripting Problems with Technology

The limitations of scripting in a complex business environment and the shift to dynamic, intelligent process automation.

From Scripting to Process Automation

Traditionally, batch files and scripts have been used to automate task-oriented processes across homo- and heterogeneous computing environments. Scripts are written to carry out routine, yet important, jobs which include backing up/clearing event logs, automating networking tasks, monitoring system performance, reading/writing to the registry, as well as managing various user accounts, computer accounts, printers, applications and services. Organizations often employ highly skilled and highly compensated programmers to write and maintain these scripts which automate various jobs that would otherwise be handled manually.

The paradigm shift from manual execution to automation by script was once considered a quantifiable business decision valued by CIOs and IT executives who aim to reduce redundancy in execution, standardize repetitive tasks, seek cost-cutting opportunities, and streamline inter-organizational processes. Nevertheless, independent research firm, Forrester Research, Inc. has stated that “Scripts lack genuine programmatic primitives that allow for changing conditions” (O’Donnell, October 2009). In the perpetual evolutionary arms race to achieve broader automation, scripting is an “extra, unnecessary layer in the automation puzzle” (O’Donnell, October 2009). You can continue using your existing investments, while adding valuable capabilities around process automation, acceleration, notification, and scheduling. Thus, as automation tools continue to evolve in a quest to deliver the most comprehensive set of automated capabilities, the notion of writing scripts will be rendered anachronistic.

The Depreciating Value of Scripting

 Let’s consider for a moment the typical approach to automating tasks by scripts. Automating in this manner does not generally consider long-term implications; it serves to address or patch an immediate need. Scripts are not aimed at broader automation in the constantly changing IT and business environment. Scripting is a patch-based approach to address problems/issues, but is not a foundation for systematic problem anticipation or productivity improvement.

Scripting technologies, such as VBScript, PHP, Perl, Javascript, and others, function as an aide-de-camp for enabling the development of applications. Scripts not only automate simple jobs that execute common O/S tasks, but also more complex IT-related processes such as automatically sending customers an invoice on the first day of each month, or updating new customers to a database nightly. Scripting is a method that IT and business units can employ as a first step in the optimization and aligning of their resources and capacity over time. Writing scripts to automate tasks often occurs organically in order to address the inefficiencies that often plague IT operations due to the tactical, “firefighting” nature of the work.

However, as organizational units extend beyond automating simple tasks to more complex processes, scalability, maintenance, cost of development and time to deployment increase dramatically. For instance, let’s look at a cost/benefit analysis (Figure 1 – Resource Allocation Spectrum) of three common methods used for process execution. With scripting and manual execution, an organization would accumulate additional costs as a result of larger wage expenditures (need for additional workers), increased time spent on code maintenance, and longer time to market. However, automation software provides more output (such as the number of processes being automated) with far less resource utilization and code dependency resulting in significantly less cost to an organization. Using a dedicated automation platform to schedule and automate complex IT and business processes eliminates having to write and manage code, saves you time by automating repetitive tasks, and optimizes the creation and deployment of automation assets across the enterprise.

 

Figure 1: Resource Allocation Spectrum

 

The transformation of IT into BT (Symons, Cameron, Leaver, & DeGennaro, August 31, 2010) has further compelled organizations to realize the limitations of scripting in terms of scalability, maintenance, and time to market as it is used to deploy automation to the enterprise. Scripting is purposefully written to minimize repetitive tasks and costs while increasing efficiency within organizations. When assessing the value of scripting in comparison to the inherent risks and challenges it poses, there is a fundamental contradiction.

The Specialized Nature of Complex Scripting

A basic script uses a simple programming language to accomplish tasks. Nevertheless, there are challenges to creating complex interactive processes across an enterprise system using a script. For instance, as a process grows in complexity, it requires more advanced scripting commands to achieve those results. As “requirements evolve, so must the CLI [Command Line Interface] and the underlying operating system that interprets the CLI” (O’Donnell, October 2009). There are filters for feeding arguments to a command, and tools for assembling the commands themselves; there are situations where command substitutions fail with too many argument errors; there are special commands for monitoring system logs, or killing processes by name.

 A script requires a special environment to run. It does not have a management framework that responds or adapts to changes dynamically. Windows, for example, provides this environment in the form of a script interpreter. The interpreter reads every line of code you write in your script and performs the task it requests. Most of the tasks performed using scripts have standard requirements. Active Directory, the Windows enterprise database, requires some special handling to work correctly. You can create a script using a text editor, such as Notepad, and a little bit of time. You can also use script-specific tools such as the Visual Basic Editor, Microsoft Script Editor, or any number of third party script editing products to create and even compile the script. Writing complex scripts requires time and many years of programming experience, especially in an enterprise software environment. Clearly, the well-known aphorism that “Hardware is Cheap, Programmers are Expensive” applies here. While experienced programmers commonly write scripts, it is not their primary responsibility and certainly not cost-effective for an organization or an ideal use of resources.

A script is intended to execute a simple task. However, basic scripts manifest into multiple script files, thousands of lines of codes, and the inevitable burden of maintaining the code. Let’s look at some of the inherent risks and challenges to writing code:

  • Time Intensive (Time to Market): Writing code is time intensive as it requires a development, debugging, and deployment phase. Whether it’s writing infrastructure code to either wait for applications to load, check for window size and position, or monitor if desktop icons have moved, it requires a programmer to spend days, weeks, or months to write cogent scripts with built-in logic.
  • Expensive: Writing code is costly to any organization. For instance, as a developer’s compensation increases, so does the cost of development and maintenance of tasks. As other projects are given to the developer, his output decreases. As such, writing code is not a scalable solution. Also, maintaining code costs money, time, and effort. As the number of interrelated tasks increase and the complexity for heterogeneous cross-platform processing is realized, the perceived value diminishes.
  • Risk of Code Abandonment: Another common risk to writing code is code abandonment. Programmers, like any opportunist, are transient and will abandon organizations and “the code” for other opportunities. This results in resources to be realigned within an organization to accommodate the sudden change, which causes a loss in productivity, loss of resources, and increase in cost to remedy the change. In addition, organizations may feel compelled to keep an otherwise expendable employee who is solely responsible for the development, upkeep, and maintenance of a particular application.

Adapting to Changing Conditions within Organizational Units

Today’s IT environment consists of multiple servers, both physical and virtual, platforms, and a wide range of applications. Organizations are in need of a centralized approach in synchronizing resources, where appropriate, into one easy-to-use scalable framework. Faced with pressures, demands, and limitations, CIOs and IT executives need solutions that can streamline and automate time-consuming and costly IT processes. Developers need user-friendly tools to shorten the application development cycle. Business analysts need an understanding of the existing business process as well as the effectiveness of producing results.

Does scripting provide a unified view of business processes across the network and ensure reliable execution of processes? The answer is “No.” Organizations need a centralized automation platform using pre-built functionality to build “end to end” workflows without having to write code. There should be the capability to perform various job types such as:

  • FTP/SFTP
  • Website Data Extraction
  • Process Jobs (exe, scripts, etc.)
  • Automate Active Directory Processes
  • File System Operations (Create, copy, delete folders, files, and more)
  • Automate SNMP-enabled network devices

The task of coordinating a series of individual jobs across servers, platforms, applications, firewalls, time zones, and wide area network (WAN) links can be daunting. When the jobs depend on one another, IT needs a system that allows these different tasks to work together seamlessly.

Solving Scripting Problems with Technology

Don’t abandon your scripts, yet. Automation software is an enabling solution that magnifies on existing script automation capabilities. Automation software can leverage existing scripts to deliver automation in a managed framework in order to gain valuable insight and reporting while allowing organizations to avoid the “hidden expense” associated with custom script development and maintenance going forward, thus reducing the overall cost of operations and delivering a positive ROI fast. Here are some benefits to implementing automation software:

  • Eliminate Code, Reduce Errors: Reduce costs and hassles associated with developing and maintaining code, scripts, and batch files. Reduce machine and employee downtime stemming from process errors or delays and errors associated with maintaining code.
  • Curb Limitations: Extend the capabilities and overcome the limitations imposed by legacy systems and applications as well as the knowledgebase of the individual(s) responsible for writing scripts whose core function may or may not be as a developer.
  • Accelerated Information Flows: Accelerate data flows, making information workers (i.e. business users and decision-makers) more effective and efficient by delivering real-time access to accurate data.
  • Free Employees: Eliminate time-consuming, labor-intensive, repetitive IT tasks; enabling employees to focus on strategic activities where human interaction and thought add the most value.
  • Centralize Resources: Improve security and control of IT processes by centralizing the management of the automation network.
  • Facilitate a Cohesive Organization: Improve collaboration between business users who benefit from automated processes and IT professionals responsible for delivering automation.

Organizations continue to place high value in the ability to reduce costs and risk through IT and business process automation across heterogeneous tools and infrastructure that exist in every data center. IT and business units are able to quickly replace external scripts and manual tasks and take their automation efforts to the next level, resulting in improved efficiency. When coupled with a ROI calculation approach, the automation strategy can be assessed quantitatively

ROI Methodology

ROI can be calculated as Total Benefit derived from automation divided by the Total Cost of automation.

ROIAuto = BenefitAuto/CostAuto

Traditionally, the costs used are the costs for automation (hardware, software, licenses, and training). The benefits are calculated by looking across a time interval at the savings of automation over manual execution (reduced time to execute tasks/processes, ability to execute 24 hrs a day, etc.). The benefits matrix below (Table 1 – Benefits Matrix) depicts the distribution of direct and indirect benefits across people, process, and technology drivers, which can be translated to customer benefits.

 

Table 1: Benefits Matrix

 

Benefits of Automation

Automation tools are capable of running continuously without any productivity loss or fatigue, with minimal or no manual intervention. This implies that organizations can plan automation activities beyond the traditional eight-hour work shift, reducing the elapsed duration for execution.

Cost Savings with Automation

CIO’s are responsible for aligning a company’s technology with its strategy and business practices. They need software that can bridge the gap between the IT department’s resources and individual business unit requirements. They need software that is extremely flexible so they can tailor it to their business requirements as they are. Yet, at the same time, they need to deploy robust automation applications quickly without the hassles of managing an army of coders and developers. This translates into significant cost savings.

For instance, Figure 2 illustrates the cost savings for executing a single job (e.g. FTP manipulation, data extraction, print job, etc.). For purposes of demonstrating the economic value1 of automation, it’s necessary to compare the Marginal Costs (MC) of achieving the same results using scripting or through manual execution. In general, the efficient level of output is where Marginal Revenue (MR) equals Marginal Cost (MC). MR is the same despite the method of execution (i.e. manual, scripting, or automation) because it is an independent variable.

 

Figure 2: Cost Savings of Automation

 

Let’s consider for a moment the output of jobs within an IT or business unit to determine how it satisfies our value proposition for automation. As a company increases their output of processes for a certain project, their total cost for the project will inevitably increase. But, at what rate will these costs increase as total process output becomes higher? Imagine automating a workflow process that combines data from two spreadsheets, uploads the results to an FTP server, and sends a confirmation e-mail. The exact sequence of steps used in the process would be:

  1. Create a temporary folder on the local drive.
  2. Locate two Microsoft Excel spreadsheets on separate drives.
  3. Copy these two files to the newly created folder.
  4. Create a new blank spreadsheet.
  5. Combine data from the two original spreadsheets into a new spreadsheet.
  6. Upload the combined spreadsheet to an offsite FTP server.
  7. Create and send an email message with the result to multiple recipients.

Basically, with automation, the difference to add or manipulate the workflow process above involves simple logistics without any change in resources. Create the process once and schedule a daily run - that’s it. Achieving the same results with manual execution mandates additional days, more resources, increased costs, and other factors. As the output of processes increase so does cost, but at a more invasive rate. This can be seen at the tail end of the manual and scripting MC curves.

With automation, there is an initial learning curve (and associated costs), represented by Ea2 , to understand, train, and implement automation assets across IT and business organizational units. Once Ea is reached, the MC for automation is minimal over time. Lastly, automation provides organizations with a far larger Economies of Scale3 when compared to other methods of process execution.

Automation in the Real World

Let’s look at a common FTP operation within IT and business organizations and compare the execution methods using a Shell Script versus AutoMate.

FTP Using a Shell Script

The following shell script connects to multiple FTP servers, makes three (3) connection attempts, uploads a file and disconnects. Note that in each connection attempt, if the maximum attempts fail, an error is generated, and there is a 10 second wait period before additional attempts are made.

 

CodeSnippet

 

FTP Using AutoMate

The sequence of steps below allows you to achieve the same results as above using AutoMate’s drag-and-drop functionality.

 

Figure 3: FTP Using AutoMate

 

The following AutoMate task (Figure 4 – Advanced FTP) goes a step further and grabs all files on the FTP server, loops through each file and deletes those greater than 30 days. The estimated time to create these additional steps in AutoMate takes less than five minutes.

Now consider the time, cost, and resource it would require to plan, write, and test a script to automate these additional steps.

 

Figure 4: Advanced FTP

 

The Shift to Dynamic, Intelligent Process Automation

The shift to dynamic, intelligent process automation stems from IT executives constantly grappling with rapidly changing business conditions and requirements. Current market conditions demand higher productivity, better customer service, and a tighter integration between the enterprise and its customers and partners. Complicating matters is that IT departments have a hard time understanding the language of business managers and users, so IT project delays are commonplace. This can be very frustrating in a competitive environment where the imperative to change along with business conditions is paramount.

Forrester Research, Inc. states that the “evolutionary paths of automation will continue unabated” and that “performance tools can now act as triggers for task automation…” (O’Donnell, October 2009). Thus, automation will bridge the gap between IT and Business units, enabling it to change the way IT departments deliver data and services to information workers, thereby adding new dimensions of flexibility and agility to their business. With automation, enterprises can quickly realize the benefits of IT process automation - driving down management and operation costs and increasing productivity - while enforcing compliant best practices aimed at improving performance and availability of critical business applications.

Clients