The USDA NFC is charged with the task of processing payroll for more than 600,000 Government employees, using more than 9,000 computer programs consisting of more 6 million lines of code. Representing in excess of 140 Federal agencies, the financial and human resources services of the NFC are highly confidential. There is no room for failure.
Following Hurricane Katrina, new equipment was acquired and systems moved at a breakneck pace as the NFC strived to meet its performance deadlines. Among their hardware and software vendors, IBM maintains what may be the largest footprint. IBM equipment formed the backbone of many of NFC’s key processes, but NFC felt that the IBM’s quality and responsiveness of maintenance services had degraded significantly post-Katrina. A task order was issued to provide a management interface capable of raising the level of performance in both the Denver-based Primary Computing Facility, and the backup facility in St. Louis.
The team was successful in implementing significant additions and improvements in the performance of maintenance services provided by IBM. The first step required nearly three months and four full time employees (FTEs). However neither IBM nor the NFC could agree upon a valid list of installed systems, which greatly delayed service calls and response time. Terry Verigan’s team generated a reconciled inventory that removed roughly 150 systems that were out of use, but had continued to be billed.
In addition, the project lead negotiated an Enhanced Entitlement Services agreement (at no cost to NFC), which forced IBM to immediately respond to trouble calls regardless of the machine’s status within IBM records. Knowing that new equipment might be “found” following reconciliation, a protocol was established to limit NFC’s financial exposure as new units were added. Two team managers monitored notices from NFC staff twenty-four hours a day regarding service related issues, issuing calls for assistance from assigned IBM support staff as needs.
Verigan personally audited IBM invoices monthly in order to ensure that billings correctly represented installed inventory and that rates were being applied correctly. Errors and excessive billings were identified monthly throughout the contract. Major cost savings were also realized by recognizing that varying service levels would be appropriate for the Primary and Backup facilities, since the backup location in St. Louis was not in use 24/7. By changing the Backup facility to Next Day Response (including the IBM z9 Mainframe, which was reviewed and approved by IBM scientists), NFC was able to save nearly $1,000,000 in annual maintenance services billing. Simultaneously, an estimates $50,000 in savings was realized by the reduction of electrical power consumption.
In order to measure the success of our efforts, Verigan directed ad co-authored the first ever IBM Performance Report for services issues impacting NFC hardware and software. This created a thorough and robust catalog of maintenance issues, response time frames, failure rates, root causes and so on for the IBM systems in NFC’s inventory. The reports indicated that revised efforts were now meeting contractual commitments. This allowed the project team to focus their efforts elsewhere.