Premiere Global Services
Automation:
Situation: Each morning (before 6AM) a sql server batch program would initiate a production sequence consisting of 1) collect data from bridge hardware, 2) process the data through required business rules, and 3) transfer data to our invoicing partner. This process was repeated in EMEA and APAC at their installations. Our finance teams were only interested when the last step was complete. This process typically collected over 2 million rows and involved nearly 100 stored procedures. But often times dirty data would cause the stored procedure to abort and would require manual intervention. In addition, design "enhancements" would incur additional processing delays which were not observed during testing prior to production release, and "mysterious" network interruptions would hang the final transfer process.
Task: Mitigate processing delays and make process status visible on demand to team members from mobile devices anywhere anytime. Additionally alert appropriate personnel when process thresholds were violated and provide timely and quantifiable feedback to database designers responsible for production process updates.
Action: Using c#/.net, developed a custom dashboard to monitor progress of each primary step in each region and provide an interface to distribute the status on demand. Stand up a dedicated server and automation user to "snapshot" process status at predetermined intervals, and to deliver the snapshot via custom email status inquiries (as mobile devices were not yet authorized to connect to the intranet through the vpn). Identify common data exceptions and prevent recurrence with procedural code updates. Benchmark and visualize process step duration with Splunk and alert on violation of preset thresholds. Finally, document the process and train team members how to request status via email and recover from fatal process errors.
Result: Significantly fewer completion delays, global process status at a glance from anywhere using any email enabled device, timely and quantifiable feedback of process changes, and quicker recovery from process aborts.
McNeil Technologies
Help Desk reporting:
Situation: ITILS project identified multiple reporting deficiencies. Help Desk Incident reporting lacked a flexible drill down status capability to include ticket age, customer satisfaction, time spent on ticket (-hold/vendor stats). Tickets also were only loosely coupled to location information because of remote submittal ownership issues and a constant flux of reporting hierarchy requirements. No BMC flashboards or analytics capabilities. Also some data was tainted because of inconsistencies in how some sites reported data.
Task: Identify where missing data could be displayed in existing report without changing layouts. Build an OLAP capability into the form to accommodate date ranges and location rollups. Supplement reporting forms to include help tips to describe what type of data went where and re-evaluated which fields and forms were required for each incident and make them required entries on the form. Build an application to identify and report inconsistencies and build a scheduled item to report anomalies in daily email to developers. Create a review process to identify causation and close the loop. Identify inconsistencies in customer survey process and make surveys easy, automatic and mandatory.
Action: Assembled a team of SMEs to include customer, Remedy programmer, and ITILs expert. Created development environment on isolated resource using VMWare and replicated a representative data sample. Developed an iterative upgrade rollout plan. Enhanced production bug tracker, and integrated VSS for production revision management. Made necessary Remedy form and field changes in dev environment. Updated SQL views to include missing information. Converted the DSR from VB to C#.net and modified the DSR to include necessary buttons for reporting flexibility. Added Customer Survey default entry with an email override capability, simplified satisfaction calculations, and added automated monthly summary and star chart viewing in support of SLA audits. Added graphics pages to DSR to provide status at a glance. Documented these for the next version of Remedy (ITSM) to be installed. and duplicated in the next version test environment. Developed all changes incrementally in dev environment before rollout to production. Provided production rollback capability in event changes had adverse effect on other systems. Developers met daily to review anomaly reports and quickly updated the process to close the gap.
Result: Customer could satisfy all ITIL deficiency objectives and we could carry them forward to next release of Remedy ITSM. Customer could manage open tickets better and close them faster. Customer satisfaction went up measurably. Less time was spent “massaging” the DSR and prepping for SLA audits.
OSI Reporting:
Situation: Firewall logs contained contact information in compressed form from every site. OSI needed a way to quickly and frequently identify specific contacts - who initiated them, what protocol was used, how much data was transferred, and when and where it occurred. Small prototype was built using SQL server BI, but data was collected and stored locally on unix systems. This manual application rarely completed and a backlog of contact information was the norm.
Task: Develop a fast robust firewall log parser and integrate with a similar process for analyzing email logs. Move logfiles to local storage to eliminate network bottlenecks and reduce single point of failure risks.
Action: Moved logs to local SAN storage. Provide user interface to identify which contacts were of interest. Develop application to quickly parse compressed log files and report on desired contact information (this required a new unix processing architecture). Massage contact information to supplement location information and integrate with geo reporting capability (similar to existing email application). Cron the process to meet federal guidelines. Archive the information for quick retrieval.
Result: Customer was able to research information in near-real time greatly exceeded expectations and automating much of the research required for contact investigations.
Intel
Design automation:
Situation: Needed a way to ensure each hardware project was using the same version of design software at multiple sites (cross-site synchronization). Allow hardware designs 5 years old to be resurrected for troubleshooting and product variants. Operating System, revision control software, 3rd party design software (Caliber, Hercules, LVS, simulators, collateral etc.), fab process details, design package (deliverable to internal/external fabs) must all be tracked by product.
Task: Create versioning for the design environment used. Simplify the documentation of which environments used which tools and which products used which environment. Automate design environment versioning based on tools used. Document specifics in public location and in product delivery. Train project owners how to decide which environment to use.
Action: Create a team of SMEs at each site – gatekeeper was mandatory participant. Developed a sophisticated rsync wrapper to ensure design software was identical at each site. Established baseline OS environments. Created project sharepoint/wiki sites and archival specifications. Monitor progress with regular conference calls.
Result: CAD team could guarantee designs using this process could be rebuilt from scratch regardless of how old they were or where they originated.
|
