Occasionally, I use this column as cheap therapy. Forgive the indulgence, but my story may save you a lot of headaches if you are planning on tinkering with your computers, network or servers any time soon. This isn’t a typical column about an industry specific topic. But most companies today have some information technology (IT) that is critical to running their business.
My story starts like many IT projects that go astray. It all began with good intentions and naive expectations that everything would go as planned. Oops!
The idea was to upgrade the server, update some sensitive database programs that had come out with new versions, migrate machines from Windows XP to Windows 7, install new anti-virus programs, and setup the user profiles to be managed by the server so they could be accessed from any machine on the network. We wanted to do all of that with an IT staff of three over the weekend. We have one server and 10 work stations. While it was an ambitious set of tasks, it seemed doable.
That was until everything just %#Zd+WZ!
Yep, you guessed it, the situation went from bad to worse. The lead IT guy got sick during the process and had to stop working due to vertigo and vision problems. I have limited networking expertise. So I had to call in a separate local tech firm to fix everything. I had to pay emergency rates that completely blew my budget. Despite all of this, we were still down for days. For more than a week, employees complained that they couldn’t find files, emails and even programs that were supposed to have come over in the migration. Key database programs that we use for ad sales and circulation management would not work on the upgraded server. It was a perfect IT storm, and I was drowning.
Now, here comes the good part. I learned a lot from this IT debacle. For starters, there was foreshadowing that the lead IT guy was not 100% healthy because he had been sick the previous week. He claimed he was healthy, but I have learned that IT guys are not as invincible as they think they are. One reason to work with an IT firm that has multiple technicians is that they can bring extra resources to a situation if necessary. I generally have worked with IT consultants who were solo operators because they tend to be cheaper and can stick with you for a longer period of time. My last IT guy worked with us for eight years on a part-time basis. The current IT guy knows his stuff, but the lack of additional support behind him meant that we were in trouble if he got sick in the middle of the process.
Mistake #2, the migration/upgrade action took place over a weekend, but we were still in the middle of production on the Pallet Enterprise. So if anything went wrong over the weekend, we had little room for error without missing deadlines. Thus, I will never do a major IT project again during the production of a magazine. You might limit IT projects to periods of the month where you have lower customer demand or production requirements.
Another major problem was that we did not have a written migration and upgrade plan that spelled out the steps, time estimates for each activity or the top objectives. I will never undertake an IT action of this magnitude again without a detailed plan. This plan should include an analysis of every machine and any hardware upgrades that may need to be made to support newer software. This includes additional RAM, new video cards, DVD drives, etc. Be aware that any change on a work station or server can have ripple effects on everything from scanners to printers to individual PCs. Newer operating systems may not run on older, slower machines. Also, older versions of software may not work on newer operating systems. You may need to download new drivers for printers or scanners. And in some cases you may have to replace this equipment if new drivers do not exist.
It is critical to develop a data backup strategy. You should have one in place right now. If you don’t, you are just asking for problems. You also need a backup plan for any major network or PC migration project. The data on each machine needs to be carefully archived. The best practice is to install a backup and recovery service on each of the machines being upgraded or refreshed. If some users will be moving to new machines, this becomes even more important, as they will need the old files placed on their new machine. Make sure to move user profiles, emails, data files, and programs from one machine to another. The backup process must be started first before making any changes.
Another thing that would have helped us was a backup server that could have functioned as a test server so that we could have tried some of the changes on it before impacting the production machine. This works best if both servers are exactly the same make and model and are configured the same way. Some companies run a backup server that mirrors the main server so that if an emergency ever occurs, the downtime can be minimized by turning on the backup server while the main one is repaired.
When it comes to migrating to a new operating system, you should create a base image that is the template for every machine. This image is then copied onto each machine. Then you can install any specialty software required by individual users on an as needed basis. You should have all software cataloged and stored in an area that is easy for IT staff to find. The IT staff will need passwords and other user account information to configure email programs and ensure that everything works properly when the office staff returns. One way to avoid some of these problems is to have your email hosted on the Web and not stored on individual machines.
When office staff comes in the next day, have them test everything to ensure that there are no problems. There will be some issues. This is generally the case no matter how thorough the preparation of your IT staff. You should have them on hand to fix issues so that the experience of your office staff is as positive as possible.
Set realistic expectations and stick to your guns. Users may want zero downtime, but this may not be possible. A promise to eliminate all downtime during major hardware migrations is one you will not likely be able to keep. This only sets you up for failure. Make sure that staff knows what is about to happen and have appropriately backed information to the server.
New software or operating systems can seem foreign to less tech savvy employees. You may need to provide some training or a basic orientation to what has changed and what resources are available to help users get oriented with any new software or interface. It may be a wise idea to send out an email with links to online tutorials or videos.
Following some of these steps could have eliminated much of my IT headaches this year. Hopefully, my experience can inspire you to turn your next IT project into a dream instead of a nightmare.
