Nuts and Bolts: Cosmos Bank rechecks IT assets

First in the series, is Vasant Manwadkar- General Manager-IT, Cosmos Bank.

The Cosmos Co-operative Bank Ltd. (Cosmos Bank), one of the leading multi-state co-operative banks selected IBM’s POWER 7 systems to run its core banking solution. The Bank consolidated its sprawling server infrastructure of over eight Power 5+ servers (which included P570, P55A & 52A servers) onto three IBM POWER 7 servers (POWER 750) to run its core banking application. With the new system, the bank expects to increase performance by 50 per cent (with 100 per cent additional headroom for growth within each server) while reducing operating costs by 40 per cent. Power consumption is also expected to be reduced by 65 per cent.

Manwadkar tells us how he and his team managed to navigate a heavy-duty implementation that brimmed with risk, scale and real-time operations’ issues. At a bank that sprawls across six states with 115 branches. And all that,  in a three-day time window of bank holidays that were captured on his radar months back. He also lets us in on how he went for a big-bang approach instead of patches, what he thinks about latency and mission-criticality concerns that virtualization throws up, why DIY IT matters and would virtualized IT iron out maintenance and upgrade wrinkles of yore. And why user-tests are so important before an implementation. In short, a migration hand-book.

An implementation of this scale would surely have been not easy, specially for a bank where real-time operations and zero-disruption are critical factors.  How did you manage it? What triggered and what substantiated this deployment?

We had a relationship with IBM since last few years. When we initially went for Core Banking, it was just about 45 branches. Now we have undergone big growth and run 115 branches. So to manage the load efficiently, we went for a new set-up. There was also an upgrade from Finacle 6 to 7. IBM came up with P7 servers, and we realized that P7 would be best for the scalability that we are looking for. So we chose three P7 servers with 12 processors each. At the same time there was Oracle database upgrade, in terms of hardware, and the Finacle bit, it were four parameters that came together. Big risk, of course and to convince everyone was a challenge, because four components were being overhauled in one shot. But the promise of 20 per cent extra capability did it. We went live in Dec 2010.

Take us through the migration journey? How easy or how tough was the whole cycle?

We outsourced it to a partner, but internally we dealt with a lot of migration on our own. It was definitely risky, because there was no possibility of shutting down operations. I had kept an eye on a three-day holiday that was coming up, and planned accordingly. The initial go-live date was December 25th, but we did it a week earlier, and that gave us room for any last-minute hiccups if any were to arise.

So did you do this in patches, and with phased roll-outs?

No. It was a Big-Bang way. We jumped to Virtual set-up in a full-fledged way. But we did three rounds of UATs (User Acceptance Testing)s. That happened two months before the go-live.

Do you prefer the DIY (Do it yourself) approach to hiring some specialist to steer the implementation?

I normally go for a combination, based on the complexity involved. Outsourcing works, where experts are really needed. But it is also crucial to build our own capabilities, especially in techno-functional areas. Core technical areas can be done through specialists from outside.

The advent of technologies like virtualization and Cloud is good. But does this genre tend to solve the traditional issues around maintenance and upgrades that their atavistic software siblings involved?

May be not at the level of virtualization that we have done so far. But next round, which is hopefully slated for this month, might succeed there. Seventeen servers may be scrapped and shrunk into a single server. It is kind of a full-fledged internal cloud set-up but at the proposal stage now. If it happens then we will achieve precisely what you just said.

What’s your take on problems around latency and mission-critical application performance that these technologies need to solve?

Ours is a critical banking operation. And that’s why we have opted for a level 1 virtualization (In IBM parlance). The hardware though is very similar to physical hardware. As to level 5, we will think about it as it would include issues like redundancy, latency etc. So for now, we will stick to level one.

Has it started showing results? What would be your expectation list here?

It has. Earlier day-end reporting used to take three hours. Now it happens in flat 20 minutes. Year-end time-guzzlers have come down to six hours. MIS reporting runs in a fraction of a minute. In so many areas, it has saved costs, from less expensive data centre maintenance, easy risk management and scale-up potential. Now, we can double the size of our operations without any worry.

Space and energy savings, come along too of course?

Yes, we have saved a lot as space and also in terms of energy. From three racks to just half a rack in size now.

How does it impact your DR (Disaster Recovery) pieces?

We had our own DR site. It required to be manned 24/7, with security guards to add up. And about Rs. 1.5 crore as costs along with Rs 70 lakh in operational costs. Now we have closed that and co-located it somewhere else with just Rs five lakh a year as required expenses. Not to mention Rs. 1.5 lakh in electricity expenses every month that is now potentially saved.

If there were three lessons that you could share with your peers out of this experience, what would be your tips for implementation?

Firstly, test the product very well. Specially, for an industry like banking, the testing has to be four times the normal rate. Second, most migration projects fail, because IT owns the project, and business does not. Business ownership has to happen, more so, because of change management issues that a migration involves. And finally, document the acceptance. I have tried video recording UATs also. Keeping a record of it really helps and adheres a user to the whole exercise.