Why Buy One When You Can Buy Two For Twice the Price

I get pretty excited when new network gear shows up at the loading dock. I get psyched when I get to configure an interesting technology that I rarely get to use. But considering our responsibility to our customer or employer, sometimes we need to put that aside in favor of the simpler (or cheaper) but more appropriate solution. Let me give you one example.

Not long ago I worked with a nonprofit in the healthcare industry to replace their network core. It was an organization of a little less than 2,000 end users on a single campus. Many people were remote part-time workers and not really information users, though they would log in occasionally to access an online timesheet or email. There was one network egress through an unreliable ISP, so part of this project would be adding a second internet connection to the mix. The main goal was to gain better network fault tolerance.

We looked at hardware and configuration options – they were stretching their budget to the max. Our initial thoughts were pretty standard: probably Catalyst 4500Xs at the core running VSS, a couple 3945 edge routers running HSRP, a failover pair of Cisco ASA 5545Xs and some switch stacks. We even discussed a building generator and moving their email to Office 365.

I was the delivery engineer for this project, not the solutions architect, so I fell out of the loop during the rest of the design process and was brought in just before kickoff. Looking over the final design I saw the 4500X switches, one 3945 router, and a pair of ASA 5585Xs in multicontext mode. They decided to skip the second ISP, increase the bandwidth with their current provider and go with larger ASAs. I had a mild disagreement with the ASA sizing and dropping the ISP, but that wasn’t my decision. Then I realized that the customer, who relied heavily on Anyconnect as their remote access solution, would be up the creek since multicontext mode didn’t support remote access VPN.

I faced a conflict here because there was a disparity between the business requirement and the technical solution.

First let’s consider the elimination of the second ISP from the plan. It was clear that the incredible cost of the bigger ASAs precluded the customer from entertaining a second provider. That made no sense to me because of how unreliable their current provider was and how heavily they relied on RA VPN. I looked at the purchase order and almost fell out of my chair when I saw how much each ASA plus licensing cost. Cisco ASA 5585Xs aren’t cheap, and each security context costs thousands. The business requirement called for local resources to be available by remote workers at all times, so eliminating the second ISP seemed strange to me.

Now let’s consider multicontext mode. (No, I won’t be explaining how to configure it because, well, Google.) Multicontext mode on a pair of Cisco ASAs allows for an Active/Active state so you can load-balance traffic as well as have subsecond failover. You also get multiple security contexts, or in other words, multiple virtual firewalls living in the same box. This is useful for multi-tenancy configurations such as with service providers and large multiple business-unit companies. This customer had no need for that whatsoever, and since multicontext mode does not support remote access VPNs and is extremely expensive I was again confused why this layer of cost and complexity was added.

Also consider the actual hardware.  Failover designs should plan for each individual firewall to be able to accommodate 100% of network traffic in case of a failover scenario. It’s important to choose a model that can alone support all network traffic. In this case each new 5585X was such an incredible overkill that I felt it was a waste of a lot of money. I know sometimes we choose hardware to meet future growth, but this was a very static organization funded mainly by government grants. There was no need to be able to accommodate huge growth within the next 7 – 10 years before the next hardware refresh. Even the lower end 5500X firewalls like the 5515X or 5525X would easily fit the bill. 

An alternative would be running the firewall pair in Active/Standby on smaller (cheaper) firewalls. This configuration also provides subsecond failover, plus it offers all the features of the ASA including remote access VPN. No, there’s no load balancing, but I really don’t think that was relevant in this case. This would have freed up all the money needed for a second router and a second ISP. Even having a second ISP but just a single router would have made sense to me.

I know that multicontext mode on Cisco ASAs has its place –  I’ve used it for several projects. I’ve also installed 5585Xs in data centers and very large networks where the traffic baselines required the larger model. But for this much smaller customer, I don’t believe the huge cost increase was worth it at all. Sticking with an Active/Standby solution would have freed up the money to do the other things necessary to meet the customer’s business requirement.

Lastly, there was no mention of replacing any of their old, daisy-chained access switches. My thought was to put in a couple basic switch stacks with 2960Xs in order to run portchannels to mission critical servers and portchannels back to the core. The 2960Xs are among the cheapest Catalyst switches you can get other than the SMB models, and in my mind they were necessary to add the access layer redundancy the customer asked for.

Though the final design cost much more in hardware than my preliminary design, it didn’t meet the customer’s business requirements. It’s my opinion that though a design governed by margin on hardware can be more lucrative, it certainly isn’t always the best solution for the customer. In the long run, that may even hurt the relationship and therefore the bottom line. I’ve seen enough one AP per classroom designs that I already had a tendency to look at final purchase orders with a bit of cynicism.

I love cool new network equipment. I really do. But consider why you choose a technology. If it isn’t truly the best solution to meet the customer’s need, then it probably isn’t the right solution.