Web 2.0 & Death of the Network Engineer

I was recently meeting with a Web 2.0 company discussing their network infrastructure plans. As I started asking questions about their racks of servers, their storage area network (SAN), their plans for routing, load-balancing and network security, the CTO of the company stopped me and made a bold statement.

He said, “The Internet is like electricity. We plug into it and all of the things that you mention are already there for us. We don’t spend any time at all on network or server infrastructure plans.”

To this CTO, knowing the details of his network and server infrastructure was like knowing the details of the local utility electricity grid – not required. Is this a bad thing, or proof that networking technologies have succeeded?

I guess I am old school, but I recall in the not-so-distant past that every startup needed a plan for their network and server infrastructure and even knew the details of their service providers network – are they using OSPF and BGP? What is the latency across the local peering point? Who are their upstream network peers? How are their firewalls and load-balancers configured? What blocks of IP addresses have I been assigned and how are they routed?

Some companies, like InterNAP and Level 3, have businesses that emphasize their network optimization and network architectures. I don’t know of any electricity optimization companies and I don’t have any idea of the architectures they have built.

My roots are in network engineering and I have spent a good part of my career building network devices and global IP-based networks and services. I’ve spent years studying routing protocols, quality of service algorithms, security mechanisms to prevent DDoS attacks and have every field of the IPv4 packet header memorized.

When the CTO of a Web 2.0 company does not know how a router or switch works (or even what layer of the OSI model they even operate on), I tend to cringe a bit.

I guess I’m reluctant to admit that my technical depth in networking has been abstracted to not being relevant in the Web 2.0 world of social networking, mash-ups, RSS and AJAX. I know that a well-architected network can have a dramatic affect on application performance – but maybe on today’s high-speed Internet it does not matter. It might be that network engineers are not relevant for today’s Internet in the same way that software optimization engineers are seemingly not relevant for Microsoft applications.

On the other hand, I see the current state of the Internet as the ultimate success of these networking technologies. You can deploy a wildly successful Web 2.0 application that serves millions of users and never know how a router, switch or load-balancer works. Even network security and firewalls that were making headline news not more than a few years ago are considered perfunctory. The success of these networking devices and technologies has enabled them to become part of the technology landscape that exists for all to use as they see fit, similar to the microprocessor or electricity.

In your opinion, has the Internet reached a level of abstraction similar to electricity? Do you use the infrastructure that is given to you by your local Internet service provider or a specialized hosting facility like Amazon without questioning how it is architected and designed?

In my role as a venture capitalist, the answers to these questions will help me determine if startups that are building optimized networking devices, improving network security, virtualizing storage, and so forth are required in today’s market.

Allan Leinwand is a venture partner with Panorama Capital and founder of Vyatta. He was also the CTO of Digital Island.