How to Evaluate New Applications and Services

530438_measure_upIt’s a great time to be a web worker. Almost every day, a new site, service or product comes on the scene that promises to make our work more efficient (or more fun). Some areas, like project management or image editing, are crowded with options. And in order to gain a following, many services are being offered inexpensively or at no cost.
But as Paisano wrote recently, current conditions won’t last forever. Many sites will eventually become fee-based; others will shut down when their funding runs out, or when their owners decide to move in a different direction.
So when I evaluate a product that I’d like to incorporate into my company’s workflow — especially a product that will be visible to clients — I try to consider the product’s feature set, along with the issues raised in Judi’s 2007 WWD post. I also ask the following questions:

  • Is the product open? Like a lot of people, I prefer open source projects. But I will consider proprietary systems if I am comfortable with how my data is stored and backed up, and whether the product allows me to do my own backups.
  • Does it use standard formats that are easy to import and export? If the product will interface with my existing data, I need to evaluate how much work it will be to prepare the data for use by the new product.
  • Can I host the product or software myself? Because my company does web hosting, we have easy access to web servers, bandwidth and backup systems. Therefore, I tend to prefer software that we can host ourselves. Maintaining a web server isn’t everyone’s cup of tea, although most of the major hosting companies make it really easy, as long as you’re willing to keep up with security patches and so on.
  • If I can’t host the software myself, how reliable and robust are the product’s servers? Of course, even major services (like Gmail (s goog)) have occasional outages. Here in Seattle, a fire caused a significant outage for many web sites just a couple of weeks ago. So I need to decide how my company will deal with such outages if we adopt the software.
  • How will using this product affect my bottom line? Managing any new product requires investing time and resources to install, set up, maintain and troubleshoot. Proprietary services are priced many different ways, and may require an up-front cost, plus continued license fees or service contracts. And while open source products don’t require license fees, many open source software companies offer service contracts. I need to decide whether I want to invest in such a contract, or whether I think I can rely on the user community. Whether open source or not, it’s always a good idea to look at the complexity and maturity of the product, how likely it is to need support, and how active the user base is.
  • What is my exit strategy should the product no longer be available? I need to decide how I will get my data, and plan for alternatives, if the product goes away.
  • How likely is it that the service provider will be available in the long run? This one’s tricky, since as a small businessperson, I’m hardly in a position to see, much less analyze, the business plans and financial statements of every producer I consider. But there is a fair amount of public information available, so I need to do what due diligence I can.

It’s never possible to plan for every contingency. I had to scramble to replace my Sunrocket VoIP service when that company ceased operation, and I still have some SyQuest backup disk cartridges somewhere for which no players are now available. But with a little common sense, it’s possible to avoid putting all of our technological eggs in one basket and becoming too dependent on any one service.
How do you evaluate what new services to include in your workflow?
Image by stock.xchng user CraigPJ