Don't listen to what users say 
A lot of web sites ask their users "what they think of the site". Some sites collect this feedback in a guest book, others ask users to mail their comments to the web master or the online editorial staff. While there is nothing wrong with asking users what they think of a web site, it is important not to overestimate the importance of this user input. 
Don't panic when one or two users criticize a certain element of your web site. You can't make a web site that satisfies every user's wants and expectations for a full 100%. It is advisable to check out the criticised points however, to make sure there isn't a problem. Taking the suggested solutions for the so-called problem into account however is always a bad idea. Users are not usability experts and cannot formulate solutions for usability problems. If you regularly receive complaints about a certain element of your web site, chances are you really do have a usability problem. Again, listen to users' complaints but don't listen to their solutions. While users can signal a problem, they cannot solve it.

Insider feedback 
A lot of companies use their employees as a barometer to find out "what users think of the site" while web masters tend to rely on their family and friends for feedback. Unfortunately, this sort of user input isn't very reliable. Very few employees are willing to go against the grain and criticise the web site their company has just spent a lot of money on. Friends and family are naturally inclined to please and say something nice rather than risk an unpleasant discussion. While it may be nice to hear such praise, it doesn't really give an objective view of the usability of a web site. 
Also take care when it comes to the opinions of people in any way involved in the project, be it the conceptualisation, development, implementation or maintenance of the web site, even if they are so-called experts like web designers or programmers. Because they know the web site from an inside perspective, their opinion of the user-friendliness of the site is not without bias and therefore they aren't objective assessors.

Formal usability test 
The only way to get a correct view of the usability of your web site is by a formal usability test. If you hold a user test, it is important that you observe users while they are surfing your site and that you interpret their comments correctly. Even users that have obvious difficulties with a web site are inclined to trivialise these problems because they think admitting the difficulties makes them look stupid. Hindsight comments about a certain element of the site aren't very valuable either. Users have to rely on their memory to make those comments and human memory is very fallible. Chances are they will leave out details that are very important to the usability of a site simply because they don't remember it. The most important thing in usability testing is an objective analysis of the collected data and that is often impossible for anyone involved in the project in any way.

Els Aerts & Karl Gilis

A more in depth version of this article has appeared in Tips & Advies Online Ondernemen, year 5, number 17 (Belgium and the Netherlands).