I feel not using www is a mistake and I'll explain why.
Trust - www has nothing to do with the trust of a site but I think your average user subconsciously takes familiarity into account when deciding whether to trust a site or not. Does your average user understand why your site doesn't have www in the URL yet their banking/shopping site does? No, but they might notice it missing and wonder why. I don't know of any sites that have discussed or studied this, it's merely intuition.
Most browers have already removed http from being displayed and while it's technically still there and can be added back, the url is now easier to read (reasoning for it's removal in the first place). This puts a bigger emphasis on what's left and more reason to leave www IMO.
Links - Ignoring power users who are the small minority, if you ask anyone for a website address, they'll begin by saying www. and I think it's fair to say most people will assume you are using www when linking to you. I don't want people linking to URL's that don't exist.
http://site.com and
http://www.site.com are different. I want links to the actual site regardless of Google's ability to figure it out.
Marketing - I've seen it a few times but don't have any specific examples but the actual website will be
http://website.com yet all the marketing and promo stuff displays www. I think it looks sloppy and I question the company's "we give a f*ck about getting stuff right" ability.
There is no downside to using www yet there potentially is when not using it. It's such a minor thing that I see no reason not to use it at all times.