My favourite neighborhood in Montreal is Outremont. It is mainly residential, with beautiful streets and cute small buildings right next to each other. Most of them have that natural yet beautiful brick color.
Then there is Bernard Street, where you'll find a few restaurants (bistro like) and Laurier Street full of cute little boutiques.
Fondest memory: Outremont is a typical Montreal residential neighborhood. It is not touristy, and therefore you really sense the local flavors...