When young women are walking the streets down town at night, be warned of dirty Montreal guys that will say just about anything to get your attention. Don't worry, look down and keep walking. It's all a lot of hot air. Lot's of talk but no action. They don't DO anything unless antagonized, (talked back too, flirted with, stared at). I Just keep on walking and don't look back and I've never had an issue.
My tip, which I discovered accidentally, makes a great experience out of being panhandled. Generally, the panhandlers will pick you out as a tourist and ask you if you speak English. I just said Non in my french accent and he came out with this beautifully accented french, probably a story of his hard times. Though there are still panhandlers, crime is low and the streets seemed alot safer to me than in my native New York.