There is very little surf worthy ocean on the east coast of Canada where I live. The only place that I am aware of is the southshore of Nova-Scotia (the hubbie province of New-Brunswick). Lawrencetown is probably the most popular place. It has a public beach that allows for surfing and aslo swimming if you are brave enough (brrrrrr).
This was something I never imagined. A friend that was surfing with us had been there once before earlier that year. She said that she had rented her surfboard and wetsuit from some guy out of his cube van. Although we didn't know how reliable it would be, we set out from Halifax to find this dude. Upon our arrival just before the beach, there was Happy dude with his cube van sitting there waiting for customers. He had all we needed for the day!
What to buy: The equipment wasn't top of the line but for begginers like us it was great. I even got an O'neil wetsuit that I think was worth over 200$.