If traveling after dark, I would definitely keep a watchful eye out for deer. It is basically wilderness outside of Ashland and it IS northern Wisconsin. Enough said.
As far as crime, I never saw anything that would make me feel uncomfortable. There are certainly no big roving gangs of bums or panhandlers. There were a few rundown looking bars, but that doesn't necessarily translate into trouble. In fact, they might be quite fun
The only warnings or dangers I can advise you to look out for is the deer and other wildlife. There are lots. And in the winter, the snow can be bad at times so driving is hazardous sometimes. Just tune in to your weather station to find out about your travel advisory. But watch the deer.