Although Gorizia is a small town, it has a good bus network. Line 1 goes from the railway station to the Piazza Transalpina and circulates every 10 minutes. There is also an international bus line connecting Gorizia to Nova Gorica, but it passes every 30 minutes, I think, and it's more expensive.
Crossing to Slovenia
If you want to cross to Slovenia on foot, do NOT ask drivers about where you can cross, especially if you are on the road coming in from the station. The crossing from Italy to Slovenia by car is outside of the city and requires a good hour of walking along the side of highways from Gorizia station to Nova Gorica station. It is far better to simply ask for the centre of town, as you can cross that way into the centre of Nova Gorica without problems.
Come by train
Gorizia lies on the railway line Trieste-Udine and is also connected to the Slovenian network. However, there are no more passengers trains from Gorizia to Slovenija, unlike what the panel in the last photo says. You can take them at the Nova Gorica station, which is not very far.
The station looks old from outside, but inside it's quite new. It even provides electronic panels showing the track of the trains.