That's one of many remains of the period - from 10 to 15 century - when all the region were part of Kingdom of Georgia. Dörtkilse, which mean "Four churches" was at that time an important monastic center. Now you can still admire quite well preserved ruins of the main church and monastery's dining hall, seminar room and chapel. All built at the begenning of 10 century.
You can see more pics of the complex in my travelogue
Dörtkilise is located in the northeastern end of Kaçkar Mountains, where dense alpin-type forests covers the lower valleys. It's beautyful but virtually without any turistic infrastructure. And to not make the thing easier the maps of the area are extremely difficult to find. Nevertheless if you want to trek a little bit in the neighbourhood of Dörtkilise you can ask for help at Cemil's Pension in Tekkale. The owner knows well the region and is willing to arrange some "expeditions" or just to share informations.