There's the South of France and then there's the South of France. For a large part it depends on what you want to do and what you like and don't like.
You could for example consider Tarn-et-Garonne region as your destination. You will have good waether for sure, but not the (numbers of) tourists that you will encounter on the Mediteranean coast or the southern part of the West-coast.
When you are based in Tarn-et-Garonne you have the beautiful rivers Tarn, Garonne and Aveyron closeby. For swimming and other watersports/activities. There's a gazillion of lovely, typical French villages. Montauban and Villefranche-de-Rouergue are great places to visit. Toulouse is closeby if you are in desperate need for a big city. The Food, like almost everywhere in France, is great. You're surrounded by nature, so hiking and mountainbiking, etc will not be a problem at all. The only thing you will not have... is the sea. Which in my experience means less annoying tourists and being able to forget that you yourself are indeed a tourist as well ;-)
Alternatively you could consider Cevennes. Also a lovely and more quiet destination