Riesling comes in so many different levels of sweetness, from bone dry all the way to dessert wine level of sugar. If you want something on the bone dry side, look for German rieslings that say “Trocken” or look for some coming from the Alsace region of France. For the former, there are regulations on how much residual sugar can be left and still classify it as a Trocken. Similarly, with champagne, there are varying levels of sweetness in the “Brut” category.
Though I don’t like even off-dry wines typically, usually it’s because the wine lacks sufficient acidity to carry that sweetness. I’ve had some off-dry chenin blancs from the Loire Valley in France where the wine was perfectly balanced due to that acidity. Think of it like a margarita: quite a bit of sugar in those, but the lime and alcohol counter act it.