You can’t really separate any and all about:config settings per-site, neither the whole setting store in it’s entirety, but there are a few workarounds.

Generally for any about:config: you can make separate firefox profiles with their own about:config variations and use those sites in their own profile.
You can use the in-browser profile manager at about:profiles, or the little window that allows selection before startup with the firefox -P command. Works on windows too, but with .exe and the full path to it.

Then there are certain about:config settings have a pair that limits or excludes certain sites (privacy.resistFingerprinting is like that)

For webgl, I’m not sure if there’s such an about:config setting, but there’s a workaround. I have reenabled webgl in about:config, and instead blocked it with a uBlock rule, and defined exceptions to that rule for a few sites where I want it to work:

*##+js(aopr, WebGLRenderingContext.prototype.getExtension)
*##+js(aopr, WebGLRenderingContext.prototype.getParameter)

demo.f4map.com#@#+js(aopr, WebGLRenderingContext.prototype.getExtension)
demo.f4map.com#@#+js(aopr, WebGLRenderingContext.prototype.getParameter)

duckduckgo.com#@#+js(aopr, WebGLRenderingContext.prototype.getExtension)
duckduckgo.com#@#+js(aopr, WebGLRenderingContext.prototype.getParameter)

The above uBO rules use scriptlets, there are various, they can do very specific funny things with websites by running code snippets preapproved by the uBO devs. The uBO wiki on github has docs on what does aopr do. There’s also 2 exceptions, both for the maps.
When I first set it up I had known this as a proper webgl blocking solution, but if anyone could verify it I would be thankful. I think I got the first 2 lines from the uBO wiki, but I don’t remember.

NB I have not used LibreWolf, I don’t think there are any site specific toggles in about:config. This probably doesn’t help but some Android Webviews do allow this.

You probably considered this already but as a workaround could you use one browser when you need WebGL and another when you do not.

You can use NoScript.
Enable WebGL in LibreWolf/Firefox, block it with NoScript and manage a white list adding manually sites you need.