You are very correct about that. However, I’ve found that there are multiple scrambles that can still yield a ‘correct’ die (where all numbers 1 through 6 are there, and opposing sides add up to 7).

Try this from a solved die/cube, left side green, right side red, and top side white, as is apparently standard by https://rubiks-cube-solver.com/

D L2 U’ F2 U2 B2 U’ F2 L2 F2 R B’ F2 L’ D R F D’ B2 R’ F’ U’

You should end up with a scrambled color cube, but technically a correct die (though the dots will be in weird positions).