Assuming it is technically easy for a game to support, should a game allow on-the-fly language switching?

I know some sites will only feature, say, the German version of a game. It is also possible to put them all in a single installer, and the user selects at install-time. But a 3rd possibility is to allow on-the-fly language switching, via a choice in the in-game options menu or via a flag icon on the game's initial screen.

This doesn't prevent the game from defaulting to a particular language based on an install-time choice, or based on an INI file flag tweaked by whoever does the builds for each language, or even based on what the game detects from the user's operating system language, but it would offer certain advantages:

1) All languages always available - no special effort required by an underpaid technician at a small site who may not follow directions correctly when setting up the installer.
2) If the user makes an error, it's easy to reverse without re-installing.
3) Some foreign users who speak, say German, but are reasonably English-fluent may not be sure whether to run the game in their native language (risking a bad translation) or in English. This allows them to try it easily at run-time.

The only downsides I can think of are:
1) A bit more work (not much, I don't think)
2) I haven't seen this in casual games downloaded from American-oriented sites. If I had this feature 'always on', it might confuse some American consumers (the bulk of the audience).

Comments on this issue and/or ways to present language choices to the user?