Teorically, you can’t guess the encoding of a file, but looking for some evidences:
If you see the “I don’t know the encoding…”, as far as I know, you can’t determine the encoding of the file. If such file contains a number “8”, it is a simple ascii file, and could be interpreted as corean, win, mac, utf-8… Perhaps you can check in some place if the file was expressly opened as corean (perhaps in TextEdit’s preference file?)
Please, note that this code is only a sample, you may find a “rtfd” file, and it will break in the second line (you can’t “read” a package), and you may also add lots of formats to check for…