Unicode text written to file is garbled

[thoatson]

In my program I want to write Unicode text to a text file (CSV). Here's a snip of my code:

int ThisWdNo;
CString ThisIndex;
CString ThisWord;
CString toWrite;

// BOM character
const char utf8BOM[] = u8"\uFEFF";

File.open(TargetFile, ios:ut | ios::binary | ios::trunc);
if (!File)
AfxMessageBox(_T("Export file cannot be opened"), MB_ICONINFORMATION);
else
{
...
// write BOM character...
pFile->write(utf8BOM, sizeof(utf8BOM));

toWrite.Format(_T("%d,%s,%s\n"), ThisWdNo, ThisIndex, ThisWord);
pFile->write((char *) toWrite.GetBuffer(), sizeof(TCHAR) * toWrite.GetLength());
}



When the text is English, everything looks great, but when the text is Greek, I get this:



My code reflects what I had gained from looking at:

How to write Unicode string to file with utf-8 BOM by C++

and

Write Unicode strings into a txt file

Originally I had:

File.open(TargetFile, ios:ut | ios::trunc);


and I didn't write the BOM character, but the results were no better...

Since Unicode is the standard (and has been for years), I would have expected writing Unicode text to a text file to be pretty much the default behavior. Shouldn't need to do something special for it...

Continue reading...