Multi-byte UTF8 characters stored in BLOB column being replace by "3F"(Ascii '?‘)

[expired user #9912]'s profile image [expired user #9912] posted 8 years ago in General Permalink

While displace Multi-byte UTF8 characters stored in BLOB column into DataGrid, or Save a UTF8 HEX String from BinEditor to BLOB column, the data being replace by "3F"(Ascii '?‘).

There is a sample in attachments.

My enviroment is windows 10, mysql, so my OS character set is GBK, but I have to store UTF8 string in mysql BLOB column.

And I already figure out the problem code is dbconnection.pas:5652

SetString(AnsiStr, FCurrentRow[Column], FColumnLengths[Column]);
      if Datatype(Column).Category in [dtcBinary, dtcSpatial] then
        Result := String(AnsiStr)
      else
  1. AnsiStr initiated by memory copy BLOB data, so there is no code page using in AnsiStr.
  2. While compile by Delphi XE5, String is corresponding to UnicodeString, a type of WideString with an UTF16 code page.
  3. When invoke String(AnsiStr), Delphi XE5 copy UTF8 bytes into String's memory, because it doesn't have code page.
  4. Anytime convert other String type into UnicodeString, it will validate the result, that's when invalide UTF8 byte being replace by "3F"(Ascii '?').

Solution: 1.Back to use older Delphi, which's String is corresponding to AnsiString, or 2.Change Col(...) return type to AnsiString.

1 attachment(s):
  • sample

Please login to leave a reply, or register at first.