Ads were blocked - no problem. But keep in mind that developing HeidiSQL, user support and hosting takes time and money. You may want to send a donation instead.

Chinese Characters cannot be dispalyed in data grid after upgraded to HeidiSQL 12

patrickwong's profile image patrickwong posted 1 month ago in General Permalink

Hi all,

After I have upgraded to version 12, Chinese characters returned in data grid are shown as '?'. No such problem in version 11. Anythings can be done in settings to fix?

Thanks in advance.

Regards, Patrcik

2 attachment(s):
  • v12
  • v11
ansgar's profile image ansgar posted 1 month ago Permalink

That could be a font issue. Your both screens look as if you were using different fonts. Try the same font, in Tools > Preferences > Grid formatting.

patrickwong's profile image patrickwong posted 1 month ago Permalink

Hi Ansgar,

Thanks for your response. However I checked the setting in Tools > Preferences > Grid formatting between two versions (11 and 12) and found no differences. They got records from the same server/database/table. Actually I did not alter anything after upgraded to version 12. Collate of the table is 'utf8_general_ci'. Are there any chances that somethings were hard-coded in version 12?

Regards, Patrick

That could be a font issue. Your both screens look as if you were using different fonts. Try the same font, in Tools > Preferences > Grid formatting.

2 attachment(s):
  • v11_20220525
  • v12_20220525
ansgar's profile image ansgar posted 1 month ago Permalink

Ok, so this is not a font issue.

There are two things which might cause that:

  • since v12.0.0.6492, HeidiSQL does not change the connection's charset if it's already starting with "utf". But I did that after the 12.0 release, in the nightly builds. Are you on the release or on the latest nightly build?
  • issue #1351 got anothe fix for critical characters in the grids (especially emojis and multibyte characters). But that did not affect the text values themselves, just the length of strings was counted wrong.

You should please check the connection charset, once you have a connection:

SHOW VARIABLES LIKE 'character_set%';
Variable_name Value
character_set_client utf8mb4
character_set_connection utf8mb4
character_set_database utf8mb4
character_set_filesystem binary
character_set_results utf8mb4
character_set_server utf8mb4
character_set_system utf8
character_sets_dir C:\Wamp\mariadb-10.3.12-winx64\share\charsets\
patrickwong's profile image patrickwong posted 1 month ago Permalink

Hi Ansgar,

My Version 12 was already on the latest nightly build 12.0.0.6507, the problem persists. The results returned from "SHOW VARIABLES LIKE 'character_set%';" are different however:

Version 12.0.0.6507 (Displays Chinese characters as '?'): Description

Version 11.3.0.6321 (Displays Chinese characters correctly):: Description

But both HeidiSQL V11 and V12 clients connect to the same server/database.

ansgar's profile image ansgar posted 1 month ago Permalink

Ok, I thought you were using the last 12.0 release. In the nightly builds, you are probably running into a new issue due to my change for issue #287.

Please restart your connection and watch out for such a line in the log panel at the bottom, saying "Changing character set from x to y":

Description

Please post a screenshot or the text you see there.

patrickwong's profile image patrickwong posted 1 month ago Permalink

Hi Ansgar,

No such line "/ Changing character set from latin1 to utf8mb4 /" as you mentioned. The screen shot is as below:

Description

ansgar's profile image ansgar posted 1 month ago Permalink

Strange. Your "character_set_client" variable is "latin1", but mysql_character_set_name() returns "utf8_mb4".

I recommend to downgrade to v12.0 until I find some fix for that.

patrickwong's profile image patrickwong posted 1 month ago Permalink

Okay, I will fallback to earlier version. Am looking forward to your fix.

Many Thanks.

Strange. Your "character_set_client" variable is "latin1", but mysql_character_set_name() returns "utf8_mb4".

I recommend to downgrade to v12.0 until I find some fix for that.

rokiko's profile image rokiko posted 1 month ago Permalink

Hi, I've the same problem with latin2 -> utf8mb4 in version 11, the characters are displayed correctly and in version 12 they are not.

After executing the instruction (SHOW VARIABLES LIKE 'character_set%';), we have the differences as shown in the attached picture.

1 attachment(s):
  • 1654172348754
ansgar's profile image ansgar posted 1 month ago Permalink

Thanks for the comparison and screenshot. That's quite strange but I will find a fix.

I recommend either downgrading to v11.3, or fire some "SET NAMES utf8mb4;" after connecting.

Code modification/commit 885bf8d from Ansgar Becker <anse@heidisql.com>, 1 month ago, revision 6512
mysql_character_set_name() from libmysql.dll reports utf8* if in fact it's a latin* charset. Force utf8* charset in that case. See https://www.heidisql.com/forum.php?t=39278
ansgar's profile image ansgar posted 1 month ago Permalink

libmysql.dll seems to cause this issue. At least I could reproduce it various times here. Changing the library in the session to "libmariadb.dll" fixed it for me. Please try the same and check if that helps.

Next build forces utf8mb4 charset when libmysql is used.

rokiko's profile image rokiko posted 1 month ago Permalink

I installed the latest version 12.0.0.6512, I changed the library to libmysql.dll and the connected characters are displayed correctly. In the case of libmariadb.dll that I had set by default, they still display badly.

Thank you for your help.

patrickwong's profile image patrickwong posted 4 weeks ago Permalink

Yes, using libmariadb.dll causes the problem, changing to libmysql.dll gets rid of the problem.

I installed the latest version 12.0.0.6512, I changed the library to libmysql.dll and the connected characters are displayed correctly. In the case of libmariadb.dll that I had set by default, they still display badly.

Thank you for your help.

ansgar's profile image ansgar posted 3 weeks ago Permalink

I thought it was the other way around, that libmariadb.dll is using the right charset from the beginning. This gets really strange.

ansgar's profile image ansgar posted 3 weeks ago Permalink

@patrickwong and @rokiko: which server versions are you running?

And can one please post the character-set related settings from your my.cnf/my.ini on the server?

ansgar's profile image ansgar posted 3 weeks ago Permalink

I could finally reproduce it here, on a super old MySQL 5.1 server. I saw that in the very first screenshot in this thread. Why the heck do you use such old server versions? These have bugs, probably serious ones.

However, I will try to find out which server version was the first to report the correct charset through mysql_character_set_name, and then make an extra call for such old versions.

Please login to leave a reply, or register at first.




Ads were blocked - no problem. But keep in mind that developing HeidiSQL, user support and hosting takes time and money. You may want to send a donation instead.