UTF-8 Support

[expired user #5742]'s profile image [expired user #5742] posted 13 years ago in General Permalink
There are quite a few threads and tickets regarding UTF-8, the upshot being that Heidi does support UTF-8.

However, in all of my tests, I cannot get utf-8 [hebrew] characters to show as anything but boxes.

The database is created using "default charset utf8".

When I click the "CREATE code" tab, the statement shown doesn't include the default charset.
It does include the collation:
COLLATE='utf8_general_ci'

When I log into the database using phpmyadmin, it shows the db as being utf-8, and when pulled from the db and displayed on a page whose meta tag is utf-8, the text shows fine.
http://teltzion.com/en/

However, in the Heidi GUI, the text shows as boxes.
Worse, if I change the text using the GUI - even if only English parts are changed - it destroys all of the existing utf8 text.

Hope that was clear.
Can someone confirm if UTF-8 is supposed to be supported, and if so, how do I get the data to show correctly.

Thanks alot for your work on this software.
ansgar's profile image ansgar posted 13 years ago Permalink
You will need to post CREATE TABLE queries if you expect a deeper analysis.
ansgar's profile image ansgar posted 13 years ago Permalink
And how do you insert rows, normally? Are you sure the content itself is not broken?
[expired user #5742]'s profile image [expired user #5742] posted 13 years ago Permalink
Thank you.

Here's the create table code:
CREATE TABLE `pages` (
`index` TINYINT(4) NOT NULL AUTO_INCREMENT,
`page` TINYTEXT NOT NULL,
`sub` TINYTEXT NULL,
`lang` ENUM('en','he','ru') NULL DEFAULT NULL,
`col0` TEXT NULL,
`col1` TEXT NULL,
`col2` TEXT NULL,
`news` TEXT NULL,
`colors` VARCHAR(20) NULL DEFAULT NULL,
`modified` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00' ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`index`)
)
DEFAULT CHARSET=utf8
COLLATE='utf8_general_ci'
ENGINE=MyISAM
AUTO_INCREMENT=4;

And the data is added using a webform:
Page has a metatag:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Added to the database using standard PHP query:

Is there some way of testing the data?
I figured that the fact it displays when the browser is set to UTF8 encoding proves that the data is valid, but am a noob - please correct me or advise.
[expired user #5742]'s profile image [expired user #5742] posted 13 years ago Permalink
BTW, I purchased Heidi before it changed name and went open source - so I'm a longtime fan!!happy

Thanks very much for your dedication.
ansgar's profile image ansgar posted 13 years ago Permalink
I hope "purchased" does not mean you gave any money for it, as it was freeware before?
ansgar's profile image ansgar posted 13 years ago Permalink
Yes, the webpage has to send UTF-8 charset, to have the communication between browser and webserver the right way, so that's very ok so far.

Additionally, the database connection has to init some "SET NAMES utf8" query before INSERTs come in, so the communication between PHP and MySQL are using the same charset. If you do not set the MySQL charset, contents arrive broken in your database (UTF8 interpreted as Latin1), but will be displayed fine on your webpage.
ansgar's profile image ansgar posted 13 years ago Permalink
You could check with MySQL Workbench to avoid having some webserver and/php inbetween.
kalvaro's profile image kalvaro posted 13 years ago Permalink
Make sure your font supports hebrew characters:

Tools-> Preferences-> Data-> Font

I've set it to "Arial Unicode MS" and it works nicely.

Seeing the chars in the browser does not really prove anything. Inspect the exact contents with the HEX() function:

http://dev.mysql.com/doc/refman/5.1/en/string-functions.html#function_hex

For instance, a field with €ÁÑ should display as E282ACC381C391.

Please login to leave a reply, or register at first.