2

How to solve this double encoding?

 3 years ago
source link: https://www.codesd.com/item/how-to-solve-this-double-encoding.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

How to solve this double encoding?

advertisements

I'm developing a website using python to preprocess request and a MySQL database to store information.

All my tables are utf8 and I also use utf8 as Content-type.

I have this code to establish connection to the db:

database_connection = MySQLdb.connect(host = database_host, user = database_username, passwd = database_password, db = database_name, use_unicode = True)
cursor = database_connection.cursor()
cursor.execute("""SET NAMES utf8;""");
cursor.execute("""SET CHARACTER SET utf8;""");
cursor.execute("""SET character_set_connection=utf8;""");

Running a simple test on my GoDaddy hosting printing the results of a simple SELECT query like this:

print results.encode("utf-8")

Shows a double encoded string. (So all non-ascii characters are transformed into two different specials). But if I leave the encode statement, it gives an encoding error for each non-ascii letter.


It sounds as though results contains a Unicode string that was incorrectly decoded from a byte string coming from the database. I.e. when you read the data from the database, it decoded the byte string as Latin-1 rather than the UTF-8 it really is.

So if you fix the decoding of the database contents, then you should be in business.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK