Skip to content

No dynamic encoding for adapted params #158

Open
@osvill

Description

@osvill

All adapted query parameters in copy_to.py line 45 have the default encoding of latin1. This comes from the psycopg2.extensions.adapt function.

This behavior leads to an implicit error if you try something like MyModel.objects.filter(name="björn").to_csv(export_path) and your database's encoding is set to UTF8, because of the ö in björn.
The generated SQL looks like: COPY (SELECT "mymodel"."id", "mymodel"."name", "mymodel"."num", "mymodel"."dt", "mymodel"."parent_id" FROM "mymodel" WHERE "mymodel"."name" = 'bj�rn ') TO STDOUT DELIMITER ',' CSV HEADER and there will be no match with name='bj�rn' for björn.

So my suggestion is to add following code to copy_to.py line 49:

48    with connections[self.using].cursor() as c:
49        # set client encoding to adapted params    # new
50        client_encoding = c.connection.encoding    # new
51        for p in adapted_params:    # new
52            p.encoding = client_encoding if client_encoding else p.encoding    # new

This will generate the right SQL:
COPY (SELECT "mymodel"."id", "mymodel"."name", "mymodel"."num", "mymodel"."dt", "mymodel"."parent_id" FROM "mymodel" WHERE "mymodel"."name" = 'björn') TO STDOUT DELIMITER ',' CSV HEADER

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions