Paperless NGX Update database needs to be updated to UTF8

[Flosen86]

Hi all,

getting this error

24-05-11 16:42:07,078] [ERROR] [paperless.tasks] ConsumeTaskPlugin failed: invoice (3).pdf: The following error occurred while storing document invoice (3).pdf after parsing: conversion between UTF8 and SQL_ASCII is not supported

LINE 1: ...511', 511, '2024-05-11 invoice (3)', NULL, 0, '', '{"custom_...

                                                         ^

Traceback (most recent call last):

File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 89, in _execute

return self.cursor.execute(sql, params)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

psycopg2.errors.FeatureNotSupported: conversion between UTF8 and SQL_ASCII is not supported

LINE 1: ...511', 511, '2024-05-11 invoice (3)', NULL, 0, '', '{"custom_...

                                                         ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/usr/local/lib/python3.11/dist-packages/asgiref/sync.py", line 327, in main_wrap

raise exc_info[1]

File "/opt/paperless/src/documents/consumer.py", line 631, in run

document = self._store(text=text, date=date, mime_type=mime_type)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/opt/paperless/src/documents/consumer.py", line 796, in _store

document = Document.objects.create(

           ^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/manager.py", line 87, in manager_method

return getattr(self.get_queryset(), name)(*args, **kwargs)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 658, in create

obj.save(force_insert=True, using=self.db)

File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 814, in save

self.save_base(

File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 892, in save_base

post_save.send(

File "/usr/local/lib/python3.11/dist-packages/django/dispatch/dispatcher.py", line 176, in send

return [

       ^

File "/usr/local/lib/python3.11/dist-packages/django/dispatch/dispatcher.py", line 177, in

(receiver, receiver(signal=self, sender=sender, **named))

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/auditlog/receivers.py", line 27, in wrapper

signal_handler(*args, **kwargs)

File "/usr/local/lib/python3.11/dist-packages/auditlog/receivers.py", line 40, in log_create

_create_log_entry(

File "/usr/local/lib/python3.11/dist-packages/auditlog/receivers.py", line 146, in _create_log_entry

raise error

File "/usr/local/lib/python3.11/dist-packages/auditlog/receivers.py", line 124, in _create_log_entry

log_entry = LogEntry.objects.log_create(

            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/auditlog/models.py", line 77, in log_create

return self.create(**kwargs)

       ^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/manager.py", line 87, in manager_method

return getattr(self.get_queryset(), name)(*args, **kwargs)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 658, in create

obj.save(force_insert=True, using=self.db)

File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 814, in save

self.save_base(

File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 877, in save_base

updated = self._save_table(

          ^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 1020, in _save_table

results = self._do_insert(

          ^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/base.py", line 1061, in _do_insert

return manager._insert(

       ^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/manager.py", line 87, in manager_method

return getattr(self.get_queryset(), name)(*args, **kwargs)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/query.py", line 1805, in _insert

return query.get_compiler(using=using).execute_sql(returning_fields)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/models/sql/compiler.py", line 1822, in execute_sql

cursor.execute(sql, params)

File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 67, in execute

return self._execute_with_wrappers(

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers

return executor(sql, params, many, context)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 84, in _execute

with self.db.wrap_database_errors:

File "/usr/local/lib/python3.11/dist-packages/django/db/utils.py", line 91, in exit

raise dj_exc_value.with_traceback(traceback) from exc_value

File "/usr/local/lib/python3.11/dist-packages/django/db/backends/utils.py", line 89, in _execute

return self.cursor.execute(sql, params)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

django.db.utils.NotSupportedError: conversion between UTF8 and SQL_ASCII is not supported

LINE 1: ...511', 511, '2024-05-11 invoice (3)', NULL, 0, '', '{"custom_...

                                                         ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/opt/paperless/src/documents/tasks.py", line 144, in consume_file

msg = plugin.run()

      ^^^^^^^^^^^^

File "/opt/paperless/src/documents/consumer.py", line 699, in run

self._fail(

File "/opt/paperless/src/documents/consumer.py", line 302, in _fail

raise ConsumerError(f"{self.filename}: {log_message or message}") from exception

documents.consumer.ConsumerError: invoice (3).pdf: The following error occurred while storing document invoice (3).pdf after parsing: conversion between UTF8 and SQL_ASCII is not supported

LINE 1: ...511', 511, '2024-05-11 invoice (3)', NULL, 0, '', '{"custom_...

According to paperless-ngx/paperless-ngx#6682 the database needs to be updated to UTF-8. Could you include it in the update script and give me a way to it manually.

[tteck]

#2925

[tteck]

c13b868

[Flosen86]

Doesn't work for me. All instructions on updating just fails.

sudo -u postgres psql -c "SELECT pid, usename, datname, client_addr, state FROM pg_stat_activity WHERE datname = 'paperlessdb';"^[[B
could not change directory to "/root": Permission denied
ERROR: syntax error at or near "
INE 1: ...tate FROM pg_stat_activity WHERE datname = 'paperlessdb';

And all the other shit. I just stay with 2.6.2

[kimmer2k]

I was able to fix this using these instructions: paperless-ngx/paperless-ngx#6651 (comment).

From your post, looks like you pasted the initial command incorrectly with ^[[B appended? The fix took me all but a few minutes, so feel free to reach out if you have any issues.