SVG support for django-filer (django-cms#1226)

* drop support for high resolution images * migrate image model fields width and height to float * SVG images: store width and height in table * fix django-cms#1216: mime-type when using dropzone * drop quality and other parameters for PNG and SVG in thumb-filename * fix django-cms#1216: mime-type when using dropzone * Fix: SVG images don’t have an exif header * Fix: SVG images don’t have an exif header * do not check for file extension if MIME type is 'application/octet-stream' * replace all occurences of force_text by force_str * upgrade to work with nodejs version 14 * move towards templatetag thumbnail * upgrade to work with nodejs version 14 * proceed with latest versions of npm and gulp * downgrade phantom and request * install gulp globally * add gulp-util to dependencies * pin nodejs to 14.15.0 * more tests in gulp ci * try with older versions * with these versions of node packages, CI runs locally * remove hacks required for retina.js’s highres images * introduce templatetag file_icon * fix some exceptions cause by bad filetype guessing * Add method ‘exists()’ on MultiStorageFieldFile * increase icon size in directory listing from 25 to 40 * use SVG icons * remove png icons and add deprecation warning if used * add properties mime_maintype and mime_subtype * make audio element behave like image * add video element * render file icon for PDF * Do not create file icons upon upload * remove focal-point.js from views without preview image * remove unused icon * handle drag & drop of images onto existing plugins * adopt drag-zone to preview images in 80x80 * adopt drag-zone to preview images in 80x80 * create prerelease of upcoming version of django-filer * fix scaling of very large images: ceil height to integer * use exists-method to check for file on disk * Pin to pre-release of easy-thumbnails (with SVG support) * fix typo * fix: add mime_type to filer’s Image rather than Django’s File model * Fields _width and _height are floats now * round width and height to one decimal unit * Drop support for Python-3.5 it is not supported by reportlabs and end of life anyway * fix E125 continuation line with same indent as next logical line * Fields _width and _height are floats now * fix sorting order reproted by isort * isort complains about one missing empty line * Bump to version 2.1rc1 * Bump to version 2.1rc1 * simplify code for property exif * fix django-cms#1227: Dropzone layout and render field label Co-authored-by: Jacob Rief <[email protected]>
kapt-labs · Feb 1, 2021 · fde463c · fde463c
1 parent b4ed668
commit fde463c
Show file tree

Hide file tree

Showing 188 changed files with 822 additions and 418 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -13,8 +13,6 @@ matrix:
     - python: 3.6
       env: TOX_ENV='frontend'
     # Django 2.2
-    - python: 3.5
-      env: DJANGO='dj22' SWAP='swap'
     - python: 3.6
       env: DJANGO='dj22' SWAP='noswap'
     - python: 3.7
@@ -39,7 +37,6 @@ matrix:
 
 install:
   - pip install coverage isort tox
-  - "if [[ $TRAVIS_PYTHON_VERSION == '3.5' ]]; then export PY_VER=py35; fi"
   - "if [[ $TRAVIS_PYTHON_VERSION == '3.6' ]]; then export PY_VER=py36; fi"
   - "if [[ $TRAVIS_PYTHON_VERSION == '3.7' ]]; then export PY_VER=py37; fi"
   - "if [[ $TRAVIS_PYTHON_VERSION == '3.8' ]]; then export PY_VER=py38; fi"
@@ -48,10 +45,9 @@ install:
 before_script:
   - if [ $TOX_ENV == 'frontend' ]; then
     pip install -r tests/requirements/frontend.txt;
-    nvm install 0.12.7 && nvm use 0.12.7;
+    nvm install 14.15.0 && nvm use node;
     npm config set spin false;
-    npm install -g npm@2;
-    npm install -g [email protected];
+    npm install -g [email protected];
     npm install;
     fi
 

diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -2,6 +2,41 @@
 CHANGELOG
 =========
 
+2.1rc1  (2020-11-25)
+====================
+
+* Add support for SVG images. They now are handled by the model
+  ``filer.imagemodels.Image`` and can be used whereever a pixel based image
+  was used. This includes scaling and cropping using existing thumbnailing
+  functionality from the
+  `easy-thumbnails <https://easy-thumbnails.readthedocs.io/en/latest/index.html>`_
+  library.
+* Drop support for high resolution images and remove ``retina.js`` from project.
+  High resolution images are handled by the HTML standard attribute in
+  ``<img srcset="..." ... />``.
+* In model ``filer.imagemodels.Image`` change ``_width`` and ``_height`` to
+  Django's ``FloatFields``; this because SVG images specify their image
+  extensions as floats rather than integers.
+* All icons for displaying folders, files (not images) have been replaced by
+  nicer looking SVG variants from `PaoMedia <https://paomedia.github.io/small-n-flat/>`_.
+* Increase size of thumbnails in the admin backend's list view from 25x25 to
+  40x40 pixels.
+* For local development switched to NodeJS version 14.
+* Add templatetag ``file_icon`` to ``file_admin_tags.py``. It now handles the
+  rendering of all file types, including folders, zip-files and missing files.
+* Remove pre-thumbnailing of images. Up to version 2.0, all images were scaled
+  immediatly after upload into many different sizes, most of which never were
+  used. Thumbnailing in the admin backend now is perfomerd lazily.
+* Uploaded audio can be listened at in their detail view.
+* Uploaded video files can be previewed in their detail view.
+* Fix scaling of very wide but short images – causing a division by zero
+  exception: ceil height to integer.
+* Add method ``exists()`` to ``MultiStorageFieldFile``, which checks if a file
+  exists on disk.
+* Drop support of Python-3.5 (Reason: Third party requirement
+  `reportlabs <https://www.reportlab.com/>`_ requires Python>=3.6).
+
+
 2.0.2 (2020-09-10)
 ==================
 

diff --git a/LICENSE b/LICENSE
@@ -21,4 +21,7 @@ DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+SVG icons supplied by courtesy of https://paomedia.github.io/small-n-flat/ and
+are subject to the Creative Commons Zero v1.0 Universal.
diff --git a/aldryn_config.py b/aldryn_config.py
@@ -27,8 +27,6 @@ def to_settings(self, data, settings):
 
         # easy-thumbnails
         settings['THUMBNAIL_QUALITY'] = env('THUMBNAIL_QUALITY', 90)
-        # FIXME: enabling THUMBNAIL_HIGH_RESOLUTION causes timeouts/500!
-        settings['THUMBNAIL_HIGH_RESOLUTION'] = False
         settings['THUMBNAIL_PRESERVE_EXTENSIONS'] = ['png', 'gif']
         settings['THUMBNAIL_PROCESSORS'] = (
             'easy_thumbnails.processors.colorspace',

diff --git a/docs/installation.rst b/docs/installation.rst
@@ -47,14 +47,6 @@ Add ``"filer"`` and related apps to your project's ``INSTALLED_APPS`` setting an
 
 Note that `easy_thumbnails`_ also has database tables and needs a ``python manage.py migrate``.
 
-For `easy_thumbnails`_ to support retina displays (recent MacBooks, iOS) add to settings.py::
-
-    THUMBNAIL_HIGH_RESOLUTION = True
-
-If you forget this, you may not see thumbnails for your uploaded files. Adding this line and
-refreshing the admin page will create the missing thumbnails.
-
-
 Static media
 ............
 

diff --git a/filer/__init__.py b/filer/__init__.py
@@ -14,6 +14,6 @@
 10. twine upload dist/django-filer-{new version}.tar.gz
 """
 
-__version__ = '2.0.2'
+__version__ = '2.1rc1'
 
 default_app_config = 'filer.apps.FilerConfig'
diff --git a/filer/admin/clipboardadmin.py b/filer/admin/clipboardadmin.py
@@ -103,45 +103,21 @@ def ajax_upload(request, folder_id=None):
                     fields=('original_filename', 'owner', 'file')
                 )
                 break
-        uploadform = FileForm({'original_filename': filename,
-                               'owner': request.user.pk},
+        uploadform = FileForm({'original_filename': filename, 'owner': request.user.pk},
                               {'file': upload})
+        uploadform.instance.mime_type = mime_type
         if uploadform.is_valid():
             file_obj = uploadform.save(commit=False)
             # Enforce the FILER_IS_PUBLIC_DEFAULT
             file_obj.is_public = filer_settings.FILER_IS_PUBLIC_DEFAULT
             file_obj.folder = folder
-            file_obj.mime_type = mime_type
             file_obj.save()
             # TODO: Deprecated/refactor
             # clipboard_item = ClipboardItem(
             #     clipboard=clipboard, file=file_obj)
             # clipboard_item.save()
 
-            # Try to generate thumbnails.
-            if not file_obj.icons:
-                # There is no point to continue, as we can't generate
-                # thumbnails for this file. Usual reasons: bad format or
-                # filename.
-                file_obj.delete()
-                # This would be logged in BaseImage._generate_thumbnails()
-                # if FILER_ENABLE_LOGGING is on.
-                return JsonResponse(
-                    {'error': 'failed to generate icons for file'},
-                    status=500,
-                )
             thumbnail = None
-            # Backwards compatibility: try to get specific icon size (32px)
-            # first. Then try medium icon size (they are already sorted),
-            # fallback to the first (smallest) configured icon.
-            for size in (['32']
-                        + filer_settings.FILER_ADMIN_ICON_SIZES[1::-1]):
-                try:
-                    thumbnail = file_obj.icons[size]
-                    break
-                except KeyError:
-                    continue
-
             data = {
                 'thumbnail': thumbnail,
                 'alt_text': '',

diff --git a/filer/admin/folderadmin.py b/filer/admin/folderadmin.py
@@ -271,9 +271,9 @@ def directory_listing(self, request, folder_id=None, viewtype=None):
                 pass
 
         # search
-        q = request.GET.get('q', None)
+        q = request.GET.get('q')
         if q:
-            search_terms = urlunquote(q).split(" ")
+            search_terms = urlunquote(q).split(' ')
             search_mode = True
         else:
             search_terms = []

diff --git a/filer/fields/multistorage_file.py b/filer/fields/multistorage_file.py
@@ -119,6 +119,12 @@ def save(self, name, content, save=True):
         content.seek(0)  # Ensure we upload the whole file
         super().save(name, content, save)
 
+    def exists(self):
+        """
+        Returns ``True`` if underlying file exists in storage.
+        """
+        return self.storage.exists(self.name)
+
 
 class MultiStorageFileField(easy_thumbnails_fields.ThumbnailerField):
     attr_class = MultiStorageFieldFile

diff --git a/filer/migrations/0013_image_width_height_to_float.py b/filer/migrations/0013_image_width_height_to_float.py
@@ -0,0 +1,23 @@
+# Generated by Django 3.0.8 on 2020-10-28 16:28
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('filer', '0012_file_mime_type'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='image',
+            name='_height',
+            field=models.FloatField(blank=True, null=True),
+        ),
+        migrations.AlterField(
+            model_name='image',
+            name='_width',
+            field=models.FloatField(blank=True, null=True),
+        ),
+    ]
diff --git a/filer/models/abstract.py b/filer/models/abstract.py
@@ -1,8 +1,11 @@
 import logging
 
 from django.db import models
+from django.utils.functional import cached_property
 from django.utils.translation import gettext_lazy as _
 
+from easy_thumbnails.VIL import Image as VILImage
+
 from .. import settings as filer_settings
 from ..utils.compatibility import PILImage
 from ..utils.filer_easy_thumbnails import FilerThumbnailer
@@ -26,8 +29,8 @@ class BaseImage(File):
     file_type = 'Image'
     _icon = "image"
 
-    _height = models.IntegerField(null=True, blank=True)
-    _width = models.IntegerField(null=True, blank=True)
+    _height = models.FloatField(null=True, blank=True)
+    _width = models.FloatField(null=True, blank=True)
 
     default_alt_text = models.CharField(_('default alt text'), max_length=255, blank=True, null=True)
     default_caption = models.CharField(_('default caption'), max_length=255, blank=True, null=True)
@@ -43,7 +46,7 @@ class BaseImage(File):
     @classmethod
     def matches_file_type(cls, iname, ifile, mime_type):
         # source: https://www.freeformatter.com/mime-types-list.html
-        image_subtypes = ['gif', 'jpeg', 'png', 'x-png']
+        image_subtypes = ['gif', 'jpeg', 'png', 'x-png', 'svg+xml']
         maintype, subtype = mime_type.split('/')
         return maintype == 'image' and subtype in image_subtypes
 
@@ -56,7 +59,10 @@ def file_data_changed(self, post_init=False):
                 except ValueError:
                     imgfile = self.file_ptr.file
                 imgfile.seek(0)
-                self._width, self._height = PILImage.open(imgfile).size
+                if self.mime_type == 'image/svg+xml':
+                    self._width, self._height = VILImage.load(imgfile).size
+                else:
+                    self._width, self._height = PILImage.open(imgfile).size
                 imgfile.seek(0)
             except Exception:
                 if post_init is False:
@@ -80,16 +86,12 @@ def sidebar_image_ratio(self):
         else:
             return 1.0
 
-    def _get_exif(self):
-        if hasattr(self, '_exif_cache'):
-            return self._exif_cache
-        else:
-            if self.file:
-                self._exif_cache = get_exif_for_file(self.file)
-            else:
-                self._exif_cache = {}
-        return self._exif_cache
-    exif = property(_get_exif)
+    @cached_property
+    def exif(self):
+        try:
+            return get_exif_for_file(self.file)
+        except:
+            return {}
 
     def has_edit_permission(self, request):
         return self.has_generic_permission(request, 'edit')
@@ -126,11 +128,11 @@ def label(self):
 
     @property
     def width(self):
-        return self._width or 0
+        return self._width or 0.0
 
     @property
     def height(self):
-        return self._height or 0
+        return self._height or 0.0
 
     def _generate_thumbnails(self, required_thumbnails):
         _thumbnails = {}

diff --git a/filer/models/filemodels.py b/filer/models/filemodels.py
@@ -9,6 +9,7 @@
 from django.db import models
 from django.urls import NoReverseMatch, reverse
 from django.utils import timezone
+from django.utils.functional import cached_property
 from django.utils.translation import gettext_lazy as _
 
 from polymorphic.managers import PolymorphicManager
@@ -107,6 +108,14 @@ def __init__(self, *args, **kwargs):
         self._old_is_public = self.is_public
         self.file_data_changed(post_init=True)
 
+    @cached_property
+    def mime_maintype(self):
+        return self.mime_type.split('/')[0]
+
+    @cached_property
+    def mime_subtype(self):
+        return self.mime_type.split('/')[1]
+
     def file_data_changed(self, post_init=False):
         """
         This is called whenever self.file changes (including initial set in __init__).

diff --git a/filer/models/mixins.py b/filer/models/mixins.py
@@ -1,3 +1,5 @@
+import warnings
+
 from django.templatetags.static import static
 
 from ..settings import FILER_ADMIN_ICON_SIZES
@@ -10,6 +12,8 @@ class IconsMixin:
     """
     @property
     def icons(self):
+        warnings.warn("Method 'icons' is deprecated and will be re moved in future versions",
+                      DeprecationWarning)
         r = {}
         if getattr(self, '_icon', False):
             for size in FILER_ADMIN_ICON_SIZES:

diff --git a/filer/private/config.rb b/filer/private/config.rb