Handle static files outside the templates folder

[maxxst]

Right now I have a folder structure like that :

. css/
. img/
. templates/
. . jinjafiles.html

I would be so happy if I could include css/ and img/ to the static path

[Ceasar]

Can you be more descriptive? If I understand correctly, using --static=css,img should solve your problem.

[maxxst]

it says everywhere in the docu, that the --static path is relative to the templates so I'm a little confused here ...

[tmewett]

... the directory (or directories) within srcpath where ...

You're right @maxxst, writing --static=../css,../img might work? Though that might copy the files to outpath/.., which isn't much use

[Ceasar]

Oops didn't fully understand the issue when I looked at this earlier.

Admittedly, I actually haven't used the staticfiles feature in my sites, so I'm a little hazy on how they work. @dominicrodger was the one who introduced them, in 4130fbb. My understanding is that this feature is for when users are building their site into a "build" directory, or for when they need to avoid compilation (because perhaps they're Javascript uses brackets {{}}).

@dominicrodger, is there a reason we're searching for static files in templates/ instead of at the top level? It seems like the default behavior should be to search for static files in a static/ directory, rather than templates/. If it were so, the staticpath would be relative to the project, which would make solving @maxxst's problem a little more intuitive.

In any case, @maxxst, if you are just building to a "build" directory, I can confirm @tmewett's solution does work.

[dominicrodger]

I think I did it that way because it seemed to make sense for to look on searchpath (which, as it happens, had previously only been used for templates). That's not a particularly good reason, that's just how I happened to implement it.

[Ceasar]

Okay cool. I'll implement a fix for this this week. Unfortunately, seems the change will be backward incompatible.

[Ceasar]

Actually, thinking about this more, I think handling static files it outside the scope of this project. I'm going to deprecate that feature. It's trivial to use Make to copy static files over. I want staticjinja to only deal with handling Jinja templates.

[NickCrews]

Actually, thinking about this more, I think handling static files it outside the scope of this project. I'm going to deprecate that feature. It's trivial to use Make to copy static files over. I want staticjinja to only deal with handling Jinja templates.

Sorry to open this back up, but I possibly have an argument for why staticjinja should be able to handle static files: I want my website src directory to directly mirror the build directory. For example

• src/
  • _base.html
  • index.html
  • base.css
  • background.jpg
  • projects/
    • _project.html
    • foo.html
    • foo/
      • foo1.jpg
      • foo2.jpg
    • bar.html
    • bar/
      • bar1.jpg
      • bar2.jpg
    • projects.css

should lead to a build/ directory that exactly mirrors src/ except that templates are rendered. In other words, I don't want to have to use a templates/ directory separately from static css/ and img/ directories for two reasons:

1. It's not obvious the location that the results will be in because the directory structure is transformed.
2. foo1.jpg would be separated from foo.html, and projects.css is separated from _project.html, even though they are related.

Also, I would argue that requiring people to use Make is just another hurdle, I would envision the point of staticjinja to be a one-stop-shop for making a simple website in plain python.

I have a PR request incoming that adds better static/partial/ignored/template filtering, I'll try yo remember to link that here when complete.

[NickCrews]

Re-opening this, because I think this still is a useful feature. This opinion is also supported by the [as of now] three 👍 's that my previous comment received.

As I look into supporting copying static files, here are some of my initial design requirements/brain dump:

1. Support for defining the types of files using regex, as is used now. The way regex are used should be consistent throughout staticjinja, for example a comma separated list of patterns. This seems adequate, and there's some precedent since clang-tidy uses this format with its -checks option.
2. If regex are inadequate some way of writing python that can do the filtering. Either implemented with subclassing of Site, or by passing in a custom function (I'm guessing at this point option two is better).
3. Ideally make things compatible with SASS and other CSS preprocessors, as I could see that being common to generate staticfiles. That might mean that we should be able to find static files in SASS's output location? IDK exactly yet.
4. Backwards compatibility:
   a. Should not break existing assumptions of . prefix meaning ignored, etc. because that's so common.
   b. Can break other uses of staticfiles, e.g. perhaps now by default treating *.js,*.css,*.jpe?g... as static (whereas now I believe they would be treated as templates)
   c. It's OK to break other use cases if they're not too common.
5. Function args, docstrings, etc might need to be updated to use phrasing like "filename" instead of "template_name". However, we should keep on always using unix "/" as path separators (as jinja2 does), so users can use their code on both Windows and Linux/OSX.
6. Support static files outside the src folder? I don't think so, because as I say above how would you merge the two input directories into the single output directory?
7. ....

Please give any feedback or suggestions that you have. Thanks!

[ghost]

Update 2022-07-08

OK, so I actually wrote a Makefile to replicate / restore the staticjinja functionality for static files. It does not have a lot of prerequisites and it is as easy to use as the staticjinja CLI. There are three caveats:

1. As this is a Makefile, there is not too much error handling. So use it with care (i. e. watch your command line parameters).
2. While I have tested this Makefile, there might still be bugs, especially as I am no Makefile expert. I will update it right here as they are found and fixed.
3. All the static stuff (files and directories) has to live in one directory, the statpath, which is separate from the directory used for the templates (the srcpath). So, in practice, the folder structure might look like bellow (although in fact, both directories might be located whereever you want them to be located). If the Makefile gets called with the build target, all the static stuff located inside the statpath will simply be copied over to the outpath. If it gets called with the watch target, the static stuff will also be copied over before being watched for changes afterwards, which will be copied over as they occur.

+ web_project
  Makefile
  + templates            <- this stuff will be handled by staticjinja
    .base.html
    index.html
  + static               <- this stuff will be handled by the Makefile
    base.css
    + static_folder_1
      static_file_1
    + static_folder_2
      static_file_2

Prerequisites

• staticjinja
• make
• inotifywait (inotify-tools)

Usage

The interface is similiar to the staticjinja CLI tool. All paths may be absolute or relative and may have a trailing slash or not. They are also all separate from each other, in contrast to the paths specified via the static parameter of the staticjinja CLI tool, which had to be located inside the srcpath.

make build
make watch outpath=/my/custom/outpath/
make build srcpath=customtemplates outpath=/path/public_html
make watch srcpath=../pathto/mytemplates outpath=overandout/ statpath=/full/path/staticfiles/

The Makefile

This code should be copied to a file named Makefile.

srcpath = ./templates
outpath = ./
statpath =

export SRCPATH=$(srcpath:/=)/
export OUTPATH=$(outpath:/=)/
export STATPATH=$(statpath:/=)/

MAKEFLAGS += -j2

build: .build_srcpath .build_statpath

.build_srcpath:
	@staticjinja build --srcpath="$${SRCPATH}" --outpath="$${OUTPATH}"

.build_statpath:
	@if [ -n "$${STATPATH%/}" ]; then \
	  echo "Copying static content..."; \
	  cp -r "$${STATPATH}"* "$${OUTPATH}" 2>/dev/null; \
	fi

watch: .watch_srcpath .watch_statpath

.watch_srcpath:
	@staticjinja watch --srcpath="$${SRCPATH}" --outpath="$${OUTPATH}"

.watch_statpath: .build_statpath
	@if [ -n "$${STATPATH%/}" ]; then \
	  echo "Watching '$$(realpath "$${STATPATH}")' for static content changes..."; \
	  inotifywait -mrq -e create,modify,moved_to "$${STATPATH}" | while read FILEPATH EVENT FILE; do \
	    RELFILEPATH="$${FILEPATH##"$${STATPATH}"}"; \
	    RELFILE="$${RELFILEPATH}$${FILE}"; \
	    echo "STATIC CONTENT $${EVENT} $${RELFILE}"; \
            echo "Copying static content $${RELFILE}..."; \
	    mkdir -p "$${OUTPATH}$${RELFILEPATH}"; \
	    cp -r "$${STATPATH}$${RELFILE}" "$${OUTPATH}$${RELFILE}" 2>/dev/null; \
	  done; \
	fi

Also, even though this approach works and is not too horrible, having this functionality right in staticjinja would make things a lot easier.

Original post

I think Make is an inappropriate solution for this problem, for several reasons:

Also, I would argue that requiring people to use Make is just another hurdle, I would envision the point of staticjinja to be a one-stop-shop for making a simple website in plain python.

This is exactly what I would expect from staticjinja (and what staticjinja actually did before the depreciation): building a whole website by executing one command. For example I could just call staticjinja build --srcpath=templates --outpath=/somehwere/public_html/ --static="images,css" and my whole website would appear at /somehwere/public_html/. Easy.

Now, this functionality could be imitated by using a Makefile to call staticjinja build and to copy the static files. All while introducing lots of complexity and a build tool most people probably do not even have installed by default. I also would argue that most people have no clue how to write a Makefile, considering this is often not required for programming in Python, Ruby, Go, Rust, ...

But what is even worse is that staticjinja watch can not be imitated using a Makefile, at least I do not think so. The reason is that staticjinja watch also tracks / tracked static files and copied them as they got added / updated. This is something Make can not easily do, at least not without using even more tooling. Of course one could revert to other methods, like using Python with the inotify module etc., which would be just as bizzare contraptions.

So, at the moment, all staticjinja allows is to continuously monitor template changes, while the monitoring of static files is left as an exercise. This is somewhat disappointing, at least if one saw staticjinja as a way to render a website (as opposed to just the HTML part of a website).

If I am mistaken and the previous functionality can actually be replicated using a Makefile: Would it be possible to add an example Makefile to the documentation? I think this is a common problem / task which would warrant to do so.