Skip to content

[6.0] Cache language files #45289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: 6.0-dev
Choose a base branch
from

Conversation

HLeithner
Copy link
Member

Summary of Changes

Add a caching layer to the languageHelper::parseIniFile method.

Benchmark the function on within the system is not easy the the performance gain is in about 1,5 to 20ms depending on language size and server configuration.

My benchmark on this method only for the backend dashboard had the following results.

Without patch: 2,6-3.0ms
With patch cold cache: 7,6-11ms
With patch warm cache: 0.3-1.11ms

Means a performance gain of 130% to 900% per page load (for language loading).

Using a synthetic 1mb big ini file (35k lines) the following benchmarks apply:
Without patch: 14,6-25,2ms
With patch cold cache: 32,9-34,5ms
With patch warm cache: 0.5-0,9ms

Means a performance gain of 1500% to 5000% per page load (for language loading).

The Benchmark are taken on a Webserver with production load so might be

The results depend heavily on the server configuration and benefits if opcache is used (which is usually active since a couple of years on most(all) hosting providers).

The cache automatically invalidates it's self as soon as the ini file has a new modification timestamp.

Testing Instructions

  • Apply patch
  • Open Frontend and Backend

Actual result BEFORE applying this Pull Request

  • should work

Expected result AFTER applying this Pull Request

  • Should Work
  • loads a bit faster.

Link to documentations

Please select:

  • Documentation link for docs.joomla.org:

  • No documentation changes for docs.joomla.org needed

  • Pull Request link for manual.joomla.org:

  • No documentation changes for manual.joomla.org needed

@brianteeman
Copy link
Contributor

The cache automatically invalidates it's self as soon as the ini file has a new modification timestamp.

What about when an override is created?

@HLeithner
Copy link
Member Author

overrides are also ini files, the caching depends on the modification time of the ini file and will automatically create a new cache based on it.

@brianteeman
Copy link
Contributor

I have tested this item 🔴 unsuccessfully on 4a7a88c

Applied the PR and no language values are loaded - only constants

Enabled language debug
Warning
: foreach() argument must be of type array|object, string given in
D:\repos\j6\plugins\system\debug\src\DataCollector\LanguageErrorsCollector.php
on line
132


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/45289.

@HLeithner
Copy link
Member Author

funny, because actually the error collector function is broken, can you please.

can you please test 2 things (because I can't replicate it).

first move this line out of the try block, so the exception is not cached and you can see it.

$strings = LanguageHelper::parseIniFile($fileName, $this->debug);

second add [] around the text string

$this->errorfiles[$fileName] = 'PHP parser errors :' . $e->getMessage();

$this->errorfiles[$fileName] = ['PHP parser errors :' . $e->getMessage()];

then your error message shouldn't be warning anymore

third be sure to use the latest version from the "Preserve directory structure" or later commit, since I fixed the that frontend languages are used in backend and vis-a-vis.

@brianteeman
Copy link
Contributor

brianteeman commented Apr 6, 2025

first move this line out of the try block, so the exception is not cached and you can see it.

Joomla\Filesystem\Folder::create: Could not create directory. Path: [ROOT]\administrator\cache\language\D:

Look at the paths

image

image

@HLeithner
Copy link
Member Author

I have no windows to test, but hope my change fixed it.

@brianteeman
Copy link
Contributor

brianteeman commented Apr 7, 2025

That change worked.

tested that overrides still work
tested site and admin
tested plugin that has language files in the plugin and not the main languages folder eg plg_sampledate_testing

Not tested any performance changes

@brianteeman
Copy link
Contributor

I have tested this item ✅ successfully on 3acc14c


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/45289.

@brbrbr
Copy link

brbrbr commented Apr 7, 2025

Reading and parsing a json file is faster.

like json_decode(file_get_contents('administrator/cache/language/administrator-language-en-GB-lib_joomla.ini.1743321429.json'),true);

about 30% on my stystem

@HLeithner
Copy link
Member Author

Reading and parsing a json file is faster.

like json_decode(file_get_contents('administrator/cache/language/administrator-language-en-GB-lib_joomla.ini.1743321429.json'),true);

about 30% on my stystem

can you please show me you measurements, because my test says it's about 8-10 times slower using json for empty frontpage and using a 1mb ini files it's 80-100 times slower.

@brbrbr
Copy link

brbrbr commented Apr 7, 2025

Hi,

I found the difference: php is a lot faster with opcache enabled. Without opache json.

So the caching strategy should depend on the opcache setting to makeit fancy?

tested on 52 language files on a (almost) clean j6 site.

opache on:

J: 2528880
A: 150734
J: 2629746
A: 141746
J: 2793574
A: 159460
J: 2343749
A: 160488
J: 2136911
A: 112294
J: 2428274
A: 131338
J: 2236474
A: 122881
J: 2356152
A: 136126
J: 2465985
A: 122388

opache off:

J: 3127872
A: 7223167
J: 3041450
A: 6905625
J: 3241212
A: 7150972
J: 3058796
A: 7086077
J: 3219969
A: 7078478
J: 3025679
A: 6968267
J: 3046686
A: 7132398
J: 3202466
A: 6967631
J: 2978589
A: 7161319

@HLeithner
Copy link
Member Author

KISS, if you don't use opcache you don't care about performance so we shouldn't blow up our codebase for nothing.

@exlemor
Copy link

exlemor commented Apr 7, 2025

I have tested this item ✅ successfully on 3acc14c

I have successfully tested this on 2 sites: non multi-lingual and 1 multi-lingual.

I can honestly say I saw consistently faster speeds however the speed improvement varied from quite a bit from 10% - 26% - I did the battery of test 5 times to get a sense but the large difference I assume comes from 1. the fact that caching is complex and 2. I have fast hosting so the speed difference would be small. Either way I trust you guys! :)


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/45289.

@richard67
Copy link
Member

RTC


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/45289.

@joomla-cms-bot joomla-cms-bot added the RTC This Pull Request is Ready To Commit label Apr 8, 2025
@laoneo
Copy link
Member

laoneo commented Apr 25, 2025

Just curious, why did you not use the Joomla cache API?

@HLeithner
Copy link
Member Author

Just curious, why did you not use the Joomla cache API?

because it's too slow, the caching api would save the content to a file or database or apcu or whatever, which means it needs to load it (over tcp as worse case) and then decode the string and transform it into an array.

when doing it directly it uses the opcache in memory and in binary/compiled format. But for this to work you need a file on the filesystem.

There is no faster format then opcache, normally I would say yes use the caching api from joomla but since it's a very specific case (it's saved in the "wrong" format ini) I have chosen this way.

Above you see the variant with json which is slower, I would expect similar results for the joomla caching layer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR-6.0-dev RTC This Pull Request is Ready To Commit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants