Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache crawler - run within a certain range of time only #621

Open
wants to merge 14 commits into
base: dev
Choose a base branch
from
Open

Cache crawler - run within a certain range of time only #621

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
3716165
Cache crawler- run within a certain range of time only
Tymotey Mar 1, 2024
06e501f
Changes to default value + instance usage
Tymotey Mar 4, 2024
c1f8d64
Time parsing update
Tymotey Mar 7, 2024
5e1b605
Added "equal to" to time comparison
Tymotey Mar 7, 2024
3e8bf12
Crawler in schedule time - change from DateTime to strtotime
Tymotey Mar 8, 2024
d5b15a2
Crawler in schedule time - change example to 24h format
Tymotey Mar 8, 2024
717b1bc
Fix setting translation
Tymotey Mar 8, 2024
4e93ce5
Tim empty spaces
Tymotey Mar 8, 2024
0193066
Improve readability
Tymotey Mar 8, 2024
4285541
Text changes
Tymotey Mar 11, 2024
b93d92b
Changed text
Tymotey Mar 11, 2024
93648f2
Review fixes
Tymotey Apr 16, 2024
a64bdd0
New Review fixes
Tymotey Apr 17, 2024
e645104
Review update
Tymotey Aug 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/base.cls.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,7 @@ class Base extends Root
## -------------- Crawler ----------------- ##
## -------------------------------------------------- ##
const O_CRAWLER = 'crawler';
const O_CRAWLER_SCHEDULE_TIME = 'crawler-schedule_time';
const O_CRAWLER_USLEEP = 'crawler-usleep';
const O_CRAWLER_RUN_DURATION = 'crawler-run_duration';
const O_CRAWLER_RUN_INTERVAL = 'crawler-run_interval';
Expand Down Expand Up @@ -514,6 +515,7 @@ class Base extends Root

// Crawler
self::O_CRAWLER => false,
self::O_CRAWLER_SCHEDULE_TIME => '',
self::O_CRAWLER_USLEEP => 0,
self::O_CRAWLER_RUN_DURATION => 0,
self::O_CRAWLER_RUN_INTERVAL => 0,
Expand Down
74 changes: 74 additions & 0 deletions src/crawler.cls.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@

namespace LiteSpeed;

use \DateTime;

defined('WPINC') || exit();

class Crawler extends Root
Expand Down Expand Up @@ -247,6 +249,72 @@ public static function async_handler($manually_run = false)
self::start($manually_run);
}

/**
* Check if crawler can run in the choosen time period
*
* @since 6.1
*/
public static function _crawler_in_schedule_time()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be static?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

function that calls this function is static

image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is different things.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

{
$class_settings = self::cls();
$schedule_times = $class_settings->conf(Base::O_CRAWLER_SCHEDULE_TIME, '');

if ($schedule_times !== '') {
$schedule_times = explode(',', $schedule_times);

if (count($schedule_times) > 0) {
$now = time();

// A single time: e.g. 1, 01, 1:1, 1:01, 1:01:1, or 1:01:01, etc.
$time_re = '(\d{1,2}(?::\d{1,2}){0,2}(?i:[AP]M)?)';
$re = '/^' . $time_re . '[-]' . $time_re . '$/';

// Allow parsing times like 1-3, 1AM-3PM, 1aM-3Pm
$with_minutes = function ($time) {
$has_meridian = stripos($time, 'm');
if (preg_match('/:\d/', $time)) {
return $time;
} else {
if ($has_meridian !== false) {
$meridian = strtoupper(substr($time, -2));
$time_only = substr($time, 0, -2);

return $time_only . ':00' . $meridian;
} else {
return $time . ':00';
}
}
};

foreach ($schedule_times as $time) {
$time = trim($time);
preg_match($re, $time, $matches);

if (!$matches) {
continue;
}

$start = strtotime($with_minutes($matches[1]));
$end = strtotime($with_minutes($matches[2]));

if (false === $start || false === $end || $start > $end) {
continue;
}

// Test start <= now <= end
if ($now <= $end && $now >= $start) {
return true;
}
}
}

self::debug('------------crawler schedule time-------------no time period found');
return false;
} else {
return true;
}
}

/**
* Proceed crawling
*
Expand All @@ -260,6 +328,12 @@ public static function start($manually_run = false)
return false;
}

$crawler_is_in_time = self::cls()->_crawler_in_schedule_time();
if (!$manually_run && !$crawler_is_in_time) {
self::debug('......crawler is NOT allowed at this time......');
return false;
}

if ($manually_run) {
self::debug('......crawler manually ran......');
}
Expand Down
1 change: 1 addition & 0 deletions src/lang.cls.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ public static function title($id)
self::O_CDN_CLOUDFLARE => __('Cloudflare API', 'litespeed-cache'),

self::O_CRAWLER => __('Crawler', 'litespeed-cache'),
self::O_CRAWLER_SCHEDULE_TIME => __('Running time', 'litespeed-cache'),
self::O_CRAWLER_USLEEP => __('Delay', 'litespeed-cache'),
self::O_CRAWLER_RUN_DURATION => __('Run Duration', 'litespeed-cache'),
self::O_CRAWLER_RUN_INTERVAL => __('Interval Between Runs', 'litespeed-cache'),
Expand Down
19 changes: 19 additions & 0 deletions tpl/crawler/settings-general.tpl.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,25 @@
</td>
</tr>

<tr>
<th>
<?php $id = Base::O_CRAWLER_SCHEDULE_TIME; ?>
<?php $this->title($id); ?>
</th>
<td>
<?php $this->build_input($id); ?>
<div class="litespeed-desc">
<?php echo __('Specify the crawler running periods.', 'litespeed-cache'); ?>
<br />
<?php echo sprintf( __('You can add multiple ranges in 24-hour format <code>HH:mm-HH:mm</code> delimited by %s', 'litespeed-cache'), '<code>,</code>'); ?>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Time format shouldn't be in translatable string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

<br />
<?php echo sprintf( __('Server time: %s', 'litespeed-cache'), '<code>'. date('H:m') . '</code>'); ?>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is redundant to use sprintf here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed and left only __

<br />
<code>00:00-06:00 , 20:00-23:59, 01:00-05:00</code>
</div>
</td>
</tr>

<tr>
<th>
<?php $id = Base::O_CRAWLER_USLEEP; ?>
Expand Down