Unable to submit research #14
Labels
bug
Something isn't working
Comments
|
Hi, and thanks for reporting this issue. I have tested this extensively on v1.1.3 and have been unable to reproduce this with Chrome and Firefox on Android 9, which is the only compatible device I currently have available. The officially supported browsers for FreeField on Android are Chrome and Firefox. I also doubt that this is related to event research as the way it is treated internally is exactly the same as other research tasks. This looks to be a pretty hard to resolve issue, and I will need quite a bit of information in order to troubleshoot it. Could you please tell me the following things about cases when this occurs?
Also, since you mentioned you host your own server:
I might have to ask for some additional information later, but for now, whatever you can provide of what I asked above will be very helpful in trying to figure out exactly what is going on here. And lastly, sorry for asking this, it might seem like a stupid question but I need to know this for verifying the issue - when reporting research that offers an "[n] prize" reward, can you confirm that the additional box labeled "Quantity" underneath the selected reward is correctly filled with a value and not left blank when clicking Report? Thanks for the report and information so far. |
|
Here is video attached what happens when submit does not go trough.
- Most importantly, what happens when research submission fails - do you
get an error message when you try to submit research, does it submit but
not show up on the map afterwards, or does nothing happen at all when you
tap the Report button?
- No error message, just unable to proceed.
-
- Are there any instances at all where reporting this type of research
from a mobile browser *does* work (not using desktop mode) when the same
task fails on another device?
- No.
-
- If reporting fails for one device, does it *always* fail for that
device as long as they do not use desktop mode?
- It fails, occasionally it goes trough when using desktop mode or pc.
-
- Is there anything in common between a lot of the affected devices, as
far you can tell? Things like device manufacturer, language, mobile network
provider used etc.
- Different devices, different android/IOS versions. Only common thing
is chrome. PWA is not enabled.
-
- For one of the devices, can you test if reporting works on WiFi vs.
mobile data?
- This is something i need to double check.
And about my own server, i try to keep running it cheap and mainly training
server management. If you need to check hardware specs, I use Raspberry Pi3.
All packages should be updated, so PHP7 is installed.
Apache+Maria Database (MySQL tables, same engine, so it should not be an
issue)
Basic firewall configurations for inbound and outbound for traffic.
Fail2Ban is installed and configured too.
HTTPS is configured.
Planning to move that to Azure at somepoint.
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Ei
viruksia. www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
to 2. toukok. 2019 klo 2.25 Marius Lindvall ([email protected])
kirjoitti:
… Hi, and thanks for reporting this issue. I have tested this extensively on
v1.1.3 and have been unable to reproduce this with Chrome and Firefox on
Android 9, which is the only compatible device I currently have available.
The officially supported browsers for FreeField on Android are Chrome and
Firefox. I also doubt that this is related to event research as the way it
is treated internally is exactly the same as other research tasks.
This looks to be a pretty hard to resolve issue, and I will need quite a
bit of information in order to troubleshoot it. Could you please tell me
the following things about cases when this occurs?
- Most importantly, what happens when research submission fails - do
you get an error message when you try to submit research, does it submit
but not show up on the map afterwards, or does nothing happen at all when
you tap the Report button?
- Are there any instances at all where reporting this type of research
from a mobile browser *does* work (not using desktop mode) when the
same task fails on another device?
- If reporting fails for one device, does it *always* fail for that
device as long as they do not use desktop mode?
- Is there anything in common between a lot of the affected devices,
as far you can tell? Things like device manufacturer, language, mobile
network provider used etc.
- For one of the devices, can you test if reporting works on WiFi vs.
mobile data?
Also, since you mentioned you host your own server:
- Are there any PHP notices/warnings/errors/fatal errors in your
server's error logs that appear around the time users try to submit
research?
- Is the server running PHP 7, or the older, outdated PHP 5?
- Are you using another HTTP and SQL daemon combination than Apache +
MySQL?
- Are you using a web application firewall (WAF), load balancer or
reverse proxy (CloudFlare-type) in front of your server, or have you
configured your server to block some connections based on the User-Agent or
IP address of connecting clients?
- Do you use HTTPS?
I might have to ask for some additional information later, but for now,
whatever you can provide of what I asked above will be very helpful in
trying to figure out exactly what is going on here.
And lastly, sorry for asking this, it might seem like a stupid question
but I need to know this for verifying the issue - when reporting research
that offers an "[n] prize" reward, can you confirm that the additional box
labeled "Quantity" underneath the selected reward is correctly filled with
a value and not left blank when clicking Report?
Thanks for the report and information so far.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#14 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ALN7B7JHCE3H3GITZQ5TJY3PTIRFHANCNFSM4HJU3VOQ>
.
|
|
It doesn't seem like the video attachment got through with your email. Could you try submitting it on this issue from the GitHub website instead? It also sounds like something on your server is blocking connections based on the browsers' user agents and/or HTTP request method. I don't know what kind of fail2ban or mod_security configurations you're running, but if this is the only filtering you're using in addition to a basic firewall, I'm somewhat inclined to believe the issue is there, but I can't tell for certain. Could you try the following in order and see if this has any effect?
When making changes to FreeField configuration, make sure you do a simple refresh of the page on the phone before you try reporting research again. Your environment otherwise looks good; Apache with PHP 7 and MySQL provided by MariaDB is well supported and is what I also run for a few FreeField installations without issue. Firewalls should not be an issue. If any of this, or none of it works, let me know and I'll see what I can do as the next step. |
|
Hi.
I did some checks and there is no mod_security installed in Apache.
Fail2ban is configured to make ssh more secure, ban if try to bruteforce.
- In the FreeField admin settings pages, go to the Security page and set
"User-agent validation" to Disabled. Save, then on the phone, sign out of
FreeField (if you're signed in), then sign back in again and try to report
research. If you're not signed in, you can try clearing browser cookies
after changing the setting.
This seems to fix the issue.
Those rest are actually checked to be ok.
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Ei
viruksia. www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
to 2. toukok. 2019 klo 12.38 Marius Lindvall ([email protected])
kirjoitti:
… It doesn't seem like the video attachment got through with your email.
Could you try submitting it on this issue from the GitHub website instead?
It also sounds like something on your server is blocking connections based
on the browsers' user agents and/or HTTP request method. I don't know what
kind of fail2ban or mod_security configurations you're running, but if this
is the only filtering you're using in addition to a basic firewall, I'm
somewhat inclined to believe the issue is there, but I can't tell for
certain.
Could you try the following in order and see if this has any effect?
- Do you use mod_security in Apache? If so, can you please ensure that
the HTTP request methods PATCH, PUT and DELETE are permitted on the same
level as GET and POST? Some configurations (see this question for
example <https://stackoverflow.com/q/48810247>) may block or
rate-limit these types of requests by default. FreeField uses these methods
because research submissions are done against a REST API built into
FreeField. If one of your fail2ban jails uses a filter regex against GET or
POST, ensure that PATCH, PUT and DELETE are also included in that regex.
- In the FreeField admin settings pages, go to the Security page and
set "User-agent validation" to Disabled. Save, then on the phone, sign out
of FreeField (if you're signed in), then sign back in again and try to
report research. If you're not signed in, you can try clearing browser
cookies after changing the setting.
- Also in the FreeField settings, go to Permissions and ensure that
the "Report field research" permission is set to the same, or a lower value
than "Default user group" at the top of the same page.
- Lastly, go to Site settings in FreeField and double-check that
"Installation URI" starts with "https".
- Whitelist the IP address that your phone is connecting from in
fail2ban and see if this has any effect. Check that the IP address is not
banned by fail2ban.
- Temporarily disable fail2ban and mod_security (if using it) and see
if submission works when these modules are turned off. If they do, then it
would indicate that the error is somewhere in the configuration of these
modules.
When making changes to FreeField configuration, make sure you do a simple
refresh of the page on the phone before you try reporting research again.
Your environment otherwise looks good; Apache with PHP 7 and MySQL
provided by MariaDB is well supported and is what I also run for a few
FreeField installations without issue. Firewalls should not be an issue.
If any of this, or none of it works, let me know and I'll see what I can
do as the next step.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#14 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ALN7B7K2YXLTH7EWIY7JX3TPTKZCJANCNFSM4HJU3VOQ>
.
|
|
Hi, If disabling user-agent validation resolves the issue, I'd recommend keeping it disabled as the resolution to this issue, as the security issue it protects against is an edge case. I'll see if I can find the root cause of the issue. Is it possible for you to do a research report from your phone, and then send me the relevant lines from the report from your Apache access log, showing the user agent? You can remove the URL, IP addresses, etc. from the log. I only need the user agents for this purpose. This will help me do local testing to see if there are any notable differences. I would be very grateful if you could provide this :-) Otherwise, I will add a note in the FAQ and make note of this resolution should it ever come up again. It's a really strange issue and I'm not able to reproduce it locally. |
|
Hi, here some info from access logs (access.log)
[08/May/2019:19:33:54 +0300] "GET /api/poi.php?updatedSince=-15 HTTP/1.1"
200 3565 "/" "Mozilla/5.0 (Android 8.0.0; Mobile; rv:66.0) Gecko/66.0
Firefox/66.0"
Mapaccess.log:
::1 - - [08/May/2019:19:31:16 +0300] "OPTIONS * HTTP/1.0" 200 110 "-"
"Apache/2 (server) OpenSSL/1.0.2r (internal dummy connection)"
Error logs are empty after this Monday, due the adjusting settings.
I'm not software developer, but it seems that there is some issues how
server sees the browsers (mobile vs desktop). That could also be at the
server side, depending how fast server can reply. Making few adjustment
that problem is more random and rare, and it seems only happen when using
mobile browser. Though this could also be user side issue (multiple active
connections, not up to date browser and so on). I also made debug stop, in
case of the need to check if some quests are not reportable.
My server settings may not be perfect, but some fine tuning and more
practice with this.
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Ei
viruksia. www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
ke 8. toukok. 2019 klo 11.32 Marius Lindvall ([email protected])
kirjoitti:
… Hi,
If disabling user-agent validation resolves the issue, I'd recommend
keeping it disabled as the resolution to this issue, as the security issue
it protects against is an edge case.
I'll see if I can find the root cause of the issue. Is it possible for you
to do a research report from your phone, and then send me the relevant
lines from the report from your Apache access log, showing the user agent?
You can remove the URL, IP addresses, etc. from the log. I only need the
user agents for this purpose. This will help me do local testing to see if
there are any notable differences. I would be very grateful if you could
provide this :-)
Otherwise, I will add a note in the FAQ and make note of this resolution
should it ever come up again. It's a really strange issue and I'm not able
to reproduce it locally.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#14 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ALN7B7I4NP3HG4OJDSMZY3TPUKFYTANCNFSM4HJU3VOQ>
.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello. Some researches cannot be submitted. I have narrowed this problem to be problem with mobile browsers, since desktop(pc) works always, some researches can be submit when using mobile browsers desktop mode.
I am using Android 8 and Chrome. Same error seems to happen many other devices and browsers. I have already give advice to use chrome since it seems to have best support. I keep asking from our community about device OS's and browser's to gather more info.
Error occurs only when prize parameter is " [n] prize". Just sometimes this happens, so it could be event related research. Happens only when you need to manually tell quantity of items to get as a prize. My guess is that it could be related to event handling.
The text was updated successfully, but these errors were encountered: