FORT RPKI Issue #133
Comments
|
Can you post the contents of |
|
{ |
Is this consistent with reality? Which version are you running? It's best not to change this value from its default unless you have a good reason for it.
Ohhhhhhhhhh. So you're using that message as a green light? I have been refactoring Fort's logging output since 1.6.0's development. This is one of the messages that changed. I left a comment in the code, actually. The log message was downgraded to INFO severity, and reworded into "Main loop: Ready for routers." Are you the person who requested that warning? I can restore it, but I would like to discuss alternate ways to solve this problem, because I'm not sure log parsing is a healthy way of doing it. The intent of the standards is that validators return an error code that means "No Data Available (yet)" while they're first building their VRP list. Clients are supposed to treat this as a temporary issue. If your routers persist in being disconnected long after Fort has finished the first cycle, then that suggests a bug somewhere. Can you please explain what happens if you connect your routers before Fort finishes the first cycle? And is there another reason why you wait for that WRN? |
|
thanks for your reply, I'm using version fort-1.6.1-1.el8.x86_64.rpm yes i'm waiting for this message to connect my routers: in my case, all VRPs have been downloaded successfully and stored in the /etc/fort/roas.csv. |
|
Ok, but I need to know the answer to these questions: What happens if you connect your routers before Fort finishes the first cycle? And what makes it so undesirable? |
|
Ok, I'm running out of time for the 1.6.2 release, so I will restore the warning for the time being. Please be aware that I will delete it again once #50 is implemented (which is currently scheduled for the 1.6.5 release), because once that happens I'll expect you to query the status to Prometheus instead of the logs. (I will document this when it happens.) But I still think there shouldn't be a reason why you'd need to wait out the first cycle to connect your routers. If unready Fort is doing something that's causing your routers to go crazy, then there's at least one bug in either of them, and if Fort is at fault, I would like to fix it. |
|
Hi Alberto, I still didn't connect my router. just waiting for the below message to appear in order to make sure that FORT is ready for router connection. my FORT status is active running. also all the VRPs have been downloaded and stored in /etc/var/roas.csv I just need to know what is the indication to know that FORT is running properly and ready for router connection. thanks for your support. |
|
I just released Fort 1.6.2. It contains the warning. |
|
Ok great, I will install it and keep you posted. thanks, |
|
hello, t=15 rsync://rpki.tools.westconnect.ca/repo/ /var/lib/fort//arin.tal/rsync/rpki.tools.westconnect.ca/repo pnic.tal: None of the URIs of the TAL '/etc/fort/tal/apnic.tal' yielded a successful traversal. below is the configuration: |
|
It's hard to debug with so little output. Please change into and into And post the log. Also, not sure if this is going to help, but try clearing the cache manually: |
|
it generates too much logs: /DaQFvPzFZIs1_iUDtwHfv6ImLGE.roa: ROAIPAddress { |
|
Ok, sorry about that. Try changing |
|
1.6.1 it was working properly for me and I was able to download the ROAs yesterday, however today i tried to re-install it after I failing with 1.6.2. but unfortunately, 1.6.1 no longer working as well:(. May 27 22:16:07 saryfortval02 fort[15764]: May 27 22:16:07 WRN [Validation]: rsync://rpki.ripe.net/repository/DEFAULT/X9ot9HVEGGls2-HKWQx9dBRj9Ww.cer: Retry> [haitham.jneid@saryfortval02 ~]$ systemctl status fort May 27 22:16:37 saryfortval02 fort[15764]: May 27 22:16:37 ERR [Validation]: rsync://ca.rg.net/rpki/RGnet-OU/NKcuY5DvQmG0vwsI6MT4nqcBcoI.gbr: Expected vCard> [haitham.jneid@saryfortval02 ~]$ systemctl status fort May 27 22:16:46 saryfortval02 fort[15764]: May 27 22:16:46 ERR [Validation]: rsync://rsync.paas.rpki.ripe.net/repository/1231fd86-c539-46c7-89e9-f3756f3075f> |
|
Well, I notice there are quite a few objects not downloading right now (even without Fort): Not sure what this is about. Temporarily, try zeroing these values, to speed up the validation cycle:
I'm guessing you're copying journalctl's output from your terminal, and that's why it's truncated? Please dump the output into a file, and post the file: |
This worked for me:
|
One other thing you can do to achieve a quick test run is to narrow down the TALs to your closest (and/or smallest) RIR. For example, in my case that would be Notice this will significantly reduce the VRP count. It's only meant to rush a validation cycle and simplify the logs.
|
|
fort.xlsx |
|
It seems your connection is a bit slow. Try this. |
|
For example, the following command is failing: Try executing that command manually, without See how long it normally takes to download it. Then adjust |
|
Also: You're still using old Not Try reducing your configuration to the following minimum, then work your way up if necessary: |
|
Hi, can you share with me the config.json that you are using for 1.6.1 or 1.6.2 thanks, |
|
Try this first: Does Fort eventually print the "now you can connect your routers" message if you configure it like that? |
|
it works, please check below output. what shall i do to fix it for all TALs? May 28 17:58:53 saryfortval02 fort[85717]: May 28 17:58:53 ERR [Validation]: rsync://rpki-repo.registro.br/repo/AYCCovAM3iMVZTC7a1ZXHjEgMEaknJfQ8goP1VTojXoz> |
|
also router will peer as RTR without specifing the IP in the config.json? |
|
also RPKI session is established: Session State Age IPv4/IPv6 record VPN |
Look at your TAL directory: First, I recommend that you try them one by one. Replace (in your configuration file) into Then run Fort until you see the message. Then replace it with the next one: Keep going until you have run them all. If one of them doesn't work, post its output. If all of them work, replace with To perform a validation run with all of them at the same time. And let's see what happens. |
When no IP is configured, Fort tries to bind itself to all available addresses. Feel free to change it to the proper address you want. |
|
[haitham.jneid@saryfortval02 fort]$ sudo service fort status May 28 18:16:22 saryfortval02 fort[86557]: May 28 18:16:22 INF [Validation]: /etc/fort/tal/apnic.tal: rsync: rsync://rpki.apnic.net/repository/ |
|
ARIN didn't worki May 28 18:20:43 saryfortval02 fort[87131]: May 28 18:20:43 INF [Validation]: /etc/fort/tal/arin.tal: rsync: rsync://rpki.arin.net/repository/ |
|
all fails except lacnic and afrinic. check logs. |
Ok. Try running this: It should print how long it took: How long does it take for you? |
|
It's weird that it rsyncs ARIN without trying HTTP first. Do you have an HTTPS URL in Mine reads like this: |
|
now Arin works, all TALs have https except apnic.tal eventhough i refreshed them. any clue? |
|
it seems apnic don't use https: |
|
but why rsync is not working? |
|
Most likely because it's timing out. Look at Fort's rsync arguments in the
configuration; It only has 60 seconds to sync. That's why I wanted you to
execute the rsync manually; to find out what's a reasonable timeout for
your connection.
If you don't want to run the command manually, try increasing --timeout
until it works.
…On Tue, May 28, 2024, 13:27 haithamjneid ***@***.***> wrote:
but why rsync is not working?
—
Reply to this email directly, view it on GitHub
<#133 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AASHNF7RA7HBRSCDOWEALETZETLAXAVCNFSM6AAAAABIDKQGWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZVHE2TSNBRGI>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
[haitham.jneid@saryfortval02 tmp]$ time rsync -rtz --delete --omit-dir-times --max-size=20MB --include=/ --include=.cer --include=.crl --include=.gbr --include=.mft --include=.roa --exclude=* rsync://rpki.arin.net/repository/ potatoes real 0m18.121s ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(644) [Receiver=3.1.3] real 7m8.195s |
Ok. That is weird, but notice that's ARIN's URL. You said ARIN was now working fine because it was using HTTP. If you want to fix APNIC, read the APNIC logs: So the APNIC rsync URL is |
|
(Remember to remove |
Ok, so... what I'm getting out of this is that you interrupted the rsync because it was taking too long. You reckon it got stuck? Tip: Add |
|
same, no progress real 1m41.876s |
|
don't know why rsync is not working and https is working. |
|
for apnic, do they have https instead of rsync? |
|
below are the rsync from Routinator RPKI. don't know if they give any clue. I run Routinator and it is working properly and downloading all ROAs. however I need to run FORT as well. why apnic don't have https? |
Works for me. Might be the work of some firewall.
Routinator's APNIC TAL does have HTTP. Looks like it's time to update Fort's. Try using Routinator's TAL: |
|
I used Routinator's TAL but it didn't work as well. I also removed the rsync link from all the TALs and kept only the https link. also not working. by the way, I have a VM for Routinator next to VM FORT, Routinator is working and FORT is not. !!! |
|
hello again, after I updated all the TALs to use only HTTPS and removed the RSYNC, it is working now as per below and all ROAs have been downloaded and fetched by the routers. May 30 12:45:07 saryfortval02 fort[129944]: May 30 12:45:07 WRN [Validation]: rsync://rpki.ripe.net/repository/DEFAULT/xZR-zIaDrQ282VqPMy8MqhNXR5A.cer: Retr>
check attached. |
Caveat: Removing rsync from the TALs does not disable rsync globally. I suspect your rsync started working suddenly, or Routinator's rsync also doesn't work. See below.
RPKI repositories have three options for serving their files: HTTP, rsync or both. When both are used, rsync is supposed to function as fallback in case HTTP fails. But note that serving files via rsync only is still an option. It's not the recommended way of doing it, but it's absolutely legal. This means that if your rsync doesn't work, you will potentially miss out on some VRPs. So yes, there is a way to completely disable rsync, but I don't recommend it: What you really want is to fix rsync. It's not clear whether your Routinator's rsync is working or not, because your output shows that Routinator and Fort ended up with a very similar amount of records. If Routinator's rsync worked and Fort's didn't, I would expect Routinator to have more records. According to Routinator's code 0 1, its rsync is (assuming your Can you please check if that works? (If it doesn't work, try removing But if it works, please confirm whether Fort's rsync still doesn't work: |
Their "current" TALs are not the ideal ones. Switch to the ones that feature HTTP. Spawned by #133.
|
hello, I want to inform you that my Routinator is downloading exactly the same number of ROAs as per below online Routinator from RIPE. as for FORT, it is also downloading the same number of ROAs. I believe both now are working properly. how many ROAs your FORT has downloaded? thanks, |
|
Okay, if you're happy with this outcome, feel free to close the issue. |
|
still RSYNC is not working, don't know why, we allowed it on Firewall and make sure access is open. FORT is working properly using HTTPS: [haitham.jneid@saryfortval02 ~]$ sudo service fort status Jun 03 16:26:09 saryfortval02 fort[280947]: Jun 3 16:26:09 ERR [Validation]: rsync://rsync.paas.rpki.ripe.net/repository/491a43bc-76ec-4abe-ba7d-e73e90d662> this is Routinator: could you please share your final vrps number to compare it with mine. |
548,806 VRPs at 2024-06-03 15:52:01 UTC. |
hello guys,
please need your support,
I'm running the following command and this is what I get:
[haitham.jneid@saryfortval02 ~]$ sudo service fort status
Redirecting to /bin/systemctl status fort.service
● fort.service - FORT RPKI validator
Loaded: loaded (/usr/lib/systemd/system/fort.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2024-05-22 12:51:42 +03; 26min ago
Docs: man:fort(8)
https://nicmx.github.io/FORT-validator/
Main PID: 146353 (fort)
Tasks: 22 (limit: 36397)
Memory: 1.3G
CGroup: /system.slice/fort.service
└─146353 /usr/bin/fort --configuration-file /etc/fort/config.json
May 22 12:51:42 saryfortval02 systemd[1]: Started FORT RPKI validator.
however for checking if FORT is ready for RTR connection, the following command ([root@validator ~]# journalctl -u fort -f
) is not providing the output stating that
Aug 12 13:34:00 validator fort[9708]: WRN: First validation cycle
successfully ended, now you can connect your router(s)
instead it provides the following output. please need your help.
[haitham.jneid@saryfortval02 ~]$ sudo journalctl -u fort -f
-- Logs begin at Thu 2024-05-16 18:27:25 +03. --
May 22 11:03:19 saryfortval02 systemd[1]: Stopping FORT RPKI validator...
May 22 11:03:19 saryfortval02 systemd[1]: fort.service: Succeeded.
May 22 11:03:19 saryfortval02 systemd[1]: Stopped FORT RPKI validator.
May 22 11:03:53 saryfortval02 systemd[1]: Started FORT RPKI validator.
May 22 11:12:24 saryfortval02 fort[142882]: May 22 11:12:24 WRN: Location '/etc/fort/' doesn't have files with extension '.slurm'
May 22 12:21:24 saryfortval02 fort[142882]: May 22 12:21:24 WRN: Location '/etc/fort/' doesn't have files with extension '.slurm'
May 22 12:51:18 saryfortval02 systemd[1]: Stopping FORT RPKI validator...
May 22 12:51:18 saryfortval02 systemd[1]: fort.service: Succeeded.
May 22 12:51:18 saryfortval02 systemd[1]: Stopped FORT RPKI validator.
May 22 12:51:42 saryfortval02 systemd[1]: Started FORT RPKI validator.
The text was updated successfully, but these errors were encountered: