Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job output not refreshing automatically in AWX 24.1.0 wit operator 2.14.0 #15071

Open
6 of 11 tasks
sysadmin-info opened this issue Apr 6, 2024 · 5 comments
Open
6 of 11 tasks

Comments

@sysadmin-info
Copy link

Please confirm the following

  • I agree to follow this project's code of conduct.
  • I have checked the current issues for duplicates.
  • I understand that AWX is open source software provided for free and that I might not receive a timely response.
  • I am NOT reporting a (potential) security vulnerability. (These should be emailed to [email protected] instead.)

Bug Summary

HI,

the bug is identical like the one described here: #9747

  • I did all the checks. So I removed completelly the older version and installed everything anew from the scratch. No luck.
  • I did not even especially installed the NGINX as ingress with hostNetwork: true what I usually do at home lab. - also does not work.
  • I tested it with hostNetwork: true. Does not work.
  • I tested it without NGINX ingress. Does not work.
  • I tested it with NGINX ingress. Does not work.
  • I also tested it with external DNS by modifying the coreDNS. Does not work.
  • I tested it also with NGINX Proxy Manager. Does not work.

No above solution works. I mean this functionality just does not work and it seems it is related to websocket.

Operator version 2.14.0
AWX version 24.1.0
Database clean installation: PostgreSQL 15
k3s version: v1.28.8+k3s1
OS: Debian 12 with kernel: 6.1.0-18-amd64

I see identical errors that the websocket is not working as expected like it is described in the URL I provided.

The additional error:

The error Uncaught DOMException: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state

indicates that the code is trying to send a message through the WebSocket before the connection has been fully established. This happens because the send method is called immediately after creating the WebSocket, but before the connection has completed the handshake with the server.

To resolve this issue, you should ensure that the send operation in the ws.onopen callback occurs only after the WebSocket connection is fully open. The onopen event is designed to handle this scenario, as it's triggered when the WebSocket connection has been established successfully.

Here's how you can modify the code to ensure the send method is called at the right time:

let ws;

export default function connectJobSocket({ type, id }, onMessage) {
  ws = new WebSocket(
    `${window.location.protocol === 'http:' ? 'ws:' : 'wss:'}//${
      window.location.host
    }${window.location.pathname}websocket/`
  );

  ws.onopen = () => {
    console.debug('WebSocket connection established.');

    const xrftoken = `; ${document.cookie}`
      .split('; csrftoken=')
      .pop()
      .split(';')
      .shift();
    const eventGroup = `${type}_events`;

    // Ensure the connection is open before sending data
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(
        JSON.stringify({
          xrftoken,
          groups: { jobs: ['summary', 'status_changed'], [eventGroup]: [id] },
        })
      );
    }
  };

  ws.onmessage = (e) => {
    onMessage(JSON.parse(e.data));
  };

  ws.onclose = (e) => {
    console.debug('WebSocket closed:', e.code);
    if (e.code !== 1000) {
      console.debug('Reconnecting WebSocket...');
      setTimeout(() => {
        connectJobSocket({ type, id }, onMessage);
      }, 1000);
    }
  };

  ws.onerror = (err) => {
    console.debug('WebSocket error:', err);
    ws.close();
  };
}

export function closeWebSocket() {
  if (ws) {
    ws.close();
  }
}

Version AWX 23.9.0 with Operator 2.12.2 and PostgreSQL 13 is working without any issue. It does not matter do you use NGINX Proxy Manager, ingress or without ingress or do you modify the coreDNS it just works. So I decided to not upgrade to the newest AWX operator, AWX and PostgreSQL.

AWX version

24.1.0

Select the relevant components

  • UI
  • UI (tech preview)
  • API
  • Docs
  • Collection
  • CLI
  • Other

Installation method

kubernetes

Modifications

yes

Ansible version

2.14.3

Operating system

Debian 12

Web browser

Chrome

Steps to reproduce

  • install newest k3s using this command:
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable traefik,servicelb" K3S_KUBECONFIG_MODE="644" sh -
  • Do not modify coreDNS. Do not install ingress.

  • Install HELM

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash 
  • Install the newest AWX version with newest AWX operator from the scratch using the below ansible playbook:
---
- name: Install AWX
  hosts: localhost
  become: yes
  vars:
    awx_namespace: awx
    project_directory: /var/lib/awx/projects
    storage_size: 2Gi

  tasks:
    - name: Download Kustomize with curl
      ansible.builtin.shell:
        cmd: curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
        creates: /usr/local/bin/kustomize

    - name: Move Kustomize to the /usr/local/bin directory
      ansible.builtin.shell:
        cmd: mv kustomize /usr/local/bin
      args:
        creates: /usr/local/bin/kustomize

    - name: Ensure namespace {{ awx_namespace }} exists
      ansible.builtin.shell:
        cmd: kubectl create namespace {{ awx_namespace }} --dry-run=client -o yaml | kubectl apply -f -

    - name: Generate AWX resource file
      ansible.builtin.copy:
        dest: "./awx.yaml"
        content: |
          ---
          apiVersion: awx.ansible.com/v1beta1
          kind: AWX
          metadata:
            name: awx
          spec:
            service_type: nodeport
            nodeport_port: 30060
            projects_persistence: true
            projects_existing_claim: awx-projects-claim          

    - name: Fetch latest release tag of AWX Operator
      ansible.builtin.shell:
        cmd: curl -s https://api.github.com/repos/ansible/awx-operator/releases/latest | grep tag_name | cut -d '"' -f 4
      register: release_tag
      changed_when: false

    - name: Generate PV and PVC resource files
      ansible.builtin.copy:
        dest: "{{ item.dest }}"
        content: "{{ item.content }}"
      loop:
        - dest: "./pv.yml"
          content: |
            ---
            apiVersion: v1
            kind: PersistentVolume
            metadata:
              name: awx-projects-volume
            spec:
              accessModes:
                - ReadWriteOnce
              persistentVolumeReclaimPolicy: Retain
              capacity:
                storage: {{ storage_size }}
              storageClassName: awx-projects-volume
              hostPath:
                path: {{ project_directory }}            
        - dest: "./pvc.yml"
          content: |
            ---
            apiVersion: v1
            kind: PersistentVolumeClaim
            metadata:
              name: awx-projects-claim
            spec:
              accessModes:
                - ReadWriteOnce
              volumeMode: Filesystem
              resources:
                requests:
                  storage: {{ storage_size }}
              storageClassName: awx-projects-volume            

    - name: Create kustomization.yaml
      ansible.builtin.copy:
        dest: "./kustomization.yaml"
        content: |
          ---
          apiVersion: kustomize.config.k8s.io/v1beta1
          kind: Kustomization
          resources:
            - github.com/ansible/awx-operator/config/default?ref={{ release_tag.stdout }}
            - pv.yml
            - pvc.yml
            - awx.yaml
          images:
            - name: quay.io/ansible/awx-operator
              newTag: {{ release_tag.stdout }}
          namespace: {{ awx_namespace }}          

    - name: Apply Kustomize configuration
      ansible.builtin.shell:
        cmd: kustomize build . | kubectl apply -f -

Perform a check using http://IP_ADDRESS:30060

Expected results

Working standard output in jobs view.

Actual results

standard output in jobs does not work and it does not scroll the results of running ansible playbook. Websocket returns the same error like it is presented in the URL I provided.

Additional information

connectJobSocket.js:17 Uncaught DOMException: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state.
    at WebSocket.onopen (https://awx.sysadmin.homes/static/js/main.13ab8549.js:2:3449626)
onopen	@	connectJobSocket.js:17
react-dom.production.min.js:54 Uncaught Error: Minified React error #188; visit https://reactjs.org/docs/error-decoder.html?invariant=188 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at react-dom.production.min.js:54:67
    at Ze (react-dom.production.min.js:55:217)
    at t.findDOMNode (react-dom.production.min.js:295:212)
    at t.value (CellMeasurer.js:102:33)
    at CellMeasurer.js:48:41    

See logs from awx-web pod

10.42.0.1 - - [05/Apr/2024:15:53:26 +0000] "GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 52 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 21|app: 0|req: 204/1288] 10.42.0.1 () {70 vars in 1359 bytes} [Fri Apr  5 15:53:25 2024] GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 52 bytes in 376 msecs (HTTP/1.1 200) 15 headers in 600 bytes (1 switches on core 0)
127.0.0.1:54360 - - [05/Apr/2024:15:45:19] "WSCONNECTING /websocket/" - -
127.0.0.1:54360 - - [05/Apr/2024:15:45:19] "WSCONNECT /websocket/" - -
127.0.0.1:54308 - - [05/Apr/2024:15:45:28] "WSDISCONNECT /websocket/" - -
127.0.0.1:54360 - - [05/Apr/2024:15:45:28] "WSDISCONNECT /websocket/" - -
127.0.0.1:49622 - - [05/Apr/2024:15:45:28] "WSCONNECTING /websocket/" - -
127.0.0.1:49622 - - [05/Apr/2024:15:45:28] "WSCONNECT /websocket/" - -
127.0.0.1:49622 - - [05/Apr/2024:15:45:30] "WSDISCONNECT /websocket/" - -
127.0.0.1:49636 - - [05/Apr/2024:15:45:30] "WSCONNECTING /websocket/" - -
127.0.0.1:49636 - - [05/Apr/2024:15:45:30] "WSCONNECT /websocket/" - -
127.0.0.1:49638 - - [05/Apr/2024:15:45:31] "WSCONNECTING /websocket/" - -
127.0.0.1:49638 - - [05/Apr/2024:15:45:31] "WSDISCONNECT /websocket/" - -
127.0.0.1:49652 - - [05/Apr/2024:15:45:32] "WSCONNECTING /websocket/" - -
127.0.0.1:49652 - - [05/Apr/2024:15:45:32] "WSCONNECT /websocket/" - -
127.0.0.1:49664 - - [05/Apr/2024:15:45:32] "WSCONNECTING /websocket/" - -
127.0.0.1:49664 - - [05/Apr/2024:15:45:32] "WSCONNECT /websocket/" - -
127.0.0.1:49664 - - [05/Apr/2024:15:45:36] "WSDISCONNECT /websocket/" - -
127.0.0.1:49202 - - [05/Apr/2024:15:45:37] "WSCONNECTING /websocket/" - -
127.0.0.1:49202 - - [05/Apr/2024:15:45:37] "WSCONNECT /websocket/" - -
127.0.0.1:49208 - - [05/Apr/2024:15:45:37] "WSCONNECTING /websocket/" - -
127.0.0.1:49208 - - [05/Apr/2024:15:45:37] "WSCONNECT /websocket/" - -
127.0.0.1:49202 - - [05/Apr/2024:15:45:37] "WSDISCONNECT /websocket/" - -
127.0.0.1:49208 - - [05/Apr/2024:15:45:37] "WSDISCONNECT /websocket/" - -
127.0.0.1:49216 - - [05/Apr/2024:15:45:38] "WSCONNECTING /websocket/" - -
127.0.0.1:49216 - - [05/Apr/2024:15:45:38] "WSCONNECT /websocket/" - -
127.0.0.1:49232 - - [05/Apr/2024:15:45:38] "WSCONNECTING /websocket/" - -
127.0.0.1:49232 - - [05/Apr/2024:15:45:38] "WSCONNECT /websocket/" - -
127.0.0.1:49636 - - [05/Apr/2024:15:46:15] "WSDISCONNECT /websocket/" - -
127.0.0.1:49232 - - [05/Apr/2024:15:46:15] "WSDISCONNECT /websocket/" - -
127.0.0.1:57920 - - [05/Apr/2024:15:46:17] "WSCONNECTING /websocket/" - -
127.0.0.1:57920 - - [05/Apr/2024:15:46:17] "WSCONNECT /websocket/" - -
127.0.0.1:57920 - - [05/Apr/2024:15:46:20] "WSDISCONNECT /websocket/" - -
127.0.0.1:57934 - - [05/Apr/2024:15:46:20] "WSCONNECTING /websocket/" - -
127.0.0.1:57934 - - [05/Apr/2024:15:46:20] "WSCONNECT /websocket/" - -
127.0.0.1:57934 - - [05/Apr/2024:15:48:20] "WSDISCONNECT /websocket/" - -
127.0.0.1:39102 - - [05/Apr/2024:15:48:20] "WSCONNECTING /websocket/" - -
127.0.0.1:39102 - - [05/Apr/2024:15:48:20] "WSCONNECT /websocket/" - -
127.0.0.1:39102 - - [05/Apr/2024:15:48:48] "WSDISCONNECT /websocket/" - -
127.0.0.1:47586 - - [05/Apr/2024:15:48:48] "WSCONNECTING /websocket/" - -
127.0.0.1:47586 - - [05/Apr/2024:15:48:48] "WSCONNECT /websocket/" - -
127.0.0.1:47586 - - [05/Apr/2024:15:48:55] "WSDISCONNECT /websocket/" - -
127.0.0.1:47592 - - [05/Apr/2024:15:48:55] "WSCONNECTING /websocket/" - -
127.0.0.1:47592 - - [05/Apr/2024:15:48:55] "WSCONNECT /websocket/" - -
127.0.0.1:45654 - - [05/Apr/2024:15:48:56] "WSCONNECTING /websocket/" - -
127.0.0.1:45654 - - [05/Apr/2024:15:48:56] "WSCONNECT /websocket/" - -
127.0.0.1:45666 - - [05/Apr/2024:15:48:56] "WSCONNECTING /websocket/" - -
127.0.0.1:45654 - - [05/Apr/2024:15:48:56] "WSDISCONNECT /websocket/" - -
127.0.0.1:45666 - - [05/Apr/2024:15:48:56] "WSDISCONNECT /websocket/" - -
127.0.0.1:45672 - - [05/Apr/2024:15:48:57] "WSCONNECTING /websocket/" - -
127.0.0.1:45672 - - [05/Apr/2024:15:48:57] "WSCONNECT /websocket/" - -
127.0.0.1:45686 - - [05/Apr/2024:15:48:57] "WSCONNECTING /websocket/" - -
127.0.0.1:45686 - - [05/Apr/2024:15:48:57] "WSCONNECT /websocket/" - -
127.0.0.1:47592 - - [05/Apr/2024:15:51:04] "WSDISCONNECT /websocket/" - -
127.0.0.1:45686 - - [05/Apr/2024:15:51:04] "WSDISCONNECT /websocket/" - -
127.0.0.1:47042 - - [05/Apr/2024:15:51:05] "WSCONNECTING /websocket/" - -
127.0.0.1:47042 - - [05/Apr/2024:15:51:05] "WSCONNECT /websocket/" - -
127.0.0.1:47042 - - [05/Apr/2024:15:51:07] "WSDISCONNECT /websocket/" - -
127.0.0.1:36972 - - [05/Apr/2024:15:51:07] "WSCONNECTING /websocket/" - -
127.0.0.1:36972 - - [05/Apr/2024:15:51:07] "WSCONNECT /websocket/" - -
127.0.0.1:36972 - - [05/Apr/2024:15:51:16] "WSDISCONNECT /websocket/" - -
127.0.0.1:50854 - - [05/Apr/2024:15:51:16] "WSCONNECTING /websocket/" - -
127.0.0.1:50854 - - [05/Apr/2024:15:51:16] "WSCONNECT /websocket/" - -
127.0.0.1:50854 - - [05/Apr/2024:15:51:21] "WSDISCONNECT /websocket/" - -
127.0.0.1:50860 - - [05/Apr/2024:15:51:22] "WSCONNECTING /websocket/" - -
127.0.0.1:50860 - - [05/Apr/2024:15:51:22] "WSCONNECT /websocket/" - -
127.0.0.1:50862 - - [05/Apr/2024:15:51:23] "WSCONNECTING /websocket/" - -
127.0.0.1:50862 - - [05/Apr/2024:15:51:23] "WSCONNECT /websocket/" - -
127.0.0.1:50864 - - [05/Apr/2024:15:51:23] "WSCONNECTING /websocket/" - -
127.0.0.1:50864 - - [05/Apr/2024:15:51:23] "WSCONNECT /websocket/" - -
127.0.0.1:50862 - - [05/Apr/2024:15:51:23] "WSDISCONNECT /websocket/" - -
127.0.0.1:50864 - - [05/Apr/2024:15:51:23] "WSDISCONNECT /websocket/" - -
127.0.0.1:50872 - - [05/Apr/2024:15:51:24] "WSCONNECTING /websocket/" - -
127.0.0.1:50872 - - [05/Apr/2024:15:51:24] "WSCONNECT /websocket/" - -
127.0.0.1:50878 - - [05/Apr/2024:15:51:24] "WSCONNECTING /websocket/" - -
127.0.0.1:50878 - - [05/Apr/2024:15:51:24] "WSCONNECT /websocket/" - -
127.0.0.1:50860 - - [05/Apr/2024:15:51:35] "WSDISCONNECT /websocket/" - -
127.0.0.1:50878 - - [05/Apr/2024:15:51:35] "WSDISCONNECT /websocket/" - -
127.0.0.1:38166 - - [05/Apr/2024:15:51:35] "WSCONNECTING /websocket/" - -
127.0.0.1:38166 - - [05/Apr/2024:15:51:35] "WSCONNECT /websocket/" - -
127.0.0.1:38166 - - [05/Apr/2024:15:51:38] "WSDISCONNECT /websocket/" - -
127.0.0.1:52776 - - [05/Apr/2024:15:51:38] "WSCONNECTING /websocket/" - -
127.0.0.1:52776 - - [05/Apr/2024:15:51:38] "WSCONNECT /websocket/" - -
127.0.0.1:52776 - - [05/Apr/2024:15:52:45] "WSDISCONNECT /websocket/" - -
127.0.0.1:54772 - - [05/Apr/2024:15:52:45] "WSCONNECTING /websocket/" - -
127.0.0.1:54772 - - [05/Apr/2024:15:52:45] "WSCONNECT /websocket/" - -
127.0.0.1:54772 - - [05/Apr/2024:15:52:51] "WSDISCONNECT /websocket/" - -
127.0.0.1:43496 - - [05/Apr/2024:15:52:52] "WSCONNECTING /websocket/" - -
127.0.0.1:43496 - - [05/Apr/2024:15:52:52] "WSCONNECT /websocket/" - -
127.0.0.1:43502 - - [05/Apr/2024:15:52:53] "WSCONNECTING /websocket/" - -
127.0.0.1:43502 - - [05/Apr/2024:15:52:53] "WSCONNECT /websocket/" - -
127.0.0.1:43518 - - [05/Apr/2024:15:52:53] "WSCONNECTING /websocket/" - -
127.0.0.1:43518 - - [05/Apr/2024:15:52:53] "WSCONNECT /websocket/" - -
127.0.0.1:43502 - - [05/Apr/2024:15:52:53] "WSDISCONNECT /websocket/" - -
127.0.0.1:43518 - - [05/Apr/2024:15:52:53] "WSDISCONNECT /websocket/" - -
127.0.0.1:43522 - - [05/Apr/2024:15:52:54] "WSCONNECTING /websocket/" - -
127.0.0.1:43522 - - [05/Apr/2024:15:52:54] "WSCONNECT /websocket/" - -
127.0.0.1:43538 - - [05/Apr/2024:15:52:54] "WSCONNECTING /websocket/" - -
127.0.0.1:43538 - - [05/Apr/2024:15:52:54] "WSCONNECT /websocket/" - -
127.0.0.1:43538 - - [05/Apr/2024:15:53:19] "WSDISCONNECT /websocket/" - -
127.0.0.1:36982 - - [05/Apr/2024:15:53:21] "WSCONNECTING /websocket/" - -
127.0.0.1:36982 - - [05/Apr/2024:15:53:21] "WSCONNECT /websocket/" - -
127.0.0.1:36992 - - [05/Apr/2024:15:53:21] "WSCONNECTING /websocket/" - -
127.0.0.1:36992 - - [05/Apr/2024:15:53:21] "WSCONNECT /websocket/" - -
127.0.0.1:43496 - - [05/Apr/2024:15:53:22] "WSDISCONNECT /websocket/" - -
127.0.0.1:36992 - - [05/Apr/2024:15:53:22] "WSDISCONNECT /websocket/" - -
127.0.0.1:37002 - - [05/Apr/2024:15:53:22] "WSCONNECTING /websocket/" - -
127.0.0.1:37002 - - [05/Apr/2024:15:53:22] "WSCONNECT /websocket/" - -
127.0.0.1:37002 - - [05/Apr/2024:15:53:24] "WSDISCONNECT /websocket/" - -
127.0.0.1:37008 - - [05/Apr/2024:15:53:24] "WSCONNECTING /websocket/" - -
127.0.0.1:37008 - - [05/Apr/2024:15:53:24] "WSCONNECT /websocket/" - -
127.0.0.1:37022 - - [05/Apr/2024:15:53:25] "WSCONNECTING /websocket/" - -
127.0.0.1:37022 - - [05/Apr/2024:15:53:25] "WSCONNECT /websocket/" - -
127.0.0.1:37022 - - [05/Apr/2024:15:53:25] "WSDISCONNECT /websocket/" - -
10.42.0.1 - - [05/Apr/2024:15:53:29 +0000] "GET /websocket/ HTTP/1.1" 101 207 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:53:30 +0000] "GET /websocket/ HTTP/1.1" 101 29 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:53:30 +0000] "GET /websocket/ HTTP/1.1" 101 29 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:53:31 +0000] "GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 4049 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 218/1289] 10.42.0.1 () {70 vars in 1359 bytes} [Fri Apr  5 15:53:30 2024] GET /api/v2/jobs/39/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 4049 bytes in 731 msecs (HTTP/1.1 200) 15 headers in 602 bytes (1 switches on core 0)

logs from google chrome browser console in developer tools

Failed to load resource: the server responded with a status of 404 ()
/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200:1   
        
Failed to load resource: the server responded with a status of 404 ()
/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200:1 
        
Failed to load resource: the server responded with a status of 404 ()
requestCommon.ts:20 
        
GET https://awx.sysadmin.homes/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200 404 (Not Found)
He @ requestCommon.ts:20
(anonimowa) @ useGet.tsx:55
t @ index.mjs:653
(anonimowa) @ index.mjs:671
(anonimowa) @ index.mjs:613
(anonimowa) @ index.mjs:245
(anonimowa) @ index.mjs:398
t.isPaused.A.revalidateOnFocus.A.revalidateOnReconnect.t.onErrorRetry.retryCount @ index.mjs:329
Pokaż jeszcze 6 ramek
Pokaż mniej
requestCommon.ts:20 

GET https://awx.sysadmin.homes/api/v2/workflow_jobs/62/workflow_nodes/?page=1&page_size=200 404 (Not Found)

See logs from k3s cluster pod for awx-web

10.42.0.1 - - [05/Apr/2024:14:59:39 +0000] "GET /websocket/ HTTP/1.1" 404 3878 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:14:59:50 +0000] "GET /websocket/ HTTP/1.1" 101 322 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:14:59:50 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 141/842] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 14:59:50 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 194 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:50 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 24|app: 0|req: 191/843] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 14:59:50 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 410 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:51 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 27264 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 260/844] 10.42.0.1 () {70 vars in 1369 bytes} [Fri Apr  5 14:59:50 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 27264 bytes in 585 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:56 +0000] "GET /api/v2/jobs/9/relaunch/ HTTP/1.1" 200 68 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 261/845] 10.42.0.1 () {70 vars in 1254 bytes} [Fri Apr  5 14:59:56 2024] GET /api/v2/jobs/9/relaunch/ => generated 68 bytes in 341 msecs (HTTP/1.1 200) 14 headers in 583 bytes (1 switches on core 0)
2024-04-05 14:59:57,986 INFO     [e7d81c60ac1a4b08af2363573ae59f3a] awx.analytics.job_lifecycle job-11 created
10.42.0.1 - - [05/Apr/2024:14:59:58 +0000] "POST /api/v2/jobs/9/relaunch/ HTTP/1.1" 201 2937 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 262/846] 10.42.0.1 () {76 vars in 1374 bytes} [Fri Apr  5 14:59:56 2024] POST /api/v2/jobs/9/relaunch/ => generated 2937 bytes in 2210 msecs (HTTP/1.1 201) 15 headers in 618 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:58 +0000] "GET /websocket/ HTTP/1.1" 101 350 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:14:59:59 +0000] "GET /api/v2/unified_jobs/?id__in=11&not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 2884 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 142/847] 10.42.0.1 () {70 vars in 1389 bytes} [Fri Apr  5 14:59:58 2024] GET /api/v2/unified_jobs/?id__in=11&not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 2884 bytes in 577 msecs (HTTP/1.1 200) 14 headers in 579 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:59 +0000] "GET /api/v2/unified_jobs/?id=11 HTTP/1.1" 200 2884 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 263/848] 10.42.0.1 () {70 vars in 1259 bytes} [Fri Apr  5 14:59:58 2024] GET /api/v2/unified_jobs/?id=11 => generated 2884 bytes in 688 msecs (HTTP/1.1 200) 14 headers in 579 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:14:59:59 +0000] "GET /api/v2/jobs/11/ HTTP/1.1" 200 3026 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 264/849] 10.42.0.1 () {70 vars in 1238 bytes} [Fri Apr  5 14:59:59 2024] GET /api/v2/jobs/11/ => generated 3026 bytes in 331 msecs (HTTP/1.1 200) 14 headers in 587 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "OPTIONS /api/v2/jobs/11/job_events/ HTTP/1.1" 200 12315 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 265/850] 10.42.0.1 () {72 vars in 1305 bytes} [Fri Apr  5 14:59:59 2024] OPTIONS /api/v2/jobs/11/job_events/ => generated 12315 bytes in 356 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
[pid: 21|app: 0|req: 139/851] 10.42.0.1 () {70 vars in 1358 bytes} [Fri Apr  5 15:00:00 2024] GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 52 bytes in 351 msecs (HTTP/1.1 200) 15 headers in 600 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 52 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 266/852] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 15:00:00 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 393 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:00 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 22|app: 0|req: 115/853] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 15:00:00 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 646 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:01 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 32260 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 143/854] 10.42.0.1 () {70 vars in 1369 bytes} [Fri Apr  5 15:00:00 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 32260 bytes in 1164 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:30 +0000] "GET /websocket/ HTTP/1.1" 101 251 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 267/855] 10.42.0.1 () {70 vars in 1260 bytes} [Fri Apr  5 15:00:30 2024] GET /api/v2/project_updates/12/ => generated 5601 bytes in 567 msecs (HTTP/1.1 200) 14 headers in 587 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:31 +0000] "GET /api/v2/project_updates/12/ HTTP/1.1" 200 5601 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:31 +0000] "OPTIONS /api/v2/project_updates/12/events/ HTTP/1.1" 200 12382 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 20|app: 0|req: 144/856] 10.42.0.1 () {72 vars in 1319 bytes} [Fri Apr  5 15:00:31 2024] OPTIONS /api/v2/project_updates/12/events/ => generated 12382 bytes in 264 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:31 +0000] "GET /api/v2/project_updates/12/events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 32639 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 22|app: 0|req: 116/857] 10.42.0.1 () {70 vars in 1373 bytes} [Fri Apr  5 15:00:31 2024] GET /api/v2/project_updates/12/events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 32639 bytes in 342 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:40 +0000] "GET /websocket/ HTTP/1.1" 101 195 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:40 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 21|app: 0|req: 140/858] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 15:00:40 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 299 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
[pid: 22|app: 0|req: 117/859] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 15:00:40 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 546 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:40 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:41 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 32590 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 268/860] 10.42.0.1 () {70 vars in 1368 bytes} [Fri Apr  5 15:00:40 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 32590 bytes in 1479 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:42 +0000] "GET /websocket/ HTTP/1.1" 101 237 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:00:43 +0000] "GET /api/v2/jobs/11/ HTTP/1.1" 200 6618 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 24|app: 0|req: 192/861] 10.42.0.1 () {70 vars in 1238 bytes} [Fri Apr  5 15:00:42 2024] GET /api/v2/jobs/11/ => generated 6618 bytes in 732 msecs (HTTP/1.1 200) 14 headers in 587 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:43 +0000] "OPTIONS /api/v2/jobs/11/job_events/ HTTP/1.1" 200 12315 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 269/862] 10.42.0.1 () {72 vars in 1305 bytes} [Fri Apr  5 15:00:43 2024] OPTIONS /api/v2/jobs/11/job_events/ => generated 12315 bytes in 232 msecs (HTTP/1.1 200) 15 headers in 603 bytes (1 switches on core 0)
[pid: 23|app: 0|req: 270/863] 10.42.0.1 () {70 vars in 1359 bytes} [Fri Apr  5 15:00:43 2024] GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 => generated 179025 bytes in 167 msecs (HTTP/1.1 200) 15 headers in 604 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:00:43 +0000] "GET /api/v2/jobs/11/job_events/?not__stdout=&order_by=counter&page=1&page_size=50 HTTP/1.1" 200 179025 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:01:28 +0000] "GET /websocket/ HTTP/1.1" 101 235 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:01:30 +0000] "GET /websocket/ HTTP/1.1" 101 223 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
10.42.0.1 - - [05/Apr/2024:15:01:30 +0000] "OPTIONS /api/v2/unified_jobs/ HTTP/1.1" 200 11658 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 24|app: 0|req: 193/864] 10.42.0.1 () {72 vars in 1293 bytes} [Fri Apr  5 15:01:30 2024] OPTIONS /api/v2/unified_jobs/ => generated 11658 bytes in 408 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:01:30 +0000] "OPTIONS /api/v2/inventory_sources/ HTTP/1.1" 200 24146 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 23|app: 0|req: 271/865] 10.42.0.1 () {72 vars in 1303 bytes} [Fri Apr  5 15:01:30 2024] OPTIONS /api/v2/inventory_sources/ => generated 24146 bytes in 569 msecs (HTTP/1.1 200) 14 headers in 586 bytes (1 switches on core 0)
10.42.0.1 - - [05/Apr/2024:15:01:31 +0000] "GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 HTTP/1.1" 200 32615 "https://awx.sysadmin.homes/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "10.10.0.114"
[pid: 21|app: 0|req: 141/866] 10.42.0.1 () {70 vars in 1369 bytes} [Fri Apr  5 15:01:30 2024] GET /api/v2/unified_jobs/?not__launch_type=sync&order_by=-finished&page=1&page_size=20 => generated 32615 bytes in 775 msecs (HTTP/1.1 200) 14 headers in 580 bytes (1 switches on core 0)

The log snippets indicate interactions with various API endpoints and websocket connections under the /websocket/ path. Here's an analysis of the relevant parts and possible issues:

Websocket Connection Codes (101): The HTTP status code 101 in your logs suggests that websocket connections are being initiated and possibly established ("Switching Protocols"). However, there's an initial 404 error for the websocket endpoint, which could indicate a misconfiguration at some point in time.

Initial 404 Error: The first attempt to access /websocket/ resulted in a 404 (Not Found) error. This could imply that either the websocket service wasn't ready at that moment or there was a misconfiguration in the URL or routing.

Successful Connections: Subsequent attempts to connect to the websocket endpoint returned a 101 status, which means the server accepted the request to switch protocols from HTTP to WebSockets.

Other API Requests: The log shows successful HTTP requests (status 200) to various API endpoints, indicating that the regular HTTP API seems to be functioning well.

@kurokobo
Copy link
Contributor

kurokobo commented Apr 6, 2024

Perhaps dup of: #15038
It's already addressed and the fix will be shipped with the next versioned release.

@rexberg
Copy link

rexberg commented Apr 11, 2024

This was addressed in the 24.2.0 release.

@bbourchanin
Copy link

bbourchanin commented Apr 11, 2024

Still have "Minified React error #188;" on 24.2.0.

When i append url with "?lang=fr" or "?lang=en", or "?no_stdout=" ... it works

Edit: Ok i have resolved my pb by deleting on ansible.cfg
#display_ok_hosts=false
#display_skipped_hosts=false

@fosterseth
Copy link
Member

@sysadmin-info are you still having this issue? if not, we can go ahead and close this ticket. Thanks!

@sysadmin-info
Copy link
Author

sysadmin-info commented Apr 14, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants