Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User Input Validation #2466

Open
wants to merge 16 commits into
base: pub/new_architecture
Choose a base branch
from

Conversation

Grace-Chang2
Copy link

Format and logic validation on all user input files.

@Omnia-svc
Copy link
Collaborator

Can one of the admins verify this patch?

- name: Find input directories
ansible.builtin.find:
paths: "{{ omnia_base_dir }}"
file_type: directory
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this need to be from NFS share

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

# See the License for the specific language governing permissions and
# limitations under the License.
---

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include input project directory utility

Copy link

@youngjae-hur7 youngjae-hur7 Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, now as default it points to /opt/omnia/input/project_default/ and grabs all json and yml files

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

project_default this can be different. Based on /opt/omnia/.data/default.yml file. Better we import utility utils/include_input_dir.yml

---
- name: Include variables
ansible.builtin.include_vars:
file: vars/vars.yml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use main.yml instead of vars.yml. this task not required

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.


# dict to hold the file names. If any file's name changes just change it here.
files = {
"accelerator_config": "accelerator_config.yml",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

acellerator_config , network_config is not required

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed accelerator_config and rhsm_config, but I left network_config just in case an user wants to validate network_config by providing its tag. It's not necessarily running everytime unless --tags network_config is provided

"passwordless_ssh_config": "passwordless_ssh_config.yml",
"provision_config_credentials": "provision_config_credentials.yml",
"provision_config": "provision_config.yml",
"rhsm_config": "rhsm_config.yml",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rhsm_config is not required

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

from datetime import datetime
import os

omnia_log = '/var/log/omnia'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/opt/omnia/log/core/playbooks/input_validator/

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

omnia_log = '/var/log/omnia'

module_log_dir = {
"omnia": omnia_log + "/_"+ datetime.now().strftime('_%d-%m-%Y.log')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to input_validator_log

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

@priti-parate
Copy link
Collaborator

priti-parate commented Feb 21, 2025

all functions should have doc string

"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Configuration Schema",
"type": "object",
"properties": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file not required

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean the network_config schema is not needed?

"type": "object",
"properties": {
"Networks": {
"type": "array",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are changes in network_spec.yml

"pattern": "^[a-zA-Z0-9._-]+$"
},
"domain_name": {
"type": "string",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

provision_config.yml will not be having iso_file_path, node_name and switch details

"items": {
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9_-]+$": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

server spec has metric value added now

},
"required": [
"nfs_client_params",
"beegfs_mgmt_server",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beegfs details are not mandatory always

],
"if": {
"properties": {
"docker_username": { "type": "string", "minLength": 1 }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file will be having changes

"required": [
"ansible_config_file_path",
"mariadb_password",
"slurm_installation_type",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing some variables in omnia_config.yml

"properties": {
"gpgkey": {
"type": "string",
"format": "uri"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repo_store_path is removed now. ubuntu and rocky urls not present

@youngjae-hur7
Copy link

youngjae-hur7 commented Feb 25, 2025

image

Added HA config input validation by providing --extra-vars "files=high_availability_config.yml"

@youngjae-hur7
Copy link

image

Successfully verify the inputs by providing '--tags high_availability'

ansible.builtin.find:
paths: "/opt/omnia/input"
file_type: directory
patterns: 'project_default'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For nfs shared input dir utils/include_input_dir.yml can be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants