Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DVX-182: Run integration tests in parallel #227

Merged
merged 7 commits into from
Jan 24, 2024
Merged

DVX-182: Run integration tests in parallel #227

merged 7 commits into from
Jan 24, 2024

Conversation

Aryamanz29
Copy link
Member

@Aryamanz29 Aryamanz29 commented Jan 22, 2024

{
  "job1": {
    "test_files": [
      "tests/integration/test_index_search.py"
    ],
    "total_time_seconds": 2511.4452063839863,
    "total_time_minutes": 41.85742010639977
  },
  "job2": {
    "test_files": [
      "tests/integration/custom_metadata_test.py"
    ],
    "total_time_seconds": 1024.8998935410011,
    "total_time_minutes": 17.08166489235002
  },
  "job3": {
    "test_files": [
      "tests/integration/test_client.py"
    ],
    "total_time_seconds": 994.4693159199942,
    "total_time_minutes": 16.57448859866657
  },
  "job4": {
    "test_files": [
      "tests/integration/glossary_test.py"
    ],
    "total_time_seconds": 558.3589443770006,
    "total_time_minutes": 9.305982406283343
  },
  "job5": {
    "test_files": [
      "tests/integration/lineage_test.py"
    ],
    "total_time_seconds": 502.56237604199987,
    "total_time_minutes": 8.376039600699999
  },
  "job6": {
    "test_files": [
      "tests/integration/test_sql_assets.py"
    ],
    "total_time_seconds": 367.6249272069981,
    "total_time_minutes": 6.1270821201166354
  },
  "job7": {
    "test_files": [
      "tests/integration/s3_asset_test.py"
    ],
    "total_time_seconds": 308.3180949139992,
    "total_time_minutes": 5.13863491523332
  },
  "job8": {
    "test_files": [
      "tests/integration/persona_test.py"
    ],
    "total_time_seconds": 261.82817225100007,
    "total_time_minutes": 4.363802870850001
  },
  "job9": {
    "test_files": [
      "tests/integration/adls_asset_test.py"
    ],
    "total_time_seconds": 253.965848124,
    "total_time_minutes": 4.2327641354
  },
  "job10": {
    "test_files": [
      "tests/integration/data_studio_asset_test.py"
    ],
    "total_time_seconds": 238.38720117000048,
    "total_time_minutes": 3.973120019500008
  },
  "job11": {
    "test_files": [
      "tests/integration/gcs_asset_test.py"
    ],
    "total_time_seconds": 224.91367779100074,
    "total_time_minutes": 3.748561296516679
  },
  "job12": {
    "test_files": [
      "tests/integration/preset_asset_test.py"
    ],
    "total_time_seconds": 205.24932687400133,
    "total_time_minutes": 3.420822114566689
  },
  "job13": {
    "test_files": [
      "tests/integration/api_asset_test.py"
    ],
    "total_time_seconds": 195.37524212400012,
    "total_time_minutes": 3.2562540354000022
  },
  "job14": {
    "test_files": [
      "tests/integration/atlan_tag_test.py"
    ],
    "total_time_seconds": 166.2517590819998,
    "total_time_minutes": 2.7708626513666634
  },
  "job15": {
    "test_files": [
      "tests/integration/file_test.py"
    ],
    "total_time_seconds": 124.11478079099993,
    "total_time_minutes": 2.0685796798499987
  },
  "job16": {
    "test_files": [
      "tests/integration/purpose_test.py"
    ],
    "total_time_seconds": 99.80052549999982,
    "total_time_minutes": 1.6633420916666637
  },
  "job17": {
    "test_files": [
      "tests/integration/requests_test.py"
    ],
    "total_time_seconds": 79.53525762599975,
    "total_time_minutes": 1.3255876270999958
  },
  "job18": {
    "test_files": [
      "tests/integration/admin_test.py"
    ],
    "total_time_seconds": 49.90503983799994,
    "total_time_minutes": 0.8317506639666656
  },
  "job19": {
    "test_files": [
      "tests/integration/connection_test.py"
    ],
    "total_time_seconds": 5.808168500999614,
    "total_time_minutes": 0.09680280834999357
  },
  "job20": {
    "test_files": [
      "tests/integration/query_parser_test.py",
      "tests/integration/custom_package_test.py"
    ],
    "total_time_seconds": 1.4749142510004276,
    "total_time_minutes": 0.02458190418334046
  }
}

@Aryamanz29 Aryamanz29 force-pushed the DVX-182 branch 5 times, most recently from bdd787d to 2faa4e7 Compare January 23, 2024 09:30
@Aryamanz29 Aryamanz29 force-pushed the DVX-182 branch 4 times, most recently from be8cb7a to 6553e9a Compare January 23, 2024 11:55
@Aryamanz29 Aryamanz29 force-pushed the DVX-182 branch 6 times, most recently from b01c963 to 4d17788 Compare January 23, 2024 13:01
@Aryamanz29
Copy link
Member Author

Hi @cmgrote,

I have noticed that after switching to the devx5 tenant, our CI builds are taking too long, approximately 2 hours. For example:

Similar results were observed when running it locally:

Screenshot

Previously, the build time was under 30 minutes:

@Aryamanz29
Copy link
Member Author

Aryamanz29 commented Jan 23, 2024

Now, the CI build time has been significantly reduced from 30 minutes to 5 minutes, making the CI build finish roughly 6 times faster. 🚀 Check it out here : https://github.com/atlanhq/atlan-python/actions/runs/7626702415?pr=227.

@Aryamanz29 Aryamanz29 marked this pull request as ready for review January 23, 2024 14:12
@Aryamanz29 Aryamanz29 force-pushed the DVX-182 branch 5 times, most recently from c0c184e to 2fe7c5c Compare January 23, 2024 18:16
@atlanhq atlanhq deleted a comment from github-advanced-security bot Jan 23, 2024
@Aryamanz29 Aryamanz29 force-pushed the DVX-182 branch 4 times, most recently from 268c18a to 867d92a Compare January 23, 2024 19:03
Copy link
Collaborator

@ErnestoLoma ErnestoLoma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@ErnestoLoma ErnestoLoma merged commit 98b5627 into main Jan 24, 2024
24 checks passed
@ErnestoLoma ErnestoLoma deleted the DVX-182 branch January 24, 2024 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants