Skip to content

Commit 213f5b0

Browse files
trumanttrumant
committed
feat: improve project security posture
This PR introduces the following security changes to the project * Dependabot configuration for daily scans/updates from the Python and GitHub Actions ecosystems * Updates to the project CI GHA workflow to improve security * Upgrade reusable actions to latest release versions * Use poetry in CI to ensure consistent dependency version usage between dev and CI * Add execution of bandit and gitleaks checks * pre-commit tooling for contributors now runs `bandit`, `snyk`, `zizmor` and `gitleaks` * CI, codebase and dependency changes made to reflect issues found in scans * `security-insights.yml` added to communicate security changes to project made above Here is the `pre-commit` output from this commit: ``` ruff.....................................................................Passed ruff-format..............................................................Passed bandit...................................................................Passed Detect hardcoded secrets.................................................Passed Snyk Test................................................................Passed - hook id: snyk-test - duration: 3.61s [pre-commit-snyk] 2025-07-22 09:00:50 level=info Snyk path: /opt/homebrew/bin/snyk [pre-commit-snyk] 2025-07-22 09:00:50 level=info Snyk version: 1.1298.0 [pre-commit-snyk] 2025-07-22 09:00:50 level=info Snyk arguments: test --file=poetry.lock --package-manager=pip --fail-on=upgradable Testing /Users/travistruman/dev/uqlm... Tested 103 dependencies for known issues, found 8 issues, 24 vulnerable paths. Issues with no direct upgrade or patch: ✗ Improper Resource Shutdown or Release [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-10332643] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available ✗ Buffer Overflow [High Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-10332644] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available ✗ Buffer Overflow [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-10332645] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available ✗ Mismatched Memory Management Routines [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-10337825] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available ✗ Out-of-bounds Write [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-10337826] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available ✗ Out-of-bounds Write [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-10337828] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available ✗ Out-of-bounds Write [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-10337834] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available ✗ Improper Check for Unusual or Exceptional Conditions [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-9726944] in torch@2.7.1 introduced by torch@2.7.1 and 2 other path(s) No upgrade or patch available Organization: trumant Package manager: poetry Target file: pyproject.toml Project name: uqlm Open source: no Project path: /Users/travistruman/dev/uqlm Licenses: enabled [pre-commit-snyk] 2025-07-22 09:00:52 level=info Snyk exit code: 0 zizmor...................................................................Passed ```
1 parent 4faa946 commit 213f5b0

File tree

10 files changed

+696
-222
lines changed

10 files changed

+696
-222
lines changed

‎.github/dependabot.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# See https://docs.github.com/en/code-security/dependabot/working-with-dependabot/dependabot-options-reference
2+
version: 2
3+
updates:
4+
- package-ecosystem: "pip"
5+
directory: "/"
6+
schedule:
7+
interval: "daily"
8+
- package-ecosystem: "github-actions"
9+
directory: "/"
10+
schedule:
11+
interval: "daily"

‎.github/workflows/ci.yaml‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ on:
66

77
jobs:
88
run-tests:
9+
permissions:
10+
contents: read
911
strategy:
1012
fail-fast: false
1113
matrix:
@@ -21,15 +23,20 @@ jobs:
2123

2224
steps:
2325
- name: Checkout code
24-
uses: actions/checkout@v3
26+
uses: actions/checkout@v4.2.2
27+
with:
28+
persist-credentials: false
2529

2630
- name: Set up Python
27-
uses: actions/setup-python@v4
31+
uses: actions/setup-python@v5.6
2832
with:
2933
python-version: ${{matrix.python-version}}
3034

3135
- name: Install dependencies
32-
run: python -m pip install pytest pytest-asyncio langchain-openai .
36+
run: |
37+
python -m pip install --upgrade pip
38+
pip install poetry
39+
poetry install --all-groups
3340
3441
- name: Run tests
35-
run: pytest -v
42+
run: poetry run pytest -v

‎.github/workflows/linting.yml‎

Lines changed: 30 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,37 @@
11
name: Linting with Ruff
22

33
on:
4-
pull_request:
5-
branches:
6-
- main
7-
- develop
8-
workflow_dispatch:
4+
pull_request:
5+
branches:
6+
- main
7+
- develop
8+
workflow_dispatch:
99

1010
concurrency:
11-
group: ${{ github.workflow }}-${{ github.ref }}
12-
cancel-in-progress: true
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
1313

1414
jobs:
15-
ruff-formatting:
16-
runs-on: ubuntu-latest
17-
steps:
18-
- uses: actions/checkout@v4
19-
- name: Set up Python
20-
uses: actions/setup-python@v5
21-
with:
22-
python-version: "3.9"
23-
cache: 'pip'
24-
- name: Get Ruff version and install
25-
run: |
26-
pip install poetry
27-
RUFF_VERSION=$(poetry show --only=dev | grep '^ruff ' | awk '{print $3}')
28-
echo "Installing ruff==$RUFF_VERSION"
29-
pip install ruff==$RUFF_VERSION
30-
- name: Lint with Ruff
31-
run: |
32-
ruff check uqlm/
33-
- name: Check for unformatted files
34-
run: |
35-
ruff format --check uqlm/
15+
ruff-formatting:
16+
permissions:
17+
contents: read
18+
runs-on: ubuntu-latest
19+
steps:
20+
- uses: actions/checkout@v4.2.2
21+
with:
22+
persist-credentials: false
23+
- name: Set up Python
24+
uses: actions/setup-python@v5.6
25+
with:
26+
python-version: "3.9"
27+
cache: 'pip'
28+
- name: Install Ruff
29+
run: |
30+
pip install poetry
31+
poetry install --all-groups
32+
- name: Lint with Ruff
33+
run: |
34+
poetry run ruff check
35+
- name: Check for unformatted files
36+
run: |
37+
poetry run ruff format --check

‎.pre-commit-config.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,25 @@ repos:
55
- id: ruff
66
args: [ --fix ]
77
- id: ruff-format
8+
- repo: https://github.com/PyCQA/bandit
9+
rev: 1.8.6
10+
hooks:
11+
- id: bandit
12+
args: [--config=bandit.yml]
13+
- repo: https://github.com/gitleaks/gitleaks
14+
rev: v8.27.2
15+
hooks:
16+
- id: gitleaks
17+
- repo: https://github.com/fabasoad/pre-commit-snyk
18+
rev: v1.0.2
19+
hooks:
20+
- id: snyk-test
21+
args:
22+
- --snyk-args=--file=poetry.lock
23+
- --snyk-args=--package-manager=pip
24+
- --snyk-args=--fail-on=upgradable
25+
- repo: https://github.com/zizmorcore/zizmor-pre-commit
26+
rev: v1.11.0
27+
hooks:
28+
# Run the linter.
29+
- id: zizmor

‎bandit.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
exclude_dirs:
2+
- assets
3+
- examples
4+
- tests
5+
skips:
6+
- B101

‎examples/judges_demo.ipynb‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,15 +66,14 @@
6666
},
6767
{
6868
"cell_type": "code",
69-
"execution_count": 1,
69+
"execution_count": null,
7070
"metadata": {
7171
"tags": []
7272
},
7373
"outputs": [],
7474
"source": [
7575
"import os\n",
7676
"from uqlm import LLMPanel\n",
77-
"from uqlm.judges import LLMJudge\n",
7877
"from uqlm.utils import load_example_dataset, math_postprocessor"
7978
]
8079
},

‎poetry.lock‎

Lines changed: 504 additions & 185 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎pyproject.toml‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@ bert-score = "^0.3.0"
3333
pandas = "^2.2.3"
3434
sentence-transformers = "^3.4.0"
3535
datasets = "^3.3.2"
36+
aiohttp = "^3.12.14"
37+
torch = "^2.7.1"
3638

3739

3840
[tool.poetry.group.dev]
@@ -44,6 +46,7 @@ ipywidgets = "^8.1.5"
4446
ruff = "0.9.7"
4547
pre-commit = "^4.1.0"
4648
ipykernel = "^6.29.5"
49+
bandit = "^1.8.6"
4750

4851
[tool.poetry.group.test]
4952
optional = true
@@ -103,7 +106,7 @@ line-length = 400
103106

104107
[tool.ruff.lint]
105108
#What rules to enable
106-
select = ["E", "F"]
109+
select = ["E", "F", "S"]
107110
# E = pycodestyle errors
108111
# F = pyflakes
109112
# I = isort (import sorting)
@@ -113,7 +116,8 @@ select = ["E", "F"]
113116
# S = bandit (security)
114117

115118
#What rules to ignore
116-
ignore = []
119+
ignore = ["S101"]
120+
per-file-ignores = { "tests/test_similarity.py" = ["S603","S607"] }
117121

118122

119123
[tool.ruff.format]

‎security-insights.yml‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
header:
2+
schema-version: 2.0.0
3+
last-updated: '2025-07-22'
4+
last-reviewed: '2025-07-22'
5+
url: https://github.com/cvs-health/uqlm
6+
7+
project:
8+
documentation:
9+
code-of-conduct: https://github.com/cvs-health/uqlm/blob/main/CODE_OF_CONDUCT.md
10+
quickstart-guide: https://cvs-health.github.io/uqlm/latest/getstarted.html
11+
homepage: https://cvs-health.github.io/uqlm/
12+
name: "uqlm: Uncertainty Quantification for Language Models"
13+
administrators:
14+
- name: Dylan Bouchard
15+
affiliation: CVS Health
16+
email: dylan.bouchard@cvshealth.com
17+
primary: true
18+
- name: Mohit Singh Chauhan
19+
affiliation: CVS Health
20+
email: mohitsingh.chauhan@cvshealth.com
21+
primary: false
22+
- name: David Skarbrevik
23+
affiliation: CVS Health
24+
email: david.skarbrevik@cvshealth.com
25+
- name: Viren Bajaj
26+
affiliation: CVS Health
27+
email: bajajv@aetna.com
28+
- name: Ho-Kyeong Ra
29+
affiliation: CVS Health
30+
email: doyajii1@gmail.com
31+
- name: Zeya Ahmad
32+
affiliation: CVS Health
33+
email: zeya.ahmad@cvshealth.com
34+
repositories:
35+
- name: LangFair
36+
url: https://github.com/cvs-health/uqlm
37+
comment: cvs-health/uqlm is the primary repository for uqlm.
38+
vulnerability-reporting:
39+
bug-bounty-available: false
40+
contact:
41+
name: Dylan Bouchard
42+
affiliation: CVS Health
43+
email: dylan.bouchard@cvshealth.com
44+
primary: true
45+
reports-accepted: true
46+
# security-policy: TODO: Add security policy URL
47+
48+
repository:
49+
url: https://github.com/cvs-health/uqlm
50+
status: active
51+
accepts-change-request: true
52+
accepts-automated-change-request: true
53+
core-team:
54+
- name: Dylan Bouchard
55+
affiliation: CVS Health
56+
email: dylan.bouchard@cvshealth.com
57+
primary: true
58+
documentation:
59+
contributing-guide: https://github.com/cvs-health/uqlm/blob/main/CONTRIBUTING.md
60+
license:
61+
url: https://github.com/cvs-health/uqlm/blob/main/LICENSE
62+
expression: "Apache-2.0"
63+
security:
64+
assessments:
65+
self:
66+
comment: Self assessment with SCA, SAST and secret leak tools completed July 2025 to establish baseline.
67+
date: '2025-07-22'
68+
tools:
69+
- name: Dependabot
70+
type: SCA
71+
version: "2"
72+
rulesets:
73+
- built-in
74+
results:
75+
adhoc:
76+
name: Scheduled SCA Scan Results
77+
predicate-uri: https://docs.github.com/en/graphql/reference/objects#repositoryvulnerabilityalert
78+
location: https://github.com/cvs-health/uqlm/security/dependabot
79+
comment: |
80+
The results of the scheduled SCA scan are available in the Dependabot tab of the Security Insights page.
81+
integration:
82+
adhoc: true
83+
ci: false
84+
release: false
85+
- name: bandit
86+
comment: Used in pre-commit and CI to check for security issues in Python code.
87+
type: SAST
88+
version: "1.8.6"
89+
rulesets:
90+
- bandit.yml
91+
integration:
92+
adhoc: true
93+
ci: true
94+
release: true
95+
- name: Snyk Open Source
96+
comment: Used in pre-commit to check for vulnerabilities in dependencies.
97+
type: SCA
98+
version: "1.1298.0"
99+
rulesets:
100+
- default
101+
integration:
102+
adhoc: true
103+
ci: false
104+
release: false

‎uqlm/black_box/bleurt.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ def _download(url, my_file_path):
9494
print(f"BLEURT checkpoint not found. Downloading to: {zip_file_path}")
9595

9696
try:
97-
response = requests.get(url)
97+
response = requests.get(url, timeout=(5, 60))
9898
if response.status_code == 200:
9999
with open(zip_file_path, "wb") as f:
100100
f.write(response.content)

0 commit comments

Comments
 (0)