Skip to content

Conversation

@chesterxgchen
Copy link
Collaborator

@chesterxgchen chesterxgchen commented Sep 7, 2025

Description

Fix data download and preparation issue.

Issues:

  1. Flamby download data doesn't work consistently, in many cases, it refuse to to download as the it has cached configure indicating download completed, and abort the operation, even through the data is already gone or not available.
    Flamby download is very convoluded requires config setup and etc.
    As result, one can't not download and finish experiments

  2. Even you manage to download, the Flamby Dataset class failed to load the data, as the datasets now contains many other data file which are in different formats.

Solution

  1. replace the Flamby download with directly download dataset from the source. Only extract the original processed the datasets.
  2. pass the data_path directly to Flamby avoid convoluted steps.

Update doc

update README and documentation hello-lr/index.rst file

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Quick tests passed locally by running ./runtest.sh.
  • In-line docstrings updated.
  • Documentation updated.

@chesterxgchen
Copy link
Collaborator Author

/build

1 similar comment
@chesterxgchen
Copy link
Collaborator Author

/build

@chesterxgchen
Copy link
Collaborator Author

/build

@chesterxgchen
Copy link
Collaborator Author

/build

@YuanTingHsieh YuanTingHsieh merged commit 8509b8e into NVIDIA:main Sep 8, 2025
20 checks passed
chesterxgchen added a commit to chesterxgchen/NVFlare that referenced this pull request Sep 30, 2025
# Description
## Fix data download and preparation issue. 

**Issues**:
1) Flamby download data doesn't work consistently, in many cases, it
refuse to to download as the it has cached configure indicating download
completed, and abort the operation, even through the data is already
gone or not available.
Flamby download is very convoluded requires config setup and etc.
As result, one can't not download and finish experiments

2) Even you manage to download, the Flamby Dataset class failed to load
the data, as the datasets now contains many other data file which are in
different formats.

**Solution**
1) replace the Flamby download with directly download dataset from the
source. Only extract the original processed the datasets.
2) pass the data_path directly to Flamby avoid convoluted steps. 

## Update  doc

update README and documentation hello-lr/index.rst file





### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Quick tests passed locally by running `./runtest.sh`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated.
@chesterxgchen chesterxgchen deleted the LR_data branch October 19, 2025 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants