DPDK: Fix source for tarball #3505
Conversation
Fixing one bug uncovered a few others. Fixes the way the Tar tool handles fetching the filename of the tar file it downloads.
|
Note: testing in progress. |
mcgov
force-pushed
the
mcgov/source_fix
branch
from
2bed636
to
a0638eb
Compare
lisa/base_tools/wget.py
Outdated
| @@ -45,8 +49,26 @@ def get( | |||
| force_run: bool = False, | |||
| timeout: int = 600, | |||
| ) -> str: | |||
| if not force_run: | |||
There was a problem hiding this comment.
The logic could be simpler.
cached_filename = self.__filename_result_cache.get(url, None)
if cached_filename:
if force_run:
del self.__filename_result_cache[url]
else:
return cached_filenameThere was a problem hiding this comment.
This is good stuff 🙂 I'll fix
lisa/tools/tar.py
Outdated
| @@ -48,6 +49,21 @@ def extract( | |||
| if strip_components: | |||
| # optionally strip N top level components from a tar file | |||
| tar_cmd += f" --strip-components={strip_components}" | |||
|
|
|||
| if skip_old_files: | |||
There was a problem hiding this comment.
How about remove existing extracted files always, or always overwrites? The skip-old-files is hard to use with other tools right.
There was a problem hiding this comment.
No, we specifically don't want to overwrite the old files here.
We have to assume that changes have been applied, we will wipe out patches that have been applied to source files in the output directory otherwise. We can't just delete the output directory and start over every time, since this will potentially wipe out a build directory.
There was a problem hiding this comment.
ok, it makes sense. But the name skip_old_files is not accurate. Please use skip_exist_files for the method argument.
There was a problem hiding this comment.
I feel strongly that options should have the same name as the flag, as long as we're just doing a passthrough like this. It allows people to look up the man page for the flag without having to check the LISA tool's source to see what the option resolves to.
There was a problem hiding this comment.
It depends on the design principle. If the goal is to be same as original commands, you are right. But LISA abstracts the commands for better understanding. In this case, you explained the term "old" in the code, but I read the original document to understand it finally. If it's called "exist", it would be clearer. When the tool is implemented by BSD, Windows and other OS, it also makes sense.
mcgov
force-pushed
the
mcgov/source_fix
branch
from
a0638eb
to
699f611
Compare
When running Wget.get(..., force_run=False) and Tar.extract it is useful to allow Tar to skip extracting existing files on the second pass. Allow the skip-old-files option, so Tar.extract will not overwrite existing files in the output directory. Note: it's important to not use this option if you are providing LISA with a default filename for your tarballs. This option could silently allow Tar.extract to not update the contents of a directory with the newer file contents. I don't see anyone using that schema now. My apologies to future devs who find this commit message while debugging that issue. Tar: chi comments
Dpdk: chi comments
mcgov
force-pushed
the
mcgov/source_fix
branch
from
699f611
to
57fd240
Compare
Fix the way the TarDownloader handles dowloading files and identifying their filenames. This required fixing a bug in the Tar tool where Linux would not infer the filename in the default case like the Windows version does. Previously, providing a directory and not a filename would result in the tool identifying an entire directory of files as the filename when the Wget tool fetches a cached result.
This PR fixes that issue by allowing the tool to infer the filename, this fixes the ability to use the Wget tool without force_run when using a default filename.