Skip to content

Automated Testing

FanDjango edited this page Oct 22, 2022 · 80 revisions

FluentFTP has an extensive automated test suite that includes unit tests for modules, and integration tests which test FluentFTP against many supported FTP servers using Docker containers to locally mount and configure these servers portably.

All credits for the development of our excellent integration test system go to Tommy Sørbråten, Michael Stiemke and Robin Rodricks.

Projects

  • FluentFTP.Tests - XUnit test suite that performs:

    • Unit testing - Tests FluentFTP components and algorithms in isolation.
    • Integration testing - Tests all major FTP functionality against containerized FTP servers.
  • FluentFTP.Dockers - Dockerfiles for our customized FTP servers used by the test suite.

  • FluentFTP.Xunit - Containerized FTP server management used by the test suite.

Supported Servers

You can help! Please file an issue if you can develop or find docker images for more FTP servers!

We have an extensive automated test suite that tests FluentFTP against these servers. We use docker to locally manage containerized FTP servers that are used for testing.

The supported servers are listed below.

Note: You don't need to download these projects manually, everything you need is in the FluentFTP repo.

Server Credits
VsFTPd Created by FluentFTP team in this PR.
Bftpd Created by FluentFTP team in this PR.
ProFTPD Created by FluentFTP team in this PR.
glFTPd Created by FluentFTP team in this PR.
FileZilla Created by FluentFTP team in this PR.
PureFTPd https://github.com/stilliard/docker-pure-ftpd
PyFtpdLib https://github.com/andriykohut/docker-pyftpdlib

How do I run the test suite?

Prerequisites

  1. Windows 10+ Home/Pro/Enterprise/Education 64-bit (or any OS supporting Docker)
  2. Virtualization in your BIOS must be enabled
  3. Hyper-V and Containers must be enabled (Windows only)
  4. Visual Studio 2022+ or VS Code
  5. Docker Desktop

Steps

  1. Open Docker Desktop to make sure it is running properly on your system.
  2. Run the Build.bat or Build.sh script inside the FluentFTP.Dockers folder to create the docker images. See below for tips.
  3. Run all the tests inside the FluentFTP.Tests project. It will automatically start and stop the docker FTP servers as required.

How does the integration test system work?

How it works:

  1. We support a variety of FTP servers which are docker images created and published by third parties
  2. Each FTP server has a container class that manages it
  3. All container classes inherit from DockerFtpContainer
  4. DockerFtpContainer has a Build method responsible for building a test container (TestcontainersContainer)
  5. The Build method creates the image using the configured DockerImage and sets the default port binding for port 21
  6. Each type of container can then add additional options in their Configure method
  7. We then use the test container data to start a docker container using the Testcontainers library
  8. After running the tests we run a system command to shut down the docker container

How do I reduce the space used by Docker on my development machine?

Docker on Windows takes up a lot of space due to this issue. WSL2 virtual disks are "dynamic" .vhdx images, which means they:

  • Are allocated to a maximum size
  • Are initialized with just a few kilobytes of structural data
  • Grow dynamically as data is added, up to their maximum allocated size

If you're willing to wipe all of your docker data, open the Docker Desktop client, click the bug icon in the top bar, and then click Clean/Purge data.

See this stackoverflow answer for more details.

Consider using Docker Desktop (Windows) and adding the "Disk Usage" extension by Docker official and cleaning up with this:

image

Of course, if you are using Docker on your dev machine for other purposes than only FluentFTP testing, you will already be aware of the considerations when removing Docker data.

Running the "Docker Builds"

The docker build batch file will build all the available images. Although our self-made images have been optimised to build as fast as possible, some of the other, public images might take a while to finish - especially when loading/installing from distro/repos that are far away.

You might want to use "REM" to comment out images that you do not want to use or that you already have built. You might want to "totally rebuild" from scratch in some cases and you might want to see more explicit progress reports when building test images. Look up the possible options for docker rebuild and the possible commands to prune or delete existing caches or images.

Read up especially on docker builder prune --all -f and docker image prune -f commands and perhaps include these in the build .bat file. Why would you want to use these commands? Although Docker tries to handle caching of build steps correctly, there are (many) cases where you would fare better when "starting from scratch". It is not so easy to convince Docker to forgo its stored (cached) layers, so beware and judiciously use these commands when needed.

The builds for our self-made images contain a preliminary step that tries to identify the closest, most up to date and hi-bandwidth mirror for debian bullseye and sets that in the etc/apt/source.list of the building step. If you have problems, disable that behaviour in the Dockerfiles.

Clone this wiki locally