I1179 create zip for submit + CI's #1180

johnbrvc · 2025-09-30T19:08:27Z

Description of what the PR does

Accepts CLICS 2023-06 compliant submissions that include the source code in a single BASE64 encoded ZIP file. To maintain backward compatibility, we also accept a submission containing an array of BASE64 encoded files if the mime type of the FILE object is null or empty.

Move getIFiles(byte []) to EventFeedUtilities from RemoteContestAPIAdapter. This method (now public static) converts a byte stream to a zip file and extracts the contents to a List of IFile objects.
Add additional getIFiles(String) method to EventFeedUtilities to accept a BASE64 encoded string consisting of a ZIP file, then decode that and call the getIFiles(byte []) method.
Change the CLICS 2023-06 submission service to accept the previous format (a list of BASE64 encoded files) for backward compatibility, and, also accept the correct format (BAS64 encoded ZIP file). The distinction is made by looking at the mime type of the FILE objects passed.
Update the pc2submit command line utility to put the source files into a zip file, then encode that as base64 and send it off. [This fix for pc2submit was provided by Nicky Gerritsen from the DOMjudge team.]
CI: Fix up the PATCH for the ContestService to set the thaw time. Better logging. Better messages returned to user. Removed and handled outstanding TODO.

Issue which the PR addresses

Fixed #1179

Environment in which the PR was developed (OS,IDE, Java version, etc.)

Windows 11
java version "1.8.0_321"
Java(TM) SE Runtime Environment (build 1.8.0_321-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.321-b07, mixed mode)

Precise steps for testing the PR (i.e., how to demonstrate that it works correctly)

Prerequisites:

Python 3.10 (or greater) installed.
Python requests package installed (pip install requests)

Load a new contest, such as SumitHello (SumH)
Start the pc2server
Start the pc2admin
Generate an event feeder account (feeder1).
Set the SHADOW_PROXY_TEAM permission on feeder1.
You should also set the Enable CCS Test Mode setting on the Configure Contest->Settings tab under Remote CCS Settings.
Switch to the Run Contest -> Runs tab on the pc2admin.
Start the pc2ef and log in as feeder1.
<< I think there is a missing step here: "Start the webserver" (jlc)
Start the contest.
Open a command prompt window
Set the SUBMITCREDENTIALS environment variable to yes so that the submit command will prompt you for the team's user and password: set SUBMITCREDENTIALS=yes
Using the bin/pc2submit command line Python submit utility, make a submission: python bin/pc2submit -y -u https://localhost:50443 -p B samps/src/hello.cpp
The first time you do this, you may be prompted for your team and password (team1/team1)
The submission should succeed and you should see it appear on the Runs tab.
Select the submission and press "View Source" to confirm the source code is there.
This confirms that BASE64 encode ZIP file worked properly.

Move getIFiles() to EventFeedUtilities from RemoteContestAPIAdapter. This class converts a byte stream to a zip file and extracts the contents to a List of IFile objects. Add additional getIFiles(String) method to EventFeedUtilities to accept a BASE64 encoded string consisting of a ZIP file, then decode that and call the getIFiles(bte []) method. Change the CLICS 2023-04 submission service to accept the previous format (a list of BASE64 encoded files) for backward compatibility, and, also accept the correct format (BAS64 encoded ZIP file). The distinction is made by looking at the mime type of the FILE objects passed. Update pc2submit command line utility to put the source files into a zip file, then encode that as base64 and send it off. [This fix for pc2submit was provided by Nicky Gerritsen from the DOMjudge team.]

Setting the contest thaw time using the Contest Service PATCH facility was a bit of a black box and it was hard to determine what happened. Now, we provide better logging. In addition, we call the proper method to set the thaw time. CLICSContestInfo: be sure to include the configured thaw time value in the contest information notification message and the contests endpoint.

nickygerritsen · 2025-10-06T17:06:24Z

src/edu/csus/ecs/pc2/clics/API202306/SubmissionService.java

+            String mimeType = firstFile.getMime();
+
+            // TODO: deprecate this "if" part and leave the "else" part.  JB
+            if(mimeType == null || "".equals(mimeType)) {


A zip with an empty mime is also allowed according to the spec, I think that's not handled now?

If you take a look at the comments in the PR, we are using the absence of the mime-type as a (temporary) indicator that the older version of pc2submit (submit) that did not zip the files is being used. We are keeping this in for now, since some images out there (ie the icpc2025 image) have the version of pc2submit that does not zip. It also does not include the mime-type, so that's how we can tell (for now). The only command line submit utilities that interact with the API are pc2submit (which has been fixed to add the mime-type and zip the files), and the playback utility, which is a private test utility.

After the next contest image cycle, we will remove the deprecated code and use the absence of the mime-type to indicate it's a zip file, as per the specification.

Makes sense!

SamanwaySadhu

Update steps to follow with how to set the credentials. Looks good otherwise!

clevengr

Reviewed src/edu/csus/ecs/pc2/clics/API202306/ContestService.java. Minor (somewhat trivial) comments on what I think are incorrect comments in the code...

clevengr · 2025-10-25T16:54:38Z

src/edu/csus/ecs/pc2/clics/API202306/ContestService.java

-            return Response.status(Status.NOT_MODIFIED).entity("Unable to set contest thaw time to " + thawTime.toString()).build();
+            Date thawDate = thawTime.getTime();
+            String thawStr = Utilities.getIso8601formatterWithMS().format(thawDate);
+            // get the local model's ContestInformation sincd we are modifying the thaw time


(trivial) typo: "sincd" -> "since"

clevengr · 2025-10-25T16:55:56Z

src/edu/csus/ecs/pc2/clics/API202306/ContestService.java

+            ContestInformation ci = model.getContestInformation();
+            if (ci != null) {
+                // set the new start date/time into the ContestInformation
+                ci.setThawed(thawDate);


The preceding comment says we are setting the "new start date/time", but I don't think that's what's actually being set here. (Copy/paste without an update?)

clevengr

reviewed src/edu/csus/ecs/pc2/clics/API202306/SubmissionService.java. Made one comment regarding handling of zero length files.

clevengr · 2025-10-25T17:08:39Z

src/edu/csus/ecs/pc2/clics/API202306/SubmissionService.java

+                    if(fileData == null || fileData.length() == 0) {
+                        // nice to put it in the log in case any questions come up.
+                        log.info(user + " POSTing empty source submission on behalf of team " + team_id);
+
+    //                    return Response.status(Response.Status.BAD_REQUEST).entity("no file data specified for " + fileName).build();
+                    }


PR #1173 (handle zero-length files in WTI) includes support for PC2 Admin configuration of "Allow zero-length files?". Once that PR is approved and merged, the above block of code should probably check the ContestInformation configuration item to determine whether to allow a zero-length file. (Unless of course the above block of code actually gets deprecated and eliminated, as suggested above...)

clevengr

reviewed src/edu/csus/ecs/pc2/convert/EventFeedUtilities.java. Made one comment on a trivial typo in a comment; otherwise looks good to me.

clevengr · 2025-10-25T17:17:45Z

src/edu/csus/ecs/pc2/convert/EventFeedUtilities.java

+    public static List<IFile> getIFiles(String base64Data) {
+
+        // use the decoder to both check the validity of, and to store, the byte data.
+        // this will throw an IllegalArgumentException if the data is basd


Typo in comment? ("basd"?)

clevengr

I've reviewed all the code changes. Other than trivial issues like typos in comments (which I've noted separately), the code looks reasonable to me. Proceeding to runtime testing.

clevengr

I completed a runtime test following the instructions in the PR (although the instructions were missing the information that I needed to have a valid .netrc file on my Windows box in order for pc2submit to work). Since I was able to view the submitted source code in the admin, it appears that the PR works as intended, so I'm approving it.

johnbrvc added 2 commits September 30, 2025 14:32

johnbrvc added this to the 9.11.0 milestone Sep 30, 2025

johnbrvc requested review from JoeTerlizzi, SamanwaySadhu, clevengr and kkarakas September 30, 2025 19:08

johnbrvc self-assigned this Sep 30, 2025

nickygerritsen reviewed Oct 6, 2025

View reviewed changes

SamanwaySadhu approved these changes Oct 13, 2025

View reviewed changes

clevengr reviewed Oct 25, 2025

View reviewed changes

clevengr approved these changes Oct 25, 2025

View reviewed changes

I1179 create zip for submit + CI's #1180

Are you sure you want to change the base?

I1179 create zip for submit + CI's #1180

Uh oh!

Conversation

johnbrvc commented Sep 30, 2025 • edited by clevengr Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of what the PR does

Issue which the PR addresses

Environment in which the PR was developed (OS,IDE, Java version, etc.)

Precise steps for testing the PR (i.e., how to demonstrate that it works correctly)

Uh oh!

nickygerritsen Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

johnbrvc Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

nickygerritsen Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

SamanwaySadhu left a comment

Choose a reason for hiding this comment

Uh oh!

clevengr left a comment

Choose a reason for hiding this comment

Uh oh!

clevengr Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

clevengr Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

clevengr left a comment

Choose a reason for hiding this comment

Uh oh!

clevengr Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

clevengr left a comment

Choose a reason for hiding this comment

Uh oh!

clevengr Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

clevengr left a comment

Choose a reason for hiding this comment

Uh oh!

clevengr left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

johnbrvc commented Sep 30, 2025 •

edited by clevengr

Loading