GitHub: Add bi-directional sync #3565

savish28 · 2021-08-12T00:12:08Z

Context: Github Bi-directional Sync

Related Pull Request: Cloud-CV/EvalAI-Starters#52

Models Considered:

Challenge
Challenge Phase

Deliverables:

Add github_token
Add sync for non-file fields
Add sync for file fields for challenge model
Add only updated fields commit

Testing without celery:
https://user-images.githubusercontent.com/32800267/129727539-80d18fd0-7cdc-489d-a3c6-0b346ce605c8.mp4

@KhalidRmb
cc: @Ram81

…ub_sync

Ram81 · 2021-08-23T03:54:05Z

apps/challenges/github_utils.py

+URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}
+
+
+class Github_Interface:


Please rename class to GithubInterface

Ram81 · 2021-08-24T15:34:25Z

apps/challenges/github_utils.py

+ return None
+ return response.json()
+
+ def return_github_url(self, url):


rename the method to get_github_url

Ram81 · 2021-08-24T15:36:30Z

apps/challenges/github_utils.py

+ url = "{0}{1}".format(base_url, url)
+ return url
+
+ def get_content_from_path(self, path):


Please add docstring for this method

Ram81 · 2021-08-24T15:36:35Z

apps/challenges/github_utils.py

+ return response
+
+ def get_data_from_path(self, path):
+ content_response = self.get_content_from_path(path)


Ram81 · 2021-08-24T15:36:42Z

apps/challenges/github_utils.py

+ return string_data
+
+ def update_content_from_path(self, path, content):
+ url = URLS.get("contents").format(self.GITHUB_REPOSITORY, path)


Ram81 · 2021-08-24T15:42:02Z

apps/challenges/github_utils.py

+
+@app.task
+def github_challenge_phase_sync(challenge_phase):
+ from .serializers import ChallengePhaseSerializer


move to global imports

Since it is a circular import.

Ram81 · 2021-08-24T15:42:19Z

apps/challenges/github_utils.py

+ from .serializers import ChallengePhaseSerializer
+
+ for obj in serializers.deserialize("json", challenge_phase):
+ challenge_phase_obj = obj.object


create a util method for deserializing and use that

Ram81 · 2021-08-24T15:42:32Z

apps/challenges/github_utils.py

+ "is_restricted_to_select_one_submission",
+ "is_partial_submission_evaluation_enabled",
+ "allowed_submission_file_types",
+ ]


move to a common util file

Ram81 · 2021-08-24T15:42:38Z

apps/challenges/github_utils.py

+ github.update_data_from_path("challenge_config.yaml", content_str)
+
+ # File fields Update
+ file_fields = ["description"]


Ram81 · 2021-08-24T15:44:04Z

apps/challenges/models.py

+def github_sync_challenge(sender, instance, created, **kwargs):
+ if instance.github_repository and instance.github_token:
+ serialized_obj = serializers.serialize("json", [instance])
+ github_challenge_sync.delay(serialized_obj)


@savish28 the method names are confusing github_challenge_sync and github_sync_challenge (post save hook) it is not clear what these methods are for. Please rename the methods.

codecov-commenter · 2021-08-24T16:33:29Z

Codecov Report

Merging #3565 (2fd2db6) into master (96968d6) will decrease coverage by 1.40%.
The diff coverage is 37.15%.

@@            Coverage Diff             @@
##           master    #3565      +/-   ##
==========================================
- Coverage   72.93%   71.53%   -1.41%     
==========================================
  Files          83       20      -63     
  Lines        5368     3235    -2133     
==========================================
- Hits         3915     2314    -1601     
+ Misses       1453      921     -532

Impacted Files	Coverage Δ
frontend/src/js/controllers/authCtrl.js	`53.91% <6.38%> (-12.95%)`	⬇️
frontend/src/js/controllers/profileCtrl.js	`79.76% <20.00%> (-13.10%)`	⬇️
frontend/src/js/controllers/permissionCtrl.js	`36.36% <22.22%> (-63.64%)`	⬇️
frontend/src/js/controllers/challengeCtrl.js	`63.54% <37.00%> (-10.15%)`	⬇️
frontend/src/js/controllers/updateProfileCtrl.js	`82.55% <44.44%> (-10.30%)`	⬇️
frontend/src/js/controllers/challengeListCtrl.js	`95.74% <50.00%> (+1.06%)`	⬆️
...ntend/src/js/controllers/challengeHostTeamsCtrl.js	`70.50% <66.66%> (-1.18%)`	⬇️
frontend/src/js/controllers/teamsCtrl.js	`71.17% <75.00%> (ø)`
frontend/src/js/controllers/ChallengeInviteCtrl.js	`100.00% <100.00%> (ø)`
frontend/src/js/controllers/SubmissionFilesCtrl.js	`95.45% <100.00%> (ø)`
... and 30 more

Impacted Files	Coverage Δ
frontend/src/js/controllers/authCtrl.js	`53.91% <6.38%> (-12.95%)`	⬇️
frontend/src/js/controllers/profileCtrl.js	`79.76% <20.00%> (-13.10%)`	⬇️
frontend/src/js/controllers/permissionCtrl.js	`36.36% <22.22%> (-63.64%)`	⬇️
frontend/src/js/controllers/challengeCtrl.js	`63.54% <37.00%> (-10.15%)`	⬇️
frontend/src/js/controllers/updateProfileCtrl.js	`82.55% <44.44%> (-10.30%)`	⬇️
frontend/src/js/controllers/challengeListCtrl.js	`95.74% <50.00%> (+1.06%)`	⬆️
...ntend/src/js/controllers/challengeHostTeamsCtrl.js	`70.50% <66.66%> (-1.18%)`	⬇️
frontend/src/js/controllers/teamsCtrl.js	`71.17% <75.00%> (ø)`
frontend/src/js/controllers/ChallengeInviteCtrl.js	`100.00% <100.00%> (ø)`
frontend/src/js/controllers/SubmissionFilesCtrl.js	`95.45% <100.00%> (ø)`
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2bc3487...2fd2db6. Read the comment docs.

RishabhJain2018 · 2021-09-08T22:42:53Z

@Ram81 @savish28 Is this PR ready for review?

savish28 · 2021-09-09T20:40:00Z

@RishabhJain2018 Yes it is!!

Ram81 · 2021-09-12T20:39:57Z

apps/challenges/github_sync_config.py

@@ -0,0 +1,37 @@
+# Fields from Challenge, ChallengePhase model to be considered for github_sync
+
+challenge_non_file_fields = [


@savish28 is there a way we can use the serializer or model to generate this list of fields? Currently, if a new field gets added we'll have to add it here manually. It might happen that some forgets adding it here which would mean there is no sync happening for that field. If we can use models/serializer for this list it would be great

@Ram81, we only wish to have only selected fields(that are editable from frontend) in the sync and not all fields, getting it from serializer would mean using all fields and that is undesirable because there are a lot of fields that are not in the challenge_config and might also be critical to share those.

Ram81 · 2021-09-12T20:40:56Z

apps/challenges/github_utils.py

+URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}
+
+
+class GithubInterface:


@savish28 can we move this class to a separate file called github_interface.py

Yeah sure :) Done.

Ram81 · 2021-09-12T20:42:20Z

apps/challenges/github_utils.py

+ if not github.is_repository():
+ return
+ try:
+ # Challenge Non-file field Update


Change to Challenge non-file field update

Ram81 · 2021-09-12T20:43:18Z

apps/challenges/github_utils.py

+ content_str = yaml.dump(challenge_config_yaml, sort_keys=False)
+ github.update_data_from_path("challenge_config.yaml", content_str)
+
+ # Challenge File fields Update


Change to Challenge file fields update

Ram81 · 2021-09-12T20:44:02Z

apps/challenges/github_utils.py

+ continue
+ github.update_data_from_path(field_path, getattr(challenge, field))
+ except Exception as e:
+ logger.info("Github Sync unsuccessful due to {}".format(e))


Why is this a info log? This should be a error log

Ram81 · 2021-09-12T20:48:19Z

apps/challenges/github_utils.py

+ field_path, getattr(challenge_phase, field)
+ )
+ break
+


remove newline

Ram81 · 2021-09-12T20:48:31Z

apps/challenges/github_utils.py

+ github.update_data_from_path(
+ field_path, getattr(challenge_phase, field)
+ )
+ break


why do we have a break here?

Because at a time only one challenge_phase is updated. Because it is initiated by post-hook of ChallengePhase

Ram81 · 2021-09-12T20:48:52Z

apps/challenges/github_utils.py

+ break
+
+ except Exception as e:
+ logger.info(


same here, Why is this not a error log?

Ram81 · 2021-09-12T20:49:29Z

apps/challenges/models.py

@@ -226,6 +231,13 @@ def create_eks_cluster_for_challenge(sender, instance, created, **kwargs):
 aws.challenge_approval_callback(sender, instance, field_name, **kwargs)


+@receiver(signals.post_save, sender="challenges.Challenge")
+def challenge_sync(sender, instance, created, **kwargs):


rename to challenge_details_sync

Ram81 · 2021-09-12T20:49:46Z

apps/challenges/models.py

@@ -347,6 +359,16 @@ def save(self, *args, **kwargs):
 return challenge_phase_instance


+@receiver(signals.post_save, sender="challenges.ChallengePhase")
+def challenge_phase_sync(sender, instance, created, **kwargs):


change to challenge_phase_details_sync

savish28 · 2021-11-28T09:09:02Z

@Ram81 Ping! :)

Ram81 · 2021-11-28T15:08:58Z

Thanks for the ping @savish28. I will review this PR today

savish28 added 2 commits August 12, 2021 05:32

Github: add bi-directional sync non-file field challenge

4851c3d

Sync: Add challenge file fields

96af768

RishabhJain2018 requested a review from Ram81 August 17, 2021 19:07

savish28 added 4 commits August 19, 2021 14:03

Support compatibility with prev Evalai-starters

9c0585b

Merge branch 'master' of https://github.com/Cloud-CV/EvalAI into gith…

7b8f5ee

…ub_sync

Merge conflicts with master

43e3eec

Sync: challenge phase file and non file updates

4f90106

savish28 changed the title ~~GitHub: add bi-directional sync for non-file challenge field~~ GitHub: Add bi-directional sync Aug 20, 2021

Ram81 requested changes Aug 24, 2021

View reviewed changes

savish28 added 2 commits August 24, 2021 21:48

Doc: decouple utils and rename functions

33cb342

doc: rename model function methods

22d312c

savish28 added 2 commits August 24, 2021 22:07

doc: add docstrings in github_sync functions

5d71989

Remove circular imports

46e4a11

Ram81 requested changes Sep 12, 2021

View reviewed changes

savish28 and others added 3 commits September 13, 2021 02:39

Change model function names

8693a33

refactor code segregating github_inference

2fd2db6

Merge branch 'master' into github_sync

77986bf

Ram81 mentioned this pull request Feb 21, 2022

Easy challenge management on EvalAI Cloud-CV/GSoC-Ideas#46

Closed

10 tasks

		URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}


		class Github_Interface:

		@@ -0,0 +1,37 @@
		# Fields from Challenge, ChallengePhase model to be considered for github_sync

		challenge_non_file_fields = [

		URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}


		class GithubInterface:

GitHub: Add bi-directional sync #3565

Are you sure you want to change the base?

GitHub: Add bi-directional sync #3565

Conversation

savish28 commented Aug 12, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 24, 2021 • edited

Codecov Report

RishabhJain2018 commented Sep 8, 2021

savish28 commented Sep 9, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

savish28 commented Nov 28, 2021

Ram81 commented Nov 28, 2021

savish28 commented Aug 12, 2021 •

edited

codecov-commenter commented Aug 24, 2021 •

edited