Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DMS-197] [DMS-291] Enabling and Fixing E2E Tests #199

Merged
merged 20 commits into from
Jul 24, 2024
Merged

Conversation

stephenfuqua
Copy link
Contributor

No description provided.

Copy link

github-actions bot commented Jul 23, 2024

Test Results

431 tests  +40   402 ✅ +30   2m 16s ⏱️ +9s
  5 suites ± 0    29 💤 +10 
  5 files   ± 0     0 ❌ ± 0 

Results for commit d2ba0ee. ± Comparison against base commit f8e9b66.

This pull request removes 44 and adds 84 tests. Note that renamed tests count towards both.
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _01PostAnEmptyRequestObject
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _02PostAnInvalidBodyForAcademicWeeksWhenWeekIdentifierLengthShouldBeAtLeast5Characters
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _03PostAnInvalidBodyForAcademicWeeksMissingSchoolidForSchoolReferenceAndTotalInstructionalDays
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _04PostAValidDescriptor
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _05PostAnInvalidBodyForAcademicWeeksMissingMoreThanOneRequiredField
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _06PostAnInvalidBodyForAcademicWeeksMissingARequiredFieldInANestedObjectSchoolidForSchoolReference
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _07PostAnInvalidBodyForAcademicWeeksMissingACommaBeforeBeginDate
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _08PostAnInvalidBodyForCourseOfferingsMissingATwoRequiredFieldsForANestedObjectCourseReferenceAndAlsoSchoolReference
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _09PostRequestBodyForAcademicWeeksWithIdentityPropertyValueContainsOnlyWhiteSpaces
EdFi.DataManagementService.Tests.E2E.Features.ErrorHandling.ValidationErrorsFeature ‑ _10PostRequestBodyForAcademicWeeksWithIdentityPropertyValueContainsWhiteSpaces
…
EdFi.DataManagementService.Tests.E2E.Features.Entities.RejectClientRequestsForAbstractEntitiesFeature ‑ _01EnsureThatClientsCannotPOSTAnAbstractEntityEducationOrganizations
EdFi.DataManagementService.Tests.E2E.Features.Entities.RejectClientRequestsForAbstractEntitiesFeature ‑ _02EnsureThatClientsCannotPOSTAnAbstractEntityGeneralStudentProgramAssociation
EdFi.DataManagementService.Tests.E2E.Features.Entities.RejectClientRequestsForAbstractEntitiesFeature ‑ _03EnsureThatClientsCannotGETAnAbstractEntityEducationOrganizations
EdFi.DataManagementService.Tests.E2E.Features.Entities.RejectClientRequestsForAbstractEntitiesFeature ‑ _04EnsureThatClientsCannotGETAnAbstractEntityStudentProgramAssociation
EdFi.DataManagementService.Tests.E2E.Features.Entities.SchoolYearReferenceValidationFeature ‑ _01TryCreatingAResourceUsingAValidSchoolYear
EdFi.DataManagementService.Tests.E2E.Features.Entities.SchoolYearReferenceValidationFeature ‑ _02TryCreatingAResourceUsingAnInvalidSchoolYear
EdFi.DataManagementService.Tests.E2E.Features.Entities.SchoolYearReferenceValidationFeature ‑ _03TryCreatingACalendarDateUsingAValidCalendarReference
EdFi.DataManagementService.Tests.E2E.Features.Entities.SchoolYearReferenceValidationFeature ‑ _04TryCreatingACalendarDateUsingAnInvalidCalendarReferenceWithAnInvalidSchoolYear
EdFi.DataManagementService.Tests.E2E.Features.General.TheDiscoveryAPIProvidesInformationAboutTheApplicationVersionSupportedDataModelSAndURLsForAdditionalMetadata_Feature ‑ _01GETReturnsTheRootDiscoveryAPIDocument
EdFi.DataManagementService.Tests.E2E.Features.General.TheDiscoveryAPIProvidesInformationAboutTheApplicationVersionSupportedDataModelSAndURLsForAdditionalMetadata_Feature ‑ _02GETMetadataReturnsTheMetadataURLList
…
This pull request removes 12 skipped tests and adds 23 skipped tests. Note that renamed tests count towards both.
EdFi.DataManagementService.Tests.E2E.Features.Paging.PagingSupportForGETRequestsForEd_FiResourcesFeature ‑ _04EnsureClientsCanGETInformationWhenFilteringWithLimitsAndProperties
EdFi.DataManagementService.Tests.E2E.Features.Properties.ValidateExtraPropertiesAreBeingRemovedFeature ‑ _01ValidateExtraPropertiesAreBeingRemovedOnPOST
EdFi.DataManagementService.Tests.E2E.Features.Properties.ValidateExtraPropertiesAreBeingRemovedFeature ‑ _02ValidateExtraPropertiesAreBeingRemovedOnPUT
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _03VerifyErrorHandlingWithPOSTUsingInvalidDataForbidden
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _05VerifyErrorHandlingWithPOSTUsingBlankSpacesInTheRequiredFields
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _06VerifyPOSTOfExistingRecordWithoutChanges
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _07VerifyPOSTOfExistingRecordChangeNon_KeyFieldWorks
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _08VerifyErrorHandlingWhenResourceIDIsIncludedInBodyOnPOST
EdFi.DataManagementService.Tests.E2E.Features.SchoolYear.SchoolYearReferenceValidationFeature ‑ _01TryCreatingAResourceUsingAValidSchoolYear
EdFi.DataManagementService.Tests.E2E.Features.SchoolYear.SchoolYearReferenceValidationFeature ‑ _02TryCreatingAResourceUsingAnInvalidSchoolYear
…
EdFi.DataManagementService.Tests.E2E.Features.References.ValidationOfDELETERequestsThatWouldCauseAForeignKeyViolationFeature ‑ _01EnsureClientsCannotDeleteAYearThatIsUsedByAnotherItem
EdFi.DataManagementService.Tests.E2E.Features.References.ValidationOfDELETERequestsThatWouldCauseAForeignKeyViolationFeature ‑ _02EnsureClientsCannotDeleteADescriptorThatIsUsedByAnotherItem
EdFi.DataManagementService.Tests.E2E.Features.References.ValidationOfDELETERequestsThatWouldCauseAForeignKeyViolationFeature ‑ _03EnsureClientsCannotDeleteADependentElementForAnItem
EdFi.DataManagementService.Tests.E2E.Features.References.ValidationOfDELETERequestsThatWouldCauseAForeignKeyViolationFeature ‑ _04EnsureClientsCannotDeleteAnElementThatIsReferenceToAnEducationOrganizationThatIsUsedByAnotherItems
EdFi.DataManagementService.Tests.E2E.Features.References.ValidationOfDELETERequestsThatWouldCauseAForeignKeyViolationFeature ‑ _05EnsureClientsCannotDeleteAResourceThatIsUsedByAnotherItems
EdFi.DataManagementService.Tests.E2E.Features.ResourceQueries.PagingSupportForGETRequestsForEd_FiResourcesFeature ‑ _04EnsureClientsCanGETInformationWhenFilteringWithLimitsAndProperties
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _02CreateADocumentWithAStringThatIsTooLongDescriptor
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _04PostADescriptorUsingAnInvalidNamespace
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _06CreateADocumentWithSpacesInRequiredFieldsDescriptor
EdFi.DataManagementService.Tests.E2E.Features.Resources.ResourcesCreateOperationValidationsFeature ‑ _07CreateADocumentWithLeadingSpacesInRequiredFieldsDescriptor
…

♻️ This comment has been updated with latest results.

Scenario: 01 GET / returns the root Discovery API document
Given a GET to the root of the API
When a GET request is made to "/"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have noticed in my prior peer review that this was mixing up Given and When.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember this, and the reason of the Given a GET to the root of the API was because we have it in that way in the Information.feature file

Feature: Information
    Retrieve information about the API

        Scenario: 01 Load information
            Given a get to the root of the API
             Then retrieves the information about the API

But I think is better to keep the When a GET request is made to "/"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I was being a little bit lazy that I didn't want to write a new step definition 😆

Then it should respond with 200
And the response body is
"""
{
"version": "1.0.0",
"applicationName": "Ed-Fi Alliance Data Management Service"
"applicationName": "Ed-Fi Alliance Data Management Service",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This JSON was invalid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Covered by the new DiscoveryAPI.feature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This had overposting, which is also covered in the ResourceCreate and ResourceUpdate features.

@@ -1,35 +1,47 @@
Feature: Validation of DELETE requests that would cause a foreign key violation

Background:
Given this School Year exists
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did not have a step definition that could use this syntax.

"currentSchoolYear": false,
"schoolYearDescription": "2021-2022"
}
"""
Copy link
Contributor Author

@stephenfuqua stephenfuqua Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an interesting little challenge. On the back-end when we want to inject an id value to replace {id}, we only had access to the last id value created in the background section above. Which probably isn't the one we need here. To get around this, I just reposted the document for the item to delete - so that the desired document's id value is the last one recorded before running the When statement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Suganya had this issue before ~2 weeks ago, but I can't tag her. Probably this could be useful for her. Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CSR2017 see above.

Copy link
Contributor

@JBrenesSimpat JBrenesSimpat Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was set up to generate the necessary Given steps and add the word references for which we should remove.

Scenario: 01 Ensure clients cannot delete a year that is used by another item
            Given the system has these "schoolYearTypes" references
                  | schoolYear | currentSchoolYear | schoolYearDescription |
                  | 2022       | false             | 2021-2022             |
             And the system has these "calendars"
                  | calendarCode   | schoolReference         | schoolYearTypeReference | calendarTypeDescriptor                                  |
                  | "2010605675"   | {"schoolId":255901107}  | {"schoolYear":2022}     | uri://ed-fi.org/CalendarTypeDescriptor#Student Specific |
             When a DELETE request is made to referenced resource "/ed-fi/schoolYearTypes/{id}"
             Then it should respond with 409

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JBrenesSimpat when you rebase on top of my changes to the "delete" feature file, please feel free to completely overwrite my changes, keeping whatever you have done.


Scenario: 04 Ensure that clients cannot GET an abstract entity (Student Program Association)
When a GET request is made to "/ed-fi/generalStudentProgramAssociations"
Then it should respond with 404
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Enabled the file.
  2. Moved it to a new directory.
  3. Removed @ignore

No other changes to the file.

| classPeriodName | schoolReference |
| 01 - Traditional | { "schoolId": 255901001 } |
| 02 - Traditional | { "schoolId": 255901001 } |
When a POST request is made to "ed-fi/bellschedules" with
When a POST request is made to "/ed-fi/bellschedules" with
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had a mix of slash and not slash. I standardized on having the leading slash, because I think that works with what we're used to seeing in Open API docs and Swagger UI.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consolidated these tests, which were about overposting, into the create and update test files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discovery API feature file replaces this one.

@stephenfuqua stephenfuqua changed the title [DMS-197] Enabling and Fixing E2E Tests [DMS-197] [DMS-291] Enabling and Fixing E2E Tests Jul 23, 2024
@@ -45,7 +45,7 @@ internal async Task GetMetadata(HttpContext httpContext)
{
dependencies = $"{baseUrl}/dependencies",
specifications = $"{baseUrl}/specifications",
xsdFiles = $"{baseUrl}/xsdMetadata"
xsdMetadata = $"{baseUrl}/xsd"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the format I had previously defined in the Discovery API standard. I was able to find this error thanks to @elopezgap's review of the API standard documents.

@stephenfuqua stephenfuqua marked this pull request as ready for review July 23, 2024 19:16
Given the system has these "Schools"
| schoolId | nameOfInstitution | educationOrganizationCategories | gradeLevels |
| 100 | School Test | [{ "educationOrganizationCategoryDescriptor": "uri://tpdm.ed-fi.org/EducationOrganizationCategoryDescriptor#School"}] | [ {"gradeLevelDescriptor": "uri://ed-fi.org/GradeLevelDescriptor#Ninth grade"}] |
And a POST request is made to "courses" with
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the leading slash /courses ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically no, but it should be failing since it does not have ed-fi/. Unfortunately, the mistake in line 66 results in a 404 - which is exactly what this test is looking for! A false positive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@JBrenesSimpat JBrenesSimpat merged commit 971b577 into main Jul 24, 2024
12 checks passed
@JBrenesSimpat JBrenesSimpat deleted the DMS-197 branch July 24, 2024 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants