Skip to content

Feature/sc descriptor mapping #188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

Feature/sc descriptor mapping #188

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
469770e
change col name to generic edfi_descriptor
Aug 8, 2025
490093b
add config for descriptor mapping files
Aug 8, 2025
cb3f90f
add docs
Aug 8, 2025
96b2ada
change descriptor name
Aug 14, 2025
5fc08ab
refactor to adjust for new col descriptor name
Aug 14, 2025
2241ac6
rethink metadata structure to allow for descriptor mapping
Aug 14, 2025
30c1e8e
separated assessmentCategoryDescriptors from the assessments.csv into…
Aug 21, 2025
1fb434d
fix metadata file
Aug 26, 2025
ce9be35
update registry for testing
johncmerfeld Sep 11, 2025
60b8674
align PSAT student IDs to grand bend
johncmerfeld Sep 11, 2025
73a6fa8
add test type input variable
johncmerfeld Sep 11, 2025
f9f265e
add test type to registry
johncmerfeld Sep 11, 2025
4a47068
handle empty edfi IDs list better
johncmerfeld Sep 11, 2025
a973a94
handle empty edfi IDs
johncmerfeld Sep 11, 2025
b5a16bd
change branch for clarity
johncmerfeld Sep 11, 2025
d1f09e4
revert ID change as it is pulled in from the other branch
johncmerfeld Sep 11, 2025
4da714f
add debug operation
johncmerfeld Sep 11, 2025
dc793fb
remove debug
johncmerfeld Sep 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions assessments/PSAT_SAT/_metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
display_name: "PSAT_SAT"
Copy link
Contributor

@johncmerfeld johncmerfeld Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jalvord1 I suppose we'll want a different display name here a la "PSAT/SAT" or something that looks a bit more human

path: "assessments/PSAT_SAT"
valid_edfi: ">=3.0"
report_resources: ["studentAssessments"]
input_files:
- display_name: "Assessment data"
env_var: "INPUT_FILE"
is_required: true
file_type: ["csv", "txt"]
Copy link
Contributor

@johncmerfeld johncmerfeld Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this true, or is the input file for this assessment always a CSV?

Copy link
Author

@gnguyen87 gnguyen87 Aug 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has always been the file_type options across bundles so i believe it is true

Copy link
Contributor

@johncmerfeld johncmerfeld Aug 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it depends on the input defined in earthmover.yaml. For example, STAAR_Interim only takes CSVs and from what I can tell the same is true here

Copy link
Collaborator

@jalvord1 jalvord1 Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah this bundle will only accept a csv file!

Copy link
Author

@gnguyen87 gnguyen87 Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be updated now!

input_params:
- display_name: "School Year"
env_var: "API_YEAR"
is_required: true
descriptor_mapping_files: ["academicSubjectDescriptors.csv", "gradeLevelDescriptors.csv"]
19 changes: 14 additions & 5 deletions assessments/PSAT_SAT/earthmover.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@ sources:
assessmentReportingMethodDescriptors:
file: ./seeds/assessmentReportingMethodDescriptors.csv
header_rows: 1
assessmentCategoryDescriptors:
file: ./seeds/assessmentCategoryDescriptors.csv
header_rows: 1
gradeLevelDescriptors:
file: ./seeds/gradeLevelDescriptors.csv
header_rows: 1
Expand All @@ -120,7 +123,7 @@ transformations:
input:
source: $sources.input
operations: []

psat_sat_student_assessment:
source: $transformations.input
operations:
Expand Down Expand Up @@ -230,7 +233,6 @@ transformations:
schoolYear: ${API_YEAR}
assessmentIdentifier: "{%raw%}${TEST_TYPE}_{{map_year_to_version(administrationDate)}}{%endraw%}"
namespace: "uri://collegeboard.org"
assessmentCategoryDescriptor: "${DESCRIPTOR_NAMESPACE}/AssessmentCategoryDescriptor#College entrance exam"
Copy link
Contributor

@johncmerfeld johncmerfeld Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jalvord1 is DESCRIPTOR_NAMESPACE something we need to account for? I see it still defaults to Ed-Fi

studentAssessmentIdentifier: "{%raw%}{{studentUniqueId}}_{{administrationDate}}_${TEST_TYPE}{%endraw%}"
- operation: join
sources:
Expand All @@ -247,7 +249,7 @@ transformations:
subject_json: >
{%raw-%}
{
"academicSubjectDescriptor": "{{academicSubjectDescriptor}}"
"academicSubjectDescriptor": "{{edfi_descriptor}}"
},
{%-endraw%}
- operation: modify_columns
Expand All @@ -268,13 +270,13 @@ transformations:
- operation: distinct_rows
columns:
- assessmentIdentifier
- gradeLevelDescriptor
- edfi_descriptor
- operation: add_columns
columns:
grade_json: >
{%raw-%}
{
"gradeLevelDescriptor": "{{gradeLevelDescriptor}}"
"gradeLevelDescriptor": "{{edfi_descriptor}}"
},
{%-endraw%}
- operation: modify_columns
Expand Down Expand Up @@ -307,6 +309,12 @@ transformations:
- operation: add_columns
columns:
assessmentFamily: "{%raw%}{{ assessmentIdentifier.split('_')[0] }}{%endraw%}"
- operation: join
sources:
- $sources.assessmentCategoryDescriptors
join_type: inner
left_key: assessmentIdentifier
right_key: assessmentIdentifier


destinations:
Expand All @@ -331,3 +339,4 @@ destinations:
source: $sources.performanceLevelDescriptors
template: ./templates/performanceLevelDescriptors.jsont
extension: jsonl

2 changes: 1 addition & 1 deletion assessments/PSAT_SAT/seeds/academicSubjectDescriptors.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
assessmentIdentifier,academicSubjectDescriptor
assessmentIdentifier,edfi_descriptor
PSAT 8/9_1600,uri://ed-fi.org/AcademicSubjectDescriptor#Composite
PSAT/NMSQT_1600,uri://ed-fi.org/AcademicSubjectDescriptor#Composite
PSAT 10_1600,uri://ed-fi.org/AcademicSubjectDescriptor#Composite
Expand Down
13 changes: 13 additions & 0 deletions assessments/PSAT_SAT/seeds/assessmentCategoryDescriptors.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
assessmentIdentifier,edfi_descriptor
PSAT 8/9_1600,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 10_1600,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT/NMSQT_1600,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
SAT_1600,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 8/9_2400,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 10_2400,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT/NMSQT_2400,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
SAT_2400,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 8/9_Adaptive,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 10_Adaptive,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT/NMSQT_Adaptive,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
SAT_Adaptive,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
26 changes: 13 additions & 13 deletions assessments/PSAT_SAT/seeds/assessments.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
assessmentIdentifier,assessmentTitle,namespace,assessmentCategoryDescriptor
PSAT 8/9_1600,PSAT 8/9,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 10_1600,PSAT 10,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT/NMSQT_1600,PSAT/NMSQT,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
SAT_1600,SAT,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 8/9_2400,PSAT 8/9,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 10_2400,PSAT 10,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT/NMSQT_2400,PSAT/NMSQT,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
SAT_2400,SAT,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 8/9_Adaptive,PSAT 8/9,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT 10_Adaptive,PSAT 10,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
PSAT/NMSQT_Adaptive,PSAT/NMSQT,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
SAT_Adaptive,SAT,uri://collegeboard.org,uri://ed-fi.org/AssessmentCategoryDescriptor#College entrance exam
assessmentIdentifier,assessmentTitle,namespace
PSAT 8/9_1600,PSAT 8/9,uri://collegeboard.org
PSAT 10_1600,PSAT 10,uri://collegeboard.org
PSAT/NMSQT_1600,PSAT/NMSQT,uri://collegeboard.org
SAT_1600,SAT,uri://collegeboard.org
PSAT 8/9_2400,PSAT 8/9,uri://collegeboard.org
PSAT 10_2400,PSAT 10,uri://collegeboard.org
PSAT/NMSQT_2400,PSAT/NMSQT,uri://collegeboard.org
SAT_2400,SAT,uri://collegeboard.org
PSAT 8/9_Adaptive,PSAT 8/9,uri://collegeboard.org
PSAT 10_Adaptive,PSAT 10,uri://collegeboard.org
PSAT/NMSQT_Adaptive,PSAT/NMSQT,uri://collegeboard.org
SAT_Adaptive,SAT,uri://collegeboard.org
2 changes: 1 addition & 1 deletion assessments/PSAT_SAT/seeds/gradeLevelDescriptors.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
psat_sat_grade,assessmentIdentifier,gradeLevelDescriptor
psat_sat_grade,assessmentIdentifier,edfi_descriptor
1,PSAT 8/9_1600,uri://ed-fi.org/GradeLevelDescriptor#Other
2,PSAT 8/9_1600,uri://ed-fi.org/GradeLevelDescriptor#Eighth grade
4,PSAT 8/9_1600,uri://ed-fi.org/GradeLevelDescriptor#Ninth grade
Expand Down
2 changes: 1 addition & 1 deletion assessments/PSAT_SAT/templates/assessments.jsont
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"assessmentFamily": "{{assessmentFamily}}",
"namespace": "{{namespace}}",
"assessedGradeLevels": [{{grade_json}}],
"assessmentCategoryDescriptor": "{{assessmentCategoryDescriptor}}",
"assessmentCategoryDescriptor": "{{edfi_descriptor}}",
"academicSubjects": [{{subject_json}}],
"scoreResults": [
{
Expand Down
4 changes: 2 additions & 2 deletions assessments/PSAT_SAT/templates/studentAssessments.jsont
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,8 @@
"studentUniqueId": "{{studentUniqueId}}"
},
"administrationDate": "{{administrationDate}}",
{% if gradeLevelDescriptor != '' and gradeLevelDescriptor is not none and gradeLevelDescriptor | length %}
"whenAssessedGradeLevelDescriptor": "{{gradeLevelDescriptor}}",
{% if edfi_descriptor != '' and edfi_descriptor is not none and edfi_descriptor | length %}
"whenAssessedGradeLevelDescriptor": "{{edfi_descriptor}}",
{% endif %}
"scoreResults": [
{% for score in all_scores %}
Expand Down