- Add array column
cohort_year_arraytodim_student, tracking student cohort designation, and add upstreambld_ef3__student_cohort_years - Add support for custom indicators on
dim_course_section, and companion audit table for testing uniqueness of custom data sources - Add
section_typedescriptor column todim_section(Ed-Fi Data Standard v5.0 addition) - Add
preferred_first_name,preferred_last_name, andgender_identitycolumns todim_staff(Ed-Fi Data Standard v5.0 additions) - Add
preferred_first_name,preferred_last_name, andgender_identitycolumns todim_parent(Ed-Fi Data Standard v5.0 additions)
- Change the source of
dim_parenttostg_ef3__contactsandfct_student_parent_associationtostg_ef3__student_contact_associationsdue to the rename from parent to contact in Ed-Fi data standard v5.0. - Add additional foreign key declarations to
fct_student_discipline_actions,fct_student_discipline_actions_summary,fct_student_discipline_incident_behaviors - Update package dependency
dbt_utilsto 1.3.0, including alignment to renamedgenerate_surrogate_key()macro. Note, this change now treats nulls and empty strings as distinct values in surrogate key generation.
- Fix model name in yaml documentation file for
dim_graduation_plan - Fix unique key test for recently changed unique key fo
fct_student_school_attendance_event
- Fix
bld_ef3__wide_school_network_assocto group across years, to correctly remove duplicates onk_school. Previously, incorrect duplicate records were created indim_schoolin cases where multiple network types are configured inxwalk_network_association_types.
- Add
fct_student_diplomaand a companion test for monitoring deduplicated data -diploma_record_duplicates - Add
dim_graduation_planand reference viafct_student_school_association.k_graduation_plan - Add some notes to dbt docs for
fct_student_special_education_program_associationandfct_student_section_association - Add
v_earned_additional_creditstofct_course_transcript
- Modify the join in
bld_ef3__stu_race_ethnicityso that students with unknown race are included and{{ var("edu:stu_demos:race_unknown_code") }}is correctly applied
- Rework and rename pivot macro to
ea_pivot()to simplify usage - Add
k_leaandk_schooltodim_course. Note - downstream queries that referencek_leaork_schoolwithout an explicit qualified column reference may break due to this change. - Add macro call that brings through extensions to all fct tables that directly reference a stg table. See here for example.
- Note: this may break in certain edge cases, if your implementation has existing configured extensions whose names collide with column names that already exist in the related fct table. This should be rare.
- Add extension columns (optional, if configured) to all fct tables. If no extensions configured, this code compiles to nothing.
- Add
has_hispanic_latino_ethnicitytodim_student. Also include in this in fields that are "immutable" (consistent across years), assuming variableedu:stu_demos:make_demos_immutableis set toTrue. - Add configurable student language columns to
dim_student. - Add configurable custom override for student grade level (some source other than student-school-assoc in Ed-Fi). Use variable
edu:stu_demos:grade_level_overrideto configure a data source and column. - Add configurable logic for override of
school_yearinfct_student_assessmentandfct_student_objective_assessment(e.g. if NULL, use thresholds to populate). By default, no override is done, but options were added to use a global variableedu:school_year:start_month,edu:school_year:start_day, or use a xwalk of date rangesxwalk_assessment_school_year_dates. - Make
stu_display_nameconfigurable (e.g. include suffix or preferred_name)
- Correct dbt docs for unique key of
fct_student_daily_attendance. - Add
k_schoolto grain of qc modelattendance_freshness.- Also restrict
attendance_freshnessmodel to current or past dates - And add column
days_since_last_attendance_eventtoattednance_freshness_test
- Also restrict
- Remove
is_latest_recordfrom auto-creation of subgroups fordim_subgroupandfct_student_subgroup.
- Add
dim_staff.race_ethnicity, using rules analogous todim_student.race_ethnicity
- Standardize ESC naming and add
k_escreference todim_lea - Add integer grade level to
dim_studentandfct_student_school_association - Add
k_student_xyeartofct_student_assessment,fct_course_transcripts,fct_student_academic_record, andfct_student_gpa. - Discipline updates:
- Add ability to report all disciplines/behaviors for a single discipline action event and all disciplines/behaviors for a single discipline incident in new, separated models.
- See more info in PR description #75
- Add some comments for existing value tests
- Add feature to conditionally remove xyear enrollments in
fct_student_school_association
- Make
k_studentnull where not joinable todim_student, forfct_student_assessment,fct_course_transcripts,fct_student_academic_record, andfct_student_gpa.
- Fix unique key of
fct_student_school_attendance_eventandfct_student_section_attendance_eventto reflect unique key in Ed-Fi.- Note: The unique key of
fct_student_daily_attendanceremains the same(k_student, k_school, calendar_date). But the way we deduplicate to this point has changed to be more consistent - The scale of impact of this fix depends on how often duplicates are found on
(k_student, k_school, calendar_date)in the underlying data. See new testanalytics.prod_dbt_test__audit.attendance_event_duplicatesfor more information.
- Note: The unique key of
- Update packages.yml version range for edu_wh to
[ ">=0.3.0", "<0.4.0" ] - Configure xwalk_grade_levels. Can copy directly from project template here,
but also check your implementation's grade level descriptors
select * from analytics.prod_stage.int_ef3__deduped_descriptors where namespace ilike '%GradeLevelDescriptor%';. If this shows custom/local descriptors, make sure to add those to the xwalk.- After running the xwalk, you can also do a check for missing values with
select * from analytics.prod_wh.dim_student where grade_level_integer is null;
- After running the xwalk, you can also do a check for missing values with
- (requires Snowflake sysadmin or transformer_prod permissions) Drop the following deprecated tables:
analytics.prod_wh.dim_education_service_centeranalytics.prod_wh.dim_discipline_incidentsanalytics.prod_wh.fct_student_discipline_incident_behaviors_actions
- Bugfix release
- Add ability to extend
dim_courseanddim_class_periodwith external data sources - Add an
is_latest_recordindicator todim_studentto identify the demographics from the most recent school year in which a student appeared
- In
dim_student, choose the grade level from the most recent school enrollment, rather than the longest duration. This better aligns with grade transition patterns seen in the wild. - Deprecate the
rlsschema in favor of extensions that cover this use-case
- Add
dim_learning_standardandfct_student_learning_standard_grades- For Learning Standard based grading, and for future support of learning standards in other areas of the warehouse
- Add
dim_cohortandfct_student_cohort_association- Not widely in use, but intended to support more complex authorization models in the future
- Allow overriding the source of daily attendance models, for more complex customizations of the daily attendance calculation
- Add incident_time to dim_discipline_incidents
- Allow some student demographics to be 'immutable', in the sense that they are stable across time
- Allow custom indicators on program memberships
- Make it optional to exclude student-school enrollments where the student exited before the first day of school
- Allow a rule where multiple calendar codes must be assigned to a date for it to be counted as a school day
- Add primary disability on student special education program associations
- Documentation improvements
- Create wide indicators from course/section characteristics in
dim_course_section
- Add defaults for discipline non-offender codes
- Minor tweaks to documentation, configuration
- Allow assessment results to come through even if they don't match a xwalk entry
- Add
k_schoolto more models - Add configuration to customize the preferred email address for staff
- Restrict the relationships between tenant and LEAs/schools
- New models covering the relationship between discipline incidents, actions, and behaviors
- Extend documentation coverage
- accordion_columns gains ability to coalesce null values with a default
- Fill False instead of NULL in some student demographics
- Configure non-offender codes. This allows for navigating the Ed-Fi deprecation
of the old student discipline model. The new model separates offenders from
non-offendors, and this config allows us to apply the same logic to the
now-deprecated model. Any
DisciplineIncidentParticipationCodeDescriptorsin use instudentDisciplineIncidentAssociationsthat refer to non-offenders should be included in this list. See here - New crosswalk for ranking the severity of Behaviors (template here). This facilitates the analysis of incidents involving multiple behaviors.
fct_discipline_actionshas been renamed tofct_student_discipline_actions. Any references to this model will need to be updated.
- Add models for education service centers
- Add support for custom indicators on dim_school
- More support for generic student programs
- More robust creation of first/last day of school table for calendaring/enrollment logic
- Generate foreign keys on optional school-network associations for BI tools
- Implement "first day school" rule for single-calendar schools in fct_student_school_association
- Allow extension columns to correctly be pulled into the stacked fct_program_service table
- Fix join between staff assignments and staff-school associations
- Bugfix release: package specification
- Add fact table for student GPAs
- Add additional program-type fact tables
- Add configurable program parsing to dim_student
- Add combined program services table
- Add minimum dbt version requirement
- Switch to ranged version pin for edu_edfi_source
- Add optional domain disabling to all non-core models, using vars in dbt_project.yml.
- Remove student_school_associations with invalid dates, such as withdraw dates before the first day of school, or before the entry date
- Add audit table to capture all invalid school enrollments
- Rework daily attendance model
- Allow for fractional attendance by changing indicators from bools to floats
- Roll attendance rates for exited students forward to the end of the year so that chronic absenteeism metrics can include exited students in daily calculations
- Required Migration: Convert T/F values in
xwalk_attendance_eventsto 1.0/0.0
- Add a single model properties file under each subdirectory, as per DBT recommendation.
- Change absenteeism metric boundaries to better match common business rules
- Improve whitespace handling and macro robustness for empty crosswalks
- Warehouse models for parents and student-parent linkage
- Bump edu_edfi_source version
- Bump edu_edfi_source version
- Added parsing for
studentEducationOrganizationAssociation.indicatorsindim_student- Allows arbitrary student indicators to be mapped into dim_student columns via a xwalk in the project template
- Fact and dimension tables for assessments
- Preserves all score results and performance levels, while allowing a customizable set to be pulled out as columns
- Added student ids to dim_student
- Fixed typo regarding Chronic Absenteeism buckets
- Add program services
- Changed chronic absenteeism threshold to be inclusive to better align with common standards
- Added more columns to attendance tables from source data
- Improved handling of extensible column-sets, such that all are optional
- Added macro
accordion_columnsto help with these cases
- Added macro
- Fixed chronic absenteeism labeling issue
Initial release