This repository contains tools for collecting and processing projects and assignments, mostly related to GitHub, GitHub Classroom, and Jupyter notebooks.
It also contains a smattering of other class-related tools, specific to the Olin College IT infrastructure.
Report issues here.
The scripts in this file are still under (occasional) active development, and are under-tested and under-documented.
$ git clone https://github.com/olin-computing/classroom-tools.gitInstall Python 3.7 or greater. [Lesser versions of Python 3 will likely work, but are untested. Python 2.x is right out.]
Testing whether Python 3.7 is installed:
- Execute
python3 --version, - Verify the output version:
$ python3 --version
Python 3.7.5- Install Poetry
poetry install
Some of these scripts use GitHub's API.
GitHub limits the rate at which a particular machine can make API calls. If you repeatedly run these scripts on a repo with many forks, you may run into these limits. (You may also run into them if you work on developing the scripts.)
You will also need a personal API token in order to access any private repositories. You will need this for use with GitHub Classroom.
To increase the limit, create a personal GitHub API
token, and set the
GITHUB_API_TOKEN environment variable to this value.
For example, my macOS and Ubuntu shells are set to zsh, so my startup files
include ~/.zshenv. My ~/.zshenv includes this line (where xxxxxxxx is my
personal GitHub API token):
export GITHUB_API_TOKEN=xxxxxxxx
The repository consists of a number of Python scripts in the scripts
directory.
Invoke a script with --help to see options that aren't listed below.
The scripts assume the following directory organization. These subdirectories are not committed to the repository.
./
├── build
│ Files that are created by script (as opposed to downloaded) are placed here.
├── config – optional configuration files
│ student-nicknames.txt
└── downloads
Scripts look here for manually downloaded files. Scripts that download files also place them here.
REPO_NAME is a GitHub repository full name, in the format
org_name/short_name. For example, this repo is
olin-computing/classroom-tools.
./scripts/clone-repo-forks REPO_NAME
./scripts/clone-repo-forks --classroom REPO_NAME
Clone the forks of a repo.
REPO_NAME is in org_name/repo_name format.
With the --classroom option, the script clones the repos org_name/repo_name-$login in the org_name account.
This is the format of repos created by GitHub Classroom.
I no longer use this script. multiclone is an alternate golang implementation, that clones the repos in parallel.
./scripts/download-repo-fork-files REPO_NAME
./scripts/download-repo-fork-files --classroom REPO_NAME
Download a repo's forks' modified files. (Files whose contents are different from the source repo.) This is suitable for collecting assignments, and quickly inspecting which files have changed.
Only files that are different from the version in the origin repository are downloaded.
With the --classroom option, the script downloads repos
org_name/repo_name-$login in the org_name account. This is the format of
repos created by GitHub Classroom.
This script also omits repos that belong to members of org_name.
(Both of these are suitable for my purposes, but could easily be turned into command-line options.)
This script downloads files into the directory structure:
./downloads/
└── ${github_organization}-${github_repo}/
└── ${student_github_id}/
└── files…
./scripts/github-fork-file-modtimes REPO_NAME
Create a spreadsheet that contains the student names and file modification dates, for each file in a forked repository.
./scripts/collate-downloaded-files (under development)
Collect downloaded notebooks into a common directory.
This script expects a directory structure created by the
download-repo-fork-files script. It creates:
./downloads/
└── ${github_organization}-${github_repo}-combined/
└── ${filename_without_extension}/
├── ${student1_github_login}.${filename_extension}
├── ${student2_github_login}.${filename_extension}
├── …
└── ${student${n}_github_login}.${filename_extension}
The name of the repository is currently hard-coded into the script.
./scripts/combine-notebooks REPO_NAME NOTEBOOK_FILE_NAME
Combine notebooks into a single notebook.
This script expects a directory structure created by the
download-repo-fork-files script. It creates:
./build/${github_organization}-${github_repo}
├── processed_notebooks/
│ └── notebook_name.ipynb
└── summaries/
├── poll1.csv
├── poll2.csv
├── …
└── poll${n}.csv
This script is derived from, and documented at, osteele/assignment-tools (which was in turn forked from paulruvolo/SoftDesSp16Prep).
The version in this directory is not the latest, and is not under active
development here. The nbcollate
package is the successor. pip install nbcollate to install this package, and the nbcollate command-line
tool.
A web application with similar functionality is at olin-computing/assignment-dashboard. That application caches the state of GitHub into a local sqlite3 store, and provides a web interface for inspect completion status by student or by question and for browsing the original and collated notebooks.
./scripts/create-enrollment-flashcards HTML_FILE
Generate a file and media directory suitable for consumption by FlashCard Deluxe, from a Course Enrollment page downloaded from the Portal.
./scripts/create-enrollment-contact-sheet HTML_FILE
Generate HTML "contact sheet" page, that displays all the student names and faces in a grid, from a Course Enrollment page downloaded from the Portal.
./scripts/summarize-scope-survey CSV_FILE
Given a SCOPE Peer and Self review spreadsheet, create an HTML report organized by student.
Some things to work on are listed here.
pip install -r requirements-dev.txt
To test in new conda environment:
$ conda create --name classroom-tools python=3.5 pip
$ source activate classroom-tools
$ pip install -r requirements-dev.txtWith exceptions listed in setup.cfg, code should conform to
PEP8,
PEP257, and the Google Python
Style Guide.
You can verify code against these style guides via:
$ pip3 install -r requirements-dev.txt # once
$ flake8 scripts # before each commit
or by setting up a git pre-commit hook to run the latter command.
These scripts are written in a Jupyter-notebook-like style, for easy development with the Hydrogen Atom plugin-in and the Python Visual Studio Code extension.
Specifically, they are light on functions and heavy on global variables.
This is an experiment, and may not have legs. For example, it would be nice to be able to re-organize the scripts as modules, and invoke them as subcommands from a single CLI entry point, or make them available to a web or desktop application. The current style may not be compatible with that.
classroom-tools/
├── build – not committed to the repo
├── config – not committed to the repo
├── downloads – not committed to the repo
├── scripts
│ Script functions, invoked from the command line.
├── src
│ Utility functions that aren't scripts; potentially shared by multiple scripts.
└── templates
HTML jinja2 template files
combine-notebooks.py is derived from a script by Paul Ruvolo at Olin
paulruvolo/SoftDesSp16Prep.
create-enrollment-flashcards is based on an idea by Ben Hill at Olin. His code
was simpler and cleaner but I added more functionality (nicknames, HTML
generation) before I saw his, and haven't yet backed out my complexity in favor
of his simplicity.
MIT