Skip to content

Commit

Permalink
add checks and CLI to README
Browse files Browse the repository at this point in the history
GitOrigin-RevId: 2d4bdb1b2d48ea70f87a1be5ede27242f712401a
  • Loading branch information
alyssachvasta authored and copybara-github committed Jan 23, 2025
1 parent fc252cc commit 253fea1
Showing 1 changed file with 16 additions and 1 deletion.
17 changes: 16 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Then to log in locally run:

`gcloud auth application-default login`

## Example Usage
## Example Usage - Javascript

Summarize Seattle’s $15 Minimum Wage Conversation.

Expand Down Expand Up @@ -90,6 +90,21 @@ const summary = mySensemaker.summarize(
);
console.log(summary.getText("MARKDOWN"));
```
## CLI Usage
There is also a simple CLI set up for testing. There are two tools:
- ./runner-cli/runner.ts: takes in a CSV representing a conversation and outputs an HTML file containing the summary. The summary is best viewed as an HTML file so that the included citations can be hovered over to see the original comment and votes.
- ./runner-cli/rerunner.ts: takes in a CSV representing a conversation and reruns summarization a number of times and outputs each of the summaries in one CSV. This is useful for testing consistency.
## Running the Checks
In the ./evals directory there are a number of checks that can be run on an unlabeled conversation. There are three categories of checks:
- Monitoring Checks: summary generation failure rate and time to run
- Quick Checks: whether the summary has an intro and conclusion, and whether all the topics and subtopics from categorization are present
- Qualitative Checks: measures how often each group is mentioned
All three checks are run using the ./evals/run_checks.ts script.
## Making Changes to the tools - Development
Expand Down

0 comments on commit 253fea1

Please sign in to comment.