Skip to content

Commit

Permalink
Move datasets to project (#1365)
Browse files Browse the repository at this point in the history
* move dataset from team to project

* move dataset from team to project

* move DatasetSourceType to lib

* add datagen to DatasetSourceType

* fix linter

* fix linter

* Update cloud_quickstart_tracing.md

* Update tutorial_tracing.md

* Update set_up_tracing.md

---------

Co-authored-by: Emeli Dral <[email protected]>
  • Loading branch information
svetleo-evidently and emeli-dral authored Nov 19, 2024
1 parent 3700d9e commit 013089f
Show file tree
Hide file tree
Showing 8 changed files with 111 additions and 73 deletions.
12 changes: 4 additions & 8 deletions docs/book/examples/tutorial_tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,12 +73,12 @@ It is recommended to pass the key as an environment variable. [See Open AI docs]

Set up configuration details:
* The `address` is the destination backend to store collected traces. In this case, it is Evidently Cloud.
* `Team_id` is the identification of the Evidently Team which defines who can access collected traces and helps organize your Projects. Go to the [Team page](https://app.evidently.cloud/teams), enter the selected Team and copy its ID.
* `Project_id` is the identification of the Evidently Project. Go to the [Projects page](https://app.evidently.cloud/), enter the selected Project and copy its ID.
* `Dataset_name` helps identify the resulting Tracing dataset. All data with the same ID would be grouped into single dataset.

```python
address="https://app.evidently.cloud/"
team_id="YOUR_TEAM_ID"
project_id="YOUR_PROJECT_ID"
dataset_name="YOUR_TRACING_DATASET_NAME"
```

Expand All @@ -87,7 +87,7 @@ Initialize tracing:
init_tracing(
address=address,
api_key=my_token,
team_id=team_id,
project_id=project_id,
export_name=dataset_name
)
```
Expand Down Expand Up @@ -196,12 +196,8 @@ Preview the dataset with `traced_data.head()`.
You can run evaluations on this dataset using the Evidently Python library. You can generate the Reports to view them locally or send them to Evidently Cloud.
For example, let’s evaluate the length and sentiment of the responses, and whether they include the word "Certainly".

Create an evaluation Project in Evidently Cloud. This will allow you to organize your evaluations and track results over time.

```python
project = ws.create_project("Trace Evals", team_id=team_id)
project.description = "Evaluation example"
project.save()
project = ws.get_project(project_id)
```

Define the evaluations:
Expand Down
4 changes: 2 additions & 2 deletions docs/book/get-started/cloud_quickstart_tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Need help? Ask on [Discord](https://discord.com/invite/xZjKRaNp8b).
Set up your Evidently Cloud workspace:
* **Sign up**. If you do not have one yet, sign up for a free [Evidently Cloud account](https://app.evidently.cloud/signup).
* **Create an Organization**. When you log in the first time, create and name your Organization.
* **Create a Team**. Click **Teams** in the left menu. Create a Team, copy and save the Team ID. ([Team page](https://app.evidently.cloud/teams)).
* **Create a Project**. Click **+** button under Project List. Create a Project, copy and save the Project ID. ([Projects page](https://app.evidently.cloud/))
* **Get your API token**. Click the **Key** icon in the left menu. Generate and save the token. ([Token page](https://app.evidently.cloud/token)).

You can now go to your Python environment.
Expand Down Expand Up @@ -62,7 +62,7 @@ Set up tracing parameters. Copy the Team ID from the [Teams page](https://app.ev
init_tracing(
address="https://app.evidently.cloud/",
api_key="EVIDENTLY_API_KEY",
team_id="YOUR_TEAM_ID",
project_id="YOUR_PROJECT_ID"
export_name="LLM tracing example"
)
```
Expand Down
4 changes: 2 additions & 2 deletions docs/book/tracing/set_up_tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Use `init_tracing` to enable tracely tracing. Example:
init_tracing(
address="https://app.evidently.cloud/",
api_key=”YOUR_EVIDENTLY_TOKEN”,
team_id="YOUR_EVIDENTLY_TEAM_ID",
project_id="YOUR_PROJECT_ID",
export_name="YOUR_TRACING_DATASET_NAME",
)
```
Expand All @@ -39,7 +39,7 @@ init_tracing(
| `address: Optional[str]` | The URL of the collector service where tracing data will be sent. For Evidently Cloud, set `https://app.evidently.cloud/`.<br>**Required:** No, **Default:** `None` |
| `exporter_type: Optional[str]` | Specifies the type of exporter to use for tracing. Options are `grpc` for gRPC protocol or `http` for HTTP protocol.<br>**Required:** No, **Default:** `None` |
| `api_key: Optional[str]` | The authorization API key for Evidently Cloud tracing. This key authenticates your requests and is necessary for sending data to Evidently Cloud.<br>**Required:** No, **Default:** `None` |
| `team_id: Optional[str]` | The ID of your team in Evidently Cloud. This is used to organize and group tracing data under your specific team in the Evidently Cloud UI.<br>**Required:** No, **Default:** `None` |
| `project_id: str` | The ID of your Project in Evidently Cloud. <br>**Required:** Yes, **Default:** `None` |
| `export_name: Optional[str]` | A string name assigned to the exported tracing data. All data with the same `export_name` will be grouped into a single dataset.<br>**Required:** No, **Default:** `None` |
| `as_global: bool = True` | Indicates whether to register the tracing provider globally for OpenTelemetry (`opentelemetry.trace.TracerProvider`) or use it locally within a scope. **Default:** `True` |

Expand Down
Loading

0 comments on commit 013089f

Please sign in to comment.