-
Notifications
You must be signed in to change notification settings - Fork 3.1k
docs(): v0.3.11 DataHub Cloud Docs #13439
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
e661955
docs(): v0.3.11 DataHub Cloud Docs
david-leifker 51f95d5
update sidebar
david-leifker 4b16164
add release notes for cloud heroes
chriscollins3456 9896dff
Update v_0_3_11.md
david-leifker ca4f76c
Update v_0_3_11.md
david-leifker 0fc0a29
Update v_0_3_11.md
david-leifker 2c877f8
Update v_0_3_11.md
david-leifker b777bcd
Update v_0_3_11.md
jjoyce0510 4726ddb
andrew release notes; refactor emitmode docs
asikowitz 56c022a
add note for proposals
chriscollins3456 0034ef3
Update v_0_3_11.md
david-leifker 7823564
Update v_0_3_11.md
david-leifker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
# v0.3.11 | ||
|
||
--- | ||
|
||
## Release Availability Date | ||
|
||
12-May-2025 | ||
|
||
## Recommended CLI/SDK | ||
|
||
1.0.0.3 | ||
|
||
## Helm Chart | ||
|
||
1.5.32 | ||
|
||
## Known Issues | ||
|
||
## Release Changelog | ||
|
||
--- | ||
|
||
### v0.3.11 | ||
|
||
- Breaking Changes | ||
|
||
- #13004: The `acryl-datahub-airflow-plugin` dropped support for Airflow 2.3 and 2.4. | ||
- #13186: NoCode Migration Removed - This code hasn't been required in many years. If needed, a user should upgrade to DataHub 1.0.x prior to upgrading to a later version. | ||
- #13397: `async_flag` removed from rest emitter, replaced with emit mode ASYNC | ||
|
||
- Bug Fixes | ||
|
||
- [Forms] Fix issues with editing metadata on an asset's sidebar while you're filling out a form for that asset. | ||
- [Forms] Fixes a bug in our validator ensuring there are not duplicate prompt IDs between Forms. | ||
- [Structured Properties] Fix edge case where an Entity type Structured Property can be edited to support no entities as allowed values. | ||
- [Structured Properties] Fix issue where Structured Properties were not appearing in Column sidebar even when settings were enabled. | ||
- [Users] Fix issue where user avatars were occasionally showing up blank due to broken URL. | ||
- [Dev Tools] Fixes for GraphiQL interface | ||
- [OpenAPI] #13047 Fix OpenAPI endpoint returning historical aspect versions | ||
- [OpenAPI] #13095 Remove required fields when default present in PDL | ||
- [Validation] Correct entity type name casing from URNs | ||
- [Ebean] #13392 Handle too large SQL statements | ||
- [Lineage] #13257 Apply more specific filters when querying elasticsearch for lineage | ||
- [Lineage] #13424 The "Show Hidden Edges" toggle is now local-storage sticky like the other toggles | ||
- [Lineage] #13323 Fixes bug with edges through queries disappearing when hiding transformations | ||
- [Stats] #13327 Fixes a bug where the incorrect total table count was displayed for Snowflake tables | ||
|
||
- Product | ||
|
||
- **Rebrand to DataHub Cloud**: To show our commitment to the success of the open source **DataHub Community**, we've officially rebranded Acryl to **DataHub Cloud**! 🎉🎉🎉 This will result in a change of the logos & copy visible across the product. Learn more at [datahub.com](https://datahub.com). | ||
- **Search Bar**: Introducing a brand new search bar experience with sleeker designs, more functionality such as filtering, and consistent results with what users expect compared to search page results. | ||
- **Glossary**: This release enhances our Business Glossary with improved scaling capabilities, ensuring support for glossaries of any size. We've also made several UX improvements, including a new search bar when browsing Terms within a Term Group. | ||
- **Tags Management**: This release includes a Tags management experience. This appears in the left navigation bar under the **Tags** section. To view tags management, users must have the new `View Manage Tags` platform privilege. To edit all tags, users must have the new `Manage Tags` platform privilege. | ||
- **Lineage**: Lineage nodes support a double click action to change the home (focused) node. | ||
|
||
- Ingestion | ||
|
||
- #13397 - Ingestion Rest Emitter - Added EmitMode parameter for write guarantees (>= 1.0.0.4) | ||
|
||
- SYNC_WAIT: Synchronously updates the primary storage (SQL) but asynchronously updates search storage (Elasticsearch). Provides a balance between consistency and performance. Suitable for updates that need to be immediately reflected in direct entity retrievals but where search index consistency can be slightly delayed. | ||
- SYNC_PRIMARY: Synchronously updates the primary storage (SQL) but asynchronously updates search storage (Elasticsearch). Provides a balance between consistency and performance. Suitable for updates that need to be immediately reflected in direct entity retrievals but where search index consistency can be slightly delayed. | ||
- ASYNC: Queues the metadata change for asynchronous processing and returns immediately. The client continues execution without waiting for the change to be fully processed. Best for high-throughput scenarios where eventual consistency is acceptable. | ||
- ASYNC_WAIT: Queues the metadata change asynchronously but blocks until confirmation that the write has been fully persisted. More efficient than fully synchronous operations due to backend parallelization and batching while still providing strong consistency guarantees. Useful when you need confirmation of successful persistence without sacrificing performance. | ||
|
||
Limitations: | ||
|
||
- Async modes (ASYNC / ASYNC_WAIT) are impacted by kafka lag. | ||
- SYNC_WAIT is only available with OpenAPI ingestion | ||
- ASYNC_WAIT: in some cases, must complete at least one ingestion run with the new service. | ||
|
||
```python | ||
from datahub.emitter.mce_builder import make_dataset_urn | ||
from datahub.ingestion.graph.client import get_default_graph | ||
from datahub.emitter.rest_emitter import EmitMode | ||
from datahub.emitter.mcp import MetadataChangeProposalWrapper | ||
from datahub.metadata.schema_classes import DatasetPropertiesClass | ||
|
||
from datahub.emitter.rest_emitter import DatahubRestEmitter | ||
|
||
graph = get_default_graph() | ||
|
||
dataset_properties = DatasetPropertiesClass(description="Testing openapi tracing") | ||
metadata_event = MetadataChangeProposalWrapper( | ||
entityUrn=make_dataset_urn("test", "openapi.tracing"), | ||
aspect=dataset_properties, | ||
) | ||
|
||
graph.emit_mcp(metadata_event, emit_mode=EmitMode.ASYNC_WAIT) | ||
``` | ||
|
||
- Platform | ||
- **Spring**: Spring Boot 3.4 | ||
- **OpenAPI**: Audit Events and #13179 Events API | ||
- **OpenAPI**: #13165 Patch Entity Added | ||
- **API Tracing**: Patch support | ||
- **Lineage**: #13257 Updated lineage query efficiency | ||
- **SSO**: Support for OIDC Implicit auth flow | ||
- **Base Image**: Updated python base images to ubuntu 24.04 | ||
- **OpenSearch**: Enable zstd compression on reindex/new indices |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.