2024-06-05: Conform app to new AI Chat Protocol

The big change in the latest release is that the app now confirms to the official Microsoft AI Chat Protocol, documented here:
https://github.com/microsoft/ai-chat-protocol/tree/main/spec#readme
There is also a JS SDK for that protocol that we will adopt soon to simplify our frontend parsing code.
You can see the pull request for more details, but basically:

• Instead of sending "stream": True to get a streaming response, we send to a different path, "/chat/stream".
• Instead of passing down the full choices list from the OpenAI response, we pass down only the first choice, specifically it's message or delta.

We have also updated the evaluator tool to work with either the new version of the backend protocol or the old, configurable via JMESPath expressions in the config JSON.

This change also includes a revamp of the Developer Settings to include tooltips, to try and make it more clear what each setting does.

What's Changed

• Bump the python-requirements group with 26 updates by @dependabot in #1673
• Update deploy_features.md by @hadirgax in #1689
• Add clickable help icons for developer settings by @pamelafox in #1522
• Upgrade to latest version of AI Chat Protocol by @pamelafox in #1682

New Contributors

• @hadirgax made their first contribution in #1689

Full Changelog: 2024-06-03b...2024-06-05

2024-06-03b: Speech output with browser

You can now optionally enable speech output using the Browser SDKs, as opposed to using the Azure Speech SDK. The speech output is generally not as fluid, but it's entirely free, so it's a great zero-cost option to increase accessibility. See docs at:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#speech-output

We've also added documentation about a Python-based load balancer for OpenAI instances. See:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/productionizing.md#openai-capacity

 ## What's Changed

• Update productionizing.md by @simonkurtz-MSFT in #1677
• Enable diagnostics for search service by @pamelafox in #1676
• feat: add low cost browser text to speech output using WebSpeechAPI by @john0isaac in #1671

New Contributors

• @simonkurtz-MSFT made their first contribution in #1677

Full Changelog: 2024-06-03...2024-06-03b

2024-06-03: GPT-4O support for optional vision feature

The optional vision feature now uses GPT-4O instead of GPT4-turbo-with-vision, due to much better performance.

Unfortunately, due to the difference in region availability, it may be difficult to switch to GPT-4O in your existing resource groups, depending on what region you originally picked. You may need to change the region for the OpenAI resource group in .azure/ENV-NAME/config.json, delete the previous OpenAI instance, and deploy a whole new OpenAI instance.

We did discover one vision-related splitting issue while debugging this change.

See pull request for performance statistics and more details on the change.

What's Changed

• Fix overlap percentage calculation by @bastbu in #1668
• Bump the python-requirements group across 1 directory with 17 updates by @dependabot in #1665
• Use gpt-4o for vision approach by @pamelafox in #1656

New Contributors

• @bastbu made their first contribution in #1668

Full Changelog: 2024-05-29...2024-06-03

2024-05-29: Speech Input/Output

The big feature in this release is the option to add speech input (via the browser) and speech output (via Azure speech SDK). See how to enable the new features in the guide here:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-speech-inputoutput

This release also includes some updates for the chat-with-vision approach.

What's Changed

• Don't set bypass rules for ComputerVision type of Cognitive Services by @pamelafox in #1657
• Update GPT-4 version in readme by @pamelafox in #1663
• Use chat model for query rewriting by @pamelafox in #1659
• Add speech recognizer and synthesis on browser interface by @sowu880 in #113

New Contributors

• @sowu880 made their first contribution in #113

Full Changelog: 2024-05-28...2024-05-29

2024-05-28: Security improvements for Document Intelligence, Free Search Tier

The most notable changes in this release relate to security:

• Disablement of API key based access for Azure Document Intelligence. Our repository does not use keys to authenticate to the DI service, only managed identity, so keys should not be needed. If you use the same Document Intelligence instance for other projects and need key-based access for it, then you should set "disableLocalAuth: false" in the Bicep.
• Removal of search keys for free tier of search service. I originally mistakenly thought that the free tier of search service did not support managed identity, but it's actually the case that it doesn't support outbound managed identity, relevant if you're using indexers or skills with it. It does support inbound managed identity, so we removed the key and key vault for that feature. This won't affect most developers, since the default deployment uses the standard tier.

What's Changed

• Fix Add Default Key for Retrieval Mode to Persist the Change by @john0isaac in #1609
• Additional pointers to docs on private endpoints and load balancing by @pamelafox in #1618
• Removing unneeded key for free search service by @pamelafox in #1620
• Bump the python-requirements group across 1 directory with 65 updates by @dependabot in #1623
• Add load testing user class for gpt-4-vision use case by @pamelafox in #1633
• Fix samples browser by @diberry in #1631
• Fix samples browser - bookmarks by @diberry in #1637
• Improve token counting for messages with package by @pamelafox in #1577
• Fix output variable for global document access by @mattgotteiner in #1645
• Updates to login docs (login step, Entra branding) by @pamelafox in #1574
• Disable key auth for Azure Document Intelligence by @pamelafox in #1629

Full Changelog: 2024-05-15B...2024-05-28

2024-05-15B: Deploying with private access

We've now made it possible for you to deploy this app with public access disabled, using Azure private endpoints and private DNS Zones. For more details, read the private deployment guide.

As always, please report any issues you encounter when trying this new feature.

What's Changed

• Optional deployment of Private Networks, Private Endpoints plus optional configuration of an ACL rule for all backend services by @tonybaloney in #864

Full Changelog: 2024-05-15...2024-05-15B

2024-05-15: Authentication improvements for public documents

This release adds new environment variables for more flexibility when using user authentication and user access control. Here is the updated section of the documentation about authentication section:

1. (Optional) To require access control when using the app, run azd env set AZURE_ENFORCE_ACCESS_CONTROL true. Authentication is always required to search on documents with access control assigned, regardless of if unauthenticated access is enabled or not.
2. (Optional) To allow authenticated users to search on documents that have no access controls assigned, even when access control is required, run azd env set AZURE_ENABLE_GLOBAL_DOCUMENT_ACCESS true.
3. (Optional) To allow unauthenticated users to use the app, even when access control is enforced, run azd env set AZURE_ENABLE_UNAUTHENTICATED_ACCESS true. AZURE_ENABLE_GLOBAL_DOCUMENT_ACCESS should also be set to true if you want unauthenticated users to be able to search on documents with no access control.

See the full steps here:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/login_and_acl.md#automatic-setup

Please file issues if you see any problems with the new capabilities.
 

What's Changed

• Add note about OPENAI_HOST and azd deployment by @pamelafox in #1479
• Update customization.md - fix broken doc links by @tamram in #1592
• Add roles descriptions to roles.sh by @cedricvidal in #1583
• Escape single quote marks for search filters by @pamelafox in #1599
• Updating load balancing instructions by @andredewes in #1598
• Allow public documents when authentication is enabled by @mattgotteiner in #1576

New Contributors

• @tamram made their first contribution in #1592
• @cedricvidal made their first contribution in #1583

Full Changelog: 2024-04-25...2024-05-15

2024-04-25: Bugfixes for free sku, ADLS2, storageUrl

Today's release includes a variety of bugfixes:

What's Changed

• Fix gunicorn config to use single worker for free sku by @pamelafox in #1551
• Adding missing adls2 variables to azd pipeline by @pamelafox in #1544
• Adding user-upload as a feature in sample comparison by @pamelafox in #1542
• Disable OpenAI key access by @pamelafox in #1555
• Add storageurl field if missing in index by @pamelafox in #1556

Full Changelog: 2024-04-19...2024-04-25

2024-04-19: Improved security for user-uploaded documents

This release improves the security of the optional user upload mechanism by adding a storageUrl field to disambiguate between user-uploaded files and admin-uploaded files with the same name. Now when running manageacl.py, you must specify the full storageUrl whose ACLs require updating.
If you have an existing search index from before this release, please follow steps in README to add the new storageUrl field and backfill it to existing chunks in the index:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-user-document-upload

What's Changed

• Adds storageURL field to track file location by @pamelafox in #1535
• Disable key-based storage account access in Bicep by @pamelafox in #1518
• Bump idna from 3.6 to 3.7 in /app/backend by @dependabot in #1525
• Bump vite from 4.5.2 to 4.5.3 in /app/frontend by @dependabot in #1488
• Fix gpt-4 vision link by @adamdougal in #1546

New Contributors

• @adamdougal made their first contribution in #1546

Full Changelog: 2024-04-10...2024-04-19

2024-04-10: Graph SDK for auth setup

We are now using msgraph-sdk instead of direct HTTP calls for our scripts that setup Entra application registrations. There should be no functional difference after the change.

What's Changed

• Bump pillow from 10.2.0 to 10.3.0 in /app/backend by @dependabot in #1486
• Port to the Graph SDK for authentication scripts by @pamelafox in #1510

Full Changelog: 2024-03-09...2024-03-10