Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve description of Generative AI elements #52

Open
schnelle opened this issue Apr 4, 2024 · 4 comments
Open

Improve description of Generative AI elements #52

schnelle opened this issue Apr 4, 2024 · 4 comments
Assignees

Comments

@schnelle
Copy link
Collaborator

schnelle commented Apr 4, 2024

The currently followed architecture of is still too closely bound to traditional NLU based voice interaction concepts. While it aimed at including LLM with speech, LLM with multimodality, ... it is potentially not visible enough.

This should be improved.

@schnelle
Copy link
Collaborator Author

schnelle commented Jun 5, 2024

Review in meetin on 2024-06-05

  • diagrams OK
  • Should add post processing component after LLM
  • Should add boxes around different dialog stacks

section 3.2.5 Prompt Adaptation

  • It may receive additional input from remote IPAs via the Provider Selection Service and may additionally augment the prompt with these inputs. --> It may receive additional input from remote IPAs via the Provider Selection Service and may additionally augment the prompt with information derived from these inputs
  • add bullet point about potential input from knowledge graphs
  • It usually receives the decoded input from the ASR --> It usually receives the decoded input from the ASR or it may receive the audio input directly if it is capable of processing it

3.2.6 LLM

  • add LLM to list of abbrreviations
  • It is not needed in NLU-based systems --> It is not needed in traditional NLU-based systems
  • Prompt aAdaptation --> Prompt Adaptation
  • It may make use of Knowledge Graphs to optimize the output, e.g. to make the output more accurate and reliable. --> It may make use of Knowledge Graphs and other post processing steps to optimize the output, e.g. to make the output more accurate and reliable.
  • last sentence should go to a new component

schnelle added a commit that referenced this issue Jun 13, 2024
schnelle added a commit that referenced this issue Jun 14, 2024
@schnelle
Copy link
Collaborator Author

Completed almost all review points.

Open

  • Update all relevant figures with layers for Generative AI and Traditional NLU
  • Clarify if dialogs are to be used for Generative AI, e.g. for error handling

@schnelle
Copy link
Collaborator Author

Comments from review

  • section 3.2.5.1 Prompt Adaptation

    • conversational state is CURRENTLY not maintained by LLM --> add a bullet point that this may be available once
  • Clarify if dialogs are to be used for Generative AI, e.g. for error handling

    • should be possble to use

@schnelle
Copy link
Collaborator Author

schnelle commented Oct 24, 2024

Review from 2024-10-23

  • section 3
    • add a note about location of components
  • section 3.2
    • add a bit more about differences between traditional NLU and Gen AI system
  • Figure 3, figure 4
    • gray out LLM and add a justification in the text
  • Add term "agentic" as this is an established term

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant