Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source prompting #40

Merged
merged 20 commits into from
Jul 24, 2024
Merged

Source prompting #40

merged 20 commits into from
Jul 24, 2024

Conversation

alnutile
Copy link
Contributor

@alnutile alnutile commented Jul 20, 2024 •
edited
Loading

Thanks to customer feedaback I am jumping on this big feature.

Any source will be 100% driven by the Prompt the user put.

So Source Email, Web, Search etc all gets passed to an LLM to do something with the text.

For example you want to check emails about xyz only, and if it is an email about that the prompt will process that text and turn it into a document in your collection. Or scrape this site for event data, the LLM will then use the HTML to just get you the event data.

There is a lot to do here:

  • All Sources Converted to this pattern (false or not, array or not)
    • WebPageSource
    • WebSearchSource
    • WebhookSource
    • SiteMapSource
    • JsonSource
    • GoogleSheetSource
    • FeedSource
    • EmailSource
    • EmailBoxSource
  • All Sources allow for a way to NOT delete emails or NOT take in data kinda of like a false return to not save as document
  • UI will surface this easily for the user so they are empowered to do this work
  • Prompts can product Array
  • SourceTask to prevent repeat on emails without deleting or marking emails.
  • Chat now shows Sources and Messages to show a log of activity to the user
  • Chunking is a generic job since all of this is text after this point

alnutile added 20 commits July 20, 2024 09:25
@alnutile
ok fix up some things thanksto stan
0bfed83
@alnutile
email source working with false option that is great
345972a
@alnutile
Ok Source works for EmailSource and Prompting now to move it to all o…
685e59c 
…ther Soruces
@alnutile
ok now to track so all sources know not to repeat
ad99c93
@alnutile
ok we track the source task in the model now we can repeat check a so…
dc71720 
…urce with different source getters eg an email box might have a source for marketing a source for customer feedback
@alnutile
ok we track the source task in the model now we can repeat check a so…
126699d 
…urce with different source getters eg an email box might have a source for marketing a source for customer feedback
@alnutile
The Email Assistant and Email Box are both working
3ce0da5
@alnutile
adds new source pattern to Get Web Feed
3f4e377
@alnutile
move Webpage source to the new pattern
e7596c3
@alnutile
move Webpage source to the new pattern
23726d6
@alnutile
ok the list feature is working now too from array
b86da75
@alnutile
add new prompt template for event data
51d554a
@alnutile
add new prompt template for event data
f31c435
@alnutile
Ok web source now has array
92b99f7
@alnutile
fix test
379b011
@alnutile
merge main
a12f32d
@alnutile
working on the webhooks interface
5ffd955
@alnutile
webhook working
b859925
@alnutile
Wow!
3c3f73a 
All sources now have the new feature to Prompt values out of the data.
Also to return false if the data does not have what you want
Lastly you can prompt out arrays to make more than one document out of that data.
@alnutile
fix the npm fail
e75649f
@alnutile alnutile merged commit 54aea8e into main Jul 24, 2024
2 checks passed
@alnutile alnutile deleted the source_prompting branch July 24, 2024 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@alnutile