Add support for Azure, OpenAI, Palm, Anthropic, Cohere, Replicate Models - using litellm #37

ishaan-jaff · 2023-08-05T00:04:06Z

This PR adds support for models from all the above mentioned providers using https://github.com/BerriAI/litellm/

Here's a sample of how it's used:

from litellm import completion, acompletion

## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# async openai
response = acompletion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion("command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

Identify the garage door by name Improve README.md Fix default password in vpn-server

Add a feature to send emails from Jarvis Avoid creating api/logs directory when running on limited mode Moves iOS related operations to a dedicated module Enable optional strict word match in word_match.py on top of regex Remove phrase being split on 'also' and make it a single command Fix a bug in windows brightness where increase and decrease were swapped Remove duplicate function arguments in listener.py Add potential future work

Remove looping in garage module when device is chosen Send an email when vehicle is unlocked remotely Include timezone in vehicle's guardian mode response Create a dedicated module for functions that only uses builtins

Change module names to make better sense

Update gmail-connector to the latest version

Set to limited mode by default for Linux Update install.sh

Write processes.yaml file regardless of limited mode state

Remove CHANGELOG from pypi and link it to github Add more classifiers for pypi

Remove forcing limited mode for linux systems Fix a bug in surveillance mode session check Restrict alarm and reminder features in limited mode Add an option to set voice and quality for speech synthesis Delete docker container spun up for speech synthesis when stopped Block ALSA errors in Linux OS during start up Simplify models.py and update install.sh Initiate speech synthesis even in limited mode Include local changes when asked for Jarvis' version Add distributor info for linux systems

Rename automation.yaml instead of removing if invalid Allow an option to brute force non-limited mode Fix release notes hyperlink in pypi Update README.md

Add one click copy to emailOTP.html (Unsupported in emails)

Implement a better way to get audio IO devices info

Add a condition check for weather location to avoid crash Remove the usage of `time.perf_counter()` Optimize globally accessible variables

Restructure the usage of `smart_devices.yaml` file Update README.md and requirements.txt

Check for existing app before launching `Home` Fix method fetching current app on TV

Iterate over a loop to power on and launch home for RokuTVs Improve type hints

More clear logging and inline comments

Move background tasks from thread to process as `called_by_offline` flag should not be set in main process Running it as a thread will raise `RuntimeError` as `runAndWait()` will not be called Remove unsafe code in `pluralize` function

Run `wifi_connector` as a process to let `socket` sleep Add an option to set hours to ignore in background tasks Upgrade `PyAudio` for `macOS` More consistent logging Get CSS and JS required for night mode in robinhood.html via URL Ignore speaker.run when called by offline Update README.md

Override logging filter to add process name Create process mapping file along with the components handled Use timed wait to optimize CPU utilization on long-running processes

Make API docs page look neater

Custom keyword mapping being overwritten Writing frequently used functions in yaml file Weather breaking when phrase is available but not place Remove redundant regex when getting capitalized

Change some HTTP requests methods from POST to GET calls Add an option to disable background tasks on demand Make `yaml` files in fileio directory to be available via API calls Update README.md

Fix native audio conversion in tts_stt.py Get plain text information for `stock-monitor` GET requests

Setup email verification service using one time passcodes Create custom email template for stock monitor OTP Set Jarvis API version to match main module

Remove giant if else condition to looping Reduce latency on weather response Replace ``PreparedRequest`` with ``OrderedDict`` Combine both conversations and keywords in lookup Extend custom units for system vitals Use ordered dict to retain keywords order Create a dedicated builtin override section Fix a bug on extracting enum object value Remove password protection on secure-send Set main process' name to JARVIS Some more minor bug fixes and improvements Release production ready version

Remove todo for NLTK since compute time is high

Use OpenAI api for unmatched phrases Remove the usage of return flag in conditions Update version_locked_requirements.txt Release beta version to test

Fix bug on meanings for words Restrict certain tasks on offline commands Add exception handler for connection error Reorder keywords_base.py

Set keywords to be read only during startup Remove API endpoint for conversation words Remove redundancy in getting weather information

Reduce top level variable declarations Create function mapping dict in place of globals Reuse stock report for robinhood summarization

Update all object level calls to dict calls for keywords Increase response time for wake word detection Remove unused indicator sounds

Avoid writing on screen when called by offline

Set retry mode for put listener status Create a new feature for custom conditions (on hold) Override timeout and phrase limit in recognizer settings

Remove redundancy in weather alert trigger Set weather alert trigger via background tasks Keep weather alert time format consistent Add an option to manually enable the listener Add automation and smart devices to files.py Cleanup keywords_handler.py Rename timeout and phrase_limit to more sensible ones Auto store LG tv's client key in smart_devices.yaml

Allow restart via ``automation.yaml`` Log vehicle auth expiration Fix missing hosted device information in logs Add inline comments Instantiate vehicle objects

Better way to distinguish smart devices Reduce redundancy and re-use variables Add more keywords for secure send Update CORS allowance for headers Update README.md and type hints Bump version

Set log file names to match the process names

Add low and high threshold for weather alerts Update README.md and bump sub-version

Remove unwanted ticker gathering methods Release version 3.2

Fix some random start up errors and delays Create an option to get stock tickers as a dict Update docs page for API endpoint Release v3.3

Upgrade version locked dependencies Update keywords for events/holidays Override defaults in listener function Update standalone speaker module to get voice by name Fix no response in failed face detection Fix one more random startup error

Fix secret access via offline communicator

ishaan-jaff · 2023-08-05T00:04:44Z

@thevickypedia can you take a look at this PR? Happy to add more comments/ info in the readme if the initial commit looks good😊

thevickypedia · 2023-08-05T13:35:18Z

@ishaan-jaff Thanks for raising a PR. I would like to run an extensive test the library.
As you may have already noticed, my package is a natural language solution and mostly related to home-automation and serving as a personal-assistant rather than an AI. So it has little to no use for ML/DL libraries.
Nevertheless, I'll go through your library and analyze the use-case scenarios before I can integrate it.

DawoodTouseef · 2023-09-17T04:22:19Z

It's best to integrate with open ai it answer the user request generating it instead of using the specfic command which you given it in jarvis/modules/conditions/conversation.py instead of that we can integrate it with open ai

dormant-user · 2024-05-21T22:13:56Z

I'm revisiting this PR after a long time. I did research about all the OpenAI solutions
Unfortunately all the APIs are behind a payment firewall (one way or the other), and I'm totally against paying for a feature solely to integrate with Jarvis which is free otherwise
So I decided to create my own private AI with ollama2, I'm still in the initial stages of customizing the model
I'll have the feature integrated in the near future, so I'm closing this PR now
Thanks again for raising a thoughtful PR

dormant-user added 30 commits November 19, 2022 10:12

Update issue templates

49135fe

Fix a bug in garage door controller

84bc851

Identify the garage door by name Improve README.md Fix default password in vpn-server

Add responses when garage door controller is offline

17a4df8

Remove looping in garage module when device is chosen Send an email when vehicle is unlocked remotely Include timezone in vehicle's guardian mode response Create a dedicated module for functions that only uses builtins

Save process IDs and name map in a yaml file

ae844eb

Change module names to make better sense

Change function names to avoid import conflicts

2239418

Set smtp flag to false while email validation

128d4c9

Update gmail-connector to the latest version

FEATURE::Add Linux support

a96fc9b

Set to limited mode by default for Linux Update install.sh

Add more Linux supporting features

ad1ca84

Write processes.yaml file regardless of limited mode state

Switch CHANGELOG to release_notes.rst

60c3725

Remove CHANGELOG from pypi and link it to github Add more classifiers for pypi

More optimizations for Linux

c76ce24

Rename automation.yaml instead of removing if invalid Allow an option to brute force non-limited mode Fix release notes hyperlink in pypi Update README.md

Load templates upon startup

8953983

Add one click copy to emailOTP.html (Unsupported in emails)

Remove Javascript from email_OTP.html

96dcd8f

Add an option to choose between microphones to use

5f7a3e4

Implement a better way to get audio IO devices info

Load PyAudio during start up to avoid ALSA errors on Linux

bc54d40

Add a condition check for weather location to avoid crash Remove the usage of `time.perf_counter()` Optimize globally accessible variables

Add support for multiple TVs and add RokuTV controls

8c29ef5

Restructure the usage of `smart_devices.yaml` file Update README.md and requirements.txt

Bug fix for RokuTV

822cad1

Check for existing app before launching `Home` Fix method fetching current app on TV

Add feature to control multiple TVs simultaneously

105a0e7

Iterate over a loop to power on and launch home for RokuTVs Improve type hints

Minor optimizations for memory usage and efficiency

725aeab

More clear logging and inline comments

Make time_converter grammatically correct

2e11ef7

Bug fix on background_tasks

a274c64

Move background tasks from thread to process as `called_by_offline` flag should not be set in main process Running it as a thread will raise `RuntimeError` as `runAndWait()` will not be called Remove unsafe code in `pluralize` function

Add custom processName to log format

66a59b7

Override logging filter to add process name Create process mapping file along with the components handled Use timed wait to optimize CPU utilization on long-running processes

Set defaults on SpeechSynthesisModal as per env vars

eee7889

Add an api endpoint to get voices available for speech synthesis

5284c19

Make API docs page look neater

Bug fixes and Remove redundancies

7238e1a

Custom keyword mapping being overwritten Writing frequently used functions in yaml file Weather breaking when phrase is available but not place Remove redundant regex when getting capitalized

Read background tasks via YAML file instead of env vars

1f46d5d

Change some HTTP requests methods from POST to GET calls Add an option to disable background tasks on demand Make `yaml` files in fileio directory to be available via API calls Update README.md

Add exception handlers in stock-monitor endpoint for JWT

e53a5c5

Fix native audio conversion in tts_stt.py Get plain text information for `stock-monitor` GET requests

Make stock-monitor endpoint open-source

b4d7902

Setup email verification service using one time passcodes Create custom email template for stock monitor OTP Set Jarvis API version to match main module

dormant-user and others added 22 commits May 5, 2023 00:04

Fix inconsistent offline compatibles

6e22dac

Remove todo for NLTK since compute time is high

Replace Wolfram-Alpha with OpenAI API wrapper

63e8579

Use OpenAI api for unmatched phrases Remove the usage of return flag in conditions Update version_locked_requirements.txt Release beta version to test

Remove restriction on offline tasks

cd17e59

Fix bug on meanings for words Restrict certain tasks on offline commands Add exception handler for connection error Reorder keywords_base.py

Add info to response on failed lights functions

45b60d6

Set keywords to be read only during startup Remove API endpoint for conversation words Remove redundancy in getting weather information

Optimize memory usage and remove globals

b11a0c3

Reduce top level variable declarations Create function mapping dict in place of globals Reuse stock report for robinhood summarization

Retain keywords as OrderedDict instead of Object

1a7d1a5

Update all object level calls to dict calls for keywords Increase response time for wake word detection Remove unused indicator sounds

Use py script to get OSName in install.sh

2fbe62c

Avoid writing on screen when called by offline

Add a feature to access all TVs at once

e66b137

Set retry mode for put listener status Create a new feature for custom conditions (on hold) Override timeout and phrase limit in recognizer settings

Fix flaky connection reset in car module

514294d

Allow restart via ``automation.yaml`` Log vehicle auth expiration Fix missing hosted device information in logs Add inline comments Instantiate vehicle objects

Store process name(s) in Settings object to reuse

80dea96

Allow secure-send to work via UI

702dee3

Better way to distinguish smart devices Reduce redundancy and re-use variables Add more keywords for secure send Update CORS allowance for headers Update README.md and type hints Bump version

Fix flaky response for failed lights

f165357

Limit value for background syncs and log retention

49013eb

Set log file names to match the process names

Multiple tasks to run simultaneously in automation

14ce4eb

Add low and high threshold for weather alerts Update README.md and bump sub-version

Implement a new feature to analyze stocks

82b0d5e

Remove unwanted ticker gathering methods Release version 3.2

Brute force timeout on ChatGPT instance creation

3d8a22c

Fix some random start up errors and delays Create an option to get stock tickers as a dict Update docs page for API endpoint Release v3.3

Add a feature to get holidays across the world

3065ca4

Upgrade version locked dependencies Update keywords for events/holidays Override defaults in listener function Update standalone speaker module to get voice by name Fix no response in failed face detection Fix one more random startup error

Change response for ChatGPT's failed authentication

f81778e

Fix secret access via offline communicator

Avoid <1 values for correction in stock-monitor

29d450d

v0 litellm

116a089

thevickypedia added the enhancement New feature or request label Aug 8, 2023

thevickypedia marked this pull request as draft August 9, 2023 19:27

thevickypedia force-pushed the master branch from 703b342 to 653589d Compare October 24, 2023 21:31

dormant-user closed this May 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Azure, OpenAI, Palm, Anthropic, Cohere, Replicate Models - using litellm #37

Add support for Azure, OpenAI, Palm, Anthropic, Cohere, Replicate Models - using litellm #37

ishaan-jaff commented Aug 5, 2023

ishaan-jaff commented Aug 5, 2023

thevickypedia commented Aug 5, 2023

DawoodTouseef commented Sep 17, 2023 •

edited

dormant-user commented May 21, 2024

Add support for Azure, OpenAI, Palm, Anthropic, Cohere, Replicate Models - using litellm #37

Add support for Azure, OpenAI, Palm, Anthropic, Cohere, Replicate Models - using litellm #37

Conversation

ishaan-jaff commented Aug 5, 2023

ishaan-jaff commented Aug 5, 2023

thevickypedia commented Aug 5, 2023

DawoodTouseef commented Sep 17, 2023 • edited

dormant-user commented May 21, 2024

DawoodTouseef commented Sep 17, 2023 •

edited