Merge remote-tracking branch 'origin/main'

README.md

@@ -1,57 +1,81 @@
-# VizChat
+<h1 align="center"> 🤖 VizChat: Enhancing Learning Analytics Dashboards with Contextualised Explanations </h1>
 
-This extension offers you a shortcut to use gpt-4-vision-preview model to analyse the opened web page.
+VizChat is an open-source Chrome extension designed to augment web-based Learning Analytics Dashboards (LADs) by providing contextualized explanations for visualizations through integrating multimodal generative AI and Retrieval-Augmented Generation (RAG). VizChat comprises three main components: 
 
-## How to install
+- A) a knowledge database that stores relevant contextual information as vector embeddings, 
 
-### Get the extension file
+- B) a prompt synthesis mechanism through a context agent that crafts context-specific prompts,
 
-You can find the newest release version in a zip format in release tab.
+- C) a contextualized explanation generation process that leverages a multimodal generative AI model to produce explanations based on the enriched prompts. 
 
-OR you can clone the project locally and compile it by yourself using. You need to install npm and node.js to compile.
-The author used npm in version 10.5.0 and node.js 21.7.2
+This architecture enables VizChat to deliver precise, relevant, and easy-to-understand explanations, enhancing the educational value of LADs by making them more accessible and informative for users.
 
-```
+<p align="center">
+<img src="./img/VizChat-system.png" width="512">
+</p>
 
-npm install
+**Use Cases**: Here is an example of using VizChat to facilitate students' in-depth understanding of a complicated LAD to support reflective practices. As shown by the examples, VizChat can provide contextualized and personal responses by utilising the insights presented in one or more visualizations and explaining the underlying data collection and analysis procedures.
 
-npm run build
+<p align="center">
+<img src="./img/VizChat-case.png" width="512">
+</p>
+
+## How to Install
+
+### Obtain the Extension
+
+You can find the extension in zip format in **releases tab**. 
+Click here to download the [Chrome extension package](https://github.com/LinxZhao/VizChat-pub/releases/download/0.2.1/chromium-v0.2.2.zip).
+
+You can also clone the repository and compile the extension yourself. Ensure npm and node.js are installed:
+- Recommended npm version: >=10.5.0
+- Recommended node.js version: >=21.7.2
+
+Commands to run:
 
 ```
+npm install
+npm run build
+```
+
+### Extension Setup
 
-### Install the extension
+Once downloaded, navigate to [your Chrome extension page](chrome://extensions/). Activate **developer mode** in the top right corner, then drag and drop the zip file (**chromium.zip**) onto the page.
 
-After downloading, go to [your chrome extension page](chrome://extensions/). Enable developer mode on the top right corner. 
-Then drag and drop zip file into this chrome extension page. 
+If installed successfully, it will appear in your extension list. Click 'Details', then enable 'Pin to toolbar' to make VizChat accessible from your extension bar.
 
-If installed correctly, you can find it in extension list. Then click the 'Details' button, then toggle the 
-'Pin to toolbar' to make the VizChat visible on your extension bar.
+To use VizChat, add an OpenAI API key in the **Advanced** options page, accessible by right-clicking the VizChat icon and selecting Option, then the Advanced tab. This key is stored locally for privacy.
 
-Before starting the chat, you need to add an openAI API key to access the model. The key is saved locally, so others would not 
-have access to it. You can add the key in Advance option page. You can access it by right click the VizChat icon and click on
-Option and click on Advanced tab.
+Now, refresh a web page, click the VizChat icon, and start a chat to ask questions about the visible part of the webpage.
 
+## Features
 
-Now you can go to a web page, refresh it, and click on the icon of VizChat to start a chat asking chatgpt questions
-about the web page. 
+### Web-based LADs Interaction
 
-## Feature
+To engage with VizChat, click its icon. Note that VizChat's analysis and discussion capabilities are limited to the portion of the Learning Analytics Dashboards (LADs) currently visible on your screen. If you wish to explore another section, you must adjust your view and initiate a new chat session. 
 
-### Ask questions of a web page
-You can click the icon on the extension tool bar. It should be on the top right corner of your chrome. Once click, a 
-conversation box will appear and you can conversate with gpt-4-vision-preview about the web page that is visible to you
-(without counting the conversation box of VizChat).
+It's important to remember that VizChat's ability to capture the webpage relies on Chrome's extension screenshot functionality, which is only triggered when you activate the chat (for instance, by right-clicking the icon). Consequently, if your LADs include multiple tabs or sections requiring separate views, you'll need to reactivate VizChat each time you switch to a different tab or section to ensure it can analyze the new content accurately.
 
-Be aware, this extension can only capture the part of the web page that is visible to you. You need
-to make the part of webpage visible to before start the chat. 
-If you want to ask question for another part of the same page, you need to close the conversation box and start a new chat.
+### Knowledge File Integration
+Add PDFs as knowledge files in the **Advanced** options. VizChat stores these locally and uses them to provide more relevant responses based on your queries. Note, only the pertinent sections of these documents are sent with your question for privacy.
 
-### Refer to knowledge files
-You can add knowledge file (currently can only receive pdf) and VizChat can save it in the local storage space of your 
-chrome to use them as potential background knowledge. VizChat will retrieve the most relevant segment of these documents
-as contextual knowledge based on your query. This means the document will be saved locally, but part of the document (the retrieved part) will be send together with your question.
+## Citation
+Please use the following citation for academic purposes:
+
+```
+@inproceedings{yan2024vizchat,
+  title={VizChat: Enhancing Learning Analytics Dashboards with Contextualised Explanations using Multimodal Generative AI Chatbots},
+  author={Yan, Lixiang and Zhao, Linxuan and Echeverria, Vanessa and Jin, Yueqiao and Alfredo, Riordan and Gasevic, Dragan and Martinez-Maldonado, Roberto},
+  booktitle={Proceedings of the 25th International Conference on Artificial Intelligence Education},
+  year={2024},
+  organization={Springer}
+}
+```
 
-The knowledge file can be added in the Advance option page. 
+## Contact
+Project Leads:
+- Lixiang Yan: [email protected]
+- Linxuan Zhao: [email protected]
 
 ## Acknowledgement
-This extension is developed based on: https://github.com/josStorer/chatGPTBox
+This extension builds upon the work found at: https://github.com/josStorer/chatGPTBox