Skip to content

ํ›„์ถ”๋Š” Langchain๐Ÿฆœ, Streamlit, Huggingface๐Ÿค— ๋“ฑ์œผ๋กœ ๊ตฌํ˜„๋œ RAG ์ถ”์ฒœ ์‹œ์Šคํ…œ์ž…๋‹ˆ๋‹ค. ์˜ฌ๋ฆฌ๋ธŒ์˜์˜ ๋ฆฌ๋ทฐ๋ฅผ ํฌ๋กค๋งํ•ด ๋ฒกํ„ฐDB๋ฅผ ์ƒ์„ฑํ•˜๊ณ , ์œ ์ €์˜ ์งˆ๋ฌธ๊ณผ ๊ฐ€์žฅ ์—ฐ๊ด€์„ฑ ๋†’์€ ๋ฆฌ๋ทฐ๋“ค์„ ๋ฐ”ํƒ•์œผ๋กœ ์œ ์ €์˜ ๊ธฐํ˜ธ๋ฅผ ๋ฐ˜์˜ํ•œ ํ™”์žฅํ’ˆ์„ ์ถ”์ฒœํ•ด์ค๋‹ˆ๋‹ค.

Notifications You must be signed in to change notification settings

Ye-eun-Kim/whochoo-1

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

69 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿง‚ Whochoo (ํ›„์ถ”); ์‹ค์‚ฌ์šฉ ๋ฆฌ๋ทฐ ๊ธฐ๋ฐ˜ ํ™”์žฅํ’ˆ ์ถ”์ฒœ ์ฑ—๋ด‡ ์„œ๋น„์Šค

ํ›„์ถ”๐Ÿง‚์˜ ๊ณต์‹ ๊นƒํ—ˆ๋ธŒ ๋ ˆํฌ์ž…๋‹ˆ๋‹ค.

drawing

Overview

ํ›„์ถ”๋Š” Langchain๐Ÿฆœ, Streamlit, Huggingface๐Ÿค— ๋“ฑ์œผ๋กœ ๊ตฌํ˜„๋œ RAG ์ถ”์ฒœ ์‹œ์Šคํ…œ์ž…๋‹ˆ๋‹ค. ์˜ฌ๋ฆฌ๋ธŒ์˜์˜ ๋ฆฌ๋ทฐ๋ฅผ ํฌ๋กค๋งํ•ด ๋ฒกํ„ฐDB๋ฅผ ์ƒ์„ฑํ•˜๊ณ , ์œ ์ €์˜ ์งˆ๋ฌธ๊ณผ ๊ฐ€์žฅ ์—ฐ๊ด€์„ฑ ๋†’์€ ๋ฆฌ๋ทฐ๋“ค์„ ๋ฐ”ํƒ•์œผ๋กœ ์œ ์ €์˜ ๊ธฐํ˜ธ๋ฅผ ๋ฐ˜์˜ํ•œ ํ™”์žฅํ’ˆ์„ ์ถ”์ฒœํ•ด์ค๋‹ˆ๋‹ค.

ํŒŒ์ผ ๋ฐ ๋””๋ ‰ํ† ๋ฆฌ ๋ชฉ๋ก

.
โ”œโ”€โ”€ assets
โ”œโ”€โ”€ chromedriver-win32
โ”œโ”€โ”€ data
โ”œโ”€โ”€ utils
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ crawling_one_json.py
โ”œโ”€โ”€ main.py
โ”œโ”€โ”€ rag.py
โ”œโ”€โ”€ rag_local.py
โ”œโ”€โ”€ requirements.txt
โ””โ”€โ”€ vectorstore.py

Content

Set-up & Installation

๊ฐœ๋ฐœ ํ™˜๊ฒฝ

  • Python 3.10.X
  • Linux, NVIDIA A100 GPU x 2, cuDNN 11.8 Python venv ์ƒ์„ฑ ํ›„, pip install -r requirements.txt

Run WhoChoo๐Ÿง‚

Run WhoChoo with Streamlit!

  1. Huggingface API Token ๋ฐœ๊ธ‰, ๊ฐœ์ธ ๋ฉ”๋ชจ, ๋กœ์ปฌ ๊ธฐ๊ธฐ ๋“ฑ์— ์ €์žฅ
  2. utils/arguments.py ๋‚ด token์˜ ๊ธฐ๋ณธ๊ฐ’ ์ˆ˜์ •
  3. Streamlit์œผ๋กœ ์‹คํ–‰:
    streamlit run main.py --server.port <port number>
    
  4. ์ฑ„ํŒ…์ฐฝ์— ์›ํ•˜๋Š” ํ™”์žฅํ’ˆ ํŠน์ง• ์ž…๋ ฅ

Test WhoChoo on Your Device!

Streamlit ํ”„๋ ˆ์ž„์›Œํฌ ์—†์ด, python ์‹คํ–‰ ํŒŒ์ผ๋งŒ์œผ๋กœ ํ›„์ถ”๋ฅผ ํ…Œ์ŠคํŠธํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์•ˆ๋‚ดํ•ฉ๋‹ˆ๋‹ค.

  1. Huggingface API ์ด์šฉํ•˜์—ฌ ํ…Œ์ŠคํŠธ
    python rag.py \
        --token [API token] \ 
        --model_id [huggingface model repo] \ # optional
        --top_k 10 \ # optional, default 5
    
  2. ๋กœ์ปฌ์—์„œ ๋ชจ๋ธ์„ ๋‹ค์šด๋กœ๋“œ ํ›„ ํ…Œ์ŠคํŠธ (GPU ์‚ฌ์šฉ ๊ถŒ์žฅ)
    python rag_local.py \
        --token [API token] \ 
        --model_id [huggingface model repo] \ # optional
        --top_k 10 \ # optional, default 5
    

Team WhoChoo

  • ๊น€์˜ˆ์€(ํŒ€์žฅ): ๊ธฐํš, ๋ฐ์ดํ„ฐ ํฌ๋กค๋ง
  • ๊น€๋ฏผ์„œ(๊ธฐ์ˆ ์ด๊ด„): RAG ๊ตฌํ˜„
  • ์œค์„ธ๋ฆผ: UI/UX, ๋ฐœํ‘œ์ž๋ฃŒ
  • ๊น€์—ฐ์ฃผ: ๊ธฐํš(์ž๋ฃŒ์กฐ์‚ฌ), ๊ฐœ๋ฐœ(ํ”„๋กฌํ”„ํŠธ ์—”์ง€๋‹ˆ์–ด๋ง), ๋ฐœํ‘œ์ž๋ฃŒ
  • ๊น€๋„ํฌ: ๊ฐœ๋ฐœ (ํ”„๋ก ํŠธ์—”๋“œ)

About

ํ›„์ถ”๋Š” Langchain๐Ÿฆœ, Streamlit, Huggingface๐Ÿค— ๋“ฑ์œผ๋กœ ๊ตฌํ˜„๋œ RAG ์ถ”์ฒœ ์‹œ์Šคํ…œ์ž…๋‹ˆ๋‹ค. ์˜ฌ๋ฆฌ๋ธŒ์˜์˜ ๋ฆฌ๋ทฐ๋ฅผ ํฌ๋กค๋งํ•ด ๋ฒกํ„ฐDB๋ฅผ ์ƒ์„ฑํ•˜๊ณ , ์œ ์ €์˜ ์งˆ๋ฌธ๊ณผ ๊ฐ€์žฅ ์—ฐ๊ด€์„ฑ ๋†’์€ ๋ฆฌ๋ทฐ๋“ค์„ ๋ฐ”ํƒ•์œผ๋กœ ์œ ์ €์˜ ๊ธฐํ˜ธ๋ฅผ ๋ฐ˜์˜ํ•œ ํ™”์žฅํ’ˆ์„ ์ถ”์ฒœํ•ด์ค๋‹ˆ๋‹ค.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •