The user can enter any topic name, the number of words the blog would be of and for which group of people it would cater to (like students or researchers or common people, etc.)
The UI of the application has been created using Python's Streamlit package.
The output is generated by indirectly by Meta's 7B Llama2 chat model. However since the original model that is desired to be used has 7 billion parameters, it restricts its usability because of resource constraints in many situations. To overcome this we use GGML version of the 7B chat model. (Link: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML)
The GGML version of this model which is only about 7GB in size, allows us to run the model locally on our computers. Besides being open source, GGML models provide some unique advantages. Some of them are:
Smaller Footprint: ggml utilizes quantization, compressing model parameters from 32-bit floats to smaller formats like 8-bit integers. This dramatically reduces model size, allowing it to fit on devices with limited storage like smartphones or laptops.
Lower Memory Needs: The compression also translates to significantly lower memory requirements during inference and training. This is crucial for running models on devices with limited RAM, opening doors for local execution.
Optimized for CPUs: ggml models are built for efficient computation on CPUs, unlike many models targeted for GPUs. This allows them to run effectively on devices without dedicated graphics cards, leading to faster inference and potentially training on personal devices.