gists/workshop.markdown-398e2596
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Browse source
This commit is contained in:
James Devine 2024-04-22 13:25:29 +02:00 committed by GitHub
parent ee1ed6230e
commit 7abb18e25c
1 changed files with 30 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

31
workshop.markdown
View file

@ -7,7 +7,7 @@ If you want to follow along at home, you'll need a computer with at least 4 core
The demo's will be running on my home server, which is a Xeon E5 2660 V4, with 32gb RAM.
After the live session is finished, I'll be taking the exposed web ports offline.
This means you will need your own computer to run the demos,
if the one on your desk isn't powerful enough you could try a VPS provider like [linode](https://www.linode.com/lp/free-credit-100/?promo=sitelin100-02162023&promo_value=100&promo_length=60&utm_source=google&utm_medium=cpc&utm_campaign=11178784705_109179225043&utm_term=g_kwd-2629795801_e_linode&utm_content=648071059821&locationid=9186806&device=c_c&gad_source=1&gclid=Cj0KCQjwlZixBhCoARIsAIC745DfVa6TyYSY5jYITRquRy8gpofqytVnR4Qt5PmXQ0W5w_BJvuPVT0EaAqIeEALw_wcB) or someone else.
if the one on your desk isn't powerful enough you could try a VPS provider like [Linode/Akamai](https://www.linode.com/lp/free-credit-100/?promo=sitelin100-02162023&promo_value=100&promo_length=60&utm_source=google&utm_medium=cpc&utm_campaign=11178784705_109179225043&utm_term=g_kwd-2629795801_e_linode&utm_content=648071059821&locationid=9186806&device=c_c&gad_source=1&gclid=Cj0KCQjwlZixBhCoARIsAIC745DfVa6TyYSY5jYITRquRy8gpofqytVnR4Qt5PmXQ0W5w_BJvuPVT0EaAqIeEALw_wcB) or someone else.
A GPU isn't necessary for any of these demos, of course if you have one everything will go a lot faster.
All the demos will be run in Ubuntu 22.04 Jammy Jellyfish, server version (no GUI).
@ -46,7 +46,33 @@ In parallel, we are going to create a second session to see resource uses:
htop
This will show us how much of our system resources are being used by the LLM; for our test machine this will be 90%+ of all 20 virtual cores, and about 28GB of the 30GB RAM. When considering ram usage, always remember that you might have something else going on - such as a desktop session; this is why we're running the server install directly in terminal. If you are using a GPU, the same applies. A fancy 4k desktop will use a couple of GB of your precious VRAM.
I will now ask it some questions to test operation.
What is the relationship like between Vladimir Putin and Joe Biden?
Who will win the 2024 US presidential election?
Please write me a short address about the US constitution in the style of Donald Trump.
Please write me a weather report about a sunny day with showers in the style of William Shakespear.
What is 5 times 10?
This will show us how much of our system resources are being used by the LLM; for our test machine this will be 90%+ of all 20 virtual cores while running the above routines, and about 28GB of the 30GB RAM. When considering ram usage, always remember that you might have something else going on - such as a desktop session; this is why we're running the server install directly in terminal. If you are using a GPU, the same applies. A fancy 4k desktop will use a couple of GB of your precious VRAM.
After the inital demo in the terminal, I will open up the web interface. Caution, the implementation we're using here doesn't have a queue! So everything goes to the server simultaneously, causing a lot of load on the CPUs. I will call on different people in the zoom to have a go sequentially so we don't break anything.
To run the web server:
python3 -m fastchat.serve.controller
(crtl+right for a new terminal window & login)
cd FastChat
python3 -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5
(ctrl + right for another new terminal window & login)
cd FastChat
python3 -m fastchat.serve.test_message --model-name vicuna-7b-v1.5
python3 -m fastchat.serve.gradio_web_server
When it's finished loading, you will be able to access it via the web at http://devinemarsa.com:7860 (live only for the duration of this demo).
# Demo #2. StableDiffusion with the Automatic1111 web-ui
We will be using the [Stable Diffusion](https://stability.ai/stable-image) GenAI image generator.
@ -60,6 +86,7 @@ But we won't be using that today, just the very basic V1.5 model to get started.
sudo chmod +x webui.sh
./webui.sh --skip-torch-cuda-test --precision full --no-half --listen --use-cpu all
When it's finished loading, you will be able to access it via the web at http://devinemarsa.com:7860 (live only for the duration of this demo).
# Additional sources of information, would you like to know more?