Create docker image

[tkellogg]

Create a docker image that runs server.py via uvicorn on all host names, exposes the port, etc. I'll push the image to docker central.

[aryannargotra]

Can you assign this to me

[tkellogg]

By the way, last night I changed it to use models via llm. llm has a configuration file where it configures things like API keys and downloaded models. The directory that holds all that can be overridden via the LLM_USER_PATH environment variable (see here. I'm unsure how well this is documented, I just found that by looking in the code.

[aryannargotra]

Hi Tim,
I read your blog about Mastodon, and it's been so long since I truly liked something and found it really meaningful. It's such a great experience, and I'll try to contribute to this cause and learn new things if I have the opportunity.

I've changed jobs three times in the past year because I wasn't finding anything interesting in my work, which led to a loss of interest in the tech field. However, your project has really got me excited.

[aryannargotra]

Having issue installing the 3.10 version in my container

[tkellogg]

i wish it was that easy to install a new version of python, but oh, no, it truly sucks. Here's a couple easier methods

• conda
• pyenv

Honestly, just change pyproject.toml to allow python >=3.9

or rather, since you're using docker anyway, use the python3.10 base image

[tkellogg]

Also, since you're in docker anyway, don't bother with the virtual environment. Docker provides all the isolation you need

[aryannargotra]

I am Using this DosckerFile but getting out of memory error while building it

[tkellogg]

Out of memory while building? 🤔 if you're out of docker cache space you can look into clearing it, that does happen a lot (although you seem to know what you're doing, so maybe you tried that already). If you're out of RAM, I'm not sure what to say, none of this is particularly big. I've used the python image before and it's quite small. llm does install a lot of packages, but I didn't think that really consumed much RAM, per se.

What stage does it die on?

[aryannargotra]

around here when it checks for vulnerabilities, it gets stuck. I changed the python version to 3.10.13 as it was in recommedations while scouting & now trying again lets see

[tkellogg]

Are you talking about the "There is not enough space on the disk" error? You might need to increase the space available to the Docker cache. Regardless, that's a giant image, are we really using all that?

[aryannargotra]

Do we also need to config the openAI key?, Its in the config file.
I've only configured the access token yet

[tkellogg]

i don't think you want any secrets in the image itself. i imagine you'd pull them in via -e environment variables or import the user's config directory via a -v volume at runtime.

i mentioned this before, but llm stores all it's configuration in your home directory, but you can override where it's stored. if you didn't import a file location as a volume, then changes made at runtime would be ephemeral (and you couldn't distribute the image because there's secrets in there).

btw, you should probably revoke the access token you posted a couple screenshots ago, just to be safe.