Skip to content

Server Offline with either llama.cpp or HF-Waitress #30

Description

@Rotoslider

Ubuntu 22.04, Installed LARS today using instruction from your github readme. Using Conda environment.
llama-server --version version: 3912 (edc26566)
built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
added to path in barshrc
model is in model directory.
if I run llama-server -m /home/asus/builds/LARS/lars_storage/models/dolphin-2.9.3-mistral-7B-32k-Q8_0.gguf -c 2048 from any where the server starts and I can chat with it.
main: server is listening on 127.0.0.1:8080 - starting the main loop
srv update_slots: all slots are idle

config.json
lars_server_log.log
llama_cpp_server_output_log.txt
The hf_waitress output log is blank and no json was created for it

What else do i need to give you to help troubleshoot?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions